272 lines
11 KiB
Markdown
272 lines
11 KiB
Markdown
# /etc/containerd/config.toml 5 04/05/2022
|
||
|
||
## NAME
|
||
|
||
containerd-config.toml - configuration file for containerd
|
||
|
||
## SYNOPSIS
|
||
|
||
The **config.toml** file is a configuration file for the containerd daemon. The
|
||
file must be placed at **/etc/containerd/config.toml** or specified with the
|
||
**--config** option of **containerd** to be used by the daemon. If the file
|
||
does not exist at the appropriate location or is not provided via the
|
||
**--config** option containerd uses its default configuration settings, which
|
||
can be displayed with the **containerd config(1)** command.
|
||
|
||
## DESCRIPTION
|
||
|
||
The TOML file used to configure the containerd daemon settings has a short
|
||
list of global settings followed by a series of sections for specific areas
|
||
of daemon configuration. There is also a section for **plugins** that allows
|
||
each containerd plugin to have an area for plugin-specific configuration and
|
||
settings.
|
||
|
||
## FORMAT
|
||
|
||
**version**
|
||
: The version field in the config file specifies the config’s version. If no
|
||
version number is specified inside the config file then it is assumed to be a
|
||
version 1 config and parsed as such. Please use version = 2 to enable version 2
|
||
config as version 1 has been deprecated.
|
||
|
||
**root**
|
||
: The root directory for containerd metadata. (Default: "/var/lib/containerd")
|
||
|
||
**state**
|
||
: The state directory for containerd (Default: "/run/containerd")
|
||
|
||
**plugin_dir**
|
||
: The directory for dynamic plugins to be stored
|
||
|
||
**[grpc]**
|
||
: Section for gRPC socket listener settings. Contains the following properties:
|
||
|
||
- **address** (Default: "/run/containerd/containerd.sock")
|
||
- **tcp_address**
|
||
- **tcp_tls_cert**
|
||
- **tcp_tls_key**
|
||
- **uid** (Default: 0)
|
||
- **gid** (Default: 0)
|
||
- **max_recv_message_size**
|
||
- **max_send_message_size**
|
||
|
||
**[ttrpc]**
|
||
: Section for TTRPC settings. Contains properties:
|
||
|
||
- **address** (Default: "")
|
||
- **uid** (Default: 0)
|
||
- **gid** (Default: 0)
|
||
|
||
**[debug]**
|
||
: Section to enable and configure a debug socket listener. Contains four properties:
|
||
|
||
- **address** (Default: "/run/containerd/debug.sock")
|
||
- **uid** (Default: 0)
|
||
- **gid** (Default: 0)
|
||
- **level** (Default: "info") sets the debug log level. Supported levels are:
|
||
"trace", "debug", "info", "warn", "error", "fatal", "panic"
|
||
- **format** (Default: "text") sets log format. Supported formats are "text" and "json"
|
||
|
||
**[metrics]**
|
||
: Section to enable and configure a metrics listener. Contains two properties:
|
||
|
||
- **address** (Default: "") Metrics endpoint does not listen by default
|
||
- **grpc_histogram** (Default: false) Turn on or off gRPC histogram metrics
|
||
|
||
**disabled_plugins**
|
||
: Disabled plugins are IDs of plugins to disable. Disabled plugins won't be
|
||
initialized and started.
|
||
|
||
**required_plugins**
|
||
: Required plugins are IDs of required plugins. Containerd exits if any
|
||
required plugin doesn't exist or fails to be initialized or started.
|
||
|
||
**[plugins]**
|
||
: The plugins section contains configuration options exposed from installed plugins.
|
||
The following plugins are enabled by default and their settings are shown below.
|
||
Plugins that are not enabled by default will provide their own configuration values
|
||
documentation.
|
||
|
||
- **[plugins."io.containerd.monitor.v1.cgroups"]** has one option __no_prometheus__ (Default: **false**)
|
||
- **[plugins."io.containerd.service.v1.diff-service"]** has one option __default__, a list by default set to **["walking"]**
|
||
- **[plugins."io.containerd.gc.v1.scheduler"]** has several options that perform advanced tuning for the scheduler:
|
||
- **pause_threshold** is the maximum amount of time GC should be scheduled (Default: **0.02**),
|
||
- **deletion_threshold** guarantees GC is scheduled after n number of deletions (Default: **0** [not triggered]),
|
||
- **mutation_threshold** guarantees GC is scheduled after n number of database mutations (Default: **100**),
|
||
- **schedule_delay** defines the delay after trigger event before scheduling a GC (Default **"0ms"** [immediate]),
|
||
- **startup_delay** defines the delay after startup before scheduling a GC (Default **"100ms"**)
|
||
- **[plugins."io.containerd.runtime.v2.task"]** specifies options for configuring the runtime shim:
|
||
- **platforms** specifies the list of supported platforms
|
||
- **sched_core** Core scheduling is a feature that allows only trusted tasks
|
||
to run concurrently on cpus sharing compute resources (eg: hyperthreads on
|
||
a core). (Default: **false**)
|
||
- **[plugins."io.containerd.service.v1.tasks-service"]** has performance options:
|
||
- **blockio_config_file** (Linux only) specifies path to blockio class definitions
|
||
(Default: **""**). Controls I/O scheduler priority and bandwidth throttling.
|
||
See [blockio configuration](https://github.com/intel/goresctrl/blob/main/doc/blockio.md#configuration)
|
||
for details of the file format.
|
||
- **rdt_config_file** (Linux only) specifies path to a configuration used for configuring
|
||
RDT (Default: **""**). Enables support for Intel RDT, a technology
|
||
for cache and memory bandwidth management.
|
||
See [RDT configuration](https://github.com/intel/goresctrl/blob/main/doc/rdt.md#configuration)
|
||
for details of the file format.
|
||
- **[plugins."io.containerd.grpc.v1.cri".containerd]** contains options for the CRI plugin, and child nodes for CRI options:
|
||
- **default_runtime_name** (Default: **"runc"**) specifies the default runtime name
|
||
- **[plugins."io.containerd.grpc.v1.cri".containerd.runtimes]** one or more container runtimes, each with a unique name
|
||
- **[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.<runtime>]** a runtime named `<runtime>`
|
||
- **[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.<runtime>.options]** options for the named `<runtime>`, most important:
|
||
- **BinaryName** specifies the path to the actual runtime to be invoked by the shim, e.g. `"/usr/bin/runc"`
|
||
|
||
|
||
|
||
|
||
**oom_score**
|
||
: The out of memory (OOM) score applied to the containerd daemon process (Default: 0)
|
||
|
||
**[cgroup]**
|
||
: Section for Linux cgroup specific settings
|
||
|
||
- **path** (Default: "") Specify a custom cgroup path for created containers
|
||
|
||
**[proxy_plugins]**
|
||
: Proxy plugins configures plugins which are communicated to over gRPC
|
||
|
||
- **type** (Default: "")
|
||
- **address** (Default: "")
|
||
|
||
**timeouts**
|
||
: Timeouts specified as a duration
|
||
|
||
<!-- [timeouts]
|
||
"io.containerd.timeout.shim.cleanup" = "5s"
|
||
"io.containerd.timeout.shim.load" = "5s"
|
||
"io.containerd.timeout.shim.shutdown" = "3s"
|
||
"io.containerd.timeout.task.state" = "2s" -->
|
||
|
||
**imports**
|
||
: Imports is a list of additional configuration files to include.
|
||
This allows to split the main configuration file and keep some sections
|
||
separately (for example vendors may keep a custom runtime configuration in a
|
||
separate file without modifying the main `config.toml`).
|
||
Imported files will overwrite simple fields like `int` or
|
||
`string` (if not empty) and will append `array` and `map` fields.
|
||
Imported files are also versioned, and the version can't be higher than
|
||
the main config.
|
||
|
||
**stream_processors**
|
||
|
||
- **accepts** (Default: "[]") Accepts specific media-types
|
||
- **returns** (Default: "") Returns the media-type
|
||
- **path** (Default: "") Path or name of the binary
|
||
- **args** (Default: "[]") Args to the binary
|
||
|
||
## EXAMPLES
|
||
|
||
### Complete Configuration
|
||
|
||
The following is a complete **config.toml** default configuration example:
|
||
|
||
```toml
|
||
version = 2
|
||
|
||
root = "/var/lib/containerd"
|
||
state = "/run/containerd"
|
||
oom_score = 0
|
||
imports = ["/etc/containerd/runtime_*.toml", "./debug.toml"]
|
||
|
||
[grpc]
|
||
address = "/run/containerd/containerd.sock"
|
||
uid = 0
|
||
gid = 0
|
||
|
||
[debug]
|
||
address = "/run/containerd/debug.sock"
|
||
uid = 0
|
||
gid = 0
|
||
level = "info"
|
||
|
||
[metrics]
|
||
address = ""
|
||
grpc_histogram = false
|
||
|
||
[cgroup]
|
||
path = ""
|
||
|
||
[plugins]
|
||
[plugins."io.containerd.monitor.v1.cgroups"]
|
||
no_prometheus = false
|
||
[plugins."io.containerd.service.v1.diff-service"]
|
||
default = ["walking"]
|
||
[plugins."io.containerd.gc.v1.scheduler"]
|
||
pause_threshold = 0.02
|
||
deletion_threshold = 0
|
||
mutation_threshold = 100
|
||
schedule_delay = 0
|
||
startup_delay = "100ms"
|
||
[plugins."io.containerd.runtime.v2.task"]
|
||
platforms = ["linux/amd64"]
|
||
sched_core = true
|
||
[plugins."io.containerd.service.v1.tasks-service"]
|
||
blockio_config_file = ""
|
||
rdt_config_file = ""
|
||
```
|
||
|
||
### Multiple Runtimes
|
||
|
||
The following is an example partial configuraton with two runtimes:
|
||
|
||
```toml
|
||
[plugins]
|
||
|
||
[plugins."io.containerd.grpc.v1.cri"]
|
||
|
||
[plugins."io.containerd.grpc.v1.cri".containerd]
|
||
default_runtime_name = "runc"
|
||
|
||
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes]
|
||
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
|
||
privileged_without_host_devices = false
|
||
runtime_type = "io.containerd.runc.v2"
|
||
|
||
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
|
||
BinaryName = "/usr/bin/runc"
|
||
|
||
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.other]
|
||
privileged_without_host_devices = false
|
||
runtime_type = "io.containerd.runc.v2"
|
||
|
||
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.other.options]
|
||
BinaryName = "/usr/bin/path-to-runtime"
|
||
```
|
||
|
||
The above creates two named runtime configurations - named `runc` and `other` - and sets the default runtime to `runc`.
|
||
The above are used _solely_ for runtimes invoked via CRI. To use the non-default "other" runtime in this example,
|
||
a spec will include the runtime handler named "other" to specify the desire to use the named runtime config.
|
||
|
||
The CRI specification includes a [`runtime_handler` field](https://github.com/kubernetes/cri-api/blob/de5f1318aede866435308f39cb432618a15f104e/pkg/apis/runtime/v1/api.proto#L476), which will reference the named runtime.
|
||
|
||
It is important to note the naming convention. Runtimes are under `[plugins."io.containerd.grpc.v1.cri".containerd.runtimes]`,
|
||
with each runtime given a unique name, e.g. `[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]`.
|
||
In addition, each runtime can have shim-specific options under `[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.<runtime>.options]`,
|
||
for example, `[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]`.
|
||
|
||
The `io.containerd.runc.v2` runtime is used to run OCI-compatible runtimes on Linux, such as runc. In the example above, the `runtime_type`
|
||
field specifies the shim to use (`io.containerd.runc.v2`) while the `BinaryName` field is a shim-specific option which specifies the path to the
|
||
OCI runtime.
|
||
|
||
For the example configuration named "runc", the shim will launch `/usr/bin/runc` as the OCI runtime. For the example configuration named
|
||
"other", the shim will launch `/usr/bin/path-to-runtime` instead.
|
||
|
||
## BUGS
|
||
|
||
Please file any specific issues that you encounter at
|
||
https://github.com/containerd/containerd.
|
||
|
||
## AUTHOR
|
||
|
||
Phil Estes <estesp@gmail.com>
|
||
|
||
## SEE ALSO
|
||
|
||
ctr(8), containerd-config(8), containerd(8)
|