A fifo on unix or named pipe on Windows will be provided to the shim.
It can be located inside the `cwd` of the shim named "log".
The shims can use the existing `github.com/containerd/containerd/log` package to log debug messages.
Messages will automatically be output in the containerd's daemon logs with the correct fiels and runtime set.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Some images like `criu` will have extra libs that it requires. This
adds lib support via LD_LIBRARY_PATH and InstallOpts
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
This adds a way for users to programatically install containerd binary
dependencies.
With runtime v2 and new shim's being built, it will be a challenge to
get those onto machines. Users would have to find the link, download,
place it in their path, yada yada yada.
With this functionality of a managed `/opt` directory, containerd can
use existing image and distribution infra. to get binarys, shims, etc
onto the system.
Configuration:
*default:* `/opt/containerd`
*containerd config:*
```toml
[plugins.opt]
path = "/opt/mypath"
```
Usage:
*code:*
```go
image, err := client.Pull(ctx, "docker.io/crosbymichael/runc:latest")
client.Install(ctx, image)
```
*ctr:*
```bash
ctr content fetch docker.io/crosbymichael/runc:latest
ctr install docker.io/crosbymichael/runc:latest
```
You can manage versions and see what is running via standard image
commands.
Images:
These images MUST be small and only contain binaries.
```Dockerfile
FROM scratch
Add runc /bin/runc
```
Containerd will only extract files in `/bin` of the image.
Later on, we can add support for `/lib`.
The code adds a service to manage an `/opt/containerd` directory and
provide that path to callers via the introspection service.
How to Test:
Delete runc from your system.
```bash
> sudo ctr run --rm docker.io/library/redis:alpine redis
ctr: OCI runtime create failed: unable to retrieve OCI runtime error (open /run/containerd/io.containerd.runtime.v1.linux/default/redis/log.json: no such file or directory): exec: "runc": executable file not found in $PATH: unknown
> sudo ctr content fetch docker.io/crosbymichael/runc:latest
> sudo ctr install docker.io/crosbymichael/runc:latest
> sudo ctr run --rm docker.io/library/redis:alpine redis
1:C 01 Aug 15:59:52.864 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 01 Aug 15:59:52.864 # Redis version=4.0.10, bits=64, commit=00000000, modified=0, pid=1, just started
1:C 01 Aug 15:59:52.864 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
1:M 01 Aug 15:59:52.866 # You requested maxclients of 10000 requiring at least 10032 max file descriptors.
1:M 01 Aug 15:59:52.866 # Server can't set maximum open files to 10032 because of OS error: Operation not permitted.
1:M 01 Aug 15:59:52.866 # Current maximum open files is 1024. maxclients has been reduced to 992 to compensate for low ulimit. If you need higher maxclients increase 'ulimit -n'.
1:M 01 Aug 15:59:52.870 * Running mode=standalone, port=6379.
1:M 01 Aug 15:59:52.870 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
1:M 01 Aug 15:59:52.870 # Server initialized
1:M 01 Aug 15:59:52.870 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
1:M 01 Aug 15:59:52.870 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
1:M 01 Aug 15:59:52.870 * Ready to accept connections
^C1:signal-handler (1533139193) Received SIGINT scheduling shutdown...
1:M 01 Aug 15:59:53.472 # User requested shutdown...
1:M 01 Aug 15:59:53.472 * Saving the final RDB snapshot before exiting.
1:M 01 Aug 15:59:53.484 * DB saved on disk
1:M 01 Aug 15:59:53.484 # Redis is now ready to exit, bye bye...
```
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Reorders the code so that it doesnt overwrite the previous allocation
when creating a NewTask via ctr.exe
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
We introduce a WithSpecFromFile option combinator to allow creation
simpler creation of OCI specs from a file name. Often used as the first
option in a `SpecOpts` slice, it simplifies choosing between a local
file and the built-in default.
The code in `ctr run` has been updated to use the new option, with out
changing the order of operations or functionality present there.
Signed-off-by: Stephen Day <stephen.day@getcruise.com>
Implements the various requirements for the runtime v2 code to abstract
away the unix/linux code into the appropriate platform level
abstractions to use the runtime v2 on Windows as well.
Adds support in the Makefile.windows to actually build the runtime v2
code for Windows by setting a shell environment BUILD_WINDOWS_V2=1
before calling make. (Note this disables the compilation of the Windows
runtime v1)
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
This patch changes the logs format to use a fixed-width timestamp,
matching the format that's used in dockerd.
Before:
$ containerd
INFO[0000] starting containerd revision=a88b6319614de846458750ff882723479ca7b1a1 version=v1.1.0-202-ga88b6319
INFO[0000] loading plugin "io.containerd.content.v1.content"... type=io.containerd.content.v1
INFO[0000] loading plugin "io.containerd.snapshotter.v1.btrfs"... type=io.containerd.snapshotter.v1
WARN[0000] failed to load plugin io.containerd.snapshotter.v1.btrfs error="path /var/lib/containerd/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter"
After:
$ containerd
INFO[2018-07-24T08:11:07.397856489Z] starting containerd revision=c3195155cacb361cd3549c4d78901b20aa19579a version=v1.1.0-203-gc3195155
INFO[2018-07-24T08:11:07.399264587Z] loading plugin "io.containerd.content.v1.content"... type=io.containerd.content.v1
INFO[2018-07-24T08:11:07.399343959Z] loading plugin "io.containerd.snapshotter.v1.btrfs"... type=io.containerd.snapshotter.v1
WARN[2018-07-24T08:11:07.399474423Z] failed to load plugin io.containerd.snapshotter.v1.btrfs error="path /var/lib/containerd/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter"
Or, when running as child-process of dockerd:
Before:
root@9637fcd85ea4:/go/src/github.com/docker/docker# dockerd --debug
DEBU[2018-07-24T08:15:16.946312436Z] Listener created for HTTP on unix (/var/run/docker.sock)
INFO[2018-07-24T08:15:16.947086499Z] libcontainerd: started new docker-containerd process pid=231
INFO[2018-07-24T08:15:16.947137166Z] parsed scheme: "unix" module=grpc
INFO[2018-07-24T08:15:16.947235001Z] scheme "unix" not registered, fallback to default scheme module=grpc
INFO[2018-07-24T08:15:16.947463403Z] ccResolverWrapper: sending new addresses to cc: [{unix:///var/run/docker/containerd/docker-containerd.sock 0 <nil>}] module=grpc
INFO[2018-07-24T08:15:16.947505954Z] ClientConn switching balancer to "pick_first" module=grpc
INFO[2018-07-24T08:15:16.947717368Z] pickfirstBalancer: HandleSubConnStateChange: 0xc420507ab0, CONNECTING module=grpc
INFO[0000] starting containerd revision=d64c661f1d51c48782c9cec8fda7604785f93587 version=v1.1.1
DEBU[0000] changing OOM score to -500
INFO[0000] loading plugin "io.containerd.content.v1.content"... type=io.containerd.content.v1
INFO[0000] loading plugin "io.containerd.snapshotter.v1.btrfs"... type=io.containerd.snapshotter.v1
WARN[0000] failed to load plugin io.containerd.snapshotter.v1.btrfs error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter"
After:
DEBU[2018-07-24T08:21:33.441741970Z] Listener created for HTTP on unix (/var/run/docker.sock)
INFO[2018-07-24T08:21:33.442428017Z] libcontainerd: started new docker-containerd process pid=232
INFO[2018-07-24T08:21:33.442510827Z] parsed scheme: "unix" module=grpc
INFO[2018-07-24T08:21:33.442598812Z] scheme "unix" not registered, fallback to default scheme module=grpc
INFO[2018-07-24T08:21:33.442681006Z] ccResolverWrapper: sending new addresses to cc: [{unix:///var/run/docker/containerd/docker-containerd.sock 0 <nil>}] module=grpc
INFO[2018-07-24T08:21:33.442770353Z] ClientConn switching balancer to "pick_first" module=grpc
INFO[2018-07-24T08:21:33.442871502Z] pickfirstBalancer: HandleSubConnStateChange: 0xc42018bc30, CONNECTING module=grpc
INFO[2018-07-24T08:21:33.457963804Z] starting containerd revision=597dd082e37f8bc6b6265ca05839d7a300861911 version=597dd082
DEBU[2018-07-24T08:21:33.458113301Z] changing OOM score to -500
INFO[2018-07-24T08:21:33.458474842Z] loading plugin "io.containerd.content.v1.content"... type=io.containerd.content.v1
INFO[2018-07-24T08:21:33.458911054Z] loading plugin "io.containerd.snapshotter.v1.btrfs"... type=io.containerd.snapshotter.v1
WARN[2018-07-24T08:21:33.459366268Z] failed to load plugin io.containerd.snapshotter.v1.btrfs error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter"
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Separate Fetch and Pull commands in client to distinguish
between platform specific and non-platform specific operations.
`ctr images pull` with all platforms will now unpack all platforms.
`ctr content fetch` now supports platform flags.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
`Ctr` interface follows the pattern `ctr <command> <subcommand>` except
for the `plugins` command which does not have subcommands. This feels
unnatural to certain users and they would expect that they can list
containerd plugins via `ctr plugins list`.
This commit implements their expectation so that `plugins` becomes a
command "group" and its `list` subcommand actually lists the plugins.
Signed-off-by: Danail Branekov <danailster@gmail.com>
This adds a `Load` Opt for cio to load a tasks io/fifos without
attaching or starting the copy routines.
It adds the load method in `ctr` by default so that fifos or other IO
are removed from disk on delete methods inbetween command runs. It is
not the default for all task loads for backwards compat. and a user may
want to keep io around to reuse or if log files are used.
Fixes#2421
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Commit 05513284e7 exposed the "rootfs"
and "no-pivot" flags for the "containers" command, but it accidentally
removed them for "run" since package-level variables are initialized
before package-level init functions in golang. Hoisting these flags to
a package imported by both commands solves the problem.
Signed-off-by: Felix Abecassis <fabecassis@nvidia.com>
This change allows implementations to resolve the location of the actual data
using OCI descriptor fields such as MediaType.
No OCI descriptor field is written to the store.
No change on gRPC API.
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
