Commit Graph

192 Commits

Author SHA1 Message Date
John Howard
e0d7ed12a5 Windows: Enable ETW logging
Signed-off-by: John Howard <jhoward@microsoft.com>
2019-02-21 14:16:44 -08:00
Veeraiah Chowdary Nuvvula
f6ac73d71e fixing typo and added a missing comment.
Signed-off-by: Veeraiah Chowdary Nuvvula <venuvvul@microsoft.com>
2019-02-18 14:13:02 -08:00
John Howard
88f19a90a8 Windows: Update stackdump event name
Signed-off-by: John Howard <jhoward@microsoft.com>
2019-02-15 15:33:29 -08:00
Wei Fu
132ee9b826 fix: linter issue
megacheck, gosimple and unused has been deprecated and subsumed by
staticcheck. And staticcheck also has been upgraded. we need to update
code for the linter issue.

close: #2945

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2019-01-23 22:54:51 +08:00
Michael Crosby
09a5b1f8af Merge pull request #2839 from jhowardmsft/jjh/service
Windows: Allow running as a service
2018-11-30 11:11:16 -05:00
Justin Terry (VM)
b4a642f6a5 Add dumpStacks support for containerd on Windows
Resolves: #1763

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-11-27 21:02:32 -08:00
John Howard
40d898a820 Windows: Allow running as a service
Signed-off-by: John Howard <jhoward@microsoft.com>

Allows containerd.exe to run as a Windows service. eg

Register: `.\containerd.exe --register-service`

Start: `net start containerd`
...
Stop: `net stop containerd`

Unregister: `.\containerd.exe --unregister-service`

When running as a service, logs will go to the Windows application
event log.
2018-11-27 17:28:04 -08:00
Justin Terry (VM)
8329a491b9 Build Windows V1 and V2 runtimes always
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-11-02 11:59:03 -07:00
Wei Fu
06616dab00 enhance: split config from server package
The github.com/containerd/containerd/services/server has a lot of
dependencies, like content, snapshots services implementation and
docker-metrics.

For the client side, it uses the config struct from server package
to start up the containerd in background. It will import a lot of
useless packages which might be conflict with existing vendor's package.

It makes integration easier with single config package.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2018-10-17 16:34:39 +08:00
Michael Crosby
1ad49689d3 Reduce publish connection timeout
Fixes #2662

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-09-19 11:21:40 -04:00
Justin Terry (VM)
0110b3c0bc Introduce the Windows lcow diff/snaphotter
Implements the Windows lcow differ/snapshotter responsible for managing
the creation and lifetime of lcow containers on Windows.

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-08-28 13:51:04 -07:00
Evan Hazlett
1537f31381 Add install support for binary images
This adds a way for users to programatically install containerd binary
dependencies.

With runtime v2 and new shim's being built, it will be a challenge to
get those onto machines.  Users would have to find the link, download,
place it in their path, yada yada yada.

With this functionality of a managed `/opt` directory, containerd can
use existing image and distribution infra. to get binarys, shims, etc
onto the system.

Configuration:

*default:* `/opt/containerd`

*containerd config:*
```toml
[plugins.opt]
	path = "/opt/mypath"

```

Usage:

*code:*

```go
image, err := client.Pull(ctx, "docker.io/crosbymichael/runc:latest")
client.Install(ctx, image)
```

*ctr:*

```bash
ctr content fetch docker.io/crosbymichael/runc:latest
ctr install docker.io/crosbymichael/runc:latest
```

You can manage versions and see what is running via standard image
commands.

Images:

These images MUST be small and only contain binaries.

```Dockerfile
FROM scratch
Add runc /bin/runc
```

Containerd will only extract files in `/bin` of the image.

Later on, we can add support for `/lib`.

The code adds a service to manage an `/opt/containerd` directory and
provide that path to callers via the introspection service.

How to Test:

Delete runc from your system.

```bash
> sudo ctr run --rm  docker.io/library/redis:alpine redis
ctr: OCI runtime create failed: unable to retrieve OCI runtime error (open /run/containerd/io.containerd.runtime.v1.linux/default/redis/log.json: no such file or directory): exec: "runc": executable file not found in $PATH: unknown

> sudo ctr content fetch docker.io/crosbymichael/runc:latest
> sudo ctr  install docker.io/crosbymichael/runc:latest

> sudo ctr run --rm  docker.io/library/redis:alpine redis
1:C 01 Aug 15:59:52.864 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 01 Aug 15:59:52.864 # Redis version=4.0.10, bits=64, commit=00000000, modified=0, pid=1, just started
1:C 01 Aug 15:59:52.864 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
1:M 01 Aug 15:59:52.866 # You requested maxclients of 10000 requiring at least 10032 max file descriptors.
1:M 01 Aug 15:59:52.866 # Server can't set maximum open files to 10032 because of OS error: Operation not permitted.
1:M 01 Aug 15:59:52.866 # Current maximum open files is 1024. maxclients has been reduced to 992 to compensate for low ulimit. If you need higher maxclients increase 'ulimit -n'.
1:M 01 Aug 15:59:52.870 * Running mode=standalone, port=6379.
1:M 01 Aug 15:59:52.870 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
1:M 01 Aug 15:59:52.870 # Server initialized
1:M 01 Aug 15:59:52.870 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
1:M 01 Aug 15:59:52.870 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
1:M 01 Aug 15:59:52.870 * Ready to accept connections
^C1:signal-handler (1533139193) Received SIGINT scheduling shutdown...
1:M 01 Aug 15:59:53.472 # User requested shutdown...
1:M 01 Aug 15:59:53.472 * Saving the final RDB snapshot before exiting.
1:M 01 Aug 15:59:53.484 * DB saved on disk
1:M 01 Aug 15:59:53.484 # Redis is now ready to exit, bye bye...
```

Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-08-02 17:11:29 -04:00
Justin Terry (VM)
d3e0c163f8 Adds runtime v2 support for Windows shim's
Implements the various requirements for the runtime v2 code to abstract
away the unix/linux code into the appropriate platform level
abstractions to use the runtime v2 on Windows as well.

Adds support in the Makefile.windows to actually build the runtime v2
code for Windows by setting a shell environment BUILD_WINDOWS_V2=1
before calling make. (Note this disables the compilation of the Windows
runtime v1)

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-07-25 14:09:26 -07:00
Sebastiaan van Stijn
da73b98b63 Set default log formatting to use RFC3339Nano with fixed width
This patch changes the logs format to use a fixed-width timestamp,
matching the format that's used in dockerd.

Before:

    $ containerd
    INFO[0000] starting containerd                           revision=a88b6319614de846458750ff882723479ca7b1a1 version=v1.1.0-202-ga88b6319
    INFO[0000] loading plugin "io.containerd.content.v1.content"...  type=io.containerd.content.v1
    INFO[0000] loading plugin "io.containerd.snapshotter.v1.btrfs"...  type=io.containerd.snapshotter.v1
    WARN[0000] failed to load plugin io.containerd.snapshotter.v1.btrfs  error="path /var/lib/containerd/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter"

After:

    $ containerd
    INFO[2018-07-24T08:11:07.397856489Z] starting containerd                           revision=c3195155cacb361cd3549c4d78901b20aa19579a version=v1.1.0-203-gc3195155
    INFO[2018-07-24T08:11:07.399264587Z] loading plugin "io.containerd.content.v1.content"...  type=io.containerd.content.v1
    INFO[2018-07-24T08:11:07.399343959Z] loading plugin "io.containerd.snapshotter.v1.btrfs"...  type=io.containerd.snapshotter.v1
    WARN[2018-07-24T08:11:07.399474423Z] failed to load plugin io.containerd.snapshotter.v1.btrfs  error="path /var/lib/containerd/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter"

Or, when running as child-process of dockerd:

Before:

    root@9637fcd85ea4:/go/src/github.com/docker/docker# dockerd --debug
    DEBU[2018-07-24T08:15:16.946312436Z] Listener created for HTTP on unix (/var/run/docker.sock)
    INFO[2018-07-24T08:15:16.947086499Z] libcontainerd: started new docker-containerd process  pid=231
    INFO[2018-07-24T08:15:16.947137166Z] parsed scheme: "unix"                         module=grpc
    INFO[2018-07-24T08:15:16.947235001Z] scheme "unix" not registered, fallback to default scheme  module=grpc
    INFO[2018-07-24T08:15:16.947463403Z] ccResolverWrapper: sending new addresses to cc: [{unix:///var/run/docker/containerd/docker-containerd.sock 0  <nil>}]  module=grpc
    INFO[2018-07-24T08:15:16.947505954Z] ClientConn switching balancer to "pick_first"  module=grpc
    INFO[2018-07-24T08:15:16.947717368Z] pickfirstBalancer: HandleSubConnStateChange: 0xc420507ab0, CONNECTING  module=grpc
    INFO[0000] starting containerd                           revision=d64c661f1d51c48782c9cec8fda7604785f93587 version=v1.1.1
    DEBU[0000] changing OOM score to -500
    INFO[0000] loading plugin "io.containerd.content.v1.content"...  type=io.containerd.content.v1
    INFO[0000] loading plugin "io.containerd.snapshotter.v1.btrfs"...  type=io.containerd.snapshotter.v1
    WARN[0000] failed to load plugin io.containerd.snapshotter.v1.btrfs  error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter"

After:

    DEBU[2018-07-24T08:21:33.441741970Z] Listener created for HTTP on unix (/var/run/docker.sock)
    INFO[2018-07-24T08:21:33.442428017Z] libcontainerd: started new docker-containerd process  pid=232
    INFO[2018-07-24T08:21:33.442510827Z] parsed scheme: "unix"                         module=grpc
    INFO[2018-07-24T08:21:33.442598812Z] scheme "unix" not registered, fallback to default scheme  module=grpc
    INFO[2018-07-24T08:21:33.442681006Z] ccResolverWrapper: sending new addresses to cc: [{unix:///var/run/docker/containerd/docker-containerd.sock 0  <nil>}]  module=grpc
    INFO[2018-07-24T08:21:33.442770353Z] ClientConn switching balancer to "pick_first"  module=grpc
    INFO[2018-07-24T08:21:33.442871502Z] pickfirstBalancer: HandleSubConnStateChange: 0xc42018bc30, CONNECTING  module=grpc
    INFO[2018-07-24T08:21:33.457963804Z] starting containerd                           revision=597dd082e37f8bc6b6265ca05839d7a300861911 version=597dd082
    DEBU[2018-07-24T08:21:33.458113301Z] changing OOM score to -500
    INFO[2018-07-24T08:21:33.458474842Z] loading plugin "io.containerd.content.v1.content"...  type=io.containerd.content.v1
    INFO[2018-07-24T08:21:33.458911054Z] loading plugin "io.containerd.snapshotter.v1.btrfs"...  type=io.containerd.snapshotter.v1
    WARN[2018-07-24T08:21:33.459366268Z] failed to load plugin io.containerd.snapshotter.v1.btrfs  error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter"

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-07-24 10:23:38 +02:00
Michael Crosby
17ab11a236 Fixes for runtimev2 and checkpoint restore
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-07-20 12:09:29 -04:00
Derek McGowan
94cfce62ba Merge pull request #2434 from crosbymichael/shimv2
Runtime v2 (shim API)
2018-07-18 13:14:10 -07:00
Akihiro Suda
02579c8c3f Merge pull request #2415 from dmcgowan/proxy-plugins
Add support for proxy plugins
2018-07-18 11:17:24 +09:00
Stephen Day
dfde5ec316 Merge pull request #2418 from crosbymichael/hook-root
Handle abs path for rootfs in oci hook
2018-07-17 12:17:28 -07:00
Michael Crosby
ed697290da Merge pull request #2463 from crosbymichael/temp-clean
Don't prevent boot on temp cleanup
2018-07-17 10:27:21 -04:00
Michael Crosby
da1b5470cd Runtime v2
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-07-17 10:21:29 -04:00
Michael Crosby
0105959c3d Don't prevent boot on temp cleanup
Fixes #2462
Fixes #2455

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-07-13 16:07:17 -04:00
Derek McGowan
cce0a46c8a Seed random on ctr and containerd startup
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-07-12 17:51:55 -07:00
Michael Crosby
e239f65590 Handle abs path for rootfs in oci hook
Fixes #2412

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-06-25 14:10:17 -04:00
Derek McGowan
7049671465 Add support for proxy plugins in configuration
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-06-21 18:57:42 -07:00
Phil Estes
df34eefa12 Merge pull request #2330 from crosbymichael/hpc
Add nvidia gpu support
2018-06-07 10:10:20 -04:00
Derek McGowan
1e8b09cfc6 Merge pull request #2353 from ehazlett/process-runtime
linux -> runtime/linux
2018-05-30 10:06:36 -07:00
Michael Crosby
c7083eed5d Merge pull request #2369 from dmcgowan/update-grpc
Update grpc to 1.12
2018-05-30 11:07:10 -04:00
Evan Hazlett
cae94b930d linux -> runtime/linux
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
2018-05-30 09:23:10 -04:00
Derek McGowan
55afe3359a Update grpc timeout and logger
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-05-29 15:32:37 -07:00
Michael Crosby
8e97da0958 Move server to services pkg
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-05-22 17:15:58 -04:00
Michael Crosby
c87ed12da5 Move restart pkg to runtime
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-05-22 17:13:40 -04:00
Michael Crosby
927517de36 Move dialer to pkg
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-05-22 13:32:25 -04:00
Michael Crosby
2b565da7ec Add restart monitor
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-05-16 13:07:23 -04:00
Michael Crosby
b949697a9c Add nvidia gpu support via libnvidia-container
This adds nvidia gpu support via the libnvidia-container project and
`nvidia-container-cli`.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-05-09 13:37:39 -04:00
Michael Crosby
544557289a Add oci-hook command to containerd
This allows many different commands to be used as OCI hooks.  It allows
these commands to template out different args and env vars so that
normal commands can accept the OCI spec State payload over stdin.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-05-08 15:58:20 -04:00
Kir Kolyshkin
9d0d4b806c context pkg: untangle
Since Go 1.7, "context" is a standard package, superceding the
"x/net/context". Since Go 1.9, the latter only provides type aliases
from the former. Therefore, it makes sense to switch to the standard
package, and the change is not disruptive in any sense.

This commit deals with a few cases where both packages happened to be
imported by the same source file. A choice between "context" and
"gocontext" was made for each file in order to minimize the patch.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-04-24 14:33:34 -07:00
Akihiro Suda
83e35b3d3a enable native (formerly naive) snapshotter by default
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-04-02 13:29:19 +09:00
Michael Crosby
a0c1abba47 Merge pull request #2228 from stevvooe/allow-configuration-msg-size
server: allow configuration default send/recv message sizes
2018-03-26 11:26:44 -04:00
Stephen J Day
acc71293c5 server: allow configuration default send/recv message sizes
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2018-03-23 16:17:23 -07:00
Stephen J Day
ab8e05ac50 cmd/containerd: include aufs by default
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2018-03-23 11:27:12 -07:00
Stephen J Day
f4113a903a containerd: include zfs plugin by default
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2018-03-14 10:29:23 -07:00
Lantao Liu
ea6a10c412 Add cri subcommand and add ctr/command package
Signed-off-by: Lantao Liu <lantaol@google.com>
2018-03-14 01:11:29 +00:00
Kunal Kushwaha
3491b9ea4a Copyright header added
Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
2018-03-05 10:07:58 +09:00
Derek McGowan
5bd99af7db Merge pull request #2097 from Random-Liu/vendor-cri-plugin
Vendor cri plugin into containerd.
2018-02-23 13:55:13 -08:00
Derek McGowan
3b4fcf771a Merge pull request #2112 from crosbymichael/temp-mounts
Add temp mount location to manage temp mounts
2018-02-21 12:56:23 -08:00
Kunal Kushwaha
b12c3215a0 Licence header added
Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
2018-02-19 10:32:26 +09:00
Lantao Liu
809a99a39e Vendor cri plugin and add critest
Signed-off-by: Lantao Liu <lantaol@google.com>
2018-02-16 23:23:47 +00:00
Michael Crosby
b2ec177bb2 Call temp mounts and unmount in containerd server
Fixes #2004

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-02-14 15:16:54 -05:00
Akihiro Suda
d7280ce2fb cmd/containerd: split package for cli.App
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-02-14 17:44:22 +09:00
Lantao Liu
050ff32dea Support trace level.
Signed-off-by: Lantao Liu <lantaol@google.com>
2018-02-06 02:30:05 +00:00