Commit Graph

4032 Commits

Author SHA1 Message Date
Ian Campbell
4a74731cd0 oci: Update docs for oci.WithUserID
The behaviour was changed in 99df1a9e11 ("Set gid 0 when no group is
specified"), part of #2529.

Take the opportunity to tighten up the grammar a bit too.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2018-08-08 15:32:59 +01:00
Phil Estes
97473edb99
Merge pull request #2534 from cyphar/manpage-rename
docs: man: rename config.toml(5) to be more descriptive
2018-08-08 10:26:25 -04:00
Aleksa Sarai
7aa132ffc7
docs: man: rename config.toml(5) to be more descriptive
The man page namespace is global, so in order to avoid colliding with
other man pages named "config.toml" rename ours to be more descriptive.
This also helps with discoverability (now tab-completion of 'man
containerd<tab>' will return the config man page), as well as making it
much cleaner from the perspective of distributions that want to package
containerd.

Signed-off-by: Aleksa Sarai <asarai@suse.de>
2018-08-08 18:33:29 +10:00
Derek McGowan
1ba4aa04b4
Merge pull request #2528 from crosbymichael/shim-debug
Add shim log pipe for log forwarding to the daemon
2018-08-07 14:14:01 -07:00
Michael Crosby
6ba4ddfdda Add shim log pipe for log forwarding to the daemon
A fifo on unix or named pipe on Windows will be provided to the shim.
It can be located inside the `cwd` of the shim named "log".
The shims can use the existing `github.com/containerd/containerd/log` package to log debug messages.
Messages will automatically be output in the containerd's daemon logs with the correct fiels and runtime set.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-08-07 15:31:00 -04:00
Wei Fu
67b54c6670 Support >= 128 layers in overlayfs snapshots
Auto-detect longest common dir in lowerdir option and compact it if the
option size is hitting one page size. If does, Use chdir + CLONE to do
mount thing to avoid hitting one page argument buffer in linux kernel
mount.

Signed-off-by: Wei Fu <fhfuwei@163.com>
2018-08-07 10:59:36 +08:00
Derek McGowan
aeb322d87d
Update release tool
Allow inclusion of sub-project changes
Order contributors by number of contributions
Add mailmap

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-08-06 17:51:10 -07:00
Phil Estes
4fb92300fe
Merge pull request #2526 from crosbymichael/managed-opt
Add docs for managed opts dir
2018-08-06 17:46:57 -04:00
Phil Estes
6bf15fa867
Merge pull request #2529 from crosbymichael/missing-gids
Set gid 0 when no group is specified
2018-08-06 17:45:42 -04:00
Michael Crosby
99df1a9e11 Set gid 0 when no group is specified
This change is to match Docker's implementaion of setting gid and groups
to 0 when no gid is specified but an explicit uid is set.

Fixes #2527

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-08-06 15:08:59 -04:00
Derek McGowan
dd97a11b6f
Merge pull request #2523 from jterry75/windows_v2_tasks_service
Refactors the TasksService requires per platform
2018-08-06 10:33:46 -07:00
Michael Crosby
2742238909 Add docs for managed opts dir
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-08-03 11:50:02 -04:00
Phil Estes
2783a19b10
Merge pull request #2518 from crosbymichael/install
Add install support for binary images
2018-08-03 08:45:02 -04:00
Akihiro Suda
74b036491a
Merge pull request #2524 from samuelkarp/time.Hour
replace 3600 seconds with 1 hour
2018-08-03 12:41:20 +09:00
Michael Crosby
5a47c5ec1d Add lib support as an option
Some images like `criu` will have extra libs that it requires.  This
adds lib support via LD_LIBRARY_PATH and InstallOpts

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-08-02 17:11:29 -04:00
Evan Hazlett
1537f31381 Add install support for binary images
This adds a way for users to programatically install containerd binary
dependencies.

With runtime v2 and new shim's being built, it will be a challenge to
get those onto machines.  Users would have to find the link, download,
place it in their path, yada yada yada.

With this functionality of a managed `/opt` directory, containerd can
use existing image and distribution infra. to get binarys, shims, etc
onto the system.

Configuration:

*default:* `/opt/containerd`

*containerd config:*
```toml
[plugins.opt]
	path = "/opt/mypath"

```

Usage:

*code:*

```go
image, err := client.Pull(ctx, "docker.io/crosbymichael/runc:latest")
client.Install(ctx, image)
```

*ctr:*

```bash
ctr content fetch docker.io/crosbymichael/runc:latest
ctr install docker.io/crosbymichael/runc:latest
```

You can manage versions and see what is running via standard image
commands.

Images:

These images MUST be small and only contain binaries.

```Dockerfile
FROM scratch
Add runc /bin/runc
```

Containerd will only extract files in `/bin` of the image.

Later on, we can add support for `/lib`.

The code adds a service to manage an `/opt/containerd` directory and
provide that path to callers via the introspection service.

How to Test:

Delete runc from your system.

```bash
> sudo ctr run --rm  docker.io/library/redis:alpine redis
ctr: OCI runtime create failed: unable to retrieve OCI runtime error (open /run/containerd/io.containerd.runtime.v1.linux/default/redis/log.json: no such file or directory): exec: "runc": executable file not found in $PATH: unknown

> sudo ctr content fetch docker.io/crosbymichael/runc:latest
> sudo ctr  install docker.io/crosbymichael/runc:latest

> sudo ctr run --rm  docker.io/library/redis:alpine redis
1:C 01 Aug 15:59:52.864 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 01 Aug 15:59:52.864 # Redis version=4.0.10, bits=64, commit=00000000, modified=0, pid=1, just started
1:C 01 Aug 15:59:52.864 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
1:M 01 Aug 15:59:52.866 # You requested maxclients of 10000 requiring at least 10032 max file descriptors.
1:M 01 Aug 15:59:52.866 # Server can't set maximum open files to 10032 because of OS error: Operation not permitted.
1:M 01 Aug 15:59:52.866 # Current maximum open files is 1024. maxclients has been reduced to 992 to compensate for low ulimit. If you need higher maxclients increase 'ulimit -n'.
1:M 01 Aug 15:59:52.870 * Running mode=standalone, port=6379.
1:M 01 Aug 15:59:52.870 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
1:M 01 Aug 15:59:52.870 # Server initialized
1:M 01 Aug 15:59:52.870 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
1:M 01 Aug 15:59:52.870 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
1:M 01 Aug 15:59:52.870 * Ready to accept connections
^C1:signal-handler (1533139193) Received SIGINT scheduling shutdown...
1:M 01 Aug 15:59:53.472 # User requested shutdown...
1:M 01 Aug 15:59:53.472 * Saving the final RDB snapshot before exiting.
1:M 01 Aug 15:59:53.484 * DB saved on disk
1:M 01 Aug 15:59:53.484 # Redis is now ready to exit, bye bye...
```

Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-08-02 17:11:29 -04:00
Derek McGowan
9ca687be63
Merge pull request #2522 from jterry75/runc_typo
Fix a typo in runc-v1 shim
2018-08-02 12:50:53 -07:00
Samuel Karp
9665a2650a *: replace 3600 seconds with 1 hour
Signed-off-by: Samuel Karp <skarp@amazon.com>
2018-08-02 11:40:03 -07:00
Justin Terry (VM)
9936370fef Refactors the TasksService requires per platform
Removes the start dependency on V1 runtimes in the TasksService for:
// +build windows_v2. For unix and windows (v1) this code remains to load all
v1 runtimes as expected.

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-08-02 11:20:35 -07:00
Justin Terry (VM)
9ff702b9a1 Fix a typo in runc-v1 shim
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-08-02 11:16:31 -07:00
Phil Estes
b8f4c7a9bd
Merge pull request #2520 from dmcgowan/use-pause-multiplatform-test
Update multi-arch image tests
2018-08-01 16:49:52 -04:00
Michael Crosby
d3887f6764
Merge pull request #2519 from jterry75/various_win_fixes
Various Windows fixes to support the runtime v2 shim workflow
2018-08-01 16:40:04 -04:00
Justin Terry (VM)
dcb905701c Adds retry support to Windows AnonDialer
Adds retry support to AnonDialer if the pipe does not exist. This will
retry up to the timeout for the pipe to exist and connect. This solves
the race between the containerd-shim-* start command and the
reinvocation.

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-08-01 13:22:25 -07:00
Justin Terry (VM)
790c3a3663 Remove extra allocation in NewTask
Reorders the code so that it doesnt overwrite the previous allocation
when creating a NewTask via ctr.exe

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-08-01 13:22:18 -07:00
Michael Crosby
123de20b59
Merge pull request #2517 from estesp/fix-travis-script
Fix loss of CRI test failure status in CI
2018-08-01 14:50:17 -04:00
Derek McGowan
438b3cb694
Update multi-arch image tests
Ensure the test pull for all platforms uses a multi-arch image
Use the pause container for testing specific platforms
Update the image unpack test to be explicit about the platform to unpack

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-08-01 11:22:07 -07:00
Phil Estes
9622369f0e
Fix loss of CRI test failure status in CI
Prior PR fixed the wrong use of `exit` built-in within a Travis script,
but lost the reporting of a failure result of CRI testing in the process.

Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
2018-08-01 13:54:16 -04:00
Justin Terry (VM)
6a252a7141 IO Windows remove unused wait group
Removes an unused and unneeded wait group.

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-08-01 09:54:31 -07:00
Derek McGowan
efb04a3361
Merge pull request #2513 from dmcgowan/set-default-platform-withplatform
Fix for empty platform
2018-07-31 17:38:23 -07:00
Derek McGowan
9dc55ea1d2
Merge pull request #2514 from jterry75/runtime_v2_shim_command
Runtime v2 absolute shim path to executable
2018-07-31 16:26:23 -07:00
Derek McGowan
d64d8a06d5
Use image constructor in client
Replace manual image struct creation with the image
constructor which is there to do just that.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-07-31 15:23:08 -07:00
Derek McGowan
3629344e6e
Ensure specifying an empty platform is treated as default
The default platform had previously been provided using the
empty string. This change ensures that the platforms
field is always filled in correctly and empty string is
properly interpreted.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-07-31 15:08:59 -07:00
Justin Terry (VM)
9f13b74f4a Runtime v2 absolute shim path to executable
Fixes an issue where the runtime v2 was not using an absolute path to
the executable but setting the .Dir field on the exec.Cmd. This causes
the executable to need to be relative to .Dir but no shim is actually
copied to the bundle directory that its work dir is set to.

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-07-31 15:07:55 -07:00
Kenfe-Mickaël Laventure
875b92c507
Merge pull request #2512 from crosbymichael/gpupath
Add nvidia Opts to lookup containerd binary or hook path
2018-07-31 09:28:33 -07:00
Phil Estes
ed0e73422d
Merge pull request #2468 from dmcgowan/set-platform-on-unpack
Update client Image to have configurable platform
2018-07-31 11:56:09 -04:00
Phil Estes
4249f44d81
Merge pull request #2493 from dmcgowan/sync-lease-removal
Add sync option to lease removal
2018-07-31 11:31:55 -04:00
Michael Crosby
d0ab8c8510
Merge pull request #2496 from dmcgowan/lease-content-uploads
Add content ingests to lease and gc
2018-07-31 11:27:57 -04:00
Michael Crosby
e4f33dcfb5 Add nvidia Opts to lookup containerd binary or hook path
This is for consumers like Docker that manage a `docker-containerd`.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-07-31 10:11:25 -04:00
Phil Estes
c55b9636f7
Merge pull request #2263 from alibaba/fix_no_gzip
fixbug: blob for schemav1 could be uncompressed
2018-07-30 18:49:28 -04:00
Phil Estes
6647e75651
Merge pull request #2509 from crosbymichael/bundle-cleanup
Cleanup workdirs on manager load
2018-07-30 18:46:21 -04:00
Derek McGowan
45d0df8fe3
Merge pull request #2510 from crosbymichael/gpucaps2
Change gpu Capability type to string
2018-07-30 14:35:59 -07:00
Stephen Day
920dc79b4d
Merge pull request #2504 from samuelkarp/ctr-t-metrics
ctr: add new metrics subcommand
2018-07-30 13:07:19 -07:00
Michael Crosby
81e2859e8b Change gpu Capability type to string
This helps with mappings so that we are not translating multiple times
from caller to hook.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-07-30 15:37:13 -04:00
Derek McGowan
ca71484793
Merge pull request #2480 from dmcgowan/proxy-plugin-doc
Document plugins
2018-07-30 11:23:24 -07:00
Stephen Day
578a236f30
Merge pull request #2506 from crosbymichael/gpucaps
nvidia GPU support for caps and multiple uuids
2018-07-30 11:21:53 -07:00
Michael Crosby
23fbdbaf13 Cleanup workdirs on manager load
This cleans up persistent work dirs on TaskManager boot.  These dirs can
be left behind in a machine reboot.  The state in /run will not exist
but the work dir in the root does, we should cleanup work dirs when
tasks are not loaded.

This also improves error handling that would prevent the task manager
from loading when a single task fails to load or cleanup.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-07-30 10:21:04 -04:00
Michael Crosby
e8f7c2af26
Merge pull request #2507 from flx42/fix-readme-runtime-v2
Remove reference to WithExit in README
2018-07-30 09:57:52 -04:00
Derek McGowan
2ebfba575c
Merge pull request #2479 from stevvooe/with-file-combinator
oci: introduce WithSpecFromFile combinator
2018-07-27 15:56:53 -07:00
Samuel Karp
9a34bb046a metrics: add optional json output
Signed-off-by: Samuel Karp <skarp@amazon.com>
2018-07-27 15:21:08 -07:00
Felix Abecassis
e22f19618c Remove reference to WithExit in README
This function was removed from the containerd package in da1b5470cd

Signed-off-by: Felix Abecassis <fabecassis@nvidia.com>
2018-07-27 15:16:43 -07:00