Commit Graph

760 Commits

Author SHA1 Message Date
Derek McGowan
cce0a46c8a
Seed random on ctr and containerd startup
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-07-12 17:51:55 -07:00
Kenfe-Mickaël Laventure
5900361791
Merge pull request #2420 from sudeeshjohn/master
Fixing the formatting directives error during compilation
2018-07-05 08:29:07 -07:00
sudeesh john
5a4f007e48 Fix the formatting directives error during compilation
Signed-off-by: Sudeesh John <sudeesh@linux.vnet.ibm.com>
2018-07-05 20:40:21 +05:30
Kenfe-Mickaël Laventure
39b6ba826a
Merge pull request #2431 from masters-of-cats/plugins-list-subcommand
Introduce plugins/list subcommand
2018-07-02 13:38:39 -07:00
Michael Crosby
6a83168157 Update ttrpc to 94dde388801693c54f88a6596f713b51a8
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-07-02 15:24:15 -04:00
Danail Branekov
3cf3881fa4 Introduce plugins/list subcommand
`Ctr` interface follows the pattern `ctr <command> <subcommand>` except
for the `plugins` command which does not have subcommands. This feels
unnatural to certain users and they would expect that they can list
containerd plugins via `ctr plugins list`.

This commit implements their expectation so that `plugins` becomes a
command "group" and its `list` subcommand actually lists the plugins.

Signed-off-by: Danail Branekov <danailster@gmail.com>
2018-07-02 10:20:20 +03:00
Michael Crosby
08150bfe76 Update ttrpc for containerd repo
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-06-28 09:53:40 -04:00
Michael Crosby
ca204317ed
Merge pull request #2422 from crosbymichael/ctr-delete
Add cio.Load for loading io set
2018-06-27 08:46:18 -04:00
Michael Crosby
fdceb13b14 Add cio.Load for loading io set
This adds a `Load` Opt for cio to load a tasks io/fifos without
attaching or starting the copy routines.

It adds the load method in `ctr` by default so that fifos or other IO
are removed from disk on delete methods inbetween command runs.  It is
not the default for all task loads for backwards compat. and a user may
want to keep io around to reuse or if log files are used.

Fixes #2421

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-06-26 11:48:26 -04:00
Michael Crosby
e239f65590 Handle abs path for rootfs in oci hook
Fixes #2412

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-06-25 14:10:17 -04:00
Derek McGowan
7049671465
Add support for proxy plugins in configuration
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-06-21 18:57:42 -07:00
Felix Abecassis
5dd22a20af Move ContainerFlags to "commands" package
Commit 05513284e7 exposed the "rootfs"
and "no-pivot" flags for the "containers" command, but it accidentally
removed them for "run" since package-level variables are initialized
before package-level init functions in golang. Hoisting these flags to
a package imported by both commands solves the problem.

Signed-off-by: Felix Abecassis <fabecassis@nvidia.com>
2018-06-20 18:33:59 -07:00
Phil Estes
df34eefa12
Merge pull request #2330 from crosbymichael/hpc
Add nvidia gpu support
2018-06-07 10:10:20 -04:00
Evan Hazlett
821c8eaa91
runtime/linux/shim -> runtime/shim
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
2018-06-06 14:35:06 -04:00
Akihiro Suda
d88de4a34f content: change Writer/ReaderAt to take OCI
This change allows implementations to resolve the location of the actual data
using OCI descriptor fields such as MediaType.

No OCI descriptor field is written to the store.

No change on gRPC API.

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-06-01 11:51:43 +09:00
Derek McGowan
1e8b09cfc6
Merge pull request #2353 from ehazlett/process-runtime
linux -> runtime/linux
2018-05-30 10:06:36 -07:00
Michael Crosby
c7083eed5d
Merge pull request #2369 from dmcgowan/update-grpc
Update grpc to 1.12
2018-05-30 11:07:10 -04:00
Evan Hazlett
cae94b930d linux -> runtime/linux
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
2018-05-30 09:23:10 -04:00
Derek McGowan
55afe3359a
Update grpc timeout and logger
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-05-29 15:32:37 -07:00
Michael Crosby
8e97da0958 Move server to services pkg
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-05-22 17:15:58 -04:00
Michael Crosby
c87ed12da5 Move restart pkg to runtime
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-05-22 17:13:40 -04:00
Michael Crosby
927517de36 Move dialer to pkg
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-05-22 13:32:25 -04:00
Michael Crosby
ae4b78d1cc Move progress into pkg
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-05-22 13:32:25 -04:00
Michael Crosby
0bafe236b4 Move reaper under shim package
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-05-22 11:38:20 -04:00
Michael Crosby
2b565da7ec Add restart monitor
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-05-16 13:07:23 -04:00
Michael Crosby
b949697a9c Add nvidia gpu support via libnvidia-container
This adds nvidia gpu support via the libnvidia-container project and
`nvidia-container-cli`.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-05-09 13:37:39 -04:00
Michael Crosby
544557289a Add oci-hook command to containerd
This allows many different commands to be used as OCI hooks.  It allows
these commands to template out different args and env vars so that
normal commands can accept the OCI spec State payload over stdin.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-05-08 15:58:20 -04:00
Derek McGowan
f701b3b960
Fix race in ctr pull
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-05-07 10:29:24 -07:00
Michael Crosby
e22c827cf5
Merge pull request #2323 from stevvooe/ping-snapshots-from-cli
cmd/snapshots: add gc.root to created snapshots
2018-05-04 13:27:44 -04:00
Tom Godkin
fc8bce59b9 Use user-specific temp directory if set
This allows non-privileged users to use containerd. This is part of a
larger track of work integrating containerd into Cloudfoundry's garden
with support for rootless.

[#156343575]

Signed-off-by: Claudia Beresford <cberesford@pivotal.io>
2018-05-04 10:27:58 +01:00
Stephen J Day
ed72059fac
cmd/snapshots: add gc.root to created snapshots
This adds gc.root label to snapshots created with prepare and commit via
the CLI. WIthout this, created snapshots get immediately garbage
collected. There may be a better solution but this seems to be a solid
stop gap.

We may also need to add more functionality around snapshot labeling for
the CLI but current use cases are unclear.

Signed-off-by: Stephen J Day <stevvooe@gmail.com>
2018-05-03 07:09:38 +02:00
Andrei Vagin
60daa414db Allow to checkpoint and restore a container with console
runc already supports this case, so we just need to run it with proper
options.

Signed-off-by: Andrei Vagin <avagin@virtuozzo.com>
2018-04-28 01:06:42 +03:00
Derek McGowan
1a5e0df98f
Merge pull request #2305 from kolyshkin/context
Switch from x/net/context -> context
2018-04-25 10:44:22 -07:00
Michael Crosby
090687916d Correctly handle reading from events channel
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-04-25 11:21:22 -04:00
Kir Kolyshkin
bbe14f0a2e Switch from x/net/context to context
Since Go 1.7, context is a standard package, superceding the
"x/net/context". Since Go 1.9, the latter only provides a few type
aliases from the former. Therefore, it makes sense to switch to the
standard package.

This commit was generated by the following script (with a couple of
minor fixups to remove extra changes done by goimports):

	#!/bin/bash

	if [ $# -ge 1 ]; then
		FILES=$*
	else
		FILES=$(git ls-files \*.go | grep -vF ".pb.go" | grep -v
	^vendor/)
	fi

	for f in $FILES; do
		printf .
		sed -i -e 's|"golang.org/x/net/context"$|"context"|' $f
		goimports -w $f
		awk '	/^$/ {e=1; next;}
			/[[:space:]]"context"$/ {e=0;}
			{if (e) {print ""; e=0}; print;}' < $f > $f.new && \
				mv $f.new $f
		goimports -w $f
	done
	echo

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-04-24 14:33:34 -07:00
Kir Kolyshkin
9d0d4b806c context pkg: untangle
Since Go 1.7, "context" is a standard package, superceding the
"x/net/context". Since Go 1.9, the latter only provides type aliases
from the former. Therefore, it makes sense to switch to the standard
package, and the change is not disruptive in any sense.

This commit deals with a few cases where both packages happened to be
imported by the same source file. A choice between "context" and
"gocontext" was made for each file in order to minimize the patch.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-04-24 14:33:34 -07:00
Julien Kassar
9d247718d7 Update ctr tasks list usage for quiet flag
Signed-off-by: Julien Kassar <github@kassisol.com>
2018-04-23 14:53:35 -04:00
Fernando Mayo
0a26b0fe43 ctr: fix --mount help message
Signed-off-by: Fernando Mayo <fermayo@gmail.com>
2018-04-17 18:29:33 -07:00
Derek McGowan
6c01529057
Clean up dependency output
Show new tag when dependencies don't have a previous version.
Align dependencies into columns.
Sort dependencies by name.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-04-13 14:42:15 -07:00
Justin Cormack
903191072e
Add --privileged option to ctr run
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2018-04-04 13:25:42 +01:00
Akihiro Suda
83e35b3d3a enable native (formerly naive) snapshotter by default
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-04-02 13:29:19 +09:00
Kenfe-Mickael Laventure
3c3a676490
Return a better error message is unix socket path is too long.
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2018-03-30 09:00:02 -07:00
Michael Crosby
a0c1abba47
Merge pull request #2228 from stevvooe/allow-configuration-msg-size
server: allow configuration default send/recv message sizes
2018-03-26 11:26:44 -04:00
Stephen J Day
acc71293c5
server: allow configuration default send/recv message sizes
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2018-03-23 16:17:23 -07:00
Stephen J Day
ab8e05ac50
cmd/containerd: include aufs by default
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2018-03-23 11:27:12 -07:00
Michael Crosby
3e8e9d3ed7
Merge pull request #2223 from dmcgowan/with-lease-context
lease: pass in context to lease done function in client
2018-03-23 10:27:39 -04:00
Derek McGowan
43d0a5cb60
Pass in context to lease done function in client
Allows the client to choose the context to finish the lease.
This allows the client to switch contexts when the main context
used to the create the lease may have been cancelled.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-03-22 14:09:24 -07:00
Stephen J Day
903ee88368
cmd/ctr/app: remove redundant package name
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2018-03-22 10:12:00 -07:00
Michael Crosby
ee84187a6a
Merge pull request #2096 from stevvooe/include-zfs-by-default
containerd: include zfs plugin by default
2018-03-15 13:56:00 -04:00
Derek McGowan
a0b818e093
Merge pull request #2200 from jessvalarezo/multiarch-pulls
allow content to be pulled for specific platform(s), all platforms
2018-03-14 14:46:30 -07:00
Stephen J Day
f4113a903a
containerd: include zfs plugin by default
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2018-03-14 10:29:23 -07:00
Phil Estes
b307df2723
Merge pull request #2207 from jessvalarezo/ctr-task-d-update
ctr: allow for force kill when deleting task process
2018-03-14 03:41:23 -04:00
Lantao Liu
ea6a10c412 Add cri subcommand and add ctr/command package
Signed-off-by: Lantao Liu <lantaol@google.com>
2018-03-14 01:11:29 +00:00
Jess Valarezo
e343006b8f ctr: allow for force kill when deleting task process
Signed-off-by: Jess Valarezo <valarezo.jessica@gmail.com>
2018-03-13 14:58:31 -07:00
Stephen J Day
ae11d8c64b
cmd/ctr/images: add i alias for images command
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2018-03-12 17:42:41 -07:00
Jess Valarezo
c3cf3d7822 allow content to be pulled for specific platform(s), all platforms
Signed-off-by: Jess Valarezo <valarezo.jessica@gmail.com>
2018-03-12 17:31:42 -07:00
Stephen Day
3013762fc5
Merge pull request #2203 from Random-Liu/support-in-process-integration
Support in process integration
2018-03-12 14:38:27 -07:00
Lantao Liu
1128b3d664 Add service plugin and support in process integration.
Signed-off-by: Lantao Liu <lantaol@google.com>
2018-03-12 18:03:50 +00:00
Michael Crosby
77a5804f6a
Merge pull request #2178 from kunalkushwaha/file-header-ci-check
CI check for file-header added
2018-03-07 10:29:26 -05:00
Stephen Day
5368984859
Merge pull request #2160 from ijc/ctr-run-unpack
ctr: unpack the image on run if necessary
2018-03-06 19:29:43 -08:00
Michael Crosby
1eabab31aa Handle SIGPIPE in shims
ref: https://github.com/moby/moby/issues/36464

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-03-06 12:54:16 -05:00
Derek McGowan
2b6b99b4a1
Merge pull request #2184 from kunalkushwaha/oci-error
better image config parse error.
2018-03-05 10:29:02 -08:00
Akihiro Suda
05513284e7 ctr: add UNIX-specific flags to ctr c create
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-03-05 15:26:03 +09:00
Kunal Kushwaha
6ed4e9e106 better image config parse error.
compatible oci runtime version printed with parse error

Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
2018-03-05 14:04:08 +09:00
Kunal Kushwaha
3491b9ea4a Copyright header added
Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
2018-03-05 10:07:58 +09:00
Michael Crosby
a2ef6952f2 Add density stress test
Running the density tool will report Pss and Rss total and per container
values for shim memory usage. Values are reported in KB.

```bash
containerd-stress density --count 500
INFO[0000] pulling docker.io/library/alpine:latest
INFO[0000] generating spec from image
{"pss":421188,"rss":2439688,"pssPerContainer":842,"rssPerContainer":4879}
```

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-02-27 16:40:37 -05:00
Ian Campbell
f48cc7d7fe ctr: unpack the image on run if necessary
Without this `ctr run` can fail with:

    ctr: parent snapshot sha256:70798fd80095f40b41baa5d107fb61532bfe494d96313fea01e8fcbf4e8743ee does not exist: not found

My image was produced by buildkit, which doesn't unpack (I think this makes
sense since buildkit doesn't know if I am going to run the image or export/push
it etc).

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2018-02-26 14:23:28 +00:00
Derek McGowan
5bd99af7db
Merge pull request #2097 from Random-Liu/vendor-cri-plugin
Vendor cri plugin into containerd.
2018-02-23 13:55:13 -08:00
Derek McGowan
3b4fcf771a
Merge pull request #2112 from crosbymichael/temp-mounts
Add temp mount location to manage temp mounts
2018-02-21 12:56:23 -08:00
Michael Crosby
d3a8055e2d Add --pid-file to ctr
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-02-20 15:10:50 -05:00
Kunal Kushwaha
b12c3215a0 Licence header added
Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
2018-02-19 10:32:26 +09:00
Lantao Liu
809a99a39e Vendor cri plugin and add critest
Signed-off-by: Lantao Liu <lantaol@google.com>
2018-02-16 23:23:47 +00:00
Phil Estes
e4e53bf486
Add --with-ns flag to ctr run/create
Adds a useful flag to `ctr` to enable joining any existing Linux
namespaces for any namespace types (network, pid, ipc, etc.) using the
existing With helper in the oci package.

Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
2018-02-16 15:23:04 -05:00
Michael Crosby
b2ec177bb2 Call temp mounts and unmount in containerd server
Fixes #2004

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-02-14 15:16:54 -05:00
Akihiro Suda
d7280ce2fb cmd/containerd: split package for cli.App
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-02-14 17:44:22 +09:00
Michael Crosby
e68bdbe9d9 Capture more error locations during stress tests
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-02-07 15:50:29 -05:00
Lantao Liu
050ff32dea Support trace level.
Signed-off-by: Lantao Liu <lantaol@google.com>
2018-02-06 02:30:05 +00:00
Michael Crosby
78bd07afef
Merge pull request #1746 from AkihiroSuda/split-differ-interface
diff: resplit Applier from Differ
2018-02-05 09:18:56 -05:00
Michael Crosby
8ee29a17e6 Bump gc threshold to 40%
Doing tests, this is a better balance for the threshold in reguards to
memory and cpu usage.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-02-02 11:00:34 -05:00
Michael Crosby
55aa0b415b Don't enable debug endpoints in default config
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-01-31 11:08:51 -05:00
Michael Crosby
98b53e0dd3 Allow tcp debug address
This uses a simple `IsAbs` check to see if we are using an on disk path
for a unix socket vs an address since we do not prefix addresses with
`unix://` or `tcp://`.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-01-31 10:19:33 -05:00
Michael Crosby
5f89502a24
Merge pull request #1790 from jessvalarezo/ctr-c-create
ctr: add container create cmd and config flag
2018-01-29 17:19:14 -05:00
Derek McGowan
b763777288
diff: rename differ to comparer
Remove combined interface and split implementations.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-01-26 16:32:09 -08:00
Phil Estes
d4fb0709c9
Add error return for missing params in ctr images
For missing required parameters adds error return before attempting any
actions to `ctr images` commands.

Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
2018-01-26 09:14:13 -05:00
Jess Valarezo
2c9ce2e693 ctr: add container create, config flag for spec
Signed-off-by: Jess Valarezo <valarezo.jessica@gmail.com>
2018-01-25 17:16:13 -08:00
Derek McGowan
7e4403540d
Merge pull request #1969 from darrenstahlmsft/WindowsSnapshotter5
Implement Windows snapshotter and differ
2018-01-24 17:02:02 -08:00
Stephen Day
3fcc52b091
Merge pull request #2055 from stevvooe/aggressive-memory-shim
cmd/containerd-shim: aggressive memory reclamation
2018-01-24 16:06:57 -08:00
Stephen J Day
71e9f6dac2
cmd/containerd-shim, reaper: reduce channel allocation
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2018-01-24 15:14:08 -08:00
Stephen J Day
0e8f08476c
cmd/containerd-shim: aggressive memory reclamation
To avoid having the shim hold on to too much memory, we've made a few
adjustments to favor more aggressive reclamation of memory from the
operating system. Typically, this would be negligible, on the order of a
few megabytes, but this is impactful when running several containers.

The first fix is to lower the threshold used to determine when to run
the garbage collector. The second runs `runtime/debug.FreeOSMemory` at a
regular interval.

Under test, this result in a sustained memory usage of around 3.7 MB.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2018-01-24 14:51:13 -08:00
Darren Stahl
dcff993653 Update Windows runtime to use snapshotter and differ layers
This changes the Windows runtime to use the snapshotter and differ
created layers, and updates the ctr commands to use the snapshotter and differ.

Signed-off-by: Darren Stahl <darst@microsoft.com>
2018-01-23 14:40:24 -08:00
Darren Stahl
a5a9f91832 Implement Windows snapshotter and differ
This implements the Windows snapshotter and diff Apply function.
This allows for Windows layers to be created, and layers to be pulled
from the hub.

Signed-off-by: Darren Stahl <darst@microsoft.com>
2018-01-23 14:40:23 -08:00
Phil Estes
d7efcbc083
Merge pull request #2036 from stevvooe/use-buffer-pools
archive, cio, cmd, linux: use buffer pools
2018-01-23 15:00:41 -05:00
Phil Estes
f47f6af585
Remove unnecessary subreaper API from sys/
Given these same exact functions are both now available in
opencontainers/runc (libcontainer/system) package, and we only use the
`SetSubreaper` today from the shim, there seems to be no reason for
duplication.

Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
2018-01-23 10:30:29 -05:00
Phil Estes
dc5964ccc2
Merge pull request #2037 from pauldotknopf/master
Made a public method to set environment variables and mounts for containers.
2018-01-22 17:05:26 -05:00
Stephen J Day
cd72819b53
archive, cio, cmd, linux: use buffer pools
To avoid buffer bloat in long running processes, we try to use buffer
pools where possible. This is meant to address shim memory usage issues,
but may not be the root cause.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2018-01-22 13:52:06 -08:00
Paul Knopf
b4c3cd7640
Add WithEnv and WithMount oci options
Signed-off-by: Paul Knopf <pauldotknopf@gmail.com>
2018-01-22 16:35:31 -05:00
Michael Crosby
e5740ca612
Merge pull request #2007 from AkihiroSuda/cio-fifo-path
cio: add WithFIFODir opt
2018-01-22 13:32:43 -05:00
Michael Crosby
4812f4be8f
Merge pull request #2030 from Random-Liu/print-plugin-default-config
Print default plugin config.
2018-01-22 11:12:20 -05:00
Lantao Liu
3d6fe5ad18 Print default plugin config.
Signed-off-by: Lantao Liu <lantaol@google.com>
2018-01-19 22:51:32 +00:00
Michael Crosby
94602aea63 Add execs to stress tests
This improves the exec support so that they can run along with the
normal stress tests.  You don't have to pick exec stres or container
stress.  They both run at the same time and report the different values.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-01-19 13:44:54 -05:00