Commit Graph

3094 Commits

Author SHA1 Message Date
Stephen J Day
d8bd6b70ce
plugin: allow querying plugin graph to be re-entrant
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-10-10 16:40:47 -07:00
Stephen J Day
363d692f35
api/services: define the introspection API
After several requests for information about platform support,
configuration introspection and feature switches, we now have a solution
that should work in all these use cases. The Introspection API hooks
into the plugin subsystem registration process. During registration,
plugins declare several pieces of information, allowing clients to
discover the cababilities and support that a containerd instance
provides, including whether or not it loaded with an error.

To allow symmetrical error reporting, the `google/rpc.Status` protobuf
definitions have been brought in from the googleapis project.
Unfortunately, we had to generate these in place to match our protobuf
system.

Once we like this design, we can add an implementation to integrate it
directly with the plugin system.

Enjoy!

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-10-10 16:40:47 -07:00
Michael Crosby
12c79cc0d9 Merge pull request #1604 from jessvalarezo/windows-listpids
ListPids returns process details in addition to process IDs.
2017-10-10 19:26:47 -04:00
Jess
061c719209 ListPids returns process ID and other info
Signed-off-by: Jess <jessica.valarezo@docker.com>
2017-10-10 22:57:15 +00:00
Phil Estes
96725a5a0b Merge pull request #1620 from mlaventure/runc-io-id
Allow setting the uid & gid of the io pipes
2017-10-10 17:12:34 -04:00
Kenfe-Mickael Laventure
c807ba8379
Allow setting the uid & gid of the io pipes
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-10-10 13:24:12 -07:00
Phil Estes
1ea8ac41d3 Merge pull request #1617 from crosbymichael/2cgroup
Use namespace in default cgroup path
2017-10-10 10:55:12 -04:00
Michael Crosby
d7864eb77b Use namespace in default cgroup path
By default, the generated spec will place containers in cgroups by their
ids, we need to use the namespace as the cgroup root to avoid
containers with the same name being placed in the same cgroup.

```
11:perf_event:/to/redis
10:freezer:/to/redis
9:memory:/to/redis
8:devices:/to/redis
7:net_cls,net_prio:/to/redis
6:pids:/to/redis
5:hugetlb:/to/redis
4:cpuset:/to/redis
3:blkio:/to/redis
2:cpu,cpuacct:/to/redis
1:name=systemd:/to/redis

11:perf_event:/te/redis
10:freezer:/te/redis
9:memory:/te/redis
8:devices:/te/redis
7:net_cls,net_prio:/te/redis
6:pids:/te/redis
5:hugetlb:/te/redis
4:cpuset:/te/redis
3:blkio:/te/redis
2:cpu,cpuacct:/te/redis
1:name=systemd:/te/redis
```

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-10-09 17:07:57 -04:00
Michael Crosby
72bb45ac46 Merge pull request #1603 from mlaventure/fix-windows-flaky-tests
windows: ensure a init process exist after "Create()"
2017-10-09 11:58:23 -04:00
Kenfe-Mickael Laventure
cfa87567a0
windows: Create init process with task
This prevents `task.Wait()` to return an error if it is called before the task
is started.

Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-10-09 08:40:12 -07:00
Kenfe-Mickael Laventure
ad5266456c
windows: Fix a few races
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-10-09 08:40:12 -07:00
Kenfe-Mickaël Laventure
b2e348231c Merge pull request #1584 from miaoyq/fix-mount-lookup-err
Fixes looking up mountinfo corresponds to path
2017-10-09 08:35:01 -07:00
Yanqiang Miao
d7c4611848 Ensure getting the correct mountinfo corresponds to path
Signed-off-by: Yanqiang Miao <miao.yanqiang@zte.com.cn>
2017-10-08 17:16:28 +08:00
Akihiro Suda
a8426ed9e7 Merge pull request #1609 from crosbymichael/attach-docs
Add docs around multiple attach
2017-10-08 11:24:57 +09:00
Derek McGowan
6c3f393765 Merge pull request #1611 from stevvooe/address-cgroup-merge-conflicty
metrics/cgroups: handle error on call to cgroup
2017-10-06 16:09:20 -07:00
Stephen J Day
77e5f6553c
metrics/cgroups: handle error on call to cgroup
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-10-06 15:51:20 -07:00
Michael Crosby
bbb01ee1ab Add docs around multiple attach
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-10-06 17:31:11 -04:00
Phil Estes
987fcd1201 Merge pull request #1598 from Random-Liu/fix-load-task
Fix task load.
2017-10-06 16:38:40 -04:00
Stephen Day
67bbfc5d1c Merge pull request #1605 from dmcgowan/metadata-db-documentation
Add documentation for metadata database
2017-10-06 13:37:00 -07:00
Derek McGowan
e4defbcb6d
Add documentation for metadata database
Add documentation and explanation for migrations.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-10-06 13:33:07 -07:00
Michael Crosby
6f351f0a85 Merge pull request #1586 from mathieui/postmortem-stats
Collect cgroup stats one last time before exit
2017-10-06 15:44:28 -04:00
Michael Crosby
d92f6eea1f Allow blocking and non-blocking metrics collection
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-10-06 12:05:56 -04:00
Mathieu Pasquet
ed519bb5ce Collect cgroup stats one last time before exit
This commit adds a collection step in the Stop() task handler which will
retrieve the metrics available for this container at that time, and
store them until the next prometheus Collect() cycle.

This allows short-lived containers to be visible in prometheus, which
would otherwise be ignored (for example, running containerd-stress would
show something like 2 or 3 containers in the end, while now we can see
all of them). It also allows for more accurate collection when
long-running containers end (for example CPU usage could spike in the
last few seconds).

A simple case illustrating this with cpu usage would be:

  ctr run -t --rm docker.io/library/alpine:latest mycontainer sh -c 'yes > /dev/null & sleep 3 && pkill yes'

Signed-off-by: Mathieu Pasquet <mathieu.pasquet@alterway.fr>
2017-10-06 15:40:38 +02:00
Stephen Day
8558b98eb1 Merge pull request #1582 from dmcgowan/metadata-db-object
Create metadata db object
2017-10-05 16:50:44 -07:00
Derek McGowan
7f657ce3de
Add database migrations
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-10-05 16:35:58 -07:00
Lantao Liu
28ca8f05d3 Fix task load.
Signed-off-by: Lantao Liu <lantaol@google.com>
2017-10-05 21:03:24 +00:00
Phil Estes
7c4bca591f Merge pull request #1600 from AkihiroSuda/defaultspec
spec: set MaskedPaths and ReadOnlyPaths by default
2017-10-05 12:16:09 -04:00
Akihiro Suda
d7b0e522c7 spec: set MaskedPaths and ReadOnlyPaths by default
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-10-05 06:01:35 +00:00
Phil Estes
72a3a019ae Merge pull request #1595 from crosbymichael/container-lock
sync container metadata before operations
2017-10-04 21:48:30 -04:00
Stephen Day
28882fddae Merge pull request #1589 from mlaventure/shim-no-newns
Add an option to prevent putting the shim in a new mount namespace
2017-10-04 17:24:26 -07:00
Michael Crosby
fa9e9bdf46 Fetch current container info before operations
This makes sure the client is always in sync with the server before
performing any type of operations on the container metadata.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-10-04 15:29:57 -04:00
Derek McGowan
8d892a651b
Update metadata plugin initialization
Updates metadata plugin to require content and
snapshotter plugins be loaded and initializes with
those plugins, keeping the metadata database structure
static after initialization. Service plugins now only
require metadata plugin access snapshotter or content
stores through metadata, which was already required
behavior of the services.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-10-04 11:25:46 -07:00
Derek McGowan
2ab70f21ac
Add content and snapshot store references
Update database object to hold reference to the
data stores.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-10-04 11:16:22 -07:00
Derek McGowan
447a0a9452
Add children bucket to back reference snapshots
Adds back links from parent to children in order to prevent
deletion of a referenced snapshot in a namespace.
Avoid removing snapshot during metadata delete to
prevent shared namespaces from being mistakenly deleted.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-10-04 11:16:22 -07:00
Derek McGowan
56c1b79a4c
Create metadata db object
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-10-04 11:16:22 -07:00
Kenfe-Mickaël Laventure
acba0f50ef Merge pull request #1594 from akimd/akim/clean
style: move definition for consistency
2017-10-04 09:18:43 -07:00
Kenfe-Mickaël Laventure
ec43dc2b46 Merge pull request #1592 from AkihiroSuda/workdir2
ctr run: add --cwd
2017-10-04 09:17:21 -07:00
Kenfe-Mickael Laventure
26d4c2c217
Add an option to prevent putting the shim in a new mount namespace
This is needed for users on kernel older than 3.18 so they can avoid EBUSY
errors when trying to unlink, rename or remove a mountpoint that is present in
a shim namespace.

Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-10-04 09:16:02 -07:00
Michael Crosby
0ae9d8fabf Merge pull request #1567 from stevvooe/image-check-function
images: support checking status of image content
2017-10-04 11:40:43 -04:00
Akim Demaille
8ceeeed4ae style: move definition for consistency
Signed-off-by: Akim Demaille <akim.demaille@docker.com>
2017-10-04 17:02:51 +02:00
Akihiro Suda
b26e2e781c ctr run: add --cwd
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-10-04 04:40:08 +00:00
Stephen Day
f6d88927d5 Merge pull request #1583 from stevvooe/image-storage-validation
metadata: validation and testing of image store
2017-10-03 16:48:38 -07:00
Stephen J Day
698b6d15f7
metadata: validation and testing of image store
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-10-03 16:34:00 -07:00
Stephen J Day
c555df54c0
images: support checking status of image content
The `Check` function returns information about an image's content components
over a content provider. From this information, one can tell which content is
required, present or missing to run an image.

The utility can be demonstrated with the `check` command:

```console
$ ctr images check
REF                               TYPE                                                      DIGEST                                                                  STATUS            SIZE
docker.io/library/alpine:latest   application/vnd.docker.distribution.manifest.list.v2+json sha256:f006ecbb824d87947d0b51ab8488634bf69fe4094959d935c0c103f4820a417d incomplete (1/2)  1.5 KiB/1.9 MiB
docker.io/library/postgres:latest application/vnd.docker.distribution.manifest.v2+json      sha256:2f8080b9910a8b4f38ff5a55a82e77cb43d88bdbb16d723c71d18493590832e9 complete (13/13)  99.3 MiB/99.3 MiB
docker.io/library/redis:alpine    application/vnd.docker.distribution.manifest.v2+json      sha256:e633cded055a94202e4ccccb8125b7f383cd6ee56527ab890db643383a2647dd incomplete (6/7)  8.1 MiB/10.0 MiB
docker.io/library/ubuntu:latest   application/vnd.docker.distribution.manifest.list.v2+json sha256:60f835698ea19e8d9d3a59e68fb96fb35bc43e745941cb2ea9eaf4ba3029ed8a unavailable (0/?) 0.0 B/?
docker.io/trollin/busybox:latest  application/vnd.docker.distribution.manifest.list.v2+json sha256:54a6424f7a2d5f4f27b3d69e5f9f2bc25fe9087f0449d3cb4215db349f77feae complete (2/2)    699.9 KiB/699.9 KiB
```

The above shows us that we have two incomplete images and one that is
unavailable. The incomplete images are those that we know the complete
size of all content but some are missing. "Unavailable" means that the
check could not get enough information about the image to get its full
size.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-10-03 15:19:22 -07:00
Kenfe-Mickaël Laventure
7c9b0eab9f Merge pull request #1590 from crosbymichael/container-update
Add container update method
2017-10-03 14:33:58 -07:00
Michael Crosby
9e85035f15 Add container update method
This adds an update method to the container interface for the client
package.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-10-03 16:59:52 -04:00
Phil Estes
1c7a042e04 Merge pull request #1566 from stevvooe/split-children
images: provide separated children function for images
2017-10-03 13:46:46 -04:00
Phil Estes
6b9aafdab1 Merge pull request #1587 from Ace-Tang/remove_blank_line
Remove a blank line
2017-10-03 11:14:47 -04:00
Ace-Tang
2231de3760 Remove a blank line
Signed-off-by: Ace-Tang <aceapril@126.com>
2017-10-03 20:58:51 +08:00
Michael Crosby
8ded4fe3a7 Merge pull request #1581 from crosbymichael/lint-2
Comment more packages to pass go lint
2017-10-02 14:46:28 -04:00