Commit Graph

8778 Commits

Author SHA1 Message Date
Derek McGowan
69e7e7fa4f
Add error wrapping for client checkpoint errors
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-10-11 10:02:29 -07:00
Derek McGowan
d9db1d112d
Refactor differ into separate package
Add differ options and package with interface.
Update optional values on diff interface to use options.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-10-11 10:02:29 -07:00
Michael Crosby
60960e1c17 Merge pull request #1597 from stevvooe/introspection-service
api/services: define the introspection API
2017-10-11 10:57:11 -04:00
Lantao Liu
e4b818ff41 Merge pull request #342 from Random-Liu/update-kubernetes-containerd
Update kubernetes and containerd.
2017-10-11 00:23:24 -07:00
Lantao Liu
bde8b0517e Update kubernetes and containerd.
Signed-off-by: Lantao Liu <lantaol@google.com>
2017-10-11 06:16:19 +00:00
Stephen J Day
0e72ce5c5b
services/introspection: implement plugin reporting
With this change, we integrate all the plugin changes into the
introspection service.

All plugins can be listed with the following command:

```console
$ ctr plugins
TYPE                            ID             PLATFORM       STATUS
io.containerd.content.v1        content        -              ok
io.containerd.metadata.v1       bolt           -              ok
io.containerd.differ.v1         walking        linux/amd64    ok
io.containerd.grpc.v1           containers     -              ok
io.containerd.grpc.v1           content        -              ok
io.containerd.grpc.v1           diff           -              ok
io.containerd.grpc.v1           events         -              ok
io.containerd.grpc.v1           healthcheck    -              ok
io.containerd.grpc.v1           images         -              ok
io.containerd.grpc.v1           namespaces     -              ok
io.containerd.snapshotter.v1    btrfs          linux/amd64    error
io.containerd.snapshotter.v1    overlayfs      linux/amd64    ok
io.containerd.grpc.v1           snapshots      -              ok
io.containerd.monitor.v1        cgroups        linux/amd64    ok
io.containerd.runtime.v1        linux          linux/amd64    ok
io.containerd.grpc.v1           tasks          -              ok
io.containerd.grpc.v1           version        -              ok
```

There are few things to note about this output. The first is that it is
printed in the order in which plugins are initialized. This useful for
debugging plugin initialization problems. Also note that even though the
introspection GPRC api is a itself a plugin, it is not listed. This is
because the plugin takes a snapshot of the initialization state at the
end of the plugin init process. This allows us to see errors from each
plugin, as they happen. If it is required to introspect the existence of
the introspection service, we can make modifications to include it in
the future.

The last thing to note is that the btrfs plugin is in an error state.
This is a common state for containerd because even though we load the
plugin, most installations aren't on top of btrfs and the plugin cannot
be used. We can actually view this error using the detailed view with a
filter:

```console
$ ctr plugins --detailed id==btrfs
Type:          io.containerd.snapshotter.v1
ID:            btrfs
Platforms:     linux/amd64
Exports:
               root      /var/lib/containerd/io.containerd.snapshotter.v1.btrfs
Error:
               Code:        Unknown
               Message:     path /var/lib/containerd/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter
```

Along with several other values, this is a valuable tool for evaluating the
state of components in containerd.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-10-10 17:29:24 -07:00
Stephen J Day
49814646cf
plugin: allow declaring a plugin depends on all others
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-10-10 16:40:47 -07:00
Stephen J Day
8508e8252b
plugin: refactor plugin system to support initialization reporting
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-10-10 16:40:47 -07:00
Stephen J Day
fe52d9369f
errdefs: use status package for mapping errors
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-10-10 16:40:47 -07:00
Stephen J Day
d8bd6b70ce
plugin: allow querying plugin graph to be re-entrant
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-10-10 16:40:47 -07:00
Stephen J Day
363d692f35
api/services: define the introspection API
After several requests for information about platform support,
configuration introspection and feature switches, we now have a solution
that should work in all these use cases. The Introspection API hooks
into the plugin subsystem registration process. During registration,
plugins declare several pieces of information, allowing clients to
discover the cababilities and support that a containerd instance
provides, including whether or not it loaded with an error.

To allow symmetrical error reporting, the `google/rpc.Status` protobuf
definitions have been brought in from the googleapis project.
Unfortunately, we had to generate these in place to match our protobuf
system.

Once we like this design, we can add an implementation to integrate it
directly with the plugin system.

Enjoy!

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-10-10 16:40:47 -07:00
Lantao Liu
5531a71873 Add prow node e2e test.
Signed-off-by: Lantao Liu <lantaol@google.com>
2017-10-10 23:29:44 +00:00
Michael Crosby
12c79cc0d9 Merge pull request #1604 from jessvalarezo/windows-listpids
ListPids returns process details in addition to process IDs.
2017-10-10 19:26:47 -04:00
Lantao Liu
d50c610947 Merge pull request #335 from Random-Liu/fix-fs-uuid
Use device number to find uuid
2017-10-10 16:06:18 -07:00
Lantao Liu
232f1e4cdc Merge pull request #340 from Random-Liu/disable-maligned
Disable maligned check.
2017-10-10 16:06:05 -07:00
Jess
061c719209 ListPids returns process ID and other info
Signed-off-by: Jess <jessica.valarezo@docker.com>
2017-10-10 22:57:15 +00:00
Lantao Liu
aab0c5df77 Disable maligned check.
Signed-off-by: Lantao Liu <lantaol@google.com>
2017-10-10 22:38:00 +00:00
Phil Estes
96725a5a0b Merge pull request #1620 from mlaventure/runc-io-id
Allow setting the uid & gid of the io pipes
2017-10-10 17:12:34 -04:00
Kenfe-Mickael Laventure
c807ba8379
Allow setting the uid & gid of the io pipes
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-10-10 13:24:12 -07:00
Phil Estes
1ea8ac41d3 Merge pull request #1617 from crosbymichael/2cgroup
Use namespace in default cgroup path
2017-10-10 10:55:12 -04:00
Michael Crosby
d7864eb77b Use namespace in default cgroup path
By default, the generated spec will place containers in cgroups by their
ids, we need to use the namespace as the cgroup root to avoid
containers with the same name being placed in the same cgroup.

```
11:perf_event:/to/redis
10:freezer:/to/redis
9:memory:/to/redis
8:devices:/to/redis
7:net_cls,net_prio:/to/redis
6:pids:/to/redis
5:hugetlb:/to/redis
4:cpuset:/to/redis
3:blkio:/to/redis
2:cpu,cpuacct:/to/redis
1:name=systemd:/to/redis

11:perf_event:/te/redis
10:freezer:/te/redis
9:memory:/te/redis
8:devices:/te/redis
7:net_cls,net_prio:/te/redis
6:pids:/te/redis
5:hugetlb:/te/redis
4:cpuset:/te/redis
3:blkio:/te/redis
2:cpu,cpuacct:/te/redis
1:name=systemd:/te/redis
```

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-10-09 17:07:57 -04:00
Lantao Liu
679868c667 Merge pull request #336 from Random-Liu/add-node-e2e-tar
Add staging tar for node e2e.
2017-10-09 13:38:47 -07:00
Michael Crosby
72bb45ac46 Merge pull request #1603 from mlaventure/fix-windows-flaky-tests
windows: ensure a init process exist after "Create()"
2017-10-09 11:58:23 -04:00
Kenfe-Mickael Laventure
cfa87567a0
windows: Create init process with task
This prevents `task.Wait()` to return an error if it is called before the task
is started.

Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-10-09 08:40:12 -07:00
Kenfe-Mickael Laventure
ad5266456c
windows: Fix a few races
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-10-09 08:40:12 -07:00
Kenfe-Mickaël Laventure
b2e348231c Merge pull request #1584 from miaoyq/fix-mount-lookup-err
Fixes looking up mountinfo corresponds to path
2017-10-09 08:35:01 -07:00
Lantao Liu
6cb3d27ed3 Use device number to find uuid
Signed-off-by: Lantao Liu <lantaol@google.com>
2017-10-09 06:10:43 +00:00
Lantao Liu
9135ef2c07 Merge pull request #334 from Random-Liu/change-versioning
Change `Version` to return cri-containerd version instead.
2017-10-08 20:28:39 -07:00
Yanqiang Miao
d7c4611848 Ensure getting the correct mountinfo corresponds to path
Signed-off-by: Yanqiang Miao <miao.yanqiang@zte.com.cn>
2017-10-08 17:16:28 +08:00
Akihiro Suda
a8426ed9e7 Merge pull request #1609 from crosbymichael/attach-docs
Add docs around multiple attach
2017-10-08 11:24:57 +09:00
Lantao Liu
2449ffdc42 Add staging tar for node e2e.
Signed-off-by: Lantao Liu <lantaol@google.com>
2017-10-07 04:45:10 +00:00
Lantao Liu
09d7d652e6 Change Version to return cri-containerd version instead.
Signed-off-by: Lantao Liu <lantaol@google.com>
2017-10-07 00:47:51 +00:00
Derek McGowan
6c3f393765 Merge pull request #1611 from stevvooe/address-cgroup-merge-conflicty
metrics/cgroups: handle error on call to cgroup
2017-10-06 16:09:20 -07:00
Stephen J Day
77e5f6553c
metrics/cgroups: handle error on call to cgroup
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-10-06 15:51:20 -07:00
Michael Crosby
bbb01ee1ab Add docs around multiple attach
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-10-06 17:31:11 -04:00
Phil Estes
987fcd1201 Merge pull request #1598 from Random-Liu/fix-load-task
Fix task load.
2017-10-06 16:38:40 -04:00
Stephen Day
67bbfc5d1c Merge pull request #1605 from dmcgowan/metadata-db-documentation
Add documentation for metadata database
2017-10-06 13:37:00 -07:00
Derek McGowan
e4defbcb6d
Add documentation for metadata database
Add documentation and explanation for migrations.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-10-06 13:33:07 -07:00
Lantao Liu
e77795b968 Merge pull request #332 from Random-Liu/use-container-update
Use container update
2017-10-06 13:20:30 -07:00
Michael Crosby
6f351f0a85 Merge pull request #1586 from mathieui/postmortem-stats
Collect cgroup stats one last time before exit
2017-10-06 15:44:28 -04:00
Lantao Liu
e78c85f76b Use new container update function
Signed-off-by: Lantao Liu <lantaol@google.com>
2017-10-06 17:57:26 +00:00
Lantao Liu
46ed19b23c Merge pull request #331 from Random-Liu/run-repodigest-node-e2e
Run repodigest test.
2017-10-06 10:18:37 -07:00
Michael Crosby
d92f6eea1f Allow blocking and non-blocking metrics collection
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-10-06 12:05:56 -04:00
Mathieu Pasquet
ed519bb5ce Collect cgroup stats one last time before exit
This commit adds a collection step in the Stop() task handler which will
retrieve the metrics available for this container at that time, and
store them until the next prometheus Collect() cycle.

This allows short-lived containers to be visible in prometheus, which
would otherwise be ignored (for example, running containerd-stress would
show something like 2 or 3 containers in the end, while now we can see
all of them). It also allows for more accurate collection when
long-running containers end (for example CPU usage could spike in the
last few seconds).

A simple case illustrating this with cpu usage would be:

  ctr run -t --rm docker.io/library/alpine:latest mycontainer sh -c 'yes > /dev/null & sleep 3 && pkill yes'

Signed-off-by: Mathieu Pasquet <mathieu.pasquet@alterway.fr>
2017-10-06 15:40:38 +02:00
Lantao Liu
f7eaa29c67 Update containerd to 8558b98eb1
Signed-off-by: Lantao Liu <lantaol@google.com>
2017-10-06 01:40:39 +00:00
Lantao Liu
3ff61f9010 Run repodigest test.
Signed-off-by: Lantao Liu <lantaol@google.com>
2017-10-05 23:55:55 +00:00
Stephen Day
8558b98eb1 Merge pull request #1582 from dmcgowan/metadata-db-object
Create metadata db object
2017-10-05 16:50:44 -07:00
Derek McGowan
7f657ce3de
Add database migrations
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-10-05 16:35:58 -07:00
Lantao Liu
28ca8f05d3 Fix task load.
Signed-off-by: Lantao Liu <lantaol@google.com>
2017-10-05 21:03:24 +00:00
Lantao Liu
d2122d62dd Merge pull request #324 from Random-Liu/installation
Add guide for installation from release tarball
2017-10-05 13:43:59 -07:00