Commit Graph

8778 Commits

Author SHA1 Message Date
Michael Crosby
bf4838eb71 Shutdown console after process exits
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-08-02 17:09:50 -04:00
Stephen J Day
610e1a250d
vendor: update OCI image-spec dependency
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-08-02 14:08:13 -07:00
Lantao Liu
502072e63e Merge pull request #107 from Random-Liu/fix-deadlink
Fix deadlink of travis.yml.
2017-08-02 11:06:40 -07:00
Kenfe-Mickaël Laventure
8b0a040004 Merge pull request #1268 from crosbymichael/exec-state
Add exec process states
2017-08-02 19:59:01 +02:00
Michael Crosby
f945cdc704 Update windows locking code
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-08-02 13:50:08 -04:00
Michael Crosby
d18af8699c Update for epoll console handling
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-08-02 13:50:08 -04:00
Michael Crosby
e827adaf11 Update windows with exec create/start actions
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-08-02 13:50:08 -04:00
Michael Crosby
83b27db923 Check status for stopped processes before wait
Fixes #1054

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-08-02 13:50:08 -04:00
Michael Crosby
cd00216fd7 Rename TaskStatus to Status
This can be renamed because both Tasks and Processes share the same
status consts

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-08-02 13:50:08 -04:00
Michael Crosby
504033e373 Add Get of task and process state
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-08-02 13:50:08 -04:00
Michael Crosby
2974e485f3 Add no-criu flag for running tests
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-08-02 13:50:08 -04:00
Michael Crosby
9f08965699 Change Exited/Status to SetExited/ExitStatus
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-08-02 13:50:08 -04:00
Michael Crosby
a2a3451925 Implement Exec + Start for tasks service
This splits up the exec creation and start in the tasks service

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-08-02 13:50:08 -04:00
Michael Crosby
63878d14ea Add create/start to exec processes in shim
This splits up the create and start of an exec process in the shim to
have two separate steps like the initial process.  This will allow
better state reporting for individual process along with a more robust
wait for execs.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-08-02 13:50:08 -04:00
Lantao Liu
4f98eac93c Fix deadlink of travis.yaml.
Signed-off-by: Lantao Liu <lantaol@google.com>
2017-08-02 16:27:59 +00:00
Michael Crosby
2533bfeaaf Merge pull request #1271 from stevvooe/autogenerate-fieldpaths
events: autogenerate fieldpath filters
2017-08-02 10:12:24 -04:00
Kenfe-Mickael Laventure
789330033a
Invoke ineffassign during CI
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-08-01 15:16:38 -07:00
Kenfe-Mickael Laventure
9526f5366d
Remove Windows cross compile from Travis
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-08-01 14:54:32 -07:00
Stephen J Day
c857ba2d0b
events: autogenerate fieldpath filters
To ensure consistent fieldpath matching for events, we generate the
fieldpath matching using protobuf definitions. This is done through a
plugin called "fieldpath" that defines a `Field` method for each type
with the plugin enabled. Generated code handles top-level envelope
fields, as well as deferred serialization for matching any types.

In practice, this means that we can cheaply match events on `topic` and
`namespace`. If we want to match on attributes within the event, we can
use the `event` prefix to address these fields. For example, the
following will match all envelopes that have a field named
`container_id` that has the value `testing`:

```
ctr events "event.container_id==testing"
```

The above will decode the underlying event and check that particular
field. Accordingly, if only `topic` or `namespace` is used, the event
will not be decoded and only match on the envelope.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-08-01 14:09:38 -07:00
Lantao Liu
6017d84040 Merge pull request #104 from Random-Liu/temporarily-remove-some-unit-test
Temporarily remove unit test relying on fake containerd services.
2017-08-01 00:37:35 -07:00
Stephen Day
bb7b41ad7b Merge pull request #1269 from stevvooe/plumb-correct-interfaces
linux/shim: use events.Publisher interface
2017-07-31 15:53:35 -07:00
Lantao Liu
ffb69423ec Temporarily remove unit test relying on fake containerd services.
Signed-off-by: Lantao Liu <lantaol@google.com>
2017-07-31 22:42:10 +00:00
Stephen J Day
7ed88c1e36
linux/shim: use events.Publisher interface
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-07-31 14:23:51 -07:00
Stephen Day
92d737f4ae Merge pull request #1259 from dqminh/epoll-io
Use Epoll to perform I/O in linux
2017-07-31 13:47:41 -07:00
Michael Crosby
834f55df78 Merge pull request #1264 from stevvooe/event-filters
events: initial support for filters
2017-07-31 16:00:58 -04:00
Stephen J Day
af2d7f0e55
events: initial support for filters
This change further plumbs the components required for implementing
event filters. Specifically, we now have the ability to filter on the
`topic` and `namespace`.

In the course of implementing this functionality, it was found that
there were mismatches in the events API that created extra serialization
round trips. A modification to `typeurl.MarshalAny` and a clear
separation between publishing and forwarding allow us to avoid these
serialization issues.

Unfortunately, this has required a few tweaks to the GRPC API, so this
is a breaking change. `Publish` and `Forward` have been clearly separated in
the GRPC API. `Publish` honors the contextual namespace and performs
timestamping while `Forward` simply validates and forwards. The behavior
of `Subscribe` is to propagate events for all namespaces unless
specifically filtered (and hence the relation to this particular change.

The following is an example of using filters to monitor the task events
generated while running the [bucketbench tool](https://github.com/estesp/bucketbench):

```
$ ctr events 'topic~=/tasks/.+,namespace==bb'
...
2017-07-28 22:19:51.78944874 +0000 UTC   bb        /tasks/start   {"container_id":"bb-ctr-6-8","pid":25889}
2017-07-28 22:19:51.791893688 +0000 UTC   bb        /tasks/start   {"container_id":"bb-ctr-4-8","pid":25882}
2017-07-28 22:19:51.792608389 +0000 UTC   bb        /tasks/start   {"container_id":"bb-ctr-2-9","pid":25860}
2017-07-28 22:19:51.793035217 +0000 UTC   bb        /tasks/start   {"container_id":"bb-ctr-5-6","pid":25869}
2017-07-28 22:19:51.802659622 +0000 UTC   bb        /tasks/start   {"container_id":"bb-ctr-0-7","pid":25877}
2017-07-28 22:19:51.805192898 +0000 UTC   bb        /tasks/start   {"container_id":"bb-ctr-3-6","pid":25856}
2017-07-28 22:19:51.832374931 +0000 UTC   bb        /tasks/exit   {"container_id":"bb-ctr-8-6","id":"bb-ctr-8-6","pid":25864,"exited_at":"2017-07-28T22:19:51.832013043Z"}
2017-07-28 22:19:51.84001249 +0000 UTC   bb        /tasks/exit   {"container_id":"bb-ctr-2-9","id":"bb-ctr-2-9","pid":25860,"exited_at":"2017-07-28T22:19:51.839717714Z"}
2017-07-28 22:19:51.840272635 +0000 UTC   bb        /tasks/exit   {"container_id":"bb-ctr-7-6","id":"bb-ctr-7-6","pid":25855,"exited_at":"2017-07-28T22:19:51.839796335Z"}
...
```

In addition to the events changes, we now display the namespace origin
of the event in the cli tool.

This will be followed by a PR to add individual field filtering for the
events API for each event type.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-07-31 12:53:18 -07:00
Michael Crosby
4ba6d8e558 Merge pull request #1267 from crosbymichael/bump-runc
Update runc to e775f0fba3ea329b8b766451c892c41a3d4
2017-07-31 14:43:28 -04:00
Michael Crosby
2386062ce1 Update runc to e775f0fba3ea329b8b766451c892c41a3d4
This resolves logrus issues with containerd not being in sync with the
version runc is using.

This also updates the OCI runtime spec to v1.0.0

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-07-31 14:33:49 -04:00
Michael Crosby
ce07fa04ac Merge pull request #1263 from crosbymichael/no-shim-events
event forwarding without shim
2017-07-31 14:26:18 -04:00
Stephen Day
f86d5ae09d Merge pull request #1262 from dmcgowan/more-snapshot-tests
Add snapshot test cases for former issues
2017-07-31 11:22:37 -07:00
Michael Crosby
7b6ff6ec89 event forwarding without shim
Fixes #1138

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-07-31 10:05:24 -04:00
Lantao Liu
cbd936b734 Merge pull request #66 from Random-Liu/refactor-metadata-store
Rewrite metadata store
2017-07-30 22:01:34 -07:00
Daniel Dao
8e53465842
use epoll to manage console i/o in linux
this adds a `platform` interface for shim service to manage platform-specific
behaviors such as I/O (which uses epoll in linux to work around bugs with applications
that closes all consoles i.e. https://github.com/opencontainers/runc/pull/1434
and https://github.com/moby/moby/issues/27202)

Its expected that we only have 1 epollfd per containerd_shim to manage all processes.
Since all the work are done outside of the container runtime, upgrading of runc
is not required and should be done separately.

Signed-off-by: Daniel Dao <dqminh89@gmail.com>
2017-07-30 10:50:39 +01:00
Lantao Liu
f4df66eaaf Remove old metadata store.
Signed-off-by: Lantao Liu <lantaol@google.com>
2017-07-28 23:35:31 +00:00
Lantao Liu
7b16a35287 Use new metadata store.
Signed-off-by: Lantao Liu <lantaol@google.com>
2017-07-28 23:35:31 +00:00
Lantao Liu
4317e6119a Remove sandbox truncindex.
Signed-off-by: Lantao Liu <lantaol@google.com>
2017-07-28 23:35:31 +00:00
Lantao Liu
a393f3a084 Add new metadata store.
Signed-off-by: Lantao Liu <lantaol@google.com>
2017-07-28 23:35:31 +00:00
Derek McGowan
d28126f05d
Handle kernel related ci failure
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-07-28 13:45:51 -07:00
Derek McGowan
a7c44f0038
Add snapshot test cases for former issues
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-07-28 13:37:38 -07:00
Stephen Day
20fa6aee2e Merge pull request #1182 from AkihiroSuda/rootfs
client, ctr: allow specifying unmanaged rootfs dir
2017-07-28 10:56:25 -07:00
Lantao Liu
e5d69aa537 Merge pull request #102 from Random-Liu/get-spec-from-metadata
Get spec from metadata
2017-07-28 09:45:43 -07:00
Lantao Liu
d8ce670109 Upgrade cri validation test.
Signed-off-by: Lantao Liu <lantaol@google.com>
2017-07-28 16:26:20 +00:00
Random-Liu
b398a161de Get runtime spec from container metadata.
Signed-off-by: Random-Liu <lantaol@google.com>
2017-07-28 16:26:20 +00:00
Michael Crosby
183a6ca888 Merge pull request #1013 from AkihiroSuda/oci-export
client: add Import() and Export() for importing/exporting image in OCI format
2017-07-28 09:22:22 -04:00
Daniel Dao
de2671b7f5
bump vendor for containerd/console update
Signed-off-by: Daniel Dao <dqminh89@gmail.com>
2017-07-28 12:23:49 +01:00
Akihiro Suda
b518f11dba client: add Import() and Export() for importing/exporting image in OCI format
Export as a tar (Note: "-" can be used for stdout):

    $ ctr images export /tmp/oci-busybox.tar docker.io/library/busybox:latest

Import a tar (Note: "-" can be used for stdin):

    $ ctr images import foo/new:latest /tmp/oci-busybox.tar

Note: media types are not converted at the moment: e.g.
  application/vnd.docker.image.rootfs.diff.tar.gzip
  -> application/vnd.oci.image.layer.v1.tar+gzip

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-07-28 04:47:53 +00:00
Akihiro Suda
752d253f40 client, ctr: allow specifying unmanaged rootfs dir
e.g. ctr run -t --rm --rootfs /tmp/busybox-rootfs foo /bin/sh
(--rm removes the container but does not remove rootfs dir, of course)

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-07-28 04:45:04 +00:00
Stephen Day
1a054c67b1 Merge pull request #1134 from crosbymichael/shim-cgroup
Allow shim to be placed inside cgroup
2017-07-27 16:09:07 -07:00
Derek McGowan
4e299a9a20 Merge pull request #1254 from stevvooe/close-channel
events: close channel explicitly
2017-07-27 15:59:50 -07:00
Stephen Day
1679727648 Merge pull request #1230 from dmcgowan/snapshot-client-testsuite
snapshot: run test suite for metadata and client implementations
2017-07-27 15:49:17 -07:00