Commit Graph

70 Commits

Author SHA1 Message Date
Stephen Day
a29c6c7ab9 Merge pull request #1386 from Random-Liu/task-service-grpc-error
Convert errors returned by task service to errdefs error.
2017-08-17 16:04:12 -07:00
Lantao Liu
f79981c2df Convert errors returned by task service to errdefs error.
Signed-off-by: Lantao Liu <lantaol@google.com>
2017-08-17 22:52:16 +00:00
Stephen Day
a64399acc2 Merge pull request #1341 from mlaventure/enable-test-parallelism
Enable test parallelism
2017-08-14 15:18:37 -07:00
Kenfe-Mickael Laventure
b02e9a844e
Fix TestContainerNoBinaryExists on windows
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-08-14 14:43:43 -07:00
Lantao Liu
8e2c95f9c1 Change IO to interface.
Signed-off-by: Lantao Liu <lantaol@google.com>
2017-08-14 18:53:12 +00:00
Kenfe-Mickael Laventure
5f36ac2093
Add test to ensure we can access tasks on restart
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-08-11 09:34:29 -07:00
Michael Crosby
d8c075aadc Add WithProcessKill DeleteOpt
Add an option that allows users for force kill and delete a process/task
when calling `Delete`

Fixes #1274

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-08-08 10:37:17 -04:00
Kenfe-Mickael Laventure
56b18c1d1f
Move client's options to separate files
This should help in discovering the available options for a given object.

Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-08-04 13:56:16 -07:00
Derek McGowan
99315f8e02 Merge pull request #1285 from crosbymichael/client-godoc
Update godoc for client package
2017-08-04 10:38:53 -07:00
Lantao Liu
8fae8a5b77 Cancel event subscribe context in Wait.
Signed-off-by: Lantao Liu <lantaol@google.com>
2017-08-04 16:46:53 +00:00
Michael Crosby
0a85f6e47d Update godoc for client package
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-08-04 11:53:31 -04:00
Michael Crosby
fdc5a475a8 Gate task.Wait status check on checkpoints
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-08-03 18:02:48 -04:00
Michael Crosby
9f13b414b9 Return exit status from Wait of stopped process
This changes Wait() from returning an error whenever you call wait on a
stopped process/task to returning the exit status from the process.

This also adds the exit status to the Status() call on a process/task so
that a user can Wait(), check status, then cancel the wait to avoid
races in event handling.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-08-03 17:22:33 -04:00
Michael Crosby
83b27db923 Check status for stopped processes before wait
Fixes #1054

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-08-02 13:50:08 -04:00
Michael Crosby
cd00216fd7 Rename TaskStatus to Status
This can be renamed because both Tasks and Processes share the same
status consts

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-08-02 13:50:08 -04:00
Michael Crosby
504033e373 Add Get of task and process state
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-08-02 13:50:08 -04:00
Michael Crosby
a2a3451925 Implement Exec + Start for tasks service
This splits up the exec creation and start in the tasks service

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-08-02 13:50:08 -04:00
Stephen J Day
af2d7f0e55
events: initial support for filters
This change further plumbs the components required for implementing
event filters. Specifically, we now have the ability to filter on the
`topic` and `namespace`.

In the course of implementing this functionality, it was found that
there were mismatches in the events API that created extra serialization
round trips. A modification to `typeurl.MarshalAny` and a clear
separation between publishing and forwarding allow us to avoid these
serialization issues.

Unfortunately, this has required a few tweaks to the GRPC API, so this
is a breaking change. `Publish` and `Forward` have been clearly separated in
the GRPC API. `Publish` honors the contextual namespace and performs
timestamping while `Forward` simply validates and forwards. The behavior
of `Subscribe` is to propagate events for all namespaces unless
specifically filtered (and hence the relation to this particular change.

The following is an example of using filters to monitor the task events
generated while running the [bucketbench tool](https://github.com/estesp/bucketbench):

```
$ ctr events 'topic~=/tasks/.+,namespace==bb'
...
2017-07-28 22:19:51.78944874 +0000 UTC   bb        /tasks/start   {"container_id":"bb-ctr-6-8","pid":25889}
2017-07-28 22:19:51.791893688 +0000 UTC   bb        /tasks/start   {"container_id":"bb-ctr-4-8","pid":25882}
2017-07-28 22:19:51.792608389 +0000 UTC   bb        /tasks/start   {"container_id":"bb-ctr-2-9","pid":25860}
2017-07-28 22:19:51.793035217 +0000 UTC   bb        /tasks/start   {"container_id":"bb-ctr-5-6","pid":25869}
2017-07-28 22:19:51.802659622 +0000 UTC   bb        /tasks/start   {"container_id":"bb-ctr-0-7","pid":25877}
2017-07-28 22:19:51.805192898 +0000 UTC   bb        /tasks/start   {"container_id":"bb-ctr-3-6","pid":25856}
2017-07-28 22:19:51.832374931 +0000 UTC   bb        /tasks/exit   {"container_id":"bb-ctr-8-6","id":"bb-ctr-8-6","pid":25864,"exited_at":"2017-07-28T22:19:51.832013043Z"}
2017-07-28 22:19:51.84001249 +0000 UTC   bb        /tasks/exit   {"container_id":"bb-ctr-2-9","id":"bb-ctr-2-9","pid":25860,"exited_at":"2017-07-28T22:19:51.839717714Z"}
2017-07-28 22:19:51.840272635 +0000 UTC   bb        /tasks/exit   {"container_id":"bb-ctr-7-6","id":"bb-ctr-7-6","pid":25855,"exited_at":"2017-07-28T22:19:51.839796335Z"}
...
```

In addition to the events changes, we now display the namespace origin
of the event in the cli tool.

This will be followed by a PR to add individual field filtering for the
events API for each event type.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-07-31 12:53:18 -07:00
Michael Crosby
b84817a29c Merge pull request #1246 from stevvooe/events-refactor
events: refactor event distribution
2017-07-26 08:31:12 -04:00
Derek McGowan
c7cc6ad2b4
Add root mounts to task info
Add WithRootFS for task creation

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-07-25 15:24:06 -07:00
Stephen J Day
a615a6fe5d
events: refactor event distribution
In the course of setting out to add filters and address some cleanup, it
was found that we had a few problems in the events subsystem that needed
addressing before moving forward.

The biggest change was to move to the more standard terminology of
publish and subscribe. We make this terminology change across the Go
interface and the GRPC API, making the behavior more familier. The
previous system was very context-oriented, which is no longer required.

With this, we've removed a large amount of dead and unneeded code. Event
transactions, context storage and the concept of `Poster` is gone. This
has been replaced in most places with a `Publisher`, which matches the
actual usage throughout the codebase, removing the need for helpers.

There are still some questions around the way events are handled in the
shim. Right now, we've preserved some of the existing bugs which may
require more extensive changes to resolve correctly.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-07-25 15:08:09 -07:00
Kenfe-Mickael Laventure
3df07cbc88
Cancel io if task is deleted without being started
This prevent a deadlock on Windows since no process is actually running
until `Start()` is called.

Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-07-21 18:19:51 +02:00
Kenfe-Mickael Laventure
a4aaa09ccc
Update ctr so it works again on windows
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-07-21 18:19:48 +02:00
Stephen J Day
6d0bcd5aec
linux, linux/shim: remove error definitions
Since we now have a common set of error definitions, mapped to existing
error codes, we no longer need the specialized error codes used for
interaction with linux processes. The main issue was that string
matching was being used to map these to useful error codes. With this
change, we use errors defined in the `errdefs` package, which map
cleanly to GRPC error codes and are recoverable on either side of the
request.

The main focus of this PR was in removin these from the shim. We may
need follow ups to ensure error codes are preserved by the `Tasks`
service.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-07-18 15:56:49 -07:00
Stephen J Day
b385798695
containerd: remove GRPC service export
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-07-12 16:57:47 -07:00
Michael Crosby
3b8018d8cf Remove protos from Go client API
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-07-12 14:32:37 -07:00
Derek McGowan
1a49f5ea79 Merge pull request #1167 from crosbymichael/multi-ss
support using multiple snapshotters simultaneously
2017-07-12 13:34:31 -07:00
Phil Estes
052a8d03ed Merge pull request #1165 from crosbymichael/delete-hang
Fix process and task io Wait hang when fails to start
2017-07-12 14:58:05 -04:00
Akihiro Suda
b06aab713a support using multiple snapshotters simultaneously
e.g. dist pull --snapshotter btrfs ...; ctr run --snapshotter btrfs ...
(empty string defaults for overlayfs)

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-07-12 11:16:12 -07:00
Michael Crosby
2b6d790ff4 Refactor runtime events into Task* types
This removes the RuntimeEvent super proto with enums into separate
runtime event protos to be inline with the other events that are output
by containerd.

This also renames the runtime events into Task* events.

Fixes #1071

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-07-12 10:57:57 -07:00
Michael Crosby
cd4e8ba448 Add exec id check on client
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-07-12 10:41:04 -07:00
Michael Crosby
96b041e1f9 Close io when Start fails
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-07-12 10:37:39 -07:00
Derek McGowan
4322664b88
Update task service to use metadata content store
Address feedback and fix issues

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-07-11 11:11:10 -07:00
Michael Crosby
f93bfb6233 Add Exec IDs
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-07-06 15:23:08 -07:00
Michael Crosby
4b9a8ee13e Require *T for typeurl interaction
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-07-06 13:14:48 -07:00
Michael Crosby
c63b69672e Add typeurl.Is to gate unmarshal
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-07-05 16:22:17 -07:00
Michael Crosby
448dc0dfa8 Replace events/convert with typeurl
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-07-05 16:05:49 -07:00
Michael Crosby
82d0208aaa Implement options for runtime specific settings
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-06-29 15:32:45 -07:00
Michael Crosby
e2d5522435 Change ListProcesses to ListPids
These rpcs only return pids []uint32 so should be named that way in
order to have other rpcs that list Processes such as Exec'd processes.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-06-28 16:10:41 -07:00
Michael Crosby
7c8acca29a Move runtime interfaces to runtime package
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-06-28 10:10:59 -07:00
Michael Crosby
f36e0193a4 Implement task update
This allows tasks to have their resources updated as they are running.

Fixes #1067

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-06-26 16:38:49 -07:00
Michael Crosby
990536f2cc Move shim protos into linux pkg
This moves the shim's API and protos out of the containerd services
package and into the linux runtime package. This is because the shim is
an implementation detail of the linux runtime that we have and it is not
a containerd user facing api.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-06-23 16:21:47 -07:00
Michael Crosby
3b9d9dfa3e Fix error on doulbe Kill calls
This returns a typed error for calls to Kill when the process has
already finished.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-06-23 13:28:48 -07:00
Kenfe-Mickael Laventure
79086a1873
Simplify task.Wait() a bit
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-06-23 12:05:24 -07:00
Stephen J Day
c05be46348
events: move types into service package
When using events, it was found to be fairly unwieldy with a number of
extra packages. For the most part, when interacting with the events
service, we want types of the same version of the service. This has been
accomplished by moving all events types into the events package.

In addition, several fixes to the way events are marshaled have been
included. Specifically, we defer to the protobuf type registration
system to assemble events and type urls, with a little bit sheen on top
of add a containerd.io oriented namespace.

This has resulted in much cleaner event consumption and has removed the
reliance on error prone type urls, in favor of concrete types.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-06-22 19:12:25 -07:00
Stephen J Day
12a6beaeeb
*: update import paths to use versioned services
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-06-21 18:29:06 -07:00
Michael Crosby
235869eb1f Rename execution service to tasks
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-06-21 14:39:25 -07:00
Michael Crosby
8b2cf6e8e6 Fix Wait() on process/tasks
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-06-21 13:48:24 -07:00
Evan Hazlett
0b06fa8518 use event service for task wait
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-06-21 13:34:24 -07:00
Michael Crosby
94eafaab60 Update GRPC for consistency
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-06-21 13:34:24 -07:00