Commit Graph

100 Commits

Author SHA1 Message Date
Derek McGowan
dce27d8c62
Remove client use of gc root label
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-11-07 12:54:22 -08:00
Derek McGowan
e13894bb7a
Add leases api
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-11-07 12:54:22 -08:00
Michael Crosby
5fd0415985 Add comments and fix common lint issues
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-10-20 13:19:14 -04:00
Michael Crosby
21c992a9c8 Reverse checkpoint url
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-10-18 14:58:28 -04:00
Michael Crosby
e833da1356 Clear root labels during checkpoint
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-10-17 16:53:44 -04:00
Michael Crosby
e201be5196 Create checkpointed image in client
Allow a user provided name for the checkpoint as well as a default
generated name for the checkpoint image.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-10-17 15:12:04 -04:00
Brian Goff
205625473f Close wait chan after sending status
This allows the caller to receive multiple times without blocking after
the first call.

This can be useful in cases like this:
```go
  ch, _  := task.Wait(ctx)
  defer func() { <-ch } // don't return until task is done
  task.Start(ctx)

  if err := doSomething(task); err != nil {
    return err
  }

  status := <- ch
  // do stuff with status
```

Since this channel is created in the `Wait()` calls and never accessible
outside, this should be safe.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-10-16 16:55:43 -04:00
Stephen Day
587f25245a Merge pull request #1563 from dmcgowan/gc-alpha
Metadata garbage collection
2017-10-11 14:31:44 -07:00
Jess Valarezo
1e69de18bd Remove ProcessInfo cast
Signed-off-by: Jess Valarezo <valarezo.jessica@gmail.com>
2017-10-11 13:01:15 -07:00
Derek McGowan
ffb03c4fe2
Add checkpoint and userns gc labels
Prevent checkpoints from getting garbage collected by
adding root labels to unreferenced checkpoint objects.
Mark checkpoints as gc roots.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-10-11 11:26:46 -07:00
Derek McGowan
d9db1d112d
Refactor differ into separate package
Add differ options and package with interface.
Update optional values on diff interface to use options.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-10-11 10:02:29 -07:00
Jess
061c719209 ListPids returns process ID and other info
Signed-off-by: Jess <jessica.valarezo@docker.com>
2017-10-10 22:57:15 +00:00
Phil Estes
cc9f58ac24 Merge pull request #1559 from mlaventure/fix-oor-panic
client: Prevent Out-Of-Range panic in task.Metrics()
2017-09-26 14:39:39 -04:00
Kenfe-Mickael Laventure
af2226c938
client: Prevent Out-Of-Range panic in task.Metrics()
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-09-25 15:46:01 -07:00
Michael Crosby
51b9240b80 Update client to pass go lint
There is one breaking change with the function naming of UidGid to
UIDGID

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-09-25 13:11:42 -04:00
Michael Crosby
d67763d922 Add wait API endpoint for waiting on process exit
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-09-21 15:03:58 -04:00
Michael Crosby
d22160c28e Vendor typeurl package
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-09-19 09:43:55 -04:00
Akihiro Suda
e30e0c8b75
api: RootFS -> SnapshotKey
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-09-06 15:22:19 -07:00
Kenfe-Mickaël Laventure
e66d1a7961 Merge pull request #1468 from crosbymichael/stats
Add Metrics endpoint to tasks
2017-09-06 14:59:55 -07:00
Michael Crosby
2ed3c62e27 Update cgroups to 5933ab4dc4f7caa3a73a1dc141bd11f4
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-09-06 16:20:19 -04:00
Kenfe-Mickaël Laventure
e1eeb0e0a2 Merge pull request #1475 from dmcgowan/content-commit-context
Add context to content commit
2017-09-06 11:04:31 -07:00
Derek McGowan
9613acb2ed
Add context to content commit
Content commit is updated to take in a context, allowing
content to be committed within the same context the writer
was in. This is useful when commit may be able to use more
context to complete the action rather than creating its own.
An example of this being useful is for the metadata implementation
of content, having a context allows tests to fully create
content in one database transaction by making use of the context.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-09-06 10:19:12 -07:00
Michael Crosby
4c5ed9c068 Move metrics requests to services
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-09-05 17:41:30 -04:00
Michael Crosby
8510512e7e Add test for client metrics
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-09-05 17:26:26 -04:00
Michael Crosby
ed6b8fb0aa Add KillOpts for killing all processes
Fixes #1431

This adds KillOpts so that a client can specify when they want to kill a
single process or all the processes inside a container.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-08-28 13:29:47 -04:00
Michael Crosby
b3303b55c1 Add LoadProcess api to Task
Fixes #1374

This adds a `LoadProcess` api to load existing exec'd processes from a
task.  It allows reattaching of IO and waiting on the process.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-08-24 16:30:34 -04:00
Kenfe-Mickael Laventure
7f6c487031
go-client: Return an ExitStatus struct when calling process.Delete()
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-08-23 10:32:51 -07:00
Brian Goff
6ab99edb71 Convert ExitStatus to use fn to get details
Instead of requiring callers to read the struct fields to check for an
error, provide the exit results via a function instead which is more
natural.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-08-22 10:48:44 -04:00
Brian Goff
026896ac4c Make Wait() async
In all of the examples, its recommended to call `Wait()` before starting
a process/task.
Since `Wait()` is a blocking call, this means it must be called from a
goroutine like so:

```go
statusC := make(chan uint32)
go func() {
  status, err := task.Wait(ctx)
  if err != nil {
    // handle async err
  }

  statusC <- status
}()

task.Start(ctx)
<-statusC
```

This means there is a race here where there is no guarentee when the
goroutine is going to be scheduled, and even a bit more since this
requires an RPC call to be made.
In addition, this code is very messy and a common pattern for any caller
using Wait+Start.

Instead, this changes `Wait()` to use an async model having `Wait()`
return a channel instead of the code itself.
This ensures that when `Wait()` returns that the client has a handle on
the event stream (already made the RPC request) before returning and
reduces any sort of race to how the stream is handled by grpc since we
can't guarentee that we have a goroutine running and blocked on
`Recv()`.

Making `Wait()` async also cleans up the code in the caller drastically:

```go
statusC, err := task.Wait(ctx)
if err != nil {
  return err
}

task.Start(ctx)

status := <-statusC
if status.Err != nil {
  return err
}
```

No more spinning up goroutines and more natural error
handling for the caller.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-08-22 09:33:07 -04:00
Kenfe-Mickael Laventure
8a1b03e525
Add ExitedAt to process proto definition
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-08-21 08:18:02 -07:00
Stephen Day
a29c6c7ab9 Merge pull request #1386 from Random-Liu/task-service-grpc-error
Convert errors returned by task service to errdefs error.
2017-08-17 16:04:12 -07:00
Lantao Liu
f79981c2df Convert errors returned by task service to errdefs error.
Signed-off-by: Lantao Liu <lantaol@google.com>
2017-08-17 22:52:16 +00:00
Stephen Day
a64399acc2 Merge pull request #1341 from mlaventure/enable-test-parallelism
Enable test parallelism
2017-08-14 15:18:37 -07:00
Kenfe-Mickael Laventure
b02e9a844e
Fix TestContainerNoBinaryExists on windows
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-08-14 14:43:43 -07:00
Lantao Liu
8e2c95f9c1 Change IO to interface.
Signed-off-by: Lantao Liu <lantaol@google.com>
2017-08-14 18:53:12 +00:00
Kenfe-Mickael Laventure
5f36ac2093
Add test to ensure we can access tasks on restart
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-08-11 09:34:29 -07:00
Michael Crosby
d8c075aadc Add WithProcessKill DeleteOpt
Add an option that allows users for force kill and delete a process/task
when calling `Delete`

Fixes #1274

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-08-08 10:37:17 -04:00
Kenfe-Mickael Laventure
56b18c1d1f
Move client's options to separate files
This should help in discovering the available options for a given object.

Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-08-04 13:56:16 -07:00
Derek McGowan
99315f8e02 Merge pull request #1285 from crosbymichael/client-godoc
Update godoc for client package
2017-08-04 10:38:53 -07:00
Lantao Liu
8fae8a5b77 Cancel event subscribe context in Wait.
Signed-off-by: Lantao Liu <lantaol@google.com>
2017-08-04 16:46:53 +00:00
Michael Crosby
0a85f6e47d Update godoc for client package
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-08-04 11:53:31 -04:00
Michael Crosby
fdc5a475a8 Gate task.Wait status check on checkpoints
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-08-03 18:02:48 -04:00
Michael Crosby
9f13b414b9 Return exit status from Wait of stopped process
This changes Wait() from returning an error whenever you call wait on a
stopped process/task to returning the exit status from the process.

This also adds the exit status to the Status() call on a process/task so
that a user can Wait(), check status, then cancel the wait to avoid
races in event handling.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-08-03 17:22:33 -04:00
Michael Crosby
83b27db923 Check status for stopped processes before wait
Fixes #1054

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-08-02 13:50:08 -04:00
Michael Crosby
cd00216fd7 Rename TaskStatus to Status
This can be renamed because both Tasks and Processes share the same
status consts

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-08-02 13:50:08 -04:00
Michael Crosby
504033e373 Add Get of task and process state
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-08-02 13:50:08 -04:00
Michael Crosby
a2a3451925 Implement Exec + Start for tasks service
This splits up the exec creation and start in the tasks service

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-08-02 13:50:08 -04:00
Stephen J Day
af2d7f0e55
events: initial support for filters
This change further plumbs the components required for implementing
event filters. Specifically, we now have the ability to filter on the
`topic` and `namespace`.

In the course of implementing this functionality, it was found that
there were mismatches in the events API that created extra serialization
round trips. A modification to `typeurl.MarshalAny` and a clear
separation between publishing and forwarding allow us to avoid these
serialization issues.

Unfortunately, this has required a few tweaks to the GRPC API, so this
is a breaking change. `Publish` and `Forward` have been clearly separated in
the GRPC API. `Publish` honors the contextual namespace and performs
timestamping while `Forward` simply validates and forwards. The behavior
of `Subscribe` is to propagate events for all namespaces unless
specifically filtered (and hence the relation to this particular change.

The following is an example of using filters to monitor the task events
generated while running the [bucketbench tool](https://github.com/estesp/bucketbench):

```
$ ctr events 'topic~=/tasks/.+,namespace==bb'
...
2017-07-28 22:19:51.78944874 +0000 UTC   bb        /tasks/start   {"container_id":"bb-ctr-6-8","pid":25889}
2017-07-28 22:19:51.791893688 +0000 UTC   bb        /tasks/start   {"container_id":"bb-ctr-4-8","pid":25882}
2017-07-28 22:19:51.792608389 +0000 UTC   bb        /tasks/start   {"container_id":"bb-ctr-2-9","pid":25860}
2017-07-28 22:19:51.793035217 +0000 UTC   bb        /tasks/start   {"container_id":"bb-ctr-5-6","pid":25869}
2017-07-28 22:19:51.802659622 +0000 UTC   bb        /tasks/start   {"container_id":"bb-ctr-0-7","pid":25877}
2017-07-28 22:19:51.805192898 +0000 UTC   bb        /tasks/start   {"container_id":"bb-ctr-3-6","pid":25856}
2017-07-28 22:19:51.832374931 +0000 UTC   bb        /tasks/exit   {"container_id":"bb-ctr-8-6","id":"bb-ctr-8-6","pid":25864,"exited_at":"2017-07-28T22:19:51.832013043Z"}
2017-07-28 22:19:51.84001249 +0000 UTC   bb        /tasks/exit   {"container_id":"bb-ctr-2-9","id":"bb-ctr-2-9","pid":25860,"exited_at":"2017-07-28T22:19:51.839717714Z"}
2017-07-28 22:19:51.840272635 +0000 UTC   bb        /tasks/exit   {"container_id":"bb-ctr-7-6","id":"bb-ctr-7-6","pid":25855,"exited_at":"2017-07-28T22:19:51.839796335Z"}
...
```

In addition to the events changes, we now display the namespace origin
of the event in the cli tool.

This will be followed by a PR to add individual field filtering for the
events API for each event type.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-07-31 12:53:18 -07:00
Michael Crosby
b84817a29c Merge pull request #1246 from stevvooe/events-refactor
events: refactor event distribution
2017-07-26 08:31:12 -04:00
Derek McGowan
c7cc6ad2b4
Add root mounts to task info
Add WithRootFS for task creation

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-07-25 15:24:06 -07:00