Commit Graph

2265 Commits

Author SHA1 Message Date
Brian Goff
6ab99edb71 Convert ExitStatus to use fn to get details
Instead of requiring callers to read the struct fields to check for an
error, provide the exit results via a function instead which is more
natural.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-08-22 10:48:44 -04:00
Brian Goff
026896ac4c Make Wait() async
In all of the examples, its recommended to call `Wait()` before starting
a process/task.
Since `Wait()` is a blocking call, this means it must be called from a
goroutine like so:

```go
statusC := make(chan uint32)
go func() {
  status, err := task.Wait(ctx)
  if err != nil {
    // handle async err
  }

  statusC <- status
}()

task.Start(ctx)
<-statusC
```

This means there is a race here where there is no guarentee when the
goroutine is going to be scheduled, and even a bit more since this
requires an RPC call to be made.
In addition, this code is very messy and a common pattern for any caller
using Wait+Start.

Instead, this changes `Wait()` to use an async model having `Wait()`
return a channel instead of the code itself.
This ensures that when `Wait()` returns that the client has a handle on
the event stream (already made the RPC request) before returning and
reduces any sort of race to how the stream is handled by grpc since we
can't guarentee that we have a goroutine running and blocked on
`Recv()`.

Making `Wait()` async also cleans up the code in the caller drastically:

```go
statusC, err := task.Wait(ctx)
if err != nil {
  return err
}

task.Start(ctx)

status := <-statusC
if status.Err != nil {
  return err
}
```

No more spinning up goroutines and more natural error
handling for the caller.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-08-22 09:33:07 -04:00
Kenfe-Mickaël Laventure
89da512692 Merge pull request #1377 from crosbymichael/exec-delete
Allow exec process to be deleted in created state
2017-08-21 15:27:17 -07:00
Derek McGowan
853fd15115 Merge pull request #1407 from crosbymichael/cgroups-bump
Update cgroups to e6d1aa8c71c6103624b2c6e6f4be0863
2017-08-21 13:47:54 -07:00
Michael Crosby
bf82de3a4e Allow exec process to be deleted in created state
Fixes #1376

This allows an exec process to be deleted in the created state.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-08-21 16:32:08 -04:00
Michael Crosby
77836a6c27 Update cgroups to e6d1aa8c71c6103624b2c6e6f4be0863
This bumps the cgroups package with various fixes to logging and net_cls
changes.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-08-21 16:17:18 -04:00
Michael Crosby
873a34649a Merge pull request #1366 from stevvooe/containers-metadata-tests
metadata: ensure correct updates on Container
2017-08-21 16:08:06 -04:00
Phil Estes
3e81b092de Merge pull request #1406 from mlaventure/add-exitedat-to-process
Add ExitedAt to process proto definition
2017-08-21 13:36:30 -04:00
Kenfe-Mickael Laventure
8a1b03e525
Add ExitedAt to process proto definition
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-08-21 08:18:02 -07:00
Michael Crosby
4c5f824167 Merge pull request #1404 from WeiZhang555/add-newline
Add newline char to avoid mix of logs
2017-08-21 11:12:42 -04:00
Zhang Wei
14fe2d5f20 Add newline char to avoid mix of logs
Add a '\n' after "unpacking xxx..." to avoid mix of logs such as:

```
unpacking sha256:a7776895af32e34b1fef997e26c79fa988b40c5cf2a3fb48dc22e0584b648d82...DEBU[0005]
Extraction not needed, layer snapshot exists
DEBU[0005] Extraction not needed, layer snapshot exists
DEBU[0005] Extraction not needed, layer snapshot exists
DEBU[0005] Extraction not needed, layer snapshot exists
DEBU[0005] Extraction not needed, layer snapshot exists
DEBU[0005] Extraction not needed, layer snapshot exists
done
```

After this commit:

```
unpacking sha256:a7776895af32e34b1fef997e26c79fa988b40c5cf2a3fb48dc22e0584b648d82...
DEBU[0008] Extraction not needed, layer snapshot exists
DEBU[0008] Extraction not needed, layer snapshot exists
DEBU[0008] Extraction not needed, layer snapshot exists
DEBU[0008] Extraction not needed, layer snapshot exists
DEBU[0008] Extraction not needed, layer snapshot exists
DEBU[0008] Extraction not needed, layer snapshot exists
done
```

Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
2017-08-20 21:03:18 +08:00
Stephen J Day
677257f032
service/snapshotter: move default to client
In order to enforce strict handling of snapshotter values on the
container object, the defaults have been moved to the client side. This
ensures that we correctly qualify the snapshotter under use when from
the container at the time it was created, rather than possibly losing
the metadata on a change of default.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-08-18 14:43:12 -07:00
Stephen J Day
783ed05057
metadata: ensure correct updates on Container
This fixes a few bugs in the container store related to reading and
writing fields. Specifically, on update, the full field set wasn't being
returned to the caller, making it appear that the store was corrupted.
We now return the correctly updated field and store the missing field
that was omitted in the original implementation. In course, we also have
defined the update semantics of each field, as well as whether or not
they are required.

The big addition here is really the container metadata testsuite. It
covers listing, filtering, creates, updates and deletes in a vareity of
scenarios.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-08-18 14:43:00 -07:00
Kenfe-Mickaël Laventure
8095244c26 Merge pull request #1396 from crosbymichael/reverts
Revert two changes to IO and cgroup handling
2017-08-18 13:51:02 -07:00
Michael Crosby
4950c26757 Revert "Wait for client side copy goroutines to start"
This reverts commit 06dc87ae59.

Revert "Change oom metric to const"

This reverts commit e800f08f9f.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-08-18 16:20:02 -04:00
Michael Crosby
e3696cfb4e Merge pull request #1395 from dmcgowan/content-testsuite-cleanup
Close writers in content test suite
2017-08-18 15:21:04 -04:00
Derek McGowan
42119539be
Close writers in content test suite
Ensure all writers are closed at end of test for content
test suite. Prevents test from leaving lingering connections
to the content store.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-08-18 12:06:38 -07:00
Kenfe-Mickaël Laventure
567c2758f8 Merge pull request #1391 from crosbymichael/stdin-block
Revert part of 06dc87ae59
2017-08-18 07:41:57 -07:00
Michael Crosby
cc3b7a5e9a Revert part of 06dc87ae59
Fixes #1389

This reverts waiting on stdin to finish its IO copy before returning
out.

This was causing `ctr` to block for every container start/exit.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-08-18 09:58:47 -04:00
Michael Crosby
0c10ce01c9 Merge pull request #1390 from AkihiroSuda/fix-typo
client: Resoveconf -> Resolvconf
2017-08-18 09:39:31 -04:00
Akihiro Suda
5f7c527ed5 client: Resoveconf -> Resolvconf
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-08-18 06:35:38 +00:00
Stephen Day
b2ee0ab34e Merge pull request #1387 from stevvooe/alpha5
release: prepare for 1.0.0-alpha5
2017-08-17 16:16:37 -07:00
Stephen J Day
a744550401
release: prepare for 1.0.0-alpha5
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-08-17 16:15:43 -07:00
Stephen Day
0fd951c2f7 Merge pull request #1379 from crosbymichael/profile
Change oom metric to const
2017-08-17 16:05:13 -07:00
Stephen Day
a29c6c7ab9 Merge pull request #1386 from Random-Liu/task-service-grpc-error
Convert errors returned by task service to errdefs error.
2017-08-17 16:04:12 -07:00
Lantao Liu
f79981c2df Convert errors returned by task service to errdefs error.
Signed-off-by: Lantao Liu <lantaol@google.com>
2017-08-17 22:52:16 +00:00
Stephen Day
0a18801468 Merge pull request #1382 from stevvooe/update-api-descriptors
api: update protobuf descriptors
2017-08-17 15:33:41 -07:00
Stephen Day
15f56727e0 Merge pull request #1369 from AkihiroSuda/ctr-snapshot-view
ctr: add `ctr snapshot view`
2017-08-17 15:32:13 -07:00
Stephen Day
c890ca9bce Merge pull request #1383 from stevvooe/document-config-function
images: document config utility function
2017-08-17 15:24:16 -07:00
Stephen J Day
2a1f7b647b
images: document config utility function
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-08-17 14:31:35 -07:00
Stephen J Day
8ae20a135a
api: update protobuf descriptors
Because we merged the PR that added the API descriptors and modified the
API at the same time. This updates the protobufs to be consistent.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-08-17 14:22:14 -07:00
Michael Crosby
e800f08f9f Change oom metric to const
This removes the metric vec that was holding onto all task id and
namespace combinations forever, until containerd was restarted.  This
was causing a memory leak with many task.

This also removes the shim cmd where the `Args` is quite large from the
reaper after the shim has been started cutting down on another leak.

This is the first pass through the reaper but more code is required to
fix all the issues when commands are added.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-08-17 16:23:20 -04:00
Phil Estes
ed1c40420e Merge pull request #1372 from jessvalarezo/d4m-gettingstarted
Supplement Building documentation for Docker for Mac users.
2017-08-17 13:45:28 -04:00
Jess Valarezo
d8abb61bed Update building doc for Docker for Mac users
Signed-off-by: Jess Valarezo <valarezo.jessica@gmail.com>
2017-08-17 10:21:03 -07:00
Phil Estes
89daacfe3f Merge pull request #1370 from crosbymichael/daemon-lock
Defer unlock for testing daemon
2017-08-17 11:44:44 -04:00
Phil Estes
240a7be607 Merge pull request #1371 from dmcgowan/btrfs-test-errors
Update mount creation to only use btrfs subvolume id
2017-08-17 11:36:25 -04:00
Michael Crosby
06dc87ae59 Wait for client side copy goroutines to start
Make sure we wait for the client side copy goroutines to start coping
from the fifos before returning from the function.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-08-17 10:58:56 -04:00
Michael Crosby
bfd62ceba9 Defer unlock for testing daemon
Make sure we use defer when unlocking the daemon during testing.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-08-16 16:29:43 -04:00
Michael Crosby
73ac66eecf Merge pull request #1352 from ijc/userns
correct container "/" ownership with userns
2017-08-16 16:09:44 -04:00
Derek McGowan
0e28f2c76e
Update mount creation to only use btrfs subvolume id
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-08-16 13:00:54 -07:00
Ian Campbell
ebafab47ca correct container "/" ownership with userns
Previously "`/`" in a container was always owned by `root:root` (0/0), even if
`withRemappedSnapshot` had been used. Meaning that if `withUserNamespace` is
used then `/` can be remapped to `nobody:nogroup` (65534/65534).

The fix is is twofold:

- incrementFS should operate on the root of the tree.
- when creating a new snapshot we must propagate the ownership of the topmost
  "lower" directory into the new "upper".

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-08-16 19:55:55 +01:00
Akihiro Suda
e03ecd30a7 ctr: add ctr snapshot view
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-08-16 08:40:11 +00:00
Derek McGowan
4703b99ac0 Merge pull request #1364 from AkihiroSuda/fix-ctr-snapshot-commit
ctr: fix `ctr snapshot commit`
2017-08-15 14:44:23 -07:00
Michael Crosby
62cebf9f1f Merge pull request #1365 from AkihiroSuda/stress
stress -> containerd-stress
2017-08-15 11:03:04 -04:00
Akihiro Suda
f8b1f4f6dc stress -> containerd-stress
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-08-15 14:54:20 +00:00
Akihiro Suda
ca352220b5 ctr: fix ctr snapshot commit
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-08-15 08:14:54 +00:00
Stephen Day
a64399acc2 Merge pull request #1341 from mlaventure/enable-test-parallelism
Enable test parallelism
2017-08-14 15:18:37 -07:00
Stephen Day
72159bf4a4 Merge pull request #1325 from dmcgowan/diffid-labels
content: commit options
2017-08-14 15:17:37 -07:00
Kenfe-Mickael Laventure
2abaf6e964
Fix possible deadlock in WithProcessKill
We were not checking the error value of `Kill` leading to deadlock if the
process didn't exist.

Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-08-14 14:43:43 -07:00
Kenfe-Mickael Laventure
b02e9a844e
Fix TestContainerNoBinaryExists on windows
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-08-14 14:43:43 -07:00