Commit Graph

192 Commits

Author SHA1 Message Date
Michael Crosby
b6e0c4f321 Fix go lint errors
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-10-25 15:26:44 -04:00
Michael Crosby
5fd0415985 Add comments and fix common lint issues
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-10-20 13:19:14 -04:00
Michael Crosby
9bd1dc78cb Unexport grpc service types
Since these are registered and the interface is what matters, these
Service types do not need to be exported.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-10-20 13:19:14 -04:00
Derek McGowan
ffb03c4fe2
Add checkpoint and userns gc labels
Prevent checkpoints from getting garbage collected by
adding root labels to unreferenced checkpoint objects.
Mark checkpoints as gc roots.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-10-11 11:26:46 -07:00
Derek McGowan
de7b281856
Update services to call garbage collection
Call garbage collection on container and image deletion.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-10-11 10:42:47 -07:00
Derek McGowan
d9db1d112d
Refactor differ into separate package
Add differ options and package with interface.
Update optional values on diff interface to use options.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-10-11 10:02:29 -07:00
Stephen J Day
0e72ce5c5b
services/introspection: implement plugin reporting
With this change, we integrate all the plugin changes into the
introspection service.

All plugins can be listed with the following command:

```console
$ ctr plugins
TYPE                            ID             PLATFORM       STATUS
io.containerd.content.v1        content        -              ok
io.containerd.metadata.v1       bolt           -              ok
io.containerd.differ.v1         walking        linux/amd64    ok
io.containerd.grpc.v1           containers     -              ok
io.containerd.grpc.v1           content        -              ok
io.containerd.grpc.v1           diff           -              ok
io.containerd.grpc.v1           events         -              ok
io.containerd.grpc.v1           healthcheck    -              ok
io.containerd.grpc.v1           images         -              ok
io.containerd.grpc.v1           namespaces     -              ok
io.containerd.snapshotter.v1    btrfs          linux/amd64    error
io.containerd.snapshotter.v1    overlayfs      linux/amd64    ok
io.containerd.grpc.v1           snapshots      -              ok
io.containerd.monitor.v1        cgroups        linux/amd64    ok
io.containerd.runtime.v1        linux          linux/amd64    ok
io.containerd.grpc.v1           tasks          -              ok
io.containerd.grpc.v1           version        -              ok
```

There are few things to note about this output. The first is that it is
printed in the order in which plugins are initialized. This useful for
debugging plugin initialization problems. Also note that even though the
introspection GPRC api is a itself a plugin, it is not listed. This is
because the plugin takes a snapshot of the initialization state at the
end of the plugin init process. This allows us to see errors from each
plugin, as they happen. If it is required to introspect the existence of
the introspection service, we can make modifications to include it in
the future.

The last thing to note is that the btrfs plugin is in an error state.
This is a common state for containerd because even though we load the
plugin, most installations aren't on top of btrfs and the plugin cannot
be used. We can actually view this error using the detailed view with a
filter:

```console
$ ctr plugins --detailed id==btrfs
Type:          io.containerd.snapshotter.v1
ID:            btrfs
Platforms:     linux/amd64
Exports:
               root      /var/lib/containerd/io.containerd.snapshotter.v1.btrfs
Error:
               Code:        Unknown
               Message:     path /var/lib/containerd/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter
```

Along with several other values, this is a valuable tool for evaluating the
state of components in containerd.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-10-10 17:29:24 -07:00
Stephen J Day
8508e8252b
plugin: refactor plugin system to support initialization reporting
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-10-10 16:40:47 -07:00
Jess
061c719209 ListPids returns process ID and other info
Signed-off-by: Jess <jessica.valarezo@docker.com>
2017-10-10 22:57:15 +00:00
Phil Estes
987fcd1201 Merge pull request #1598 from Random-Liu/fix-load-task
Fix task load.
2017-10-06 16:38:40 -04:00
Lantao Liu
28ca8f05d3 Fix task load.
Signed-off-by: Lantao Liu <lantaol@google.com>
2017-10-05 21:03:24 +00:00
Derek McGowan
8d892a651b
Update metadata plugin initialization
Updates metadata plugin to require content and
snapshotter plugins be loaded and initializes with
those plugins, keeping the metadata database structure
static after initialization. Service plugins now only
require metadata plugin access snapshotter or content
stores through metadata, which was already required
behavior of the services.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-10-04 11:25:46 -07:00
Derek McGowan
56c1b79a4c
Create metadata db object
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-10-04 11:16:22 -07:00
Michael Crosby
451421b615 Comment more packages to pass go lint
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-10-02 13:54:56 -04:00
Stephen Day
df896c92e7 Merge pull request #1528 from jessvalarezo/labels-validate
Labels are consistently validated across services.
2017-09-22 10:33:17 -07:00
Lantao Liu
54521126aa Fix container CreatedAt and UpdatedAt.
Signed-off-by: Lantao Liu <lantaol@google.com>
2017-09-22 05:57:02 +00:00
Jess Valarezo
18c4322bb3 Labels are consistently validated across services
* The combined size of a key/value pair cannot exceed 4096 bytes

Signed-off-by: Jess Valarezo <valarezo.jessica@gmail.com>
2017-09-21 15:11:46 -07:00
Michael Crosby
d67763d922 Add wait API endpoint for waiting on process exit
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-09-21 15:03:58 -04:00
Michael Crosby
d22160c28e Vendor typeurl package
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-09-19 09:43:55 -04:00
Kenfe-Mickaël Laventure
e517952bc7 Merge pull request #1378 from cpuguy83/container_store_extra_data
Add `Container` field to store client-defined data
2017-09-18 07:32:07 -07:00
Michael Crosby
72f380af45 Merge pull request #1517 from mlaventure/remove-toml-omitempty
Remove omitempty from toml tags
2017-09-15 11:30:01 -04:00
Kenfe-Mickael Laventure
f2d1459929
Remove omitempty from toml tags
The encoder only support changing the name of the key.

Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-09-14 20:49:22 -07:00
Michael Crosby
14c36cca60 Set ctx ns before publish
Fixes #1497

This sets the namespace on the context when deleting a namespace so that
the publish event does not fail.  Use the same namespace as you are
deleting for the context.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-09-14 13:54:57 -04:00
Derek McGowan
8785f3f002 Merge pull request #1403 from stevvooe/platform-selectors
platforms: define selectors for platforms
2017-09-13 11:50:11 -07:00
Stephen J Day
94f6be5f10
platforms: implement matcher support
Matching support is now implemented in the platforms package. The
`Parse` function now returns a matcher object that can be used to
match OCI platform specifications. We define this as an interface to
allow the creation of helpers oriented around platform selection.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-09-08 19:22:29 -07:00
Brian Goff
3552ce5688 Add field to Container for client-defined data
This field allows a client to store specialized information in the
container metadata rather than having to store this itself and keep
the data in sync with containerd.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-09-07 16:54:47 -04:00
Michael Crosby
951c129bf1 Handle locking and errors for process state
ref: #1464

This tries to solve issues with races around process state.  First it
adds the process mutex around the state call so that any state changes,
deletions, etc will be handled in order.

Second, for IsNoExist errors from the runtime, return a stopped state if
a process has been removed from the underlying OCI runtime but not from
the shim yet.  This shouldn't happen with the lock from above but its
hare to verify this issue.

Third, handle shim disconnections and return an ErrNotFound.

Forth, don't abort returning all tasks if one task is unable to return
its state.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-09-07 16:22:00 -04:00
Akihiro Suda
e30e0c8b75
api: RootFS -> SnapshotKey
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-09-06 15:22:19 -07:00
Kenfe-Mickaël Laventure
e66d1a7961 Merge pull request #1468 from crosbymichael/stats
Add Metrics endpoint to tasks
2017-09-06 14:59:55 -07:00
Kenfe-Mickaël Laventure
e1eeb0e0a2 Merge pull request #1475 from dmcgowan/content-commit-context
Add context to content commit
2017-09-06 11:04:31 -07:00
Derek McGowan
9613acb2ed
Add context to content commit
Content commit is updated to take in a context, allowing
content to be committed within the same context the writer
was in. This is useful when commit may be able to use more
context to complete the action rather than creating its own.
An example of this being useful is for the metadata implementation
of content, having a context allows tests to fully create
content in one database transaction by making use of the context.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-09-06 10:19:12 -07:00
Michael Crosby
4c5ed9c068 Move metrics requests to services
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-09-05 17:41:30 -04:00
Michael Crosby
8510512e7e Add test for client metrics
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-09-05 17:26:26 -04:00
Michael Crosby
697dcdd407 Refactor task service metrics
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-09-05 17:26:26 -04:00
Michael Crosby
f5d81a631e Return grpc errs from task service
Closes #1201

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-09-05 16:10:19 -04:00
Stephen J Day
5cc108605f
service/containers: correctly plumb fieldpaths
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-09-01 13:17:31 -07:00
Kenfe-Mickaël Laventure
0daa593b3a Merge pull request #1393 from dmcgowan/multiple-differs
Add support for multiple differs
2017-08-23 14:45:17 -07:00
Michael Crosby
873a34649a Merge pull request #1366 from stevvooe/containers-metadata-tests
metadata: ensure correct updates on Container
2017-08-21 16:08:06 -04:00
Kenfe-Mickael Laventure
8a1b03e525
Add ExitedAt to process proto definition
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-08-21 08:18:02 -07:00
Stephen J Day
677257f032
service/snapshotter: move default to client
In order to enforce strict handling of snapshotter values on the
container object, the defaults have been moved to the client side. This
ensures that we correctly qualify the snapshotter under use when from
the container at the time it was created, rather than possibly losing
the metadata on a change of default.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-08-18 14:43:12 -07:00
Derek McGowan
2fa366d6af
Add support for multiple differs
Updates the differ service to support calling and configuring
multiple differs. The differs are configured as an ordered list
of differs which will each be attempting until a supported differ
is called.

Additionally a not supported error type was added to allow differs
to be selective of whether the differ arguments are supported by
the differ. This error type corresponds to the GRPC unimplemented error.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-08-18 11:19:08 -07:00
Derek McGowan
dee8dc2cda
Add support for content labels on commit
Add commit options which allow for setting labels on commit.
Prevents potential race between garbage collector reading labels
after commit and labels getting set.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-08-11 14:15:20 -07:00
Derek McGowan
c4387a159e
Add content test suite run to client
Fix bugs in content deletion and upload status

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-08-11 14:08:34 -07:00
Stephen Day
e6ba48f25c Merge pull request #1257 from stevvooe/content-changes
content: remove Provider.Reader
2017-08-09 17:12:44 -07:00
Stephen J Day
8be340e37b
content: remove Provider.Reader
After some analysis, it was found that Content.Reader was generally
redudant to an io.ReaderAt. This change removes `Content.Reader` in
favor of a `Content.ReaderAt`. In general, `ReaderAt` can perform better
over interfaces with indeterminant latency because it avoids remote
state for reads. Where a reader is required, a helper is provided to
convert it into an `io.SectionReader`.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-08-09 14:32:28 -07:00
Michael Crosby
29a4dd7f46 Merge pull request #1276 from dmcgowan/snapshot-labels
Snapshot labels
2017-08-08 21:59:28 -04:00
Michael Crosby
2fb2168bae Merge pull request #1224 from epilatow/solaris.cc.20170719.0
Add solaris build support
2017-08-07 16:15:07 -04:00
Edward Pilatowicz
56c1f5c184 Add solaris build support
Signed-off-by: Edward Pilatowicz <edward.pilatowicz@oracle.com>
2017-08-07 12:40:21 -07:00
Michael Crosby
9f13b414b9 Return exit status from Wait of stopped process
This changes Wait() from returning an error whenever you call wait on a
stopped process/task to returning the exit status from the process.

This also adds the exit status to the Status() call on a process/task so
that a user can Wait(), check status, then cancel the wait to avoid
races in event handling.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-08-03 17:22:33 -04:00
Derek McGowan
efca0c5347
Update GRPC api to support labels and snapshot updates
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-08-02 16:28:41 -07:00