Commit Graph

375 Commits

Author SHA1 Message Date
Derek McGowan
b6d58f63a8
Support for ingest namespacing
Move content status to list statuses and add single status
to interface.
Updates API to support list statuses and status
Updates snapshot key creation to be generic

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-07-11 11:11:10 -07:00
Derek McGowan
106c7504f4 Merge pull request #1151 from stevvooe/image-metadata-service
images, containers: converge metadata API conventions
2017-07-11 11:05:01 -07:00
Stephen J Day
7f4c4aecf7
images, containers: converge metadata API conventions
The primary feature we get with this PR is support for filters and
labels on the image metadata store. In the process of doing this, the
conventions for the API have been converged between containers and
images, providing a model for other services.

With images, `Put` (renamed to `Update` briefly) has been split into a
`Create` and `Update`, allowing one to control the behavior around these
operations. `Update` now includes support for masking fields at the
datastore-level across both the containers and image service. Filters
are now just string values to interpreted directly within the data
store. This should allow for some interesting future use cases in which
the datastore might use the syntax for more efficient query paths.

The containers service has been updated to follow these conventions as
closely as possible.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-07-11 10:45:12 -07:00
Michael Crosby
f39693eabe Add Post to events service
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-07-07 14:23:10 -07:00
Michael Crosby
f93bfb6233 Add Exec IDs
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-07-06 15:23:08 -07:00
Michael Crosby
a60511d5aa Use typeurl package for spec types
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-07-05 15:37:26 -07:00
Zachary Adam Kaplan
a28a871d44 fixes typo in comment
Signed-off-by: Zachary Adam Kaplan <razic@viralkitty.com>
2017-07-03 17:48:14 +00:00
Stephen J Day
396d89e423
cmd/ctr, service/containers: implement container filter
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-06-30 11:49:16 -07:00
Michael Crosby
96dbb08ec4 Change runtime options to *Any
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-06-29 15:35:13 -07:00
Michael Crosby
82d0208aaa Implement options for runtime specific settings
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-06-29 15:32:45 -07:00
Michael Crosby
e2d5522435 Change ListProcesses to ListPids
These rpcs only return pids []uint32 so should be named that way in
order to have other rpcs that list Processes such as Exec'd processes.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-06-28 16:10:41 -07:00
Michael Crosby
040558cf81 Remove runtime.Event types
This uses the events service types for runtime events

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-06-28 10:47:22 -07:00
Michael Crosby
f36e0193a4 Implement task update
This allows tasks to have their resources updated as they are running.

Fixes #1067

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-06-26 16:38:49 -07:00
Michael Crosby
990536f2cc Move shim protos into linux pkg
This moves the shim's API and protos out of the containerd services
package and into the linux runtime package. This is because the shim is
an implementation detail of the linux runtime that we have and it is not
a containerd user facing api.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-06-23 16:21:47 -07:00
Stephen J Day
12c0daa9c9
api/types: consolidate types package
To simplify use of types, we have consolidate the packages for the mount
and descriptor protobuf types into a single Go package. We also drop the
versioning from the type packages, as these types will remain the same
between versions.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-06-23 13:50:28 -07:00
Stephen J Day
c05be46348
events: move types into service package
When using events, it was found to be fairly unwieldy with a number of
extra packages. For the most part, when interacting with the events
service, we want types of the same version of the service. This has been
accomplished by moving all events types into the events package.

In addition, several fixes to the way events are marshaled have been
included. Specifically, we defer to the protobuf type registration
system to assemble events and type urls, with a little bit sheen on top
of add a containerd.io oriented namespace.

This has resulted in much cleaner event consumption and has removed the
reliance on error prone type urls, in favor of concrete types.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-06-22 19:12:25 -07:00
Stephen J Day
782d78346d
api/services: update go packages and regenerate
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-06-21 18:22:32 -07:00
Stephen J Day
4784c3962a
api/services: move services into version directories
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-06-21 18:17:27 -07:00
Michael Crosby
235869eb1f Rename execution service to tasks
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-06-21 14:39:25 -07:00
Michael Crosby
8b2cf6e8e6 Fix Wait() on process/tasks
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-06-21 13:48:24 -07:00
Evan Hazlett
0b06fa8518 use event service for task wait
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-06-21 13:34:24 -07:00
Michael Crosby
94eafaab60 Update GRPC for consistency
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-06-21 13:34:24 -07:00
Evan Hazlett
935645b03a events: add protos
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>

events: update events package to include emitter and use envelope proto

Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>

events: add events service

Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>

events: enable events service and update ctr events to use events service

Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>

event listeners

Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>

events: helper func for emitting in services

Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>

events: improved cli for containers and tasks

Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>

create event envelope with poster

Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>

events: introspect event data to use for type url

Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>

events: use pb encoding; add event types

Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>

events: instrument content and snapshot services with events

Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>

events: instrument image service with events

Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>

events: instrument namespace service with events

Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>

events: add namespace support

Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>

events: only send events from namespace requested from client

Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>

events: switch to go-events for broadcasting

Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
2017-06-20 10:47:28 -04:00
Michael Crosby
ff598449d1 Add DeleteProcess API for removing execs
We need a separate API for handing the exit status and deletion of
Exec'd processes to make sure they are properly cleaned up within the
shim and daemon.

Fixes #973

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-06-12 09:32:23 -07:00
Stephen J Day
af2718b01f
namespaces: support within containerd
To support multi-tenancy, containerd allows the collection of metadata
and runtime objects within a heirarchical storage primitive known as
namespaces. Data cannot be shared across these namespaces, unless
allowed by the service. This allows multiple sets of containers to
managed without interaction between the clients that management. This
means that different users, such as SwarmKit, K8s, Docker and others can
use containerd without coordination. Through labels, one may use
namespaces as a tool for cleanly organizing the use of containerd
containers, including the metadata storage for higher level features,
such as ACLs.

Namespaces

Namespaces cross-cut all containerd operations and are communicated via
context, either within the Go context or via GRPC headers. As a general
rule, no features are tied to namespace, other than organization. This
will be maintained into the future. They are created as a side-effect of
operating on them or may be created manually. Namespaces can be labeled
for organization. They cannot be deleted unless the namespace is empty,
although we may want to make it so one can clean up the entirety of
containerd by deleting a namespace.

Most users will interface with namespaces by setting in the
context or via the `CONTAINERD_NAMESPACE` environment variable, but the
experience is mostly left to the client. For `ctr` and `dist`, we have
defined a "default" namespace that will be created up on use, but there
is nothing special about it. As part of this PR we have plumbed this
behavior through all commands, cleaning up context management along the
way.

Namespaces in Action

Namespaces can be managed with the `ctr namespaces` subcommand. They
can be created, labeled and destroyed.

A few commands can demonstrate the power of namespaces for use with
images. First, lets create a namespace:

```
$ ctr namespaces create foo mylabel=bar
$ ctr namespaces ls
NAME LABELS
foo  mylabel=bar
```

We can see that we have a namespace `foo` and it has a label. Let's pull
an image:

```
$ dist pull docker.io/library/redis:latest
docker.io/library/redis:latest: resolved       |++++++++++++++++++++++++++++++++++++++|
manifest-sha256:548a75066f3f280eb017a6ccda34c561ccf4f25459ef8e36d6ea582b6af1decf: done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:d45bc46b48e45e8c72c41aedd2a173bcc7f1ea4084a8fcfc5251b1da2a09c0b6: done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:5b690bc4eaa6434456ceaccf9b3e42229bd2691869ba439e515b28fe1a66c009: done           |++++++++++++++++++++++++++++++++++++++|
config-sha256:a858478874d144f6bfc03ae2d4598e2942fc9994159f2872e39fae88d45bd847: done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:4cdd94354d2a873333a205a02dbb853dd763c73600e0cf64f60b4bd7ab694875: done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:10a267c67f423630f3afe5e04bbbc93d578861ddcc54283526222f3ad5e895b9: done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:c54584150374aa94b9f7c3fbd743adcff5adead7a3cf7207b0e51551ac4a5517: done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:d1f9221193a65eaf1b0afc4f1d4fbb7f0f209369d2696e1c07671668e150ed2b: done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:71c1f30d820f0457df186531dc4478967d075ba449bd3168a3e82137a47daf03: done           |++++++++++++++++++++++++++++++++++++++|
elapsed: 0.9 s total:   0.0 B (0.0 B/s)
INFO[0000] unpacking rootfs
INFO[0000] Unpacked chain id: sha256:41719840acf0f89e761f4a97c6074b6e2c6c25e3830fcb39301496b5d36f9b51
```

Now, let's list the image:

```
$ dist images ls
REF                            TYPE  DIGEST SIZE
docker.io/library/redis:latest application/vnd.docker.distribution.manifest.v2+json sha256:548a75066f3f280eb017a6ccda34c561ccf4f25459ef8e36d6ea582b6af1decf 72.7 MiB
```

That looks normal. Let's list the images for the `foo` namespace and see
this in action:

```
$ CONTAINERD_NAMESPACE=foo dist images ls
REF TYPE DIGEST SIZE
```

Look at that! Nothing was pulled in the namespace `foo`. Let's do the
same pull:

```
$ CONTAINERD_NAMESPACE=foo dist pull docker.io/library/redis:latest
docker.io/library/redis:latest: resolved       |++++++++++++++++++++++++++++++++++++++|
manifest-sha256:548a75066f3f280eb017a6ccda34c561ccf4f25459ef8e36d6ea582b6af1decf: done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:d45bc46b48e45e8c72c41aedd2a173bcc7f1ea4084a8fcfc5251b1da2a09c0b6: done           |++++++++++++++++++++++++++++++++++++++|
config-sha256:a858478874d144f6bfc03ae2d4598e2942fc9994159f2872e39fae88d45bd847: done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:4cdd94354d2a873333a205a02dbb853dd763c73600e0cf64f60b4bd7ab694875: done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:c54584150374aa94b9f7c3fbd743adcff5adead7a3cf7207b0e51551ac4a5517: done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:71c1f30d820f0457df186531dc4478967d075ba449bd3168a3e82137a47daf03: done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:d1f9221193a65eaf1b0afc4f1d4fbb7f0f209369d2696e1c07671668e150ed2b: done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:10a267c67f423630f3afe5e04bbbc93d578861ddcc54283526222f3ad5e895b9: done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:5b690bc4eaa6434456ceaccf9b3e42229bd2691869ba439e515b28fe1a66c009: done           |++++++++++++++++++++++++++++++++++++++|
elapsed: 0.8 s total:   0.0 B (0.0 B/s)
INFO[0000] unpacking rootfs
INFO[0000] Unpacked chain id: sha256:41719840acf0f89e761f4a97c6074b6e2c6c25e3830fcb39301496b5d36f9b51
```

Wow, that was very snappy! Looks like we pulled that image into out
namespace but didn't have to download any new data because we are
sharing storage. Let's take a peak at the images we have in `foo`:

```
$ CONTAINERD_NAMESPACE=foo dist images ls
REF                            TYPE DIGEST SIZE
docker.io/library/redis:latest application/vnd.docker.distribution.manifest.v2+json sha256:548a75066f3f280eb017a6ccda34c561ccf4f25459ef8e36d6ea582b6af1decf 72.7 MiB
```

Now, let's remove that image from `foo`:

```
$ CONTAINERD_NAMESPACE=foo dist images rm
docker.io/library/redis:latest
```

Looks like it is gone:

```
$ CONTAINERD_NAMESPACE=foo dist images ls
REF TYPE DIGEST SIZE
```

But, as we can see, it is present in the `default` namespace:

```
$ dist images ls
REF                            TYPE DIGEST SIZE
docker.io/library/redis:latest application/vnd.docker.distribution.manifest.v2+json sha256:548a75066f3f280eb017a6ccda34c561ccf4f25459ef8e36d6ea582b6af1decf 72.7 MiB
```

What happened here? We can tell by listing the namespaces to get a
better understanding:

```
$ ctr namespaces ls
NAME    LABELS
default
foo     mylabel=bar
```

From the above, we can see that the `default` namespace was created with
the standard commands without the environment variable set. Isolating
the set of shared images while sharing the data that matters.

Since we removed the images for namespace `foo`, we can remove it now:

```
$ ctr namespaces rm foo
foo
```

However, when we try to remove the `default` namespace, we get an error:

```
$ ctr namespaces rm default
ctr: unable to delete default: rpc error: code = FailedPrecondition desc = namespace default must be empty
```

This is because we require that namespaces be empty when removed.

Caveats

- While most metadata objects are namespaced, containers and tasks may
exhibit some issues. We still need to move runtimes to namespaces and
the container metadata storage may not be fully worked out.
- Still need to migrate content store to metadata storage and namespace
the content store such that some data storage (ie images).
- Specifics of snapshot driver's relation to namespace needs to be
worked out in detail.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-06-06 13:50:33 -07:00
Stephen Day
f105db9626 Merge pull request #933 from kunalkushwaha/timestamp
Timestamp added to container object.
2017-06-05 12:00:33 -07:00
Kunal Kushwaha
0008ac7f3d Timestamp added to container object.
Fix for #912

Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
2017-06-05 14:30:49 +09:00
Michael Crosby
00734ab04a Return fifo paths from Shim
This allows attach of existing fifos to be done without any information
stored on the client side.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-06-01 14:12:02 -07:00
Michael Crosby
ee90a77f63 Rename Image to CheckpointPath in shim
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-05-26 10:06:53 -07:00
Stephen J Day
319b5c5ea2
api/services/namespaces: define the Namespaces service
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-05-25 17:39:41 -07:00
Stephen J Day
539742881d
api/services: define the container metadata service
Working from feedback on the existing implementation, we have now
introduced a central metadata object to represent the lifecycle and pin
the resources required to implement what people today know as
containers. This includes the runtime specification and the root
filesystem snapshots. We also allow arbitrary labeling of the container.
Such provisions will bring the containerd definition of container closer
to what is expected by users.

The objects that encompass today's ContainerService, centered around the
runtime, will be known as tasks. These tasks take on the existing
lifecycle behavior of containerd's containers, which means that they are
deleted when they exit. Largely, there are no other changes except for
naming.

The `Container` object will operate purely as a metadata object. No
runtime state will be held on `Container`. It only informs the execution
service on what is required for creating tasks and the resources in use
by that container. The resources referenced by that container will be
deleted when the container is deleted, if not in use. In this sense,
users can create, list, label and delete containers in a similar way as
they do with docker today, without the complexity of runtime locks that
plagues current implementations.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-05-22 23:27:53 -07:00
Michael Crosby
7cc1b64bd8 Add checkpoint and restore
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Update go-runc to 49b2a02ec1ed3e4ae52d30b54a291b75

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Add shim to restore creation

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Keep checkpoint path in service

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Add C/R to non-shim build

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Checkpoint rw and image

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Pause container on bind checkpoints

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Return dump.log in error on checkpoint failure

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Pause container for checkpoint

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Update runc to 639454475cb9c8b861cc599f8bcd5c8c790ae402

For checkpoint into to work you need runc version
639454475cb9c8b861cc599f8bcd5c8c790ae402 + and criu 3.0 as this is what
I have been testing with.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Move restore behind create calls

This remove the restore RPCs in favor of providing the checkpoint
information to the `Create` calls of a container.  If provided, the
container will be created/restored from the checkpoint instead of an
existing container.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Regen protos after rebase

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-05-22 15:34:45 -07:00
Phil Estes
5ee77fc281 Merge pull request #876 from stevvooe/nice-enum-values
api/types/container: nice enum values for status
2017-05-19 11:19:22 -04:00
Stephen J Day
cb68150a2f
protos: remove need for customname on ID fields
Credit to Aaron Lehmann for the implementation from SwarmKit.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-05-18 18:50:59 -07:00
Stephen J Day
178b4275ca
api/types/container: nice enum values for status
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-05-18 17:48:20 -07:00
Evan Hazlett
70dd7fcc60 shim: Ps -> Processes
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
2017-05-18 11:05:48 -04:00
Evan Hazlett
395d21018f PsRequest/PsResponse -> ProcessesRequest/ProcessesResponse; added Any to types.Process
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
2017-05-18 11:05:48 -04:00
Evan Hazlett
ef158f8b5e add support to kill container process by pid
This adds support for signalling a container process by pid.

Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>

make Ps more extensible

Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>

ps: windows support

Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
2017-05-18 11:05:48 -04:00
Derek McGowan
098ff94b24
Add snapshot and diff service
Remove rootfs service in place of snapshot service. Adds
diff service for extracting and creating diffs. Diff
creation is not yet implemented. This service allows
pulling or creating images without needing root access to
mount. Additionally in the future this will allow containerd
to ensure extractions happen safely in a chroot if needed.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-05-15 16:50:16 -07:00
Akihiro Suda
2562aca1a3 new service: version
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-05-11 09:07:43 +00:00
Stephen J Day
193abed96e
content: unify provider and ingester
The split between provider and ingester was a long standing division
reflecting the client-side use cases. For the most part, we were
differentiating these for the algorithms that operate them, but it made
instantation and use of the types challenging. On the server-side, this
distinction is generally less important. This change unifies these types
and in the process we get a few benefits.

The first is that we now completely access the content store over GRPC.
This was the initial intent and we have now satisfied this goal
completely. There are a few issues around listing content and getting
status, but we resolve these with simple streaming and regexp filters.
More can probably be done to polish this but the result is clean.

Several other content-oriented methods were polished in the process of
unification. We have now properly seperated out the `Abort` method to
cancel ongoing or stalled ingest processes. We have also replaced the
`Active` method with a single status method.

The transition went extremely smoothly. Once the clients were updated to
use the new methods, every thing worked as expected on the first
compile.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-05-10 17:05:53 -07:00
Phil Estes
91410874e7
Add pause/unpause implementation
This adds pause and unpause to containerd's execution service and the
same commands to the `ctr` client.

Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
2017-05-06 16:05:37 -04:00
Michael Crosby
5d3bc1a4a7 windows process exit timestamp
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-04-14 11:22:07 -07:00
Michael Crosby
efe177ed77 Add exited at to events and delete requests
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-04-13 13:55:58 -07:00
Michael Crosby
a7da08b7ba Add Pty and CloseStdin RPCs
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-04-07 11:20:10 -07:00
Michael Crosby
7715ddcefa Add exec APIs
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-04-05 11:51:56 -07:00
Qiang Huang
4764429cbe Add kill API for execution service
Signed-off-by: Qiang Huang <h.huangqiang@huawei.com>
2017-04-05 09:03:07 +08:00
Qiang Huang
6132bec05a Add Kill API for shim service
Signed-off-by: Qiang Huang <h.huangqiang@huawei.com>
2017-04-05 08:26:35 +08:00
Stephen J Day
a5c9d6d41b
api/services/images: define images metadata service
This is a first pass at the metadata required for supporting an image
store. We use a shallow approach to the problem, allowing this
component to centralize the naming. Resources for this image can then be
"snowballed" in for actual implementations. This is better understood
through example.

Let's take pull. One could register the name "docker.io/stevvooe/foo" as
pointing at a particular digest. When instructed to pull or fetch, the
system will notice that no components of that image are present locally.
It can then recursively resolve the resources for that image and fetch
them into the content store. Next time the instruction is issued, the
content will be present so no action will be taken.

Another example is preparing the rootfs. The requirements for a rootfs
can be resolved from a name. These "diff ids" will then be compared with
what is available in the snapshot manager. Any parts of the rootfs, such
as a layer, that isn't available in the snapshotter can be unpacked.
Once this process is satisified, the image will be runnable as a
container.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-04-04 10:35:11 -07:00
Michael Crosby
4f2b443a27 Rewrite imports for new github org
This rewrites the Go imports after switching to the new github org.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-04-03 14:05:44 -07:00
Derek McGowan
7473b2125a Call go fmt in travis
Not all checks are passing yet but we should still enforce
the most basic one.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-03-21 23:36:18 -07:00
Ophir Sneh
87f94bf79b Update shim exec rpc to take fewer args
Signed-off-by: Ophir Sneh <otsneh@gmail.com>
2017-03-21 15:34:02 -07:00
Derek McGowan
b1bc82726f
Rename prepare to unpack and init to prepare
Unpack and prepare better map to the actions done by rootfs.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-03-15 16:32:21 -07:00
Derek McGowan
340e56ecd0
Update comments in rootfs and for desciptor
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-03-15 16:32:21 -07:00
Derek McGowan
5d4577534a
Add init and get mounts to rootfs service
Update rootfs package to add mount initializer function.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-03-15 16:32:21 -07:00
Derek McGowan
ead425f426
Add rootfs service
Add rootfs service to allow creation of rootfs from layer array

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-03-15 16:32:21 -07:00
Akihiro Suda
899a52d655 api: implement fetching the container status
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-03-06 08:26:35 +00:00
Ian Campbell
186a9a2acd Return DeleteResponse from ContainerService.Delete
The message was defined but the method was returning empty, plumb through the
result from the shim layer.

Compile tested only.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-03-01 14:59:29 +00:00
Stephen Day
317b884110 Merge pull request #576 from stevvooe/seek-reset-handling
content: allow reset via Truncate
2017-02-28 10:42:21 -08:00
Stephen J Day
d99756a8a2
content: allow reset via Truncate
To make restarting after failed pull less racy, we define `Truncate(size
int64) error` on `content.Writer` for the zero offset. Truncating a
writer will dump any existing data and digest state and start from the
beginning. All subsequent writes will start from the zero offset.

For the service, we support this by defining the behavior for a write
that changes the offset. To keep this narrow, we only support writes out
of order at the offset 0, which causes the writer to dump existing data
and reset the local hash.

This makes restarting failed pulls much smoother when there was a
previously encountered error and the source doesn't support arbitrary
seeks or reads at arbitrary offsets. By allowing this to be done while
holding the write lock on a ref, we can restart the full download
without causing a race condition.

Once we implement seeking on the `io.Reader` returned by the fetcher,
this will be less useful, but it is good to ensure that our protocol
properly supports this use case for when streaming is the only option.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-28 10:40:02 -08:00
Stephen J Day
706c629354
api/services/content: define delete method
Allow deletion of content over the GRPC interface. For now, we are going
with a model that conducts reference management outside of the content
store, in the metadata store but this design is valid either way.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-27 20:06:29 -08:00
Ian Campbell
cc253b0f18 Add ContainerService.Info to get info about a single container
For clients which only want to know about one container this is simpler than
searching the result of execution.List.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-02-24 17:50:46 +00:00
Stephen J Day
c062a85782
content: cleanup service and interfaces
After implementing pull, a few changes are required to the content store
interface to make sure that the implementation works smoothly.
Specifically, we work to make sure the predeclaration path for digests
works the same between remote and local writers. Before, we were
hesitent to require the the size and digest up front, but it became
clear that having this provided significant benefit.

There are also several cleanups related to naming. We now call the
expected digest `Expected` consistently across the board and `Total` is
used to mark the expected size.

This whole effort comes together to provide a very smooth status
reporting workflow for image pull and push. This will be more obvious
when the bulk of pull code lands.

There are a few other changes to make `content.WriteBlob` more broadly
useful. In accordance with addition for predeclaring expected size when
getting a `Writer`, `WriteBlob` now supports this fully. It will also
resume downloads if provided an `io.Seeker` or `io.ReaderAt`. Coupled
with the `httpReadSeeker` from `docker/distribution`, we should only be
a lines of code away from resumable downloads.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-22 13:30:01 -08:00
Stephen J Day
baaf7543dc
api/services/content: define the content service
Bring the content service into the containerd API. This allows the
content store to be coordinated in the containerd daemon with minimal
effort. For the most part, this API follows the conventions and behavior
of the existing content store implementation with a few caveats.
Specifically, we remove the object oriented transaction mechanism in
favor of a very rich `Write` call.

Pains are taken to reduce race conditions around when having multiple
writers to a single piece of content. Clients should be able to race
towards getting a write lock on a reference, then wait on each other.

For the most part, this should be generically pluggable to allow
implementations of the content store to be swapped out.

We'll follow this up with an implementation to validate the model.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-21 13:10:10 -08:00
Stephen J Day
4cb31d9615
vendor: update gogo/protobuf dependency
Updates to the gogo/protobuf dependency are required to correctly
generate time types. We also remove an unused windows dependency.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-21 13:09:58 -08:00
Michael Crosby
5db3da6c2c Change shim proto package
This is not really a service like the other rpcs that we expose so lets
change the import paths for it.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-02-17 09:55:22 -08:00
Michael Crosby
ab8586b7c5 Remove bundles from API
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-02-15 13:56:41 -08:00
Stephen J Day
a3120172b0
cmd/protoc-gen-gogoctrd: turn off GoString generation
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-15 13:37:48 -08:00
Stephen J Day
83e7610194
cmd/ctrd-protobuild: create proper command for building protos
After trying to explain the complexities of developing with protobuf, I
have now created a command that correctly calculates the import paths
for each package and runs the protobuf command.

The Makefile has been updated accordingly, expect we now no longer use
`go generate`. A new target `protos` has been defined. We alias the two,
for the lazy. We leave `go generate` in place for cases where we will
actually use `go generate`.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-15 12:05:27 -08:00
Akihiro Suda
bf8abef70f api: introduce api/{types,services} for deduplication of API definition
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-02-14 02:42:13 +00:00
Stephen J Day
a9edb0d230
api/types/mount: bring back the mount type
The mount type is used across common GRPC services to express a deferred
access of a filesystem. Right now, they are generated by snapshotters,
but eventually, they can be passed for containers for creation at
runtime. With this flow, we can separate the generation and use of a
root container filesystem.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-09 18:23:08 -08:00
Kenfe-Mickael Laventure
78d7e8b256 supervisor: implement monitoring
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-02-06 15:16:55 -08:00
Kenfe-Mickael Laventure
6f9eda1134 api/execution: remove ProcessID from rpc calls
Now that the shim handles all container's processes the system pid is
sufficient.

Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-02-02 15:36:10 -08:00
Kenfe-Mickael Laventure
40b0b211b7 api/execution: add Container suffix to relevant rpc calls
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-02-02 14:40:23 -08:00
Michael Crosby
f187da9485 Port over supervisor to use grpc shim
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-02-01 16:01:53 -08:00
Michael Crosby
f431bf4ad4 Add state rpc to shim
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-01-26 15:09:59 -08:00
Michael Crosby
b59bd59d8a Working tty and io support in shim
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-01-26 11:31:17 -08:00
Michael Crosby
07c81ccac4 Add events api to shim
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-01-26 11:31:17 -08:00
Michael Crosby
e09b0b0c35 Add exec functionality to shim
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-01-26 11:31:17 -08:00
Michael Crosby
d5d2e586cd Refactor shim terminal and io handling
This also finishes the service implementation of the shim behind GRPC

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-01-26 11:31:17 -08:00
Michael Crosby
c08e0e610c Add grpc service to shim
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-01-26 11:31:17 -08:00
Michael Crosby
e6de7ea4b5 Add shim proto generation
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-01-26 11:30:54 -08:00
Stephen Day
23a644de8a Merge pull request #390 from estesp/init-vendoring
Add vendoring to containerd master
2017-01-11 14:16:57 -08:00
Phil Estes
f867156a38
Update protoc generation to use vendored protos
Use vendored-in protos in the project path rather
than expecting developers to have them in a local
path on the host. This made a generated change
in the FileDescriptor content, but everything else
matches and binaries are building properly.

Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
2017-01-11 16:59:20 -05:00
Davanum Srinivas
4f8a784b41 Indent proto files with tabs only
fix "make fmt" failure by switching from spaces to tabs

Signed-Off-By: Davanum Srinivas <davanum@gmail.com>
2016-12-16 17:20:41 -05:00
Antonio Murdaca
a5fa2c5536
api/shim: fix indent
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-12-15 10:33:03 +01:00
Kenfe-Mickael Laventure
2ef399b315 Add event support to execution subsystem
The implementation relies on nats.io

Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2016-12-12 09:21:29 -08:00
Kenfe-Mickael Laventure
2bee4ac5cd Only use IDs for execution RPC requests
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2016-12-09 11:32:45 -08:00
Michael Crosby
642ed8bb48 Add shim proto
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-12-09 10:13:57 -08:00
Kenfe-Mickael Laventure
0aad42f5cf Add exec and terminal support
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2016-12-09 10:00:34 -08:00
Kenfe-Mickael Laventure
5a86eae247 Merge execution and container service
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2016-12-07 19:44:22 -08:00
Michael Crosby
723a72bdf8 Fix execution build
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-12-05 16:18:05 -08:00
Michael Crosby
21a53c1d70 Add container start and supervisor
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-12-05 16:18:05 -08:00
Kenfe-Mickael Laventure
c857213b4c move work on execution service
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2016-12-05 14:15:03 -08:00
Stephen J Day
d9a2f40463
api/execution: use Empty well-known type
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2016-12-02 16:04:13 -08:00
Michael Crosby
32bf0f69fd Start work on Container and Process model
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-12-02 15:37:16 -08:00
Michael Crosby
6641888667 Start work on execution server implementation
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-12-02 11:33:58 -08:00
Michael Crosby
d0e2f3778a Update execution and container services
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-12-02 10:28:16 -08:00
Stephen J Day
f898628330 api/mount: add common mount type for services
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2016-12-01 15:27:52 -08:00
Kenfe-Mickael Laventure
ef03000b27 Revert "proto: move all protos under a single root"
This reverts commit 352c6b590c.
2016-12-01 14:25:03 -08:00
Kenfe-Mickael Laventure
352c6b590c proto: move all protos under a single root
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2016-12-01 14:17:13 -08:00
Kenfe-Mickael Laventure
4cf40db861 proto: rename container service to execution
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2016-12-01 13:57:51 -08:00
Kenfe-Mickael Laventure
62ca43566e proto: rename container to execution
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2016-12-01 13:38:29 -08:00
Stephen J Day
8a36e1c6d1
api: use correct generation helper
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2016-12-01 13:31:57 -08:00
Stephen J Day
9f90fdbf07
api: generate protobufs using go generate
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2016-12-01 12:36:19 -08:00
Stephen J Day
4d1a30ff41
api: protos now pass lint checks
Generation is likely broken, but the Makefile linter now passes. Another
pass will have a fully working toolchain.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2016-12-01 12:22:05 -08:00
Kenfe-Mickael Laventure
6c7b4b4c17 proto: make container.proto compile
Replaced tags with owner. This assumes we will provide a Metadata
endpoint since we're planning on using it for distribution.

Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2016-12-01 10:28:38 -08:00
Michael Crosby
3ef3f74878 Add ctr run functions for mocking client
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-11-29 16:52:04 -08:00
Michael Crosby
0806a0d8db Update container protos
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-11-29 15:36:25 -08:00
Michael Crosby
fc577a1cbb Add protos for initial API
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-11-28 15:28:38 -08:00
Michael Crosby
e115b52ce2 Remove containerd files
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-11-07 13:10:09 -08:00
Michael Crosby
b6bf350a07 Merge container.protos
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-09-22 11:29:20 -07:00
Stephen J Day
32a25d5523
api: begin to define the containerkit api
This commit cuts out the structure for defining grpc services for this
project. To provide compatibility with go package generation and support
reuse, we use a single protobuf file per package and make the import
paths relative to the GOPATH.

This first pass attempts to position the Mount type as the lingua franca
of ContainerKit. The Images service will provide paths prepared for use
as a set of mounts of the container service.

We'll need to merge the container service in place with new file defined
here.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2016-09-20 20:22:23 -07:00
Michael Crosby
0dcd3a034d Add additional request/response protos
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-09-20 12:01:28 -07:00
Michael Crosby
ce7702f2db Add initial container protos and service
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-09-20 11:20:48 -07:00
Michael Crosby
97dbfe9e1f Move cputicks into osutils package
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-09-20 11:20:17 -07:00
Michael Crosby
435a1c825d Refactor api/http to just pprof
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-09-20 10:22:19 -07:00
Mike Danese
3492af35ba grpc: regenerate protos and adjust for new package API
Signed-off-by: Mike Danese <mikedanese@google.com>
2016-09-12 11:37:24 -07:00
Yu, Zou
79c7673858 add more blkio subsystem parameters to control according to ones provided by runc. (#304)
Signed-off-by: Yu Zou <zouyu7@huawei.com>
2016-08-26 11:07:45 -07:00
Harry Zhang
9df37c23e4 Fix wrong field name
Signed-off-by: Harry Zhang <harryz@hyper.sh>
2016-07-14 17:06:01 +08:00
Kenfe-Mickaël Laventure
90f827ca10 Streamline events (#287)
* Sync process.State() with the matching events

Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>

* Allow requesting events for a specific container

Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>

* Sync container state retrieval with other events

Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>

* Let containerd take care of calling runtime delete on exit

Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>

* Take care of possible race in TestBusyboxTopExecTopKillInit

Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2016-07-13 11:01:07 -07:00
Kenfe-Mickael Laventure
dfb626dccc Use protobuf Timestamp type instead of uint64
This will ensure nanoseconds are taken in account.

Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2016-07-01 09:37:21 -07:00
Kenfe-Mickaël Laventure
098f9f1a1b Use uin64 for UpdateResource rpc (#265)
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2016-06-13 10:59:12 -07:00
boucher
a135e1093d Support the new --empty-ns flags in RunC for checkpoint/restore.
Signed-off-by: Ross Boucher <rboucher@gmail.com>
2016-06-09 16:49:26 -04:00
Kenfe-Mickaël Laventure
16a8dfe6a8 Add support for updating Kernel Memory Limits (#256)
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2016-06-06 11:26:07 -07:00
Kenfe-Mickaël Laventure
5624732128 Add golint to test (#255)
* Add a new lint rule to the Makefile

Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>

* Fix linter errors

Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>

* Allow replacing the default apt mirror

Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2016-06-03 15:00:49 -07:00
Michael Crosby
7fa8fc1477 Regen proto for checkpoint pr (#249)
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-05-25 17:52:46 -07:00
Ross Boucher
e756ae42d1 Add support for specifying a checkpoint directory. (#245)
Signed-off-by: Ross Boucher <rboucher@gmail.com>
2016-05-25 17:42:37 -07:00
Kenfe-Mickaël Laventure
126e5c403e Allow specifying runtime to be used when creating container (#247)
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2016-05-23 15:43:47 -07:00
Amit Krishnan
62e1370f91 containerd build clean on Solaris (#203)
* containerd build clean on Solaris

Signed-off-by: Amit Krishnan <krish.amit@gmail.com>

* Vendor golang.org/x/sys

Signed-off-by: Amit Krishnan <krish.amit@gmail.com>
2016-05-19 10:12:50 -07:00
Michael Crosby
614df93b92 Update grpc and protobufs dep (#243)
* Update grpc and protobufs dep

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

* Fix grpc error check

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-05-18 09:16:55 -07:00
Kenfe-Mickaël Laventure
57b7c3da91 Merge pull request #237 from crosbymichael/create-status
Use state for container create
2016-05-09 14:55:08 -07:00
Michael Crosby
6fae7fd8fc Use state for container create
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-05-09 14:51:19 -07:00
Kenfe-Mickael Laventure
ba465c17a7 Add new GetServerVersion rpc method
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2016-05-09 11:38:28 -07:00
Tonis Tiigi
62c89c1c50 Fix including memory stats field
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2016-04-27 18:42:16 -07:00
Michael Crosby
9ed8422e8c Use runc events --stats for collecting stats
runc `events --stats` now has stable output so we don't need to bind to
libcontainer directly to get stats output for the containers.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-04-26 14:57:30 -07:00
Michael Crosby
f7f4d8677f Remove windows code and simplify linux
Windows will not use containerd and its just unused code and unneed
complexity to keep it all around.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-04-26 14:10:41 -07:00
Michael Crosby
31270bba69 Remove windows files
Windows is not going to use containerd because there is already a
similar implementation on windows.  This removes all the windows files
because there is no reason to keep this overhead when its not going to
be used.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-04-26 14:10:33 -07:00
Michael Crosby
79c35e0306 Merge pull request #218 from crosbymichael/bump_runc
Update runc and runtime-specs dependencies
2016-04-26 14:05:44 -07:00
Michael Crosby
7a1d28e42b Bump runc and runtime-specs deps
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-04-26 13:38:36 -07:00
HuKeping
ca7c504068 Bugfix: ctr container list can not get the proper status of container
Prior to this patch, when list containers by "ctr containers" or
"ctr containers xxx", it will not get the proper status of conatinser(s).

That was caused by the wrong implementation of State() for structure process,
it only send a signal "0" to ping the "init" process and do nothing.

Since the OCI/runc has implemented an interface Status(), we can use that.
And I think this is more compatible with the design for containerd:
- containerd -> runtime -> fun()

Signed-off-by: Hu Keping <hukeping@huawei.com>
2016-04-26 07:04:26 -04:00
Michael Crosby
63001ee20d Merge pull request #185 from LK4D4/fix_pprof_address
api: fix pprof-address usage
2016-04-18 11:01:31 -07:00
Alexander Morozov
9416996835 api: fix pprof-address usage
those handlers already added by init functions

Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2016-04-05 11:36:51 -07:00
Aleksa Sarai
696cb97c28 api: grpc: add Limit to PidsStats
This will be filled by the value of pids.max, allowing clients to render
usage statistics for PIDs in a container.

Signed-off-by: Aleksa Sarai <asarai@suse.de>
2016-04-05 01:47:32 +10:00
Michael Crosby
47f239706c Add no pivot root support
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-03-30 21:52:15 -07:00
Michael Crosby
3d72dc1e4e Merge pull request #161 from calavera/enable_pprof
Enable http pprof connections when pprof-address flag is set
2016-03-25 17:27:08 -07:00
David Calavera
06bcd7d03c Enable http pprof connections when pprof-address is set.
Add flag to specify the address where those connections listen.

Signed-off-by: David Calavera <david.calavera@gmail.com>
2016-03-25 17:46:10 -04:00
Kenfe-Mickael Laventure
bac78851e0 Append error message to internal errors
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2016-03-25 11:03:11 -07:00
Ben Firshman
9979502ca8 Fix spacing in api.proto
Signed-off-by: Ben Firshman <ben@firshman.co.uk>
2016-03-18 17:37:59 -07:00
Michael Crosby
a4844a68a6 Add exec rlimit support to containerd
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-03-15 11:46:47 -07:00
Michael Crosby
e637c1ea80 Update specs imports for specs-go
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-03-15 11:31:32 -07:00
Tonis Tiigi
ab5eae56bf Add pids stats
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2016-03-08 16:13:10 -08:00
Michael Crosby
0dd075a47b Add update rpc for resource updates
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-03-07 15:23:52 -08:00
Michael Crosby
44d6a60e7e Add process cap support to add process rpc
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-03-07 11:01:28 -08:00
John Howard
d4ca79c978 Getting ctr closer to compiling on Windows
Signed-off-by: John Howard <jhoward@microsoft.com>
2016-02-29 12:42:30 -08:00
John Howard
b044ff0f29 Refactor process.go for platform specific
Signed-off-by: John Howard <jhoward@microsoft.com>

Move process sorter to new file

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Sort containers by id

This will not be the most accurate sorting but atleast the list will be
consistent inbetween calls.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Allow runtime to be configurable via daemon start

This allows people to pass an alternate name or location to the runtime
binary to start containers.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Fix state output for containers

Return the proper state/status for a container by checking if the pid is
still alive.  Also fix the cleanup handling in the shim to make sure
containers are not left behind.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Properly wait for container start

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-02-29 10:43:35 -08:00
John Howard
dd192f4cd9 Final bit of compiling on Windows
Signed-off-by: John Howard <jhoward@microsoft.com>
2016-02-26 15:38:05 -08:00
John Howard
2b8f022950 runtime compiles on Windows
Signed-off-by: John Howard <jhoward@microsoft.com>
2016-02-26 14:01:06 -08:00
Kenfe-Mickael Laventure
d8fce0997e Fill forgotten KernelUsage field of CgroupStats.MemoryStats
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2016-02-23 10:51:28 -08:00
Kenfe-Mickael Laventure
9f8e542234 Add Limit field to rpc MemoryStats.MemoryData
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2016-02-23 10:51:28 -08:00
Michael Crosby
a2461f1b6b Merge pull request #99 from mlaventure/system-usage-stats
Add SystemUsage to returned stats
2016-02-23 10:15:42 -08:00
Kenfe-Mickael Laventure
da5d876d91 Add SystemUsage to returned CpuStats
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2016-02-20 23:40:09 -08:00
Michael Crosby
4e05bf491a Remove eventloop package
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-02-19 13:21:21 -08:00
Michael Crosby
d19c818cdc Don't get pids on create
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-02-12 14:01:24 -08:00
Michael Crosby
1d8dc2a072 Fix time parsing from API default value
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-02-12 13:29:53 -08:00
Michael Crosby
7b182bd898 Remove name from container proto
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-02-12 13:08:38 -08:00
Michael Crosby
9341a95c26 add event log and timestamp to events api
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-02-12 10:17:59 -08:00
Michael Crosby
3dc59d565a Rename Events to task in supervisor
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-02-11 17:26:24 -08:00
Michael Crosby
d2bf71043f Fix memory value in stats api
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-02-11 15:55:19 -08:00
Michael Crosby
bdeb87a090 Implement stats for containerd
This is a single endpoint that clients will poll

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-02-11 14:36:32 -08:00
Michael Crosby
532697f32f Add all pids to state output
Also update libcontainer dep

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-02-11 14:07:34 -08:00
Michael Crosby
d6bd304c92 Add labels for containers
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-02-11 13:44:25 -08:00
Michael Crosby
76d69b4ed8 Add id to state command to return single container
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-02-11 12:20:29 -08:00
Michael Crosby
58c5eb5ecb Return container on start
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-02-11 11:56:11 -08:00
Michael Crosby
01176f2d7f Add pid and stdio to process state
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-02-04 14:40:35 -08:00
Michael Crosby
36eb83cb99 Move fifo creation to client
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-02-03 13:56:15 -08:00
Michael Crosby
a42eb9fd63 Add close stdin
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-02-02 16:00:22 -08:00
Michael Crosby
277cc920a4 Implement checkpoint / restore for shim
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-02-02 16:00:22 -08:00
Michael Crosby
835f3b6a97 Update shim for exec
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-02-02 16:00:22 -08:00
Michael Crosby
fe38efda50 Add shim for reattach of processes
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Remove runtime files from containerd

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Update supervisor for orphaned containers

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Remove ctr/container.go back to rpc calls

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Add attach to loaded container

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Add monitor based on epoll for process exits

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Convert pids in containerd to string

This is so that we no longer care about linux or system level pids and
processes in containerd have user defined process id(pid) kinda like the
exec process ids that docker has today.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Add reaper back to containerd

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Implement list containers with new process model

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Implement restore of processes

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Add NONBLOCK to exit fifo open

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Implement tty reattach

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Fix race in exit pipe creation

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Add delete to shim

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Update shim to use pid-file and not stdout

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-02-02 16:00:22 -08:00
Burcu Dogan
167219101f runtime: remove unnecessary intermediate type, Status
Signed-off-by: Burcu Dogan <jbd@google.com>
2016-01-28 00:31:33 +01:00
Antonio Murdaca
6bc0dcf4e9 api: grpc: types: align fields in api.proto
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-01-22 21:29:37 +01:00
Alexander Morozov
69f8f566a2 Move supervisor to it's own package
It allows to keep main namespace cleaner

Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2015-12-17 16:18:48 -08:00
Michael Crosby
76cf593212 Add oom notifications and pid to create response
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2015-12-15 16:22:53 -08:00
Michael Crosby
ec31245f54 Add exec process log support
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2015-12-15 11:43:51 -08:00
Michael Crosby
d542ece69c Add tty support from client
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2015-12-14 16:47:42 -08:00
Michael Crosby
b2e649b164 Add console to proto
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2015-12-14 16:27:28 -08:00
Alexander Morozov
4bc322397f Add initial implementation of stats
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2015-12-14 14:45:06 -08:00
Alexander Morozov
ce387dc505 Add datastructures for stats API
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2015-12-14 14:43:56 -08:00
Michael Crosby
92c0790899 Add stdin support for client and daemon
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2015-12-14 14:40:50 -08:00
Michael Crosby
c4aa39a818 Update protos with stdin for container create
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2015-12-14 14:18:42 -08:00
Michael Crosby
e9f63fc9a4 Add basic fifo support for IO copy
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2015-12-14 14:15:26 -08:00
Michael Crosby
e5545a1461 Add basic logging to file support
This currently logs to a json file with the stream type.  This is slow
and hard on the cpu and memory so we need to swich this over to
something like protobufs for the binary logs but this is just a start.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2015-12-11 10:26:49 -08:00
Alexander Morozov
04d2034a83 Add initial documentation for api
Cool that protoc generates docstrings for pb.go file

Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2015-12-10 14:43:23 -08:00
Michael Crosby
71ef776082 Add events support in client
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2015-12-10 12:30:20 -08:00
Michael Crosby
1d3349128e Replace rest api with grpc api
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2015-12-09 15:39:27 -08:00
Michael Crosby
11c27935d0 Replace api cli with grpc cli
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2015-12-09 15:37:04 -08:00
Alexander Morozov
e14e4576c9 Prototype of grpc API
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2015-12-09 15:05:50 -08:00
Michael Crosby
e87f8336c4 Add more client methods
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2015-12-08 13:31:20 -08:00
Michael Crosby
d9e8fe62cb Add concurrency and id flag for daemon
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2015-12-07 15:19:56 -08:00