Move historical docs to docs/historical
To clarify that end users do not need to read these docs, and that these docs do not need to be updated Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
This commit is contained in:
Akihiro Suda
|
Before Width: | Height: | Size: 52 KiB After Width: | Height: | Size: 52 KiB |
|
Before Width: | Height: | Size: 69 KiB After Width: | Height: | Size: 69 KiB |
46
docs/historical/reports/2017-01-13.md
Normal file
46
docs/historical/reports/2017-01-13.md
Normal file
@@ -0,0 +1,46 @@
|
||||
# Development Report for Jan 13, 2017
|
||||
|
||||
### Preface
|
||||
|
||||
I wanted to try something different for this project during its development by giving a weekly summary on the progress and what is being working on. I figured the best place is to keep the reports in the repo. Please give your feedback on the format and let me know if this is a waste of time or if there is something better we can do.
|
||||
|
||||
## Quick Start and Vendoring
|
||||
|
||||
This week we merged PRs adding a quick start guide with build instructions to help get new contributors onboard quicker. We also merged a vendoring PR so everyone has the same revision of packages for the project. It can add a little burden on development having vendoring at this early stage, especially the way we have to do vendoring in Go, but it ensures that everyone contributing has a consistent environment.
|
||||
|
||||
* https://github.com/containerd/containerd/pull/400
|
||||
* https://github.com/containerd/containerd/pull/390
|
||||
|
||||
## Milestones
|
||||
|
||||
I spent some time yesterday working on milestones. The milestones are more goal focused instead of being based on a versions or a date. We have goals that we want to hit in each of the phases and the milestones are organized that way.
|
||||
|
||||
We are currently in the bootstrap phase. Much of the design work, getting the project's infrastructure up (ci, vendoring, guides for development, etc) are part of this work.
|
||||
|
||||
The milestones will give you the most up-to-date view on the project and what is being worked on when.
|
||||
|
||||
* https://github.com/containerd/containerd/milestones
|
||||
|
||||
## Networking
|
||||
|
||||
Containerd will not support networking in its core. Based on the feedback we received and our own experience with networking we can confidently leave this out of containerd without impacting functionality. You can see the full discussion in the issue linked below.
|
||||
|
||||
Networking means many different things today and is becoming more software based. On the runtime level, overlay networking is very simple, its just veth interfaces. All of the overlay magic happens at the upper levels with routing tables and termination outside of the runtime and interfaces. Networking also contains various forms of service discovery which should not be at the runtime level as things like health checks come into play to make sure you are only discovering healthy services.
|
||||
|
||||
I personally feel like this is a good move for the project. We have good primitives at the runtime level with the create/start split that allows a container's network namespace to be created and time for a network plugin to create and add interfaces to the container before the user's process is started. We also have the ability for containers to join pre-existing network namespaces that have already been populated by a networking implementation.
|
||||
|
||||
* https://github.com/containerd/containerd/issues/362
|
||||
|
||||
## Runtime and Shim
|
||||
|
||||
We finished porting over the shim from the existing containerd implementation that is being used today. This makes sure that the containers are not tied to the daemon's lifecycle. This allows containerd to reattach to the STDIO and receive exit events for the containers when it is no longer the parent of these processes.
|
||||
|
||||
* https://github.com/containerd/containerd/pull/417
|
||||
|
||||
## What's Next?
|
||||
|
||||
Next week we will be working towards a full PoC with the runtime, storage, and fetching of images. Getting the core functionality up and running quickly is important to us to ensure that integration between the different subsystems in the core flow well together. We want to make sure the responsibilities of pulling an image from a remote source do not spill into the storage layer and vice-versa.
|
||||
|
||||
We still have more documentation work to do on the design and lifecycle of components in the core which is another focus for next week. You can find the current design docs in the [repo here](https://github.com/containerd/containerd/tree/main/design).
|
||||
|
||||
I hope this status report helps. If you are looking for ways to contribute, check out the issues on the current milestone.
|
||||
22
docs/historical/reports/2017-01-20.md
Normal file
22
docs/historical/reports/2017-01-20.md
Normal file
@@ -0,0 +1,22 @@
|
||||
# Development Report for Jan 20, 2017
|
||||
|
||||
This week we worked on refactoring the storage and runtime aspects of containerd, much of this work won't be completed until next week.
|
||||
|
||||
## Distribution tool
|
||||
|
||||
@stevvooe started work on a `dist` tool for fetching images and other distribution aspects. One of the problems we wanted to solve with containerd is the ability to decouple the fetch of image contents, the unpacking of the contents, and the storage of the contents. Separating these lets us download content as non-root on machines and only elevate privileges when the contents need to be unpacked on disk.
|
||||
|
||||
* https://github.com/containerd/containerd/pull/452
|
||||
|
||||
## Shim Work
|
||||
|
||||
A large part of my week I have been working on the shim. It is a very core aspect of containerd and allows containers to run without being tied to the lifecycle of the containerd daemon. With this work we will end up with one shim per container that manages all additional processes in the container. This saves resources on the system as well as makes interacting with the shim much easier. We are placing the shim behind a GRPC API instead of the pipe based protocol that we have today.
|
||||
|
||||
I don't have a PR open at this time but expect something next week.
|
||||
|
||||
## General Improvements
|
||||
|
||||
We worked on the build process this week as well as improvements across the codebase. There were 31 commits merged this week from various contributors.
|
||||
|
||||
Overall, there were no large features hitting master this week. Many of those are still in the works but a lot was done to simplify existing code and reduce complexity where we can.
|
||||
|
||||
256
docs/historical/reports/2017-01-27.md
Normal file
256
docs/historical/reports/2017-01-27.md
Normal file
@@ -0,0 +1,256 @@
|
||||
# Development Report for Jan 27, 2017
|
||||
|
||||
This week we made a lot of progress on tools to work with local content storage
|
||||
and image distribution. These parts are critical in forming an end to end proof
|
||||
of concept, taking docker/oci images and turning them into bundles.
|
||||
|
||||
We also have defined a new GRPC protocol for interacting with the
|
||||
container-shim, which is used for robust container management.
|
||||
|
||||
## Maintainers
|
||||
|
||||
* https://github.com/containerd/containerd/pull/473
|
||||
|
||||
Derek McGowan will be joining the containerd team as a maintainer. His
|
||||
extensive experience in graphdrivers and distribution will be invaluable to the
|
||||
containerd project.
|
||||
|
||||
## Shim over GRPC
|
||||
|
||||
* https://github.com/containerd/containerd/pull/462
|
||||
|
||||
```
|
||||
NAME:
|
||||
containerd-shim -
|
||||
__ _ __ __ _
|
||||
_________ ____ / /_____ _(_)___ ___ _________/ / _____/ /_ (_)___ ___
|
||||
/ ___/ __ \/ __ \/ __/ __ `/ / __ \/ _ \/ ___/ __ /_____/ ___/ __ \/ / __ `__ \
|
||||
/ /__/ /_/ / / / / /_/ /_/ / / / / / __/ / / /_/ /_____(__ ) / / / / / / / / /
|
||||
\___/\____/_/ /_/\__/\__,_/_/_/ /_/\___/_/ \__,_/ /____/_/ /_/_/_/ /_/ /_/
|
||||
|
||||
shim for container lifecycle and reconnection
|
||||
|
||||
|
||||
USAGE:
|
||||
containerd-shim [global options] command [command options] [arguments...]
|
||||
|
||||
VERSION:
|
||||
1.0.0
|
||||
|
||||
COMMANDS:
|
||||
help, h Shows a list of commands or help for one command
|
||||
|
||||
GLOBAL OPTIONS:
|
||||
--debug enable debug output in logs
|
||||
--help, -h show help
|
||||
--version, -v print the version
|
||||
|
||||
```
|
||||
|
||||
This week we completed work on porting the shim over to GRPC. This allows us
|
||||
to have a more robust way to interface with the shim. It also allows us to
|
||||
have one shim per container where previously we had one shim per process. This
|
||||
drastically reduces the memory usage for exec processes.
|
||||
|
||||
We also had a lot of code in the containerd core for syncing with the shims
|
||||
during execution. This was because we needed ways to signal if the shim was
|
||||
running, the container was created or any errors on create and then starting
|
||||
the container's process. Getting this right and syncing was hard and required
|
||||
a lot of code. With the new flow it is just function calls via rpc.
|
||||
|
||||
```proto
|
||||
service Shim {
|
||||
rpc Create(CreateRequest) returns (CreateResponse);
|
||||
rpc Start(StartRequest) returns (google.protobuf.Empty);
|
||||
rpc Delete(DeleteRequest) returns (DeleteResponse);
|
||||
rpc Exec(ExecRequest) returns (ExecResponse);
|
||||
rpc Pty(PtyRequest) returns (google.protobuf.Empty);
|
||||
rpc Events(EventsRequest) returns (stream Event);
|
||||
rpc State(StateRequest) returns (StateResponse);
|
||||
}
|
||||
```
|
||||
|
||||
With the GRPC service it allows us to decouple the shim's lifecycle from the
|
||||
containers, in the way that we get synchronous feedback if the container failed
|
||||
to create, start, or exec from shim errors.
|
||||
|
||||
The overhead for adding GRPC to the shim is actually less than the initial
|
||||
implementation. We already had a few pipes that allowed you to control
|
||||
resizing of the pty master and exit events, now all replaced by one unix
|
||||
socket. Unix sockets are cheap and fast and we reduce our open fd count with
|
||||
way by not relying on multiple fifos.
|
||||
|
||||
We also added a subcommand to the `ctr` command for testing and interfacing
|
||||
with the shim. You can interact with a shim directly via `ctr shim` and get
|
||||
events, start containers, start exec processes.
|
||||
|
||||
## Distribution Tool
|
||||
|
||||
* https://github.com/containerd/containerd/pull/452
|
||||
* https://github.com/containerd/containerd/pull/472
|
||||
* https://github.com/containerd/containerd/pull/474
|
||||
|
||||
Last week, @stevvooe committed the first parts of the distribution tool. The main
|
||||
component provided there was the `dist fetch` command. This has been followed
|
||||
up by several other low-level commands that interact with content resolution
|
||||
and local storage that can be used together to work with parts of images.
|
||||
|
||||
With this change, we add the following commands to the dist tool:
|
||||
|
||||
- `ingest`: verify and accept content into storage
|
||||
- `active`: display active ingest processes
|
||||
- `list`: list content in storage
|
||||
- `path`: provide a path to a blob by digest
|
||||
- `delete`: remove a piece of content from storage
|
||||
- `apply`: apply a layer to a directory
|
||||
|
||||
When this is more solidified, we can roll these up into higher-level
|
||||
operations that can be orchestrated through the `dist` tool or via GRPC.
|
||||
|
||||
As part of the _Development Report_, we thought it was a good idea to show
|
||||
these tools in depth. Specifically, we can show going from an image locator to
|
||||
a root filesystem with the current suite of commands.
|
||||
|
||||
### Fetching Image Resources
|
||||
|
||||
The first component added to the `dist` tool is the `fetch` command. It is a
|
||||
low-level command for fetching image resources, such as manifests and layers.
|
||||
It operates around the concept of `remotes`. Objects are fetched by providing a
|
||||
`locator` and an object identifier. The `locator`, roughly analogous to an
|
||||
image name or repository, is a schema-less URL. The following is an example of
|
||||
a `locator`:
|
||||
|
||||
```
|
||||
docker.io/library/redis
|
||||
```
|
||||
|
||||
When we say the `locator` is a "schema-less URL", we mean that it starts with
|
||||
hostname and has a path, representing some image repository. While the hostname
|
||||
may represent an actual location, we can pass it through arbitrary resolution
|
||||
systems to get the actual location. In that sense, it acts like a namespace.
|
||||
|
||||
In practice, the `locator` can be used to resolve a `remote`. Object
|
||||
identifiers are then passed to this remote, along with hints, which are then
|
||||
mapped to the specific protocol and retrieved. By dispatching on this common
|
||||
identifier, we should be able to support almost any protocol and discovery
|
||||
mechanism imaginable.
|
||||
|
||||
The actual `fetch` command currently provides anonymous access to Docker Hub
|
||||
images, keyed by the `locator` namespace `docker.io`. With a `locator`,
|
||||
`identifier` and `hint`, the correct protocol and endpoints are resolved and the
|
||||
resource is printed to stdout. As an example, one can fetch the manifest for
|
||||
`redis` with the following command:
|
||||
|
||||
```
|
||||
$ ./dist fetch docker.io/library/redis latest mediatype:application/vnd.docker.distribution.manifest.v2+json
|
||||
```
|
||||
|
||||
Note that we have provided a mediatype "hint", nudging the fetch implementation
|
||||
to grab the correct endpoint. We can hash the output of that to fetch the same
|
||||
content by digest:
|
||||
|
||||
```
|
||||
$ ./dist fetch docker.io/library/redis sha256:$(./dist fetch docker.io/library/redis latest mediatype:application/vnd.docker.distribution.manifest.v2+json | shasum -a256)
|
||||
```
|
||||
|
||||
The hint now elided on the outer command, since we have affixed the content to
|
||||
a particular hash. The above shows us effectively fetches by tag, then by hash
|
||||
to demonstrate the equivalence when interacting with a remote.
|
||||
|
||||
This is just the beginning. We should be able to centralize configuration
|
||||
around fetch to implement a number of distribution methodologies that have been
|
||||
challenging or impossible up to this point.
|
||||
|
||||
Keep reading to see how this is used with the other commands to fetch complete
|
||||
images.
|
||||
|
||||
### Fetching all the layers of an image
|
||||
|
||||
If you are not yet entertained, let's bring `jq` and `xargs` into the mix for
|
||||
maximum fun. Our first task will be to collect the layers into a local content
|
||||
store with the `ingest` command.
|
||||
|
||||
The following incantation fetches the manifest and downloads each layer:
|
||||
|
||||
```
|
||||
$ ./dist fetch docker.io/library/redis latest mediatype:application/vnd.docker.distribution.manifest.v2+json | \
|
||||
jq -r '.layers[] | "./dist fetch docker.io/library/redis "+.digest + "| ./dist ingest --expected-digest "+.digest+" --expected-size "+(.size | tostring) +" docker.io/library/redis@"+.digest' | xargs -I{} -P10 -n1 sh -c "{}"
|
||||
```
|
||||
|
||||
The above fetches a manifest, pipes it to jq, which assembles a shell pipeline
|
||||
to ingest each layer into the content store. Because the transactions are keyed
|
||||
by their digest, concurrent downloads and downloads of repeated content are
|
||||
ignored. Each process is then executed parallel using xargs. If you run the
|
||||
above command twice, it will not download the layers because those blobs are
|
||||
already present in the content store.
|
||||
|
||||
What about status? Let's first remove our content so we can monitor a download.
|
||||
`dist list` can be combined with xargs and `dist delete` to remove that
|
||||
content:
|
||||
|
||||
```
|
||||
$ ./dist list -q | xargs ./dist delete
|
||||
```
|
||||
|
||||
In a separate shell session, could monitor the active downloads with the following:
|
||||
|
||||
```
|
||||
$ watch -n0.2 ./dist active
|
||||
```
|
||||
|
||||
For now, the content is downloaded into `.content` in the current working
|
||||
directory. To watch the contents of this directory, you can use the following:
|
||||
|
||||
```
|
||||
$ watch -n0.2 tree .content
|
||||
```
|
||||
|
||||
Now, run the fetch pipeline from above. You'll see the active downloads, keyed
|
||||
by locator and object, as well as the ingest transactions resulting blobs
|
||||
becoming available in the content store. This will help to understand what is
|
||||
going on internally.
|
||||
|
||||
### Getting to a rootfs
|
||||
|
||||
While we haven't yet integrated full snapshot support for layer application, we
|
||||
can use the `dist apply` command to start building out rootfs for inspection
|
||||
and testing. We'll build up a similar pipeline to unpack the layers and get an
|
||||
actual image rootfs.
|
||||
|
||||
To get access to the layers, you can use the path command:
|
||||
|
||||
```
|
||||
$./dist path sha256:010c454d55e53059beaba4044116ea4636f8dd8181e975d893931c7e7204fffa
|
||||
sha256:010c454d55e53059beaba4044116ea4636f8dd8181e975d893931c7e7204fffa /home/sjd/go/src/github.com/containerd/containerd/.content/blobs/sha256/010c454d55e53059beaba4044116ea4636f8dd8181e975d893931c7e7204fffa
|
||||
```
|
||||
|
||||
This returns the a direct path to the blob to facilitate fast access. We can
|
||||
incorporate this into the `apply` command to get to a rootfs for `redis`:
|
||||
|
||||
```
|
||||
$ mkdir redis-rootfs
|
||||
$ ./dist fetch docker.io/library/redis latest mediatype:application/vnd.docker.distribution.manifest.v2+json | \
|
||||
jq -r '.layers[] | "sudo ./dist apply ./redis-rootfs < $(./dist path -q "+.digest+")"' | xargs -I{} -n1 sh -c "{}"
|
||||
```
|
||||
|
||||
The above fetches the manifest, then passes each layer into the `dist apply`
|
||||
command, resulting in the full redis container root filesystem. We do not do
|
||||
this in parallel, since each layer must be applied sequentially. Also, note
|
||||
that we have to run `apply` with `sudo`, since the layers typically have
|
||||
resources with root ownership.
|
||||
|
||||
Alternatively, you can just read the manifest from the content store, rather
|
||||
than fetching it. We use fetch above to avoid having to lookup the manifest
|
||||
digest for our demo.
|
||||
|
||||
Note that this is mostly a POC. This tool has a long way to go. Things like
|
||||
failed downloads and abandoned download cleanup aren't quite handled. We'll
|
||||
probably make adjustments around how content store transactions are handled to
|
||||
address this. We still need to incorporate snapshotting, as well as the ability
|
||||
to calculate the `ChainID` under subsequent unpacking. Once we have some tools
|
||||
to play around with snapshotting, we'll be able to incorporate our
|
||||
`rootfs.ApplyLayer` algorithm that will get us a lot closer to a production
|
||||
worthy system.
|
||||
|
||||
From here, we'll build out full image pull and create tooling to get runtime
|
||||
bundles from the fetched content.
|
||||
71
docs/historical/reports/2017-02-10.md
Normal file
71
docs/historical/reports/2017-02-10.md
Normal file
@@ -0,0 +1,71 @@
|
||||
# Development Report for Feb 10, 2017
|
||||
|
||||
Sorry for slacking off last week on the report. We totally spaced it.
|
||||
This week we will go over what happened last week and this week.
|
||||
|
||||
## Snapshot Design Changes
|
||||
|
||||
* https://github.com/containerd/containerd/pull/496
|
||||
|
||||
After receiving feedback on the `snapshot.Driver` interface, now known as the
|
||||
`Snapshotter`, it was found that the behavior of active and committed snapshots
|
||||
was confusing. Specifically, it was hard to tell which methods to use based on
|
||||
the state of the snapshot. It was also confusing which identifier to use based
|
||||
on the state.
|
||||
|
||||
To clean this up, we moved to using "Active" and "Committed" as the type of a
|
||||
snapshot, as opposed to the state of the snapshot. These lifecycle
|
||||
relationships are such that "Active" snapshots can only be created from a
|
||||
"Committed" snapshot and "Committed" snapshots must come from an "Active"
|
||||
snapshot. We retain the concept of a parent for "Active" and "Committed"
|
||||
snapshots, but clarified that only "Committed" snapshots can be a parent.
|
||||
|
||||
|
||||
As part of this, we unified the keyspace of snapshots. For common operations,
|
||||
such as removal and stat, we only have a single method that works for both
|
||||
active and committed snapshots. For methods that take one or the other, the
|
||||
restriction is called out. `Remove` and `Delete` were consolidated as part of
|
||||
this. This also makes it easier to handle scenarios that use the snapshot
|
||||
identity as a lock to coordinate multiple writers.
|
||||
|
||||
`Exists` and `Parent` have also be consolidated into single `Stat` call. This
|
||||
returns an `Info` structure which includes information about state and
|
||||
parentage. We also simplify the `Walk` method as part of this.
|
||||
|
||||
Effectively, we now have snapshots that are either active or committed and a
|
||||
much smaller interface!
|
||||
|
||||
## Bundles Bundles Bundles
|
||||
|
||||
We spend time talking with people implementing Windows support as well as a few other users.
|
||||
One the major issues with our current approach was that bundles were a central part of our architecture.
|
||||
The content and storage subsystems would produce bundles and the execution subsystem would consume them.
|
||||
However, with a bundle being on the filesystem, having this concept does not work as smoothly on Windows
|
||||
as it would for Unix platforms.
|
||||
|
||||
So the major design change is that bundles will be an implementation detail of the runtime and not a core
|
||||
part of the API. You will no longer pass the bundle path to containerd, it will manage bundles internally
|
||||
and the root filesystem mounts along with the spec, passed via the `Any` type, will be API fields for the
|
||||
create request of a container.
|
||||
|
||||
## Runtimes
|
||||
|
||||
With the bundles change above we also need to make sure changes for where containers are created and who
|
||||
does the supervision after creation.
|
||||
|
||||
The runtimes in containerd, things such as Linux, Windows, and Solaris, will be responsible for the
|
||||
creation of containers and loading of containers when containerd boots.
|
||||
|
||||
The containerd core will be responsible for interfacing with the GRPC API and managing a common `Container`
|
||||
interface that the runtimes produce. The supervision of containers will be handled in the core.
|
||||
This is not much of a change from today, just refining where responsibilities lie.
|
||||
|
||||
## Progress and POC
|
||||
|
||||
Overall design has been a large focus for us at the moment. While containerd can run containers today it
|
||||
is not optimized in terms of speed or design. With the work we did this week to make sure that containerd
|
||||
will work across many different platforms with first class support, not as an after thought, we are in a good
|
||||
position to really start on the development work and get a POC out within the next few weeks.
|
||||
|
||||
This POC should give you a good idea of what containerd can do, its APIs, and how users will interact with
|
||||
its various subsystems.
|
||||
67
docs/historical/reports/2017-02-24.md
Normal file
67
docs/historical/reports/2017-02-24.md
Normal file
@@ -0,0 +1,67 @@
|
||||
# Development Report for Feb 24, 2017
|
||||
|
||||
## containerd Summit
|
||||
|
||||
Yesterday, Feb. 23, we had the containerd summit for contributors and maintainers. We started off by getting everyone up to speed on the project, roadmap, and goals before diving down into specific issues and design of containerd. We should have the videos up soon for the various sessions and Q&A.
|
||||
|
||||
## Runtime Breakout Session
|
||||
|
||||
### Shim
|
||||
|
||||
We discussed the use of the containerd shim, the costs that it adds, as well as its benefits. Overall, most of us agreed that the extra memory overhead is worth it for the feature set that it provides.
|
||||
|
||||
However, we did make a few changes to the shim in the master branch; a grpc API so that the shim is straightforward to interact with and much more robust in terms of error handling. We also have one shim per container vs the one shim per process, which was the current implementation.
|
||||
|
||||
There were a couple of additional changes we discussed. One being launching the shim inside the cgroup of the container so that any IO, CPU, and memory that the shim consumes is "charged" to the container. I'll expand a little more on why this matters in the next section when it comes to logging.
|
||||
|
||||
|
||||
### Logging
|
||||
|
||||
Logging is a big issue for containers and it’s not exactly straightforward. Since containers can be used in many different ways the support of logging and connecting to a containers interactively can add complexity to the runtime. Right now, the shim is responsible for handling the IO of a container and making sure that it is kept open in the event that containerd ( the daemon ) or clients above containerd crash and need to reconnect to containers that they launched.
|
||||
|
||||
You can use fifos for reconnecting to container logs but there is concerns about the buffer limit when nothing is reading from the fifo. If the fifo fills up, then the application in the container will block. In the end, most container systems log to files on disk but in various formats. The only part of the container's IO that needs to be a pipe, or fifo for reconnection is STDIN. STDOUT and STDERR can go directly to a file in most cases. However, logging the raw output for a container directly to a file is not ideal as both Docker and Kube have specific on disk formats.
|
||||
|
||||
So our initial idea that was discussed is to move some type of "log formatter" into the shim. This will allow a single container to have its logs written directly to disk from the shim without using fifos for OUT and ERR. The other problem that this would solve is when a single container has excessive logs. If a single daemon is writing all the logs to disk, you will see high CPU on the entire daemon. If we place the shim in the cgroup for the container, then only that container is affected when it starts to log excessively without affecting the performance of the entire system. The CPU and IO could be charged to the container.
|
||||
|
||||
We still have to work out how we will specify the log format in the shim. Go 1.8 plugins are not looking like a good solution for this. I’ll be opening an issue so that we can figure this out on the repo.
|
||||
|
||||
|
||||
### Multitenancy Metadata
|
||||
|
||||
A use case for containerd is that it’s a solid base for multiple container platforms. When you have a single containerd installed on a system but have Kube, Docker, and Swarm all using it, different operations such as listing of containers and access to content need to be scoped appropriately so that Docker does not return containers created by Kube and vice versa.
|
||||
|
||||
We were already looking to design a metadata store for clients and internal data of containerd and this seems like a good fit for this problem. In the breakout session we discussed having system details like content and containers stay in a flat namespace inside the containerd core but query and create operations by clients should be namespaced.
|
||||
|
||||
As simple example would be:
|
||||
|
||||
`/docker/images/redis/3.2`
|
||||
`/docker/containers/redis-master`
|
||||
`/kube/services/lb`
|
||||
`/kube/pods/redis/containers/logger`
|
||||
`/kube/pods/redis/containers/master`
|
||||
|
||||
These are just simple examples and much of the format will be left up to the clients for storing metadata within the system. Clients should be able to query based on the namespace of keys as well as queries of `/` to see the entire system as a whole.
|
||||
|
||||
### Plugins in Go 1.8
|
||||
|
||||
Looking at the latest Go 1.8 release plugins looked like a perfect solution for extending containerd by third parties. However, after discussions at the summit and with help from Tim to get clarification on the feature from the Go team it does not not look very promising in terms of implementation. We currently have the code for plugins via go 1.8 merged into master but we will have to rethink our extension method going forward.
|
||||
|
||||
https://github.com/containerd/containerd/issues/563
|
||||
|
||||
## Storage and Distribution Breakout Session
|
||||
|
||||
### Snapshots
|
||||
|
||||
During the image distribution breakout we first covered the current state of snapshots and where we expect to go next. The discussion started with how new snapshotters will be integrated using Go plugins or package importing rather than relying on grpc. Snapshots themselves are not currently exposed as a grpc service but there is some desire to have lower level access for debugging, content sideloading, or building. The exposed interface for snapshots beyond the existing pull/push/run workflows will be designed with this in mind. Additionally there was some discussion related to the possibility of having the mounter be pluggable and opening up snapshots to volumes use cases. The snapshot model was discussed in depth including how we will differentiate between terminal commits and online snapshots. Differentiating between the two could be important to optimize different use cases around building, importing, and backing up container changes.
|
||||
|
||||
### Distribution
|
||||
|
||||
The next major topic of discussion was around distribution, primarily focused on the resolver and fetcher interfaces. We discussed what a locator will look for finding remote content as well as how they can be integrated with a `git remote`-like model. Stephen expressed his intent to keep the locator as opaque as possible, but to stay away from including protocol definitions in the locator value. Having a remotes table was proposed was proposed as a way to keep naming simple but allow defining more specific protocols. The current fetch interface was found to be unclear. It was proposed to remove the hints and integrate the media type checks and expectations into the resolver. Role of content store discussed for validating content and converging ingests of duplicate data.
|
||||
|
||||
## Upcoming work
|
||||
|
||||
1. Next week we are still working towards and end to end PoC with the various subsystems of containerd working together behind the GRPC api. The API is still under development but the PoC should give people a good idea how the various subsystems will interact with one another.
|
||||
2. We will also try to get issues up on the repository from discussions at the summit so we can continue the discussions during the summit with the broader community.
|
||||
3. Rethinking an extension model. Extending containerd is important for many people. It makes sure that third parties do not have to commit code to the core to gain the functionality that they require. However, it’s not simple in Go and Go 1.8 plugins do not look like the solution. We need to work more on this.
|
||||
|
||||
|
||||
43
docs/historical/reports/2017-03-10.md
Normal file
43
docs/historical/reports/2017-03-10.md
Normal file
@@ -0,0 +1,43 @@
|
||||
# Development Report for Mar 10, 2017
|
||||
|
||||
## Runtime
|
||||
|
||||
This week I (@crosbymichael) have been working on the OCI runtime specifications. Since containerd is to be fully compliant with the OCI specs we need to make sure we reach a 1.0 and have a solid base to build on. We are getting very close on the spec side and have a final rc out for review. If you have time please check it out.
|
||||
|
||||
I have also been working on the `runc` side of things, fixing terminal handling for `exec` and implementing the changes in our go bindings.
|
||||
|
||||
## Reaper
|
||||
|
||||
I have also worked on adding a global process monitor/reaper to containerd. The issues before with having a `SIGCHLD` reaper is that the Go `exec.Cmd` and especially its `Wait` method did not play well together. This would cause races between the reaper doing a `waitpid` and the `exec.Cmd` doing a `Wait`. I think we solved this problem fully now as long as code is written against the `reaper` api. It is a little more of a burden on developers of containerd but makes it much more robust when dealing with processes being reparented to `containerd` and its `shim`.
|
||||
|
||||
## Snapshot Dynamic Registration
|
||||
|
||||
We merged a PR making the snapshot registration dynamic. This allows users to compile containerd with additional shapshotters than the ones we have builtin to the core. Because of the current state of Go 1.8 plugins the best course of action for adding additional runtimes, snapshotters, and other extensions is to modify the `builtins.go` file for the `containerd` command and recompile. Hopefully the short comings will be fixed in later Go releases.
|
||||
|
||||
```go
|
||||
package main
|
||||
|
||||
// register containerd builtins here
|
||||
import (
|
||||
_ "github.com/containerd/containerd/linux"
|
||||
_ "github.com/containerd/containerd/services/content"
|
||||
_ "github.com/containerd/containerd/services/execution"
|
||||
_ "github.com/containerd/containerd/snapshot/btrfs"
|
||||
_ "github.com/containerd/containerd/snapshot/overlay"
|
||||
)
|
||||
```
|
||||
|
||||
## Update on Dates
|
||||
|
||||
We are working towards being feature complete mid April, with a Q2 1.0. The Beta milestone on github should reflect this goal and the dates associated with it.
|
||||
|
||||
After we hit a feature complete state we hope to finish the Docker, swarm, and kube CRI integrations to make sure that we provide the feature set that is required for each, before locking down the APIs.
|
||||
|
||||
## Containerd Summit
|
||||
|
||||
There is going to be another containerd summit at Dockecon this year. I created a document in the repository so that everyone can add discussion points for the breakout sessions. We should have a much larger crowd than the previous summit, therefore, having a few pre-defined discussion points will help. We will still have ad-hoc discussions but it never hurts to be prepared.
|
||||
|
||||
|
||||
## Next
|
||||
|
||||
We are still moving forward so that we can have the distribution, content, and snapshotters all working together over the API. One of the challenges is to make sure things work well over GRPC. We need to be able to support things like `docker build` and making sure that it performs as good or better than what you expect.
|
||||
197
docs/historical/reports/2017-03-17.md
Normal file
197
docs/historical/reports/2017-03-17.md
Normal file
@@ -0,0 +1,197 @@
|
||||
# Development Report for Mar 17, 2017
|
||||
|
||||
## Testing Plan
|
||||
|
||||
Thanks to @gianarb for starting the discussion around a test and CI plan for containerd. We want to make sure that users of containerd can feel secure depending on containerd; having a solid test plan is a must.
|
||||
|
||||
Testing a project like containerd is always a challenge because of the systems that it needs to support. ARM, Windows, Linux, and Power (as well as many more variations) are all examples of platforms that we support today and will need a CI.
|
||||
|
||||
You can view the issue and contribute to the testing plan [here](https://github.com/containerd/containerd/issues/634).
|
||||
|
||||
## Windows Runtime
|
||||
|
||||
Work has started on porting over the Windows execution code. There is still a lot of testing to do after the port but a PR should be coming soon.
|
||||
|
||||
## Metrics
|
||||
|
||||
We started the work to get container level metrics exported over prometheus. You can see the initial output here:
|
||||
|
||||
```
|
||||
containerd_container_blkio_io_service_bytes_recursive_bytes{id="test",major="8",minor="0",op="Async"} 958464
|
||||
containerd_container_blkio_io_service_bytes_recursive_bytes{id="test",major="8",minor="0",op="Read"} 958464
|
||||
containerd_container_blkio_io_service_bytes_recursive_bytes{id="test",major="8",minor="0",op="Sync"} 0
|
||||
containerd_container_blkio_io_service_bytes_recursive_bytes{id="test",major="8",minor="0",op="Total"} 958464
|
||||
containerd_container_blkio_io_service_bytes_recursive_bytes{id="test",major="8",minor="0",op="Write"} 0
|
||||
containerd_container_blkio_io_serviced_recursive_total{id="test",major="8",minor="0",op="Async"} 17
|
||||
containerd_container_blkio_io_serviced_recursive_total{id="test",major="8",minor="0",op="Read"} 17
|
||||
containerd_container_blkio_io_serviced_recursive_total{id="test",major="8",minor="0",op="Sync"} 0
|
||||
containerd_container_blkio_io_serviced_recursive_total{id="test",major="8",minor="0",op="Total"} 17
|
||||
containerd_container_blkio_io_serviced_recursive_total{id="test",major="8",minor="0",op="Write"} 0
|
||||
containerd_container_cpu_kernel_nanoseconds{id="test"} 1e+07
|
||||
containerd_container_cpu_throttle_periods_total{id="test"} 0
|
||||
containerd_container_cpu_throttled_periods_total{id="test"} 0
|
||||
containerd_container_cpu_throttled_time_nanoseconds{id="test"} 0
|
||||
containerd_container_cpu_total_nanoseconds{id="test"} 2.1428791e+07
|
||||
containerd_container_cpu_user_nanoseconds{id="test"} 0
|
||||
containerd_container_hugetlb_failcnt_total{id="test",page="1GB"} 0
|
||||
containerd_container_hugetlb_failcnt_total{id="test",page="2MB"} 0
|
||||
containerd_container_hugetlb_max_bytes{id="test",page="1GB"} 0
|
||||
containerd_container_hugetlb_max_bytes{id="test",page="2MB"} 0
|
||||
containerd_container_hugetlb_usage_bytes{id="test",page="1GB"} 0
|
||||
containerd_container_hugetlb_usage_bytes{id="test",page="2MB"} 0
|
||||
containerd_container_memory_active_anon_bytes{id="test"} 0
|
||||
containerd_container_memory_active_file_bytes{id="test"} 659456
|
||||
containerd_container_memory_cache_bytes{id="test"} 925696
|
||||
containerd_container_memory_dirty_bytes{id="test"} 0
|
||||
containerd_container_memory_hierarchical_memory_limit_bytes{id="test"} 9.223372036854772e+18
|
||||
containerd_container_memory_hierarchical_memsw_limit_bytes{id="test"} 9.223372036854772e+18
|
||||
containerd_container_memory_inactive_anon_bytes{id="test"} 73728
|
||||
containerd_container_memory_inactive_file_bytes{id="test"} 266240
|
||||
containerd_container_memory_kernel_failcnt_total{id="test"} 0
|
||||
containerd_container_memory_kernel_limit_bytes{id="test"} 9.223372036854772e+18
|
||||
containerd_container_memory_kernel_max_bytes{id="test"} 0
|
||||
containerd_container_memory_kernel_usage_bytes{id="test"} 0
|
||||
containerd_container_memory_kerneltcp_failcnt_total{id="test"} 0
|
||||
containerd_container_memory_kerneltcp_limit_bytes{id="test"} 9.223372036854772e+18
|
||||
containerd_container_memory_kerneltcp_max_bytes{id="test"} 0
|
||||
containerd_container_memory_kerneltcp_usage_bytes{id="test"} 0
|
||||
containerd_container_memory_mapped_file_bytes{id="test"} 577536
|
||||
containerd_container_memory_oom_total{id="test"} 0I
|
||||
containerd_container_memory_pgfault_bytes{id="test"} 770
|
||||
containerd_container_memory_pgmajfault_bytes{id="test"} 6
|
||||
containerd_container_memory_pgpgin_bytes{id="test"} 651
|
||||
containerd_container_memory_pgpgout_bytes{id="test"} 407
|
||||
containerd_container_memory_rss_bytes{id="test"} 73728
|
||||
containerd_container_memory_rss_huge_bytes{id="test"} 0
|
||||
containerd_container_memory_swap_failcnt_total{id="test"} 0
|
||||
containerd_container_memory_swap_limit_bytes{id="test"} 9.223372036854772e+18
|
||||
containerd_container_memory_swap_max_bytes{id="test"} 1.527808e+06
|
||||
containerd_container_memory_swap_usage_bytes{id="test"} 999424
|
||||
containerd_container_memory_total_active_anon_bytes{id="test"} 0
|
||||
containerd_container_memory_total_active_file_bytes{id="test"} 659456
|
||||
containerd_container_memory_total_cache_bytes{id="test"} 925696
|
||||
containerd_container_memory_total_dirty_bytes{id="test"} 0
|
||||
containerd_container_memory_total_inactive_anon_bytes{id="test"} 73728
|
||||
containerd_container_memory_total_inactive_file_bytes{id="test"} 266240
|
||||
containerd_container_memory_total_mapped_file_bytes{id="test"} 577536
|
||||
containerd_container_memory_total_pgfault_bytes{id="test"} 770
|
||||
containerd_container_memory_total_pgmajfault_bytes{id="test"} 6
|
||||
containerd_container_memory_total_pgpgin_bytes{id="test"} 651
|
||||
containerd_container_memory_total_pgpgout_bytes{id="test"} 407
|
||||
containerd_container_memory_total_rss_bytes{id="test"} 73728
|
||||
containerd_container_memory_total_rss_huge_bytes{id="test"} 0
|
||||
containerd_container_memory_total_unevictable_bytes{id="test"} 0
|
||||
containerd_container_memory_total_writeback_bytes{id="test"} 0
|
||||
containerd_container_memory_unevictable_bytes{id="test"} 0
|
||||
containerd_container_memory_usage_failcnt_total{id="test"} 0
|
||||
containerd_container_memory_usage_limit_bytes{id="test"} 9.223372036854772e+18
|
||||
containerd_container_memory_usage_max_bytes{id="test"} 1.527808e+06
|
||||
containerd_container_memory_usage_usage_bytes{id="test"} 999424
|
||||
containerd_container_memory_writeback_bytes{id="test"} 0
|
||||
containerd_container_per_cpu_nanoseconds{cpu="0",id="test"} 7.530139e+06
|
||||
containerd_container_per_cpu_nanoseconds{cpu="1",id="test"} 4.586408e+06
|
||||
containerd_container_per_cpu_nanoseconds{cpu="2",id="test"} 5.076059e+06
|
||||
containerd_container_per_cpu_nanoseconds{cpu="3",id="test"} 4.236185e+06
|
||||
containerd_container_pids_current{id="test"} 1
|
||||
containerd_container_pids_limit{id="test"} 0
|
||||
```
|
||||
|
||||
The `id` label will be the id of the container so users can filter on the metrics for only the containers that they care about.
|
||||
|
||||
The frequency of metric collection is configurable via the prometheus scrape time. Every time the `/metrics` API is hit, that is when container metrics are collected. There is no internal timer in containerd, you only pay the cost of collecting metrics when you are asking for them. If you never ask for metrics the collection never happens.
|
||||
|
||||
There should be a PR up soon so that we can discuss the metrics and label names.
|
||||
|
||||
## Image Pull
|
||||
|
||||
* https://github.com/containerd/containerd/pull/640
|
||||
|
||||
We now have a proof of concept of end to end pull. Up to this point, the
|
||||
relationship between subsystems has been somewhat theoretical. We now leverage
|
||||
fetching, the snapshot drivers, the rootfs service, image metadata and the
|
||||
execution service, validating the proposed model for containerd. There are a
|
||||
few caveats, including the need to move some of the access into GRPC services,
|
||||
but the basic components are there.
|
||||
|
||||
The first command we will cover here is `dist pull`. This is the analog of
|
||||
`docker pull` and `git pull`. It performs a full resource fetch for an image
|
||||
and unpacks the root filesystem into the snapshot drivers. An example follows:
|
||||
|
||||
``` console
|
||||
$ sudo ./bin/dist pull docker.io/library/redis:latest
|
||||
docker.io/library/redis:latest: resolved |++++++++++++++++++++++++++++++++++++++|
|
||||
manifest-sha256:4c8fb09e8d634ab823b1c125e64f0e1ceaf216025aa38283ea1b42997f1e8059: done |++++++++++++++++++++++++++++++++++++++|
|
||||
layer-sha256:3b281f2bcae3b25c701d53a219924fffe79bdb74385340b73a539ed4020999c4: done |++++++++++++++++++++++++++++++++++++++|
|
||||
config-sha256:e4a35914679d05d25e2fccfd310fde1aa59ffbbf1b0b9d36f7b03db5ca0311b0: done |++++++++++++++++++++++++++++++++++++++|
|
||||
layer-sha256:4b7726832aec75f0a742266c7190c4d2217492722dfd603406208eaa902648d8: done |++++++++++++++++++++++++++++++++++++++|
|
||||
layer-sha256:338a7133395941c85087522582af182d2f6477dbf54ba769cb24ec4fd91d728f: done |++++++++++++++++++++++++++++++++++++++|
|
||||
layer-sha256:83f12ff60ff1132d1e59845e26c41968406b4176c1a85a50506c954696b21570: done |++++++++++++++++++++++++++++++++++++++|
|
||||
layer-sha256:693502eb7dfbc6b94964ae66ebc72d3e32facd981c72995b09794f1e87bac184: done |++++++++++++++++++++++++++++++++++++++|
|
||||
layer-sha256:622732cddc347afc9360b4b04b46c6f758191a1dc73d007f95548658847ee67e: done |++++++++++++++++++++++++++++++++++++++|
|
||||
layer-sha256:19a7e34366a6f558336c364693df538c38307484b729a36fede76432789f084f: done |++++++++++++++++++++++++++++++++++++++|
|
||||
elapsed: 1.6 s total: 0.0 B (0.0 B/s)
|
||||
INFO[0001] unpacking rootfs
|
||||
```
|
||||
|
||||
Note that we haven't integrated rootfs unpacking into the status output, but we
|
||||
pretty much have what is in docker today (:P). We can see the result of our
|
||||
pull with the following:
|
||||
|
||||
```console
|
||||
$ sudo ./bin/dist images
|
||||
REF TYPE DIGEST SIZE
|
||||
docker.io/library/redis:latest application/vnd.docker.distribution.manifest.v2+json sha256:4c8fb09e8d634ab823b1c125e64f0e1ceaf216025aa38283ea1b42997f1e8059 1.8 kB
|
||||
```
|
||||
|
||||
The above shows that we have an image called "docker.io/library/redis:latest"
|
||||
mapped to the given digest marked with a specific format. We get the size of
|
||||
the manifest right now, not the full image, but we can add more as we need it.
|
||||
For the most part, this is all that is needed, but a few tweaks to the model
|
||||
for naming may need to be added. Specifically, we may want to index under a few
|
||||
different names, including those qualified by hash or matched by tag versions.
|
||||
We can do more work in this area as we develop the metadata store.
|
||||
|
||||
The name shown above can then be used to run the actual container image. We can
|
||||
do this with the following command:
|
||||
|
||||
```console
|
||||
$ sudo ./bin/ctr run --id foo docker.io/library/redis:latest /usr/local/bin/redis-server
|
||||
1:C 17 Mar 17:20:25.316 # Warning: no config file specified, using the default config. In order to specify a config file use /usr/local/bin/redis-server /path/to/redis.conf
|
||||
1:M 17 Mar 17:20:25.317 * Increased maximum number of open files to 10032 (it was originally set to 1024).
|
||||
_._
|
||||
_.-``__ ''-._
|
||||
_.-`` `. `_. ''-._ Redis 3.2.8 (00000000/0) 64 bit
|
||||
.-`` .-```. ```\/ _.,_ ''-._
|
||||
( ' , .-` | `, ) Running in standalone mode
|
||||
|`-._`-...-` __...-.``-._|'` _.-'| Port: 6379
|
||||
| `-._ `._ / _.-' | PID: 1
|
||||
`-._ `-._ `-./ _.-' _.-'
|
||||
|`-._`-._ `-.__.-' _.-'_.-'|
|
||||
| `-._`-._ _.-'_.-' | http://redis.io
|
||||
`-._ `-._`-.__.-'_.-' _.-'
|
||||
|`-._`-._ `-.__.-' _.-'_.-'|
|
||||
| `-._`-._ _.-'_.-' |
|
||||
`-._ `-._`-.__.-'_.-' _.-'
|
||||
`-._ `-.__.-' _.-'
|
||||
`-._ _.-'
|
||||
`-.__.-'
|
||||
|
||||
1:M 17 Mar 17:20:25.326 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
|
||||
1:M 17 Mar 17:20:25.326 # Server started, Redis version 3.2.8
|
||||
1:M 17 Mar 17:20:25.326 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
|
||||
1:M 17 Mar 17:20:25.326 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
|
||||
1:M 17 Mar 17:20:25.326 * The server is now ready to accept connections on port 6379
|
||||
```
|
||||
|
||||
Wow! So, now we are running `redis`!
|
||||
|
||||
There are still a few things to work out. Notice that we have to specify the
|
||||
command as part of the arguments to `ctr run`. This is because are not yet
|
||||
reading the image config and converting it to an OCI runtime config. With the
|
||||
base laid in this PR, adding such functionality should be straightforward.
|
||||
|
||||
While this is a _little_ messy, this is great progress. It should be easy
|
||||
sailing from here.
|
||||
|
||||
|
||||
|
||||
64
docs/historical/reports/2017-03-24.md
Normal file
64
docs/historical/reports/2017-03-24.md
Normal file
@@ -0,0 +1,64 @@
|
||||
# Development Report for Mar 24, 2017
|
||||
|
||||
## Container Level Metrics
|
||||
|
||||
* https://github.com/containerd/containerd/pull/646
|
||||
|
||||
We merged a PR to add our first pass of container level metrics to our
|
||||
prometheus output in containerd. We still have to review the metric names and
|
||||
structure before having something we are all comfortable supporting in the long
|
||||
run but we now have comprehensive metrics for all containers running on
|
||||
containerd.
|
||||
|
||||
## Distribution
|
||||
|
||||
After hitting a major milestone of getting a proof of concept for end to end
|
||||
pull and run, this week the focus was on getting that code in the right place
|
||||
and figure out where the implementation gaps are.
|
||||
|
||||
### Image to OCI Spec
|
||||
|
||||
* https://github.com/containerd/containerd/pull/660
|
||||
|
||||
We merged support for getting an image config that was pulled off of a registry
|
||||
and generating a spec based on the image properties in the `ctr` command. This
|
||||
will let you pull images off of a registry and run them based on the config and
|
||||
how the image was built.
|
||||
|
||||
Its very simple at the moment but will will be porting over the default spec
|
||||
and generation code from Docker soon into a package that can be easily consumed
|
||||
by clients of containerd.
|
||||
|
||||
You can test this by running:
|
||||
|
||||
```console
|
||||
bash
|
||||
sudo dist pull docker.io/library/redis:alpine
|
||||
sudo ctr run --id redis -t docker.io/library/redis:alpine
|
||||
```
|
||||
|
||||
### Image handlers
|
||||
|
||||
* https://github.com/containerd/containerd/pull/638
|
||||
|
||||
We refactored the fetch command into a more generic image handler interface. As
|
||||
we look forward to supporting the full oci image spec as well as the Docker
|
||||
distribution specifications, we are removing any opinionated code to make
|
||||
distribution as generalized and efficient as possible.
|
||||
|
||||
### Image listing now with full image size
|
||||
|
||||
```console
|
||||
$ dist images
|
||||
REF TYPE DIGEST SIZE
|
||||
docker.io/library/redis:latest application/vnd.docker.distribution.manifest.v2+json sha256:1b358a2b0dc2629af3ed75737e2f07e5b3408eabf76a8fa99606ec0c276a93f8 71.0 MiB
|
||||
```
|
||||
|
||||
## Snapshot driver improvements
|
||||
|
||||
* https://github.com/containerd/containerd/pull/635
|
||||
|
||||
The `overlay` and `btrfs` driver implementations are now fully implemented and
|
||||
share an implementation for metadata storage. This new metadata storage package
|
||||
allows not only making snapshot drivers easier, but allow us to focus on making
|
||||
our existing drivers more resilient and stable once.
|
||||
88
docs/historical/reports/2017-04-28.md
Normal file
88
docs/historical/reports/2017-04-28.md
Normal file
@@ -0,0 +1,88 @@
|
||||
# Development Report for April 28, 2017
|
||||
|
||||
Sorry for the slow reports lately. Last week was Dockercon and many people inside Docker and external contributors where in Austin for the week. It was a long week and we didn't get much work done on the project but it was good meeting everyone IRL and putting a face to the github handle. Also, thank you to everyone that stayed a day late to attend the summit on Thursday. You can see the summit notes at the bottom of this report.
|
||||
|
||||
## New Projects to the Org.
|
||||
|
||||
We are looking to move some of our dependencies used by containerd into the org. We want to have a simple maintainer model for containerd and any other repository that is added to the org. Basically, if you are a maintainer of containerd, you are a maintainer on all of the repos. We don't want to have separate maintainers for the various parts as we should all share the responsibility for quality and stability for the project as a whole.
|
||||
|
||||
We have an issue on github discussing the various projects that we want to add. If you have any feedback please let us know on the issue below.
|
||||
|
||||
[projects!](https://github.com/containerd/containerd/issues/772)
|
||||
[process](https://github.com/containerd/containerd/pull/779)
|
||||
|
||||
## Darwin/BSD Port
|
||||
|
||||
Justin has begun work on porting over containerd to Darwin and BSD. `ctr` compiles and there were a few changes in terminal handling that had to be made but overall its going fairly smooth. You can ping @justincormack for more information or if you are interested in working on something like a BSD jails port of `runc`.
|
||||
|
||||
## Introspection
|
||||
|
||||
@AkihiroSuda has been working on introspection RPCs to make sure consumers know the services and features that they are interacting with. There is still a lot to consider when exposing service versions, containerd versions and functionality; if you have any input you can take a look at the current PR [here](https://github.com/containerd/containerd/pull/776).
|
||||
|
||||
## Expanded Registry Support
|
||||
|
||||
@dmcgowan has been working on expanding registry support with authentication in [783](https://github.com/containerd/containerd/pull/783). This expands registry support for gcr.io, Dockerhub, and local registries.
|
||||
|
||||
|
||||
## Task List for week of May 1
|
||||
|
||||
So, now that we are all back from Dockercon and not rested at all, what's on the agenda for next week?
|
||||
|
||||
* Docker Integration Pass
|
||||
- Replace the current v0.2.x branch of containerd in Docker for execution with the 1.0 execution service
|
||||
* Converge Event Handling
|
||||
* Move new projects into containerd organization
|
||||
* New Layer creation for supporting build
|
||||
* Continue work on metadata service
|
||||
- container
|
||||
- layer
|
||||
- image
|
||||
- namespace support
|
||||
|
||||
If are you looking for areas to contribute to containerd just take a pass through the issue tracker. There are issues such as [770](https://github.com/containerd/containerd/issues/770) that could use some help in solving as well as other. Also feel free to stop by the slack channel for containerd to chat and find ideas on what needs to be developed next.
|
||||
|
||||
## containerd Summit Notes
|
||||
|
||||
Here are the notes from the containerd summit that was held after Dockercon. If there are important to you and you would like to start working on a feature, fix, request, please go ahead and open an issue on github. Just remember, we shouldn't have to have a summit to get things into containerd. Open issues, discuss with other members of the community and write the code ;)
|
||||
|
||||
* “Since containerd is one of the bottom bricks in the stack, how can we setup automated integration tests for consumers of containerd?”
|
||||
- Looking for others to provide compute (testing within Docker, K8s, Microsoft)
|
||||
- Would we publish results of integration tests, not likely per PR
|
||||
- K8s had a similar problem, solved by pushing builds to buckets and creating dashboards from results pushed back to buckets
|
||||
- Need tests for projects containerd is integrated with (Windows, runv), testing containerd connection to shims
|
||||
* “We'd like to propose an Authorization plugin to containerd that would allow an external component to police events like container start & stop (and have a discussion about the best way to go about it)”
|
||||
- Need to look into how it could be implemented with containerd
|
||||
- Possible to be done by MITM the GRPC socket
|
||||
- Currently seen as out of scope for server, can be done via client (or proxy), we could build a client package to support this so it does not have to be implemented by each integrator.
|
||||
- Currently only unix socket allowed which prevents unauthorized access
|
||||
* “Should containerd provide image filesystem metrics? If yes, what metrics should be included? How to implement that?”
|
||||
- Previously discussed, yes, PR to provide this information through the snapshot interface.
|
||||
- Figuring out what is being shared, could figure out it from mapping layers to images
|
||||
* “Support for disk quotas: How? What is the role of containerd? How is it going to be integrated with volume managers that want to be in the same quota group?”
|
||||
- Expanding question, “will each snapshot driver have a quota system? What is the granularity, per container, global?”
|
||||
- Stephen’s hope is that quota management can be done based on the returned mounts, otherwise need to figure out quota abstraction
|
||||
- Could we rely on work being done for volumes with CSI
|
||||
- Need to open up issue to discuss project quotas and general snapshot quotas
|
||||
* “Checkpoint/Restore: how can we support more use cases? One of the big issues here is the large number of options that can be passed to CRIU.”
|
||||
- Not currently in Containerd, requires runc update as well
|
||||
- Need to figure out a way to pass in options opaquely
|
||||
- Needs to be discuss with Michael for what best solution for runc will be
|
||||
- Today caller must know runtime since runtime spec is passed opaquely on create to runtime
|
||||
* “How to support multi-OS docker images, for example, Linux Vs Windows using one graph driver plugin properly?”
|
||||
- From looking into implementing linux containers on HyperV
|
||||
- Missing options when creating rootfs to specify the snapshot driver
|
||||
- Containerd will support running multiple snapshotter
|
||||
- Timeline, Docker will first use update runtime after stabilizing
|
||||
* “Containerd is acting as init system in LinuxKit, is there a plan to support what init systems support”
|
||||
- Containerd was not implemented as an init system, we are not trying to compete with it. A higher level init system could be implemented with it.
|
||||
* “Will containerd be able to leverage runtime plugins specified within Docker?”
|
||||
- Docker would have to orchestrate re-configuring containerd with the new runtime
|
||||
- Currently cannot hotswap plugins, runtime and shim are built in
|
||||
- Would require defining plugin interface or getting go 1.8 plugins working
|
||||
- We could have a plugin which connects to a grpc address
|
||||
- Resolved in rocket by first allowing downloading a runtime and running it as privileged
|
||||
- Need to open up an issue to discuss this
|
||||
* “What do we want from orchestrator who are looking at integration?”
|
||||
- Feedback is biggest
|
||||
- Currently targeting K8s and SwarmKit to keep API grounded
|
||||
- Starting PoCs now, we have already made changes based on early feedback
|
||||
47
docs/historical/reports/2017-05-05.md
Normal file
47
docs/historical/reports/2017-05-05.md
Normal file
@@ -0,0 +1,47 @@
|
||||
# Development Report for May 05, 2017
|
||||
|
||||
## runc and runtime-spec 1.0 final
|
||||
|
||||
This week we have been working on getting Docker integrated with the final RCs for the OCI runtime-spec and runc. We currently have a [PR open on Docker](https://github.com/moby/moby/pull/33007) that is ready for review.
|
||||
|
||||
This is important for the containerd project because we need a stable runtime like runc and have a spec that is not constantly changing so that people can begin integrating with containerd and know that the APIs we expose and APIs/specs that we depend on are stable for the long term.
|
||||
|
||||
## New Projects
|
||||
|
||||
We finished moving a few of our external dependencies into the containerd organization this week. There were a few projects that we built outside but wanted to bring these under the project to ensure that our dependencies are held to the same standards as the rest of our codebase.
|
||||
|
||||
### go-runc
|
||||
|
||||
[go-runc](https://github.com/containerd/go-runc)
|
||||
|
||||
This project contains the runc bindings that we consume in containerd to interface with runc and other OCI runtimes. It lets us interact with the binary and handles many of the common options like working with the console socket and other IO operations.
|
||||
|
||||
### fifo
|
||||
|
||||
[fifo](https://github.com/containerd/fifo)
|
||||
|
||||
This package contains helpers for handling fifos. Fifos are a little more complex than regular pipes and sometimes requires special handling and blocking semantics depending on the flags used with opening the fifo on either end. This package helps to handle many of the common use cases that we use in containerd.
|
||||
|
||||
### console
|
||||
|
||||
[console](https://github.com/containerd/console)
|
||||
|
||||
The console package is a refresh of the `term` package from Docker. It provides a cleaner API for working with the current console for a program or creating new terminals and keeping the flags in sync for proxying reads and writes between the two sides of the console.
|
||||
|
||||
### cgroups
|
||||
|
||||
[cgroups](https://github.com/containerd/cgroups)
|
||||
|
||||
The cgroups package is currently used in containerd for collecting stats from cgroups that are created for a container. It exposes a package for exporting cgroup level metrics to prometheus for containers.
|
||||
|
||||
### btrfs
|
||||
|
||||
[btrfs](https://github.com/containerd/btrfs)
|
||||
|
||||
The btrfs package handles interfacing with btrfs for our snapshotter. It binds to the btrfs C library to create subvolumes and handle any other interaction with the filesystem.
|
||||
|
||||
### continuity
|
||||
|
||||
[continuity](https://github.com/containerd/continuity)
|
||||
|
||||
continuity provides a transport agnostic filesystem metadata manifest. This allows us to work with filesystems at the file level instead of interacting with a "layer". We also intend to concentrate a rich set of file system utility packages for use in containerd.
|
||||
75
docs/historical/reports/2017-05-19.md
Normal file
75
docs/historical/reports/2017-05-19.md
Normal file
@@ -0,0 +1,75 @@
|
||||
# Development Report for May 19, 2017
|
||||
|
||||
## Containers and Tasks
|
||||
|
||||
Moving more functionality into containerd means more requirements from users. One thing that we have ran into was the disconnect of what our Container model is and what users expect, `docker create;docker start;docker rm` vs a container that is destroyed when it exits.
|
||||
|
||||
To users, containers are more of a metadata object that resources(rw, configuration) and information(state, last exit status) are attached to. We have been reworking what we call a "container" today to be called a "task" and a Container metadata object. The task only has runtime state: a pid, namespaces, cgroups, etc. A container has an id, root filesystem, configuration, and other metadata from a user.
|
||||
|
||||
Managing static state and runtime state in the same object is very tricky so we choose to keep execution and metadata separate. We are hoping to not cause more confusion with this additional task object. You can see a mockup of a client interacting with a container and how the task is handled below:
|
||||
|
||||
```go
|
||||
container, err := client.NewContainer(id, spec, rootfs)
|
||||
|
||||
task, err := container.CreateTask(containerd.Stdio())
|
||||
task.Start()
|
||||
task.Pid()
|
||||
task.Kill(syscall.SIGKILL)
|
||||
|
||||
container.Delete()
|
||||
```
|
||||
|
||||
[container metadata PR](https://github.com/containerd/containerd/pull/859)
|
||||
|
||||
## Checkpoint && Restore
|
||||
|
||||
There is a PR open for checkpoint and restore within containerd. We needed to port over this code from the existing containerd branch but we are able to do more in terms of functionality since we have filesystem and distribution built in. The overall functionality for checkpoint/restore is that you can still checkpoint containers but instead of having a directory on disk with the checkpointed data, it is checkpointed to the content store.
|
||||
|
||||
Having checkpoints in the content store allows you to push checkpoints that include the container's memory data, rw layer with the file contents that the container has written, and other resources like bind(volumes) to a registry that is running so that you can live migrate containers to other hosts in your cluster. And you can do all this with your existing registries; no other services required to migrate containers around your data center.
|
||||
|
||||
[checkpoint restore PR](https://github.com/containerd/containerd/pull/862)
|
||||
|
||||
## Snapshot and Diff Service
|
||||
|
||||
The RootFS service has been removed and replaced with a snapshot service and
|
||||
diff service. The snapshot service provides access to all snapshotter methods
|
||||
and allows clients to operate directly against the snapshotter interface. This
|
||||
enables clients to prepare RW layers as it could before with the RootFS service,
|
||||
but also has full access to commit or remove those snapshots directly. The diff
|
||||
service provides 2 methods, extract and diff. The extract takes in a set of
|
||||
mounts and a descriptor to a layer tar, mounts, then extracts the tar from the
|
||||
content store into the mount. The diff service takes in 2 sets of mounts,
|
||||
computes the diff, sends the diff to the content store and returns the content
|
||||
descriptor for the computed diff. The diff service is designed to allow clients
|
||||
to pull content into snapshotters without requiring the privileges to mount and
|
||||
handle root-owned files.
|
||||
|
||||
[Snapshot and Diff Service PR](https://github.com/containerd/containerd/pull/849)
|
||||
[Diff implementation PR](https://github.com/containerd/containerd/pull/863)
|
||||
|
||||
## Roadmap for May and June
|
||||
|
||||
We have a few remaining tasks to finish up in the next few weeks before we consider containerd to be feature complete.
|
||||
|
||||
### Namespaces
|
||||
|
||||
We want the ability to have a single containerd running on a system but allow multiple consumers like Docker, swarmkit, and Kube all consuming the same containerd daemon without stepping on each others containers and images. We will do this with namespaces.
|
||||
|
||||
```go
|
||||
client, err := containerd.NewClient(address, namespace)
|
||||
```
|
||||
|
||||
### Push Support
|
||||
|
||||
Right now we have pull support for fetching content but we need to have the ability to produce content in terms of builds and pushing checkpoints.
|
||||
|
||||
### Daemon Wide Events
|
||||
|
||||
We need to finish the events service so that consumers have a single service that they can consume to handle events produced by containerd, images, and their containers.
|
||||
|
||||
|
||||
After we have these features completed, the majority of the time in June will be spent reviewing the API and client packages so that we have an API we are comfortable supporting in an LTS release.
|
||||
|
||||
We currently have a few integrations in the works for using containerd with Kube, Swarmkit, and Docker and in June we hope to complete some of these integrations and have testing infrastructure setup for these.
|
||||
|
||||
So that is the basic plan for the next 6 weeks or so. Feature complete with the last remaining tasks then iterate on our API and clients to make sure we have the features that consumers need and an API that we can support for the long term.
|
||||
131
docs/historical/reports/2017-05-26.md
Normal file
131
docs/historical/reports/2017-05-26.md
Normal file
@@ -0,0 +1,131 @@
|
||||
# Development Report for May 26, 2017
|
||||
|
||||
## Project Status
|
||||
|
||||
Overall, containerd is very close to being feature complete. The final features like namespace support are being worked on. We have a couple consumers that are working on containerd implementations such as a CRI for kube, Docker execution, Swarmkit, and Linuxkit consuming containerd.
|
||||
|
||||
We hope to be feature complete by the end of this month. That will leave the month of June to focus on quality of the API and making sure that client needs are met. We have some work on error codes and reviewing our protos to make sure we have an API that we can support for the life of containerd 1.x.
|
||||
|
||||
# Image push
|
||||
|
||||
We added a `push` and `push-object` command to the dist tool. The `push` command
|
||||
can be used to push a manifest and all the related objects to a registry. The
|
||||
`push-object` can push individual blobs from the content store to a registry.
|
||||
|
||||
The push command does not require first tagging an image, just specify the
|
||||
remote name of the image as well as the local name. If the remote and local
|
||||
name are the same, only one argument is needed. The remote name may exclude
|
||||
the object identifier (Docker-style tag or digest), in which case the manifest
|
||||
may be repulled using the manifest digest as the object identifier. Push only
|
||||
handles pushing images which already exist, it will not create an image manifest
|
||||
if one does not exist. We will add a separate command for creating new images in
|
||||
the future.
|
||||
|
||||
Example pushing image to a local registry.
|
||||
```
|
||||
$ dist image list
|
||||
REF TYPE DIGEST SIZE
|
||||
docker.io/library/ubuntu:latest application/vnd.docker.distribution.manifest.v2+json sha256:382452f82a8bbd34443b2c727650af46aced0f94a44463c62a9848133ecb1aa8 44.7 MiB
|
||||
$ dist push localhost:5000/ubuntu docker.io/library/ubuntu:latest
|
||||
manifest-sha256:382452f82a8bbd34443b2c727650af46aced0f94a44463c62a9848133ecb1aa8: done |++++++++++++++++++++++++++++++++++++++|
|
||||
layer-sha256:cf9722e506aada1109f5c00a9ba542a81c9e109606c01c81f5991b1f93de7b66: done |++++++++++++++++++++++++++++++++++++++|
|
||||
layer-sha256:b6f892c0043b37bd1834a4a1b7d68fe6421c6acbc7e7e63a4527e1d379f92c1b: done |++++++++++++++++++++++++++++++++++++++|
|
||||
layer-sha256:55010f332b047687e081a9639fac04918552c144bc2da4edb3422ce8efcc1fb1: done |++++++++++++++++++++++++++++++++++++++|
|
||||
layer-sha256:3deef3fcbd3072b45771bd0d192d4e5ff2b7310b99ea92bce062e01097953505: done |++++++++++++++++++++++++++++++++++++++|
|
||||
config-sha256:ebcd9d4fca80e9e8afc525d8a38e7c56825dfb4a220ed77156f9fb13b14d4ab7: done |++++++++++++++++++++++++++++++++++++++|
|
||||
layer-sha256:2955fb827c947b782af190a759805d229cfebc75978dba2d01b4a59e6a333845: done |++++++++++++++++++++++++++++++++++++++|
|
||||
elapsed: 6.5 s total: 44.7 M (6.9 MiB/s)
|
||||
```
|
||||
|
||||
[#886 Add dist push-object](https://github.com/containerd/containerd/pull/886)
|
||||
|
||||
[#911 Add dist push](https://github.com/containerd/containerd/pull/911)
|
||||
|
||||
## Client
|
||||
|
||||
We started work on the initial containerd client. Since we will have multiple consumers using the API, we want to reduce duplicated code for users of containerd. Another goal for our client is to give users a stable client API that does not feel like they are working with a remote daemon. Containers are already complex for most people but this does not need to be the case. Building a platform on top of containerd should be effortless. We want to handle the low level system aspects so that you can build a platform for your users and not worry about the underlying system, unless you want to.
|
||||
|
||||
|
||||
### Creating a client for Docker
|
||||
|
||||
```go
|
||||
client, err := containerd.New(address, containerd.WithNamespace("docker"))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer client.Close()
|
||||
```
|
||||
|
||||
### Pulling an Image from DockerHub
|
||||
|
||||
```go
|
||||
// pull && unpack the image to your snapshot ( overlayfs default ) of choice
|
||||
image, err := client.Pull(ctx, "docker.io/library/redis:alpine", containerd.WithPullUnpack)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
```
|
||||
|
||||
### Generate an OCI runtime specification
|
||||
|
||||
```go
|
||||
// generate the spec based on the image that we pulled
|
||||
spec, err := containerd.GenerateSpec(containerd.WithImageConfig(ctx, image))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
```
|
||||
|
||||
### Create a new container based on the image
|
||||
|
||||
```go
|
||||
// create the container with a persistent ReadWrite layer based on the image and spec
|
||||
container, err := client.NewContainer(ctx, "redis", spec, containerd.WithNewRootFS("redis-rootfs", image))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer container.Delete(ctx)
|
||||
```
|
||||
|
||||
### Run the container
|
||||
|
||||
```go
|
||||
// use the current process's stdio
|
||||
task, err := container.NewTask(ctx, containerd.Stdio)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer task.Delete(ctx)
|
||||
|
||||
pid := task.Pid()
|
||||
|
||||
// start the redis process
|
||||
if err := task.Start(ctx); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
task.Kill(ctx, syscall.SIGTERM)
|
||||
|
||||
status, err := task.Wait(ctx)
|
||||
|
||||
os.Exit(status)
|
||||
```
|
||||
|
||||
This is a very simple but common usecase for containerd. Pull an image and run a container based on that image. The client lets you write code, use `defer` statements and any other logic you want without having to do complex filesystem operations, deal with tar files or setting up root filesystems. The API is split up and allows you to look into the lifecycle of an executing container without the need to register hooks and provide callbacks.
|
||||
|
||||
Over the next few weeks we will be expanding the client to support the full containerd feature set.
|
||||
|
||||
## Testing
|
||||
|
||||
With the initial client merged into the code base we are also starting work on integration tests. We have full integration tests running on travis ci. You don't have to mess with any other test frameworks to write tests for containerd, just the standard Go testing package.
|
||||
|
||||
We also designed the client to allow mocking of calls so that our tests can be mocked out or users of our client to easily mock their own code for testing.
|
||||
|
||||
You can view the PR [here](https://github.com/containerd/containerd/pull/910).
|
||||
|
||||
|
||||
## What's Next?
|
||||
|
||||
We still need to finish Events. This is one of the last major features that we need before we consider containerd feature complete. The namespace work is in progress with the initial service spec'd.
|
||||
|
||||
Within the next few weeks we should have namespaces done, a complete client, and hopefully a stable events service.
|
||||
35
docs/historical/reports/2017-06-09.md
Normal file
35
docs/historical/reports/2017-06-09.md
Normal file
@@ -0,0 +1,35 @@
|
||||
# Development Report for June 09, 2017
|
||||
|
||||
## Docker Schema 1 Pull
|
||||
|
||||
We added support for pulling Docker images which use the schema1 manifest
|
||||
format in the containerd client. This allows pulling images from older
|
||||
registries as well as images pushed from older versions of Docker. With this
|
||||
change any image from the Docker Hub or registry supporting the v2 API can be
|
||||
pulled. As part of our commitment to support OCI images, the schema 1 images
|
||||
pulled by the client are converted to OCI images before being stored by
|
||||
containerd. The containerd client will only support pushing these images as
|
||||
OCI.
|
||||
|
||||
[#968 Schema 1 pull](https://github.com/containerd/containerd/pull/968)
|
||||
|
||||
## Namespaces for multitenancy
|
||||
|
||||
One of the goals of containerd is to support multiple consumers. The ability to have swarm, docker, kube, and more all running on the same system and using the same containerd without having naming and actions conflict with one another. We have the namespace API merged in and most of the underlying components updated to support namespaces. You can view the namespace PR below for more information on the functionality of namespaces.
|
||||
|
||||
[namespaces](https://github.com/containerd/containerd/pull/963)
|
||||
|
||||
## Client Updates
|
||||
|
||||
We added updates to the client to support attach, checkpoint and restore, exec of additional processes, and fixes. We also ported over the `ctr` and `dist` tools to use the client this week.
|
||||
|
||||
[exec](https://github.com/containerd/containerd/pull/940)
|
||||
[tty](https://github.com/containerd/containerd/pull/948)
|
||||
[checkpoint](https://github.com/containerd/containerd/pull/958)
|
||||
[attach](https://github.com/containerd/containerd/pull/976)
|
||||
|
||||
# What's Next?
|
||||
|
||||
We only have a few features left to implement, such as [events](https://github.com/containerd/containerd/pull/956), for our 1.0 release. The rest of the month we are working on usability, bug fixes, and stability.
|
||||
|
||||
We will be helping out with integrations of containerd into other platforms to make sure that everyone has the functionality that they need. If you run into any problems please open an issue on github. Also Pull Requests for the problems you are having are welcome. If you have any question or help working on a change, stop by the #containerd slack channel as everyone is friendly and helpful.
|
||||
173
docs/historical/reports/2017-06-23.md
Normal file
173
docs/historical/reports/2017-06-23.md
Normal file
@@ -0,0 +1,173 @@
|
||||
# Development Report for June 23, 2017
|
||||
|
||||
## Events Service
|
||||
|
||||
We added an events service for getting events across various services including
|
||||
containers, content, execution, images, namespaces, and snapshots. Additionally
|
||||
we added `ctr events` to view events emitted by the service.
|
||||
|
||||
```
|
||||
$ ctr events
|
||||
2017-06-23 23:21:30.271802153 +0000 UTC /snapshot/prepare key=registry parent=sha256:dc22a13eb565d14bfe2b16f6fa731a05da0eeff02a52059c7b59cdc2c232a2b2
|
||||
2017-06-23 23:21:30.28045193 +0000 UTC /containers/create id=registry image=docker.io/library/registry:latest runtime=&ContainerCreate_Runtime{Name:io.containerd.runtime.v1.linux,Options:map[string]string{},}
|
||||
2017-06-23 23:21:30.347842352 +0000 UTC /runtime/task-create id=registry type=CREATE pid=30411 status=0 exited=0001-01-01 00:00:00 +0000 UTC
|
||||
2017-06-23 23:21:30.355290368 +0000 UTC /runtime/create id=registry bundle=/var/lib/containerd/io.containerd.runtime.v1.linux/default/registry rootfs=type=overlay:src=overlay checkpoint=
|
||||
2017-06-23 23:21:30.362012776 +0000 UTC /tasks/create id=registry
|
||||
2017-06-23 23:21:30.369742117 +0000 UTC /runtime/task-start id=registry type=START pid=30411 status=0 exited=0001-01-01 00:00:00 +0000 UTC
|
||||
2017-06-23 23:21:30.369793151 +0000 UTC /tasks/start id=registry
|
||||
2017-06-23 23:21:40.169884207 +0000 UTC /runtime/task-exit id=registry type=EXIT pid=30411 status=130 exited=2017-06-23 23:21:40.16962998 +0000 UTC
|
||||
2017-06-23 23:21:40.185850194 +0000 UTC /runtime/delete id=registry runtime=io.containerd.runtime.v1.linux status=130 exited=2017-06-23 23:21:40.16962998 +0000 UTC
|
||||
2017-06-23 23:21:40.225633455 +0000 UTC /tasks/delete id=registry pid=30411 status=130
|
||||
2017-06-23 23:21:42.053154149 +0000 UTC /snapshot/remove key=registry
|
||||
2017-06-23 23:21:42.061579495 +0000 UTC /containers/delete id=registry
|
||||
```
|
||||
|
||||
[#956: Events service](https://github.com/containerd/containerd/pull/956)
|
||||
|
||||
## Filters
|
||||
|
||||
We added the syntax to use for filtration of items over the containerd API. The
|
||||
`filter` package defines a syntax and parser that can be used across types and
|
||||
use cases in a uniform manner. This will be used commonly across the API for
|
||||
images, containers, events, snapshots, etc.
|
||||
|
||||
The syntax is fairly familiar, if you've used container ecosystem
|
||||
projects. At the core, we base it on the concept of protobuf field
|
||||
paths, augmenting with the ability to quote portions of the field path
|
||||
to match arbitrary labels. These "selectors" come in the following
|
||||
syntax:
|
||||
|
||||
```
|
||||
<fieldpath>[<operator><value>]
|
||||
```
|
||||
|
||||
A basic example is as follows:
|
||||
|
||||
```
|
||||
name==foo
|
||||
```
|
||||
|
||||
This would match all objects that have a field `name` with the value
|
||||
`foo`. If we only want to test if the field is present, we can omit the
|
||||
operator. This is most useful for matching labels in containerd. The
|
||||
following will match objects that has the field labels and have the
|
||||
label "foo" defined:
|
||||
|
||||
```
|
||||
labels.foo
|
||||
```
|
||||
|
||||
We also allow for quoting of parts of the field path to allow matching
|
||||
of arbitrary items:
|
||||
|
||||
```
|
||||
labels."very complex label"==something
|
||||
```
|
||||
|
||||
We also define `!=` and `~=` as operators. The `!=` operator will match
|
||||
all objects that don't match the value for a field and `~=` will compile
|
||||
the target value as a regular expression and match the field value
|
||||
against that.
|
||||
|
||||
Selectors can be combined using a comma, such that the resulting
|
||||
selector will require all selectors are matched for the object to match.
|
||||
The following example will match objects that are named `foo` and have
|
||||
the label `bar`:
|
||||
|
||||
```
|
||||
name==foo,labels.bar
|
||||
```
|
||||
|
||||
[#995: Add package for filters](https://github.com/containerd/containerd/pull/995)
|
||||
|
||||
[#1050: Container list filters](https://github.com/containerd/containerd/pull/1050)
|
||||
|
||||
[#1069: Event filters](https://github.com/containerd/containerd/pull/1069)
|
||||
|
||||
## Support for pushing and pulling OCI indexes
|
||||
|
||||
We added support for pushing and pulling OCI indexes. Currently all content
|
||||
referenced by the list are pulled and further work on the client will be done
|
||||
to allow selection of the pulled manifest to extract and run.
|
||||
|
||||
[#1022: Support pull for OCI indexes](https://github.com/containerd/containerd/pull/1022)
|
||||
|
||||
## Snapshot list
|
||||
|
||||
We added `ctr snapshot list` to snapshots from containerd. This will output all
|
||||
snapshots, not just the active snapshots used by containers.
|
||||
|
||||
```
|
||||
$ ctr snapshot list
|
||||
ID Parent State Readonly
|
||||
registry2 sha256:dc22a13eb565d14bfe2b16f6fa731a05da0eeff02a52059c7b59cdc2c232a2b2 active false
|
||||
registry3 sha256:dc22a13eb565d14bfe2b16f6fa731a05da0eeff02a52059c7b59cdc2c232a2b2 active false
|
||||
sha256:4ac69ce655ab8aa97362915793348d31361fb3c047e223c2b58be706e89c48fc sha256:ba2cc2690e31f63847e4bc0d266b354f8f11dc04474d45d44312ff70edae9c98 committed true
|
||||
sha256:ba2cc2690e31f63847e4bc0d266b354f8f11dc04474d45d44312ff70edae9c98 committed true
|
||||
sha256:bfe0b04fc169b94099b29dbf5a527f6a11db627cd0a6126803edf8f42bd7b4b3 sha256:4ac69ce655ab8aa97362915793348d31361fb3c047e223c2b58be706e89c48fc committed true
|
||||
sha256:d959def87dadbb9ba85070c09e99b46d994967b12f5748f617c377073b8d1e39 sha256:bfe0b04fc169b94099b29dbf5a527f6a11db627cd0a6126803edf8f42bd7b4b3 committed true
|
||||
sha256:dc22a13eb565d14bfe2b16f6fa731a05da0eeff02a52059c7b59cdc2c232a2b2 sha256:d959def87dadbb9ba85070c09e99b46d994967b12f5748f617c377073b8d1e39 committed true
|
||||
```
|
||||
|
||||
[#1058: Add list snapshot subcommand](https://github.com/containerd/containerd/pull/1058)
|
||||
|
||||
## API cleanup
|
||||
|
||||
As part of our API review process we have started implementing some changes to
|
||||
make the API clearer and more consistent.
|
||||
|
||||
[#1040: Update GRPC APIs](https://github.com/containerd/containerd/pull/1040)
|
||||
|
||||
[#1047: Rename execution service to tasks](https://github.com/containerd/containerd/pull/1047)
|
||||
|
||||
[#1059: enforce a character set for namespaces](https://github.com/containerd/containerd/pull/1059)
|
||||
|
||||
[#1052: version protobuf service packages](https://github.com/containerd/containerd/pull/1062)
|
||||
|
||||
[#1072: move events types into service package](https://github.com/containerd/containerd/pull/1072)
|
||||
|
||||
[#1073: consolidate API types packages](https://github.com/containerd/containerd/pull/1073)
|
||||
|
||||
[#1074: replace whale with D](https://github.com/containerd/containerd/pull/1074)
|
||||
|
||||
## Containerd notes from Moby Summit
|
||||
|
||||
At the Moby summit on 06/19/2017 there was a containerd round table meeting.
|
||||
This was a good opportunity to discuss the upcoming API freeze and release of
|
||||
containerd with others working with it in the community. Always remember that
|
||||
these summit round tables are not the only opportunity to have these topics
|
||||
discussed and everyone is encouraged to open issues and engage the community on
|
||||
Slack.
|
||||
|
||||
* "What are the plans for a resolver-resolver and image reference namespacing?"
|
||||
- Maintainers are trying to figure out what everyone’s plans/needs are for a resolver-resolver.
|
||||
- A resolver-resolver allows configuring where push/pull happens. Could take in a configuration format which specifies how to push, pull, and authenticate. Needed in order to do discover of names to registry configurations.
|
||||
- Stephen confirms we are thinking of more configuration driven rather programmatic. The resolver-resolver and any configuration is always client side, allowing the integrator to design any interface or configuration suits their needs. But we are also looking to define something that could be consistently applied by containerd clients.
|
||||
- The resolver-resolver will be compatible with Docker, but could allow Docker to expand its resolution support.
|
||||
|
||||
* "What is the plan for garbage collection?"
|
||||
- Current design is oriented around being garbage collected. The data model is designed around references which are used to walk a root set. Everything unreferenced would be garbage collected.
|
||||
- Another more complicated aspect of garbage collection is around policy, allowing to clean up resources based on age or client specified policies.
|
||||
- Client side implementations would allow establishing policies.
|
||||
- Containerd will require a stop the world to do the garbage collection.
|
||||
- A heavy delete which does not garbage collect is an option and similar to to the interface today. The API does not guarantee that disk is cleaned up after a deletion, only that the resource is freed. Inline deletion would require reference counting to know when to delete. This would also require global locking to protect the references.
|
||||
- How to handle content which is active but unreferenced, leasing vs pre-allocate. This has not been decided on.
|
||||
|
||||
* "What will need to change in Docker in regards to graphdrivers for accommodating the containerd snapshotters?"
|
||||
- The goal is to have graphdrivers be able to be used as snapshot drivers.
|
||||
- To accomplish this graphdrivers need to be able to return mounts without needing to require action on unmount. Unmount will no longer contact the graphdriver since snapshotters to do not control the mount/unmount lifecycle.
|
||||
- For implementation, in the repository tree only overlayfs and btrfs are supported. Everything else will be out of tree and require special builds or proxying.
|
||||
|
||||
* "Version numbers on events and API objects"
|
||||
- Are objects mutable? Some objects are mutable but not necessarily relevant to clients.
|
||||
- Are containers mutable? It can have multiple tasks, can be started and stopped.
|
||||
- We may need to have task id separate from the container to differentiate between tasks within a container.
|
||||
- Tasks have a pid which could be used to identify the task. Someone had a customer that ran out of memory from running too many tasks, could be caused by repeated execs in the same container.
|
||||
- Getting state of a containers task involves requesting containers and tasks, or just tasks could be listed.
|
||||
- What are the API costs, is there measurements of the cost of API calls and comparisons with Docker? Calling docker ps can be expensive due to locks, containerd should be much cheaper and faster to call. This need to be verified and measurements added.
|
||||
|
||||
* "How can clients understand the capabilities of the containerd instance it is talking to?"
|
||||
- As an orchestrator, interested in understanding what can be done with the plugins. Example, docker log drivers change feature set but the log driver names do not change. Stuck on checking docker version.
|
||||
- There have been problems in the past with requiring multiple clients of docker to handle changes.
|
||||
- GRPC is adding introspection, plan is to wait for this work rather than making something ourselves.
|
||||
- The maintainers would like to understand what were the really bad changes in Docker that caused problems with version support for users. Problems around needing to bump the whole API to get a new feature. Containerd API versions each service and v1 interface will be stable and supported.
|
||||
Reference in New Issue
Block a user