Commit Graph

82 Commits

Author SHA1 Message Date
Stephen J Day
539742881d api/services: define the container metadata service
Working from feedback on the existing implementation, we have now
introduced a central metadata object to represent the lifecycle and pin
the resources required to implement what people today know as
containers. This includes the runtime specification and the root
filesystem snapshots. We also allow arbitrary labeling of the container.
Such provisions will bring the containerd definition of container closer
to what is expected by users.

The objects that encompass today's ContainerService, centered around the
runtime, will be known as tasks. These tasks take on the existing
lifecycle behavior of containerd's containers, which means that they are
deleted when they exit. Largely, there are no other changes except for
naming.

The `Container` object will operate purely as a metadata object. No
runtime state will be held on `Container`. It only informs the execution
service on what is required for creating tasks and the resources in use
by that container. The resources referenced by that container will be
deleted when the container is deleted, if not in use. In this sense,
users can create, list, label and delete containers in a similar way as
they do with docker today, without the complexity of runtime locks that
plagues current implementations.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-05-22 23:27:53 -07:00
Derek McGowan
098ff94b24 Add snapshot and diff service
Remove rootfs service in place of snapshot service. Adds
diff service for extracting and creating diffs. Diff
creation is not yet implemented. This service allows
pulling or creating images without needing root access to
mount. Additionally in the future this will allow containerd
to ensure extractions happen safely in a chroot if needed.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-05-15 16:50:16 -07:00
Phil Estes
bd432461d3 Shorten no-btrfs build tag name
Remove the `containerd_` portion of the build tag name as it is
unnecessary.

Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
2017-05-12 09:53:57 -05:00
Daniel Dao
4b2ce72aaf remove unused import
With this `make vet` should pass

Signed-off-by: Daniel Dao <dqminh89@gmail.com>
2017-05-11 19:20:00 -05:00
Daniel, Dao Quang Minh
6df793797e Merge pull request #846 from crosbymichael/linux-container
Move container and runtime to plugin pkg
2017-05-11 19:07:58 -05:00
Michael Crosby
01b9f5ec67 Move Container and runtime to plugin pkg
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-05-11 15:24:12 -07:00
Akihiro Suda
2562aca1a3 new service: version
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-05-11 09:07:43 +00:00
Stephen J Day
193abed96e content: unify provider and ingester
The split between provider and ingester was a long standing division
reflecting the client-side use cases. For the most part, we were
differentiating these for the algorithms that operate them, but it made
instantation and use of the types challenging. On the server-side, this
distinction is generally less important. This change unifies these types
and in the process we get a few benefits.

The first is that we now completely access the content store over GRPC.
This was the initial intent and we have now satisfied this goal
completely. There are a few issues around listing content and getting
status, but we resolve these with simple streaming and regexp filters.
More can probably be done to polish this but the result is clean.

Several other content-oriented methods were polished in the process of
unification. We have now properly seperated out the `Abort` method to
cancel ongoing or stalled ingest processes. We have also replaced the
`Active` method with a single status method.

The transition went extremely smoothly. Once the clients were updated to
use the new methods, every thing worked as expected on the first
compile.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-05-10 17:05:53 -07:00
Kenfe-Mickael Laventure
180eaf724e Don't print a warning when getting a healtcheck request
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-05-09 15:32:05 -07:00
Vincent Batts
55bf3c10f0 containerd: compile tags to exclude btrfs
Leave in btrfs by default, but add go build tags to exclude it.
`go build -tags containerd_no_btrfs` will leave that driver out.
As the current containerd/btrfs code needs link to libbtrfs*.so, but not
all distros provide it.

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2017-05-04 17:30:23 -04:00
Justin Cormack
1d27259777 Partial port to Darwin and FreeBSD
This mainly fixes Linux vs generic Unix differences, with some
differences between Darwin and Freebsd (which are close bit not
identical). Should make fixing for other Unix platforms easier.

Note there are not yet `runc` equivalents for these platforms;
my current use case is image manipulation for the `moby` tool.
However there is interest in OCI runtime ports for both platforms.

Current status is that MacOS can build and run `ctr`, `dist`
and `containerd` and some operations are supported. FreeBSD 11
still needs some more fixes to continuity for extended attributes.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-05-04 18:11:53 +01:00
Derek McGowan
4cb0839e41 Ensure grpc socket is group writable
Updates the filemode on the grpc socket to have group write
permission which is needed to perform GRPC. Additionally, ensure
the run directory has the specified group ownership and has group
read and enter permission.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-04-26 14:40:36 -07:00
CuiHaozhi
d51976b60a Correct the usage of containerd --config
Signed-off-by: CuiHaozhi <cuihz@wise2c.com>
2017-04-17 06:44:18 -04:00
Michael Crosby
634f0c0c83 Set oom score for containerd and shims
This adds a config option to set the oom score for the containerd daemon
as well as automatically setting the oom score for the shim's lauched so
that they are not killed until the very end of an out of memory
condition.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-04-12 11:34:32 -07:00
Phil Estes
62918511f3 Merge pull request #712 from crosbymichael/sysunix
Replace syscall with /x/sys/unix in execution code
2017-04-10 16:30:32 -04:00
Michael Crosby
3db1ea8d07 [bin] Replace syscall with /x/sys/unix
Replace syscall usage with /sys/unix in the binaries and their packages

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-04-10 12:01:33 -07:00
Vincent Batts
f58f047b7f containerd: revert the auto config creation
rather than automagically doing this, it is the user's responsibility to
review the output of `containerd config default` and create the config
themselves.

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2017-04-10 14:54:36 -04:00
Vincent Batts
741c471753 containerd: make a config subcommand
For the purpose of reviewing the default config

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2017-04-10 14:33:29 -04:00
Kenfe-Mickael Laventure
c5843b7615 Initial windows runtime work
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-04-07 09:20:44 -07:00
Derek McGowan
e2b042e7c1 Merge pull request #675 from stevvooe/images-service
api/services/images: define images metadata service
2017-04-04 16:10:54 -07:00
Vincent Batts
93417d446c containerd: missing config creates a default
This fixes silently ignore the config file not existing as well.

```bash
sudo ./bin/containerd --config farts
$ sudo ./bin/containerd --config farts
INFO[0000] config "farts" does not exist. Creating it.   module=containerd
INFO[0000] starting containerd boot...                   module=containerd
INFO[0000] starting debug API...                         debug="/run/containerd/debug.sock" module=containerd
INFO[0000] loading monitor plugin "cgroups"...           module=containerd
INFO[0000] loading runtime plugin "linux"...             module=containerd
INFO[0000] loading snapshot plugin "snapshot-overlay"...  module=containerd
INFO[0000] loading grpc service plugin "content-grpc"...  module=containerd
INFO[0000] loading grpc service plugin "metrics-grpc"...  module=containerd
INFO[0000] loading grpc service plugin "runtime-grpc"...  module=containerd
INFO[0000] loading grpc service plugin "healthcheck-grpc"...  module=containerd
INFO[0000] loading grpc service plugin "rootfs-grpc"...  module=containerd
INFO[0000] starting GRPC API server...                   module=containerd
INFO[0000] containerd successfully booted in 0.001465s   module=containerd
^C$ cat farts
state = "/run/containerd"
root = "/var/lib/containerd"
snapshotter = "overlay"
subreaper = false

[grpc]
  socket = "/run/containerd/containerd.sock"
  uid = 0
  gid = 0

[debug]
  socket = "/run/containerd/debug.sock"
  level = "info"

[metrics]
  address = ""
```

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2017-04-04 17:32:05 -04:00
Vincent Batts
28d012ea00 containerd: add access to default config
when wanting to craft a custom config, but based on the default config,
add a route to output the containerd config to a tempfile.

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2017-04-04 16:32:12 -04:00
Stephen J Day
8c74da3983 cmd/dist, cmd/ctr: move image store access to GRPC
With this changeset, image store access is now moved to completely
accessible over GRPC. No clients manipulate the image store database
directly and the GRPC client is fully featured. The metadata database is
now managed by the daemon and access coordinated via services.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-04-04 10:35:55 -07:00
Michael Crosby
4f2b443a27 Rewrite imports for new github org
This rewrites the Go imports after switching to the new github org.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-04-03 14:05:44 -07:00
unclejack
7b4d238ee3 archive,cmd,images: clean up some code
Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com>
2017-03-30 11:41:22 +03:00
unclejack
d2a6630658 utils: remove utils & migrate code to sys
Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com>
2017-03-29 21:25:02 +03:00
Michael Crosby
5e5479718c Merge pull request #619 from crosbymichael/no-shim
Add `no_shim` config for not running with a shim
2017-03-22 13:49:03 -07:00
Michael Crosby
f36feb2ed4 Add prometheus container level metrics
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-03-22 11:44:25 -07:00
Michael Crosby
ddbeb9f936 Add monitor plugin loading
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-03-22 11:44:05 -07:00
Derek McGowan
38a6f90f2b Add rootfs command to dist
Commands allows preparing a rootfs from a manifest hash

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-03-15 16:32:21 -07:00
Derek McGowan
ead425f426 Add rootfs service
Add rootfs service to allow creation of rootfs from layer array

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-03-15 16:32:21 -07:00
Michael Crosby
f9212348e4 Add grpc prometheus metrics
This provideds generic grpc metrics via prometheus

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-03-14 16:16:49 -07:00
Qiang Huang
e663cbddf5 Add grpc health check service
Fixes: #615

Signed-off-by: Qiang Huang <h.huangqiang@huawei.com>
2017-03-14 15:39:29 +08:00
Michael Crosby
1d7fa45403 Add no_shim config for not running with a shim
This reuses the exiting shim code and services to let containerd run as
the reaper for all container processes without the use of a shim.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-03-10 15:28:21 -08:00
Michael Crosby
61263bd77e Set containerd as subreaper
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-03-09 16:26:14 -08:00
Michael Crosby
9f3240364f Implement reaper with runc support in shim
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-03-09 16:11:57 -08:00
Michael Crosby
df48983fe7 Add reaper code for daemon
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-03-09 16:07:35 -08:00
Michael Crosby
fc45d9c119 Chown grpc socket based on config
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-03-08 11:42:36 -08:00
Derek McGowan
a4247e2aa9 Add snapshot plugin type
Update existing snapshot drivers to register as plugins.
Load snapshot driver at containerd startup.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-03-07 14:55:36 -08:00
Derek McGowan
0b44e24c07 Move plugin registration to separate package
Only compile in plugin support on non-windows 1.8.
Re-enable go 1.7.x tests.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-03-06 17:23:00 -08:00
Stephen J Day
0c00d4a64c cmd/containerd: set correct module on interceptor for content
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-28 16:47:41 -08:00
Michael Crosby
e693cad926 Change linux runtime(w shim) to builtin
Because of the plugin findings and having the default runtime builtin
this makes it much better for development and testing.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-02-24 15:50:59 -08:00
Phil Estes
a463ba33fc Merge pull request #561 from stevvooe/correct-versioning
version: finish version setup
2017-02-22 13:48:05 -08:00
Stephen J Day
935144fadd version: finish version setup
This setup will now correctly set the version number from the git tag.
When using `--version`, we will see the binary name, the package it was
built from and a git hash based on the tag:

```console
$./bin/dist -v
./bin/dist github.com/docker/containerd 0b45d91.m
```

Note that in the above example, if we set a tag of `v1.0.0-dev`, that
will show up in the version number, as follows:

```console
$./bin/dist -v
./bin/dist github.com/docker/containerd v1.0.0-dev
```

Once commits are made past that tag, the version number will be
expressed relative to that tag and include a git hash:

```console
$./bin/dist -v
./bin/dist github.com/docker/containerd v1.0.0-dev-1-g7953e96.m
```

Some these examples include a `.m` postfix. This indicates that the
binary was build from a source tree with local modifications.

We can add a dev tag to start getting 1.0 version numbers for test
builds.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-22 13:16:06 -08:00
Michael Crosby
3101be93bc Load runtimes dynamically via go1.8 plugins
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Add registration for more subsystems via plugins

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Move content service to separate package

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-02-21 16:29:46 -08:00
Stephen J Day
baa307cd4b cmd/containerd: remove cli.Context dependence for content.Store
We've moved to using the config, directly. This removes the argument and
gets rid of a few extra lines of code.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-21 15:27:12 -08:00
Stephen J Day
e6efb397cf cmd/dist: port commands over to use GRPC content store
Following from the rest of the work in this branch, we now are porting
the dist command to work directly against the containerd content API.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-02-21 13:10:31 -08:00
Michael Crosby
f01434283c Merge pull request #553 from crosbymichael/config
Add config file
2017-02-21 11:52:08 -08:00
Michael Crosby
a9950aedcf Add config file
This adds a config file for containerd configuration.  It is hard to
have structure data on cli flags and the config file should be used for
the majority of fields when configuring containerd.

There are still a few flags on the daemon that override config file
values but flags should take a back seat going forward and should be
kept at a minimum.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-02-21 11:22:08 -08:00
Qiang Huang
7206427df4 Fix interceptor handler parameter
Signed-off-by: Qiang Huang <h.huangqiang@huawei.com>
2017-02-21 08:41:31 -08:00