Adds a useful flag to `ctr` to enable joining any existing Linux
namespaces for any namespace types (network, pid, ipc, etc.) using the
existing With helper in the oci package.
Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
This uses a simple `IsAbs` check to see if we are using an on disk path
for a unix socket vs an address since we do not prefix addresses with
`unix://` or `tcp://`.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
For missing required parameters adds error return before attempting any
actions to `ctr images` commands.
Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
To avoid having the shim hold on to too much memory, we've made a few
adjustments to favor more aggressive reclamation of memory from the
operating system. Typically, this would be negligible, on the order of a
few megabytes, but this is impactful when running several containers.
The first fix is to lower the threshold used to determine when to run
the garbage collector. The second runs `runtime/debug.FreeOSMemory` at a
regular interval.
Under test, this result in a sustained memory usage of around 3.7 MB.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
This changes the Windows runtime to use the snapshotter and differ
created layers, and updates the ctr commands to use the snapshotter and differ.
Signed-off-by: Darren Stahl <darst@microsoft.com>
This implements the Windows snapshotter and diff Apply function.
This allows for Windows layers to be created, and layers to be pulled
from the hub.
Signed-off-by: Darren Stahl <darst@microsoft.com>
Given these same exact functions are both now available in
opencontainers/runc (libcontainer/system) package, and we only use the
`SetSubreaper` today from the shim, there seems to be no reason for
duplication.
Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
To avoid buffer bloat in long running processes, we try to use buffer
pools where possible. This is meant to address shim memory usage issues,
but may not be the root cause.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
This improves the exec support so that they can run along with the
normal stress tests. You don't have to pick exec stres or container
stress. They both run at the same time and report the different values.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
After comtemplation, the complexity of the logging module system
outweighs its usefulness. This changeset removes the system and restores
lighter weight code paths. As a concession, we can always provide more
context when necessary to log messages to understand them without having
to fork the context for a certain set of calls.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
This allows other packages and plugins to easily exec things without
racing with the reaper.
The reaper is mostly needed in the shim but can be removed in containerd
in favor of the `exec.Cmd` apis
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
This linter checks for unnecessary type convertions.
Some convertions are whitelisted because their type is different
on 32bit platforms
Signed-off-by: Daniel Nephin <dnephin@gmail.com>
Preserves the order of the tree output between each execution. Slightly
refactored the behavior to be more "object oriented".
Signed-off-by: Stephen J Day <stephen.day@docker.com>
- Use lease API (previoisly, GC was not supported)
- Refactored interfaces for ease of future Docker v1 importer support
For usage, please refer to `ctr images import --help`.
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
This subreaper should always be turned on for containerd unless
explicitly needed for it to be off.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Could issues where when exec processes fail the wait block is not
released.
Second, you could not dump stacks if the reaper loop locks up.
Third, the publisher was not waiting on the correct pid.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
The shim doesn't need massive concurrency and a bunch of CPUs to do its
job correctly. We can reduce the number of threads to save memory at
little cost to performance.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
By replacing grpc with ttrpc, we can reduce total memory runtime
requirements and binary size. With minimal code changes, the shim can
now be controlled by the much lightweight protocol, reducing the total
memory required per container.
When reviewing this change, take particular notice of the generated shim
code.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Synchronous image delete provides an option image delete to wait
until the next garbage collection deletes after an image is removed
before returning success to the caller.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Add garbage collection as a background process and policy
configuration for configuring when to run garbage collection.
By default garbage collection will run when deletion occurs
and no more than 20ms out of every second.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
The binary name used for executing "containerd publish" was hard-coded
in the shim code, and hence it did not work with customized daemon
binary name. (e.g. `docker-containerd`)
This commit allows specifying custom daemon binary via `containerd-shim
-containerd-binary ...`.
The daemon invokes this command with `os.Executable()` path.
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
Currently the output for a non-existent image reference and a valid
image reference is exactly the same on `ctr images remove`. Instead of
outputting the target ref input, if it is "not found" we should alert
the user in case of a mispelling, but continue not to make it a failure
for the command (given it supports multiple ref entries)
Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
This patch changes the output of `ctr version` to align version and revision.
It also changes `.Printf()` to `.Println()`, to make the code slightly easier
to read.
Before this change:
$ ctr version
Client:
Version: v1.0.0-beta.2-132-g564600e.m
Revision: 564600ee79aefb0f24cbcecc90d4388bd0ea59de.m
Server:
Version: v1.0.0-beta.2-132-g564600e.m
Revision: 564600ee79aefb0f24cbcecc90d4388bd0ea59de.m
With this patch applied:
$ ctr version
Client:
Version: v1.0.0-beta.2-132-g564600e.m
Revision: 564600ee79aefb0f24cbcecc90d4388bd0ea59de.m
Server:
Version: v1.0.0-beta.2-132-g564600e.m
Revision: 564600ee79aefb0f24cbcecc90d4388bd0ea59de.m
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
To reduce the binary size of containerd, we no longer import the
`server` package for only a few defaults. This reduces the size of `ctr`
by 2MB. There are probably other gains elsewhere.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
This keeps the semantics the same as the other commands to only list
containers, tasks, images by calling the list subcommand.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
This allows a project to have a TEMPLATE file in the root of the repo to
be used with the release tool. If they don't have this file and did not
specify a custom file then it will use the compiled in template in the
release tool.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Allow a user provided name for the checkpoint as well as a default
generated name for the checkpoint image.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
This allows one to manage the checkpoints by using the `ctr image`
command.
The image is created with label "containerd.io/checkpoint". By
default, it is not included in the output of `ctr images ls`.
We can list the images by using the following command:
$ ctr images ls labels.containerd.\"io/checkpoint\"==true
Fixes#1026
Signed-off-by: Jacob Wen <jian.w.wen@oracle.com>
This tool makes our standard release template easy to generate. It also
adds a few features like marking changed dependnencies for packages and
others to know what updated from the last release.
usage:
`containerd-release -n releases/v1.0.0-beta.2.toml`
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Add differ options and package with interface.
Update optional values on diff interface to use options.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
With this change, we integrate all the plugin changes into the
introspection service.
All plugins can be listed with the following command:
```console
$ ctr plugins
TYPE ID PLATFORM STATUS
io.containerd.content.v1 content - ok
io.containerd.metadata.v1 bolt - ok
io.containerd.differ.v1 walking linux/amd64 ok
io.containerd.grpc.v1 containers - ok
io.containerd.grpc.v1 content - ok
io.containerd.grpc.v1 diff - ok
io.containerd.grpc.v1 events - ok
io.containerd.grpc.v1 healthcheck - ok
io.containerd.grpc.v1 images - ok
io.containerd.grpc.v1 namespaces - ok
io.containerd.snapshotter.v1 btrfs linux/amd64 error
io.containerd.snapshotter.v1 overlayfs linux/amd64 ok
io.containerd.grpc.v1 snapshots - ok
io.containerd.monitor.v1 cgroups linux/amd64 ok
io.containerd.runtime.v1 linux linux/amd64 ok
io.containerd.grpc.v1 tasks - ok
io.containerd.grpc.v1 version - ok
```
There are few things to note about this output. The first is that it is
printed in the order in which plugins are initialized. This useful for
debugging plugin initialization problems. Also note that even though the
introspection GPRC api is a itself a plugin, it is not listed. This is
because the plugin takes a snapshot of the initialization state at the
end of the plugin init process. This allows us to see errors from each
plugin, as they happen. If it is required to introspect the existence of
the introspection service, we can make modifications to include it in
the future.
The last thing to note is that the btrfs plugin is in an error state.
This is a common state for containerd because even though we load the
plugin, most installations aren't on top of btrfs and the plugin cannot
be used. We can actually view this error using the detailed view with a
filter:
```console
$ ctr plugins --detailed id==btrfs
Type: io.containerd.snapshotter.v1
ID: btrfs
Platforms: linux/amd64
Exports:
root /var/lib/containerd/io.containerd.snapshotter.v1.btrfs
Error:
Code: Unknown
Message: path /var/lib/containerd/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter
```
Along with several other values, this is a valuable tool for evaluating the
state of components in containerd.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
By default, the generated spec will place containers in cgroups by their
ids, we need to use the namespace as the cgroup root to avoid
containers with the same name being placed in the same cgroup.
```
11:perf_event:/to/redis
10:freezer:/to/redis
9:memory:/to/redis
8:devices:/to/redis
7:net_cls,net_prio:/to/redis
6:pids:/to/redis
5:hugetlb:/to/redis
4:cpuset:/to/redis
3:blkio:/to/redis
2:cpu,cpuacct:/to/redis
1:name=systemd:/to/redis
11:perf_event:/te/redis
10:freezer:/te/redis
9:memory:/te/redis
8:devices:/te/redis
7:net_cls,net_prio:/te/redis
6:pids:/te/redis
5:hugetlb:/te/redis
4:cpuset:/te/redis
3:blkio:/te/redis
2:cpu,cpuacct:/te/redis
1:name=systemd:/te/redis
```
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
This makes sure the client is always in sync with the server before
performing any type of operations on the container metadata.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
The `Check` function returns information about an image's content components
over a content provider. From this information, one can tell which content is
required, present or missing to run an image.
The utility can be demonstrated with the `check` command:
```console
$ ctr images check
REF TYPE DIGEST STATUS SIZE
docker.io/library/alpine:latest application/vnd.docker.distribution.manifest.list.v2+json sha256:f006ecbb824d87947d0b51ab8488634bf69fe4094959d935c0c103f4820a417d incomplete (1/2) 1.5 KiB/1.9 MiB
docker.io/library/postgres:latest application/vnd.docker.distribution.manifest.v2+json sha256:2f8080b9910a8b4f38ff5a55a82e77cb43d88bdbb16d723c71d18493590832e9 complete (13/13) 99.3 MiB/99.3 MiB
docker.io/library/redis:alpine application/vnd.docker.distribution.manifest.v2+json sha256:e633cded055a94202e4ccccb8125b7f383cd6ee56527ab890db643383a2647dd incomplete (6/7) 8.1 MiB/10.0 MiB
docker.io/library/ubuntu:latest application/vnd.docker.distribution.manifest.list.v2+json sha256:60f835698ea19e8d9d3a59e68fb96fb35bc43e745941cb2ea9eaf4ba3029ed8a unavailable (0/?) 0.0 B/?
docker.io/trollin/busybox:latest application/vnd.docker.distribution.manifest.list.v2+json sha256:54a6424f7a2d5f4f27b3d69e5f9f2bc25fe9087f0449d3cb4215db349f77feae complete (2/2) 699.9 KiB/699.9 KiB
```
The above shows us that we have two incomplete images and one that is
unavailable. The incomplete images are those that we know the complete
size of all content but some are missing. "Unavailable" means that the
check could not get enough information about the image to get its full
size.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
The SIGUNUSED constant was removed from golang.org/x/sys/unix in
https://go-review.googlesource.com/61771 as it is also removed from the
respective glibc headers.
This means the command
ctr tasks kill SIGUNUSED ...
will no longer work. However, the same effect can be achieved with
ctr tasks kill SIGSYS ...
as SIGSYS has the same value as SIGUNUSED used to have.
Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
Fixes pulling of multi-arch images by limiting the expansion
of the index by filtering to the current default platform.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
This handles signals first thing on boot so that plugins are able to
boot with the reaper enabled.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Content commit is updated to take in a context, allowing
content to be committed within the same context the writer
was in. This is useful when commit may be able to use more
context to complete the action rather than creating its own.
An example of this being useful is for the metadata implementation
of content, having a context allows tests to fully create
content in one database transaction by making use of the context.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
The labels can be very long (e.g. cri-containerd stores a large JSON metadata
blob as `io.cri-containerd.container.metadata`) which renders the output
useless due to all the line wrapping etc.
The information is still available in `ctr containers info «name»`.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
This also fix the type used for RuncOptions.SystemCgroup, hence introducing
an API break.
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
Fixes#1431
This adds KillOpts so that a client can specify when they want to kill a
single process or all the processes inside a container.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
