Fix restoring from a checkpoint image always getting skipped when no checkpoint image path is set. As one restores either from an image OR a path, this makes no sense.
Signed-off-by: Jakob Schrettenbrunner <dev@schrej.net>
When containerd is restarted, only v1 tasks are monitored again. This
leads to the lack of existing v2 task metrics.
Signed-off-by: Li Yuxuan <liyuxuan04@baidu.com>
Prevent error messages from being output to stderr.
Return illegal token when a quoted string is invalid and
capture the error.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Cleanup is an optional method a snapshotter may implement.
Cleanup can be used to cleanup resources after a snapshot
has been removed. This function allows a snapshotter to defer
longer resource cleanup until after snapshot removals are
completed. Adding this to the API allows proxy snapshotters
to leverage this enhancement.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
* only shim v2 runc v2 ("io.containerd.runc.v2") is supported
* only PID metrics is implemented. Others should be implemented in separate PRs.
* lots of code duplication in v1 metrics and v2 metrics. Dedupe should be separate PR.
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
config imports is in random order after LoadConfig call so that the
testing check should be done after sort.Strings(out.Imports).
Signed-off-by: Wei Fu <fuweid89@gmail.com>
Proxy plugins are keyed only on the identifier, the type
is specified within the proxy plugin configuration which
maps to the full URI. The proxy plugin configuration is
not passed to the plugin for configuration like other
plugins.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Do not rely on toml metadata when decoding plugin's configs as it's not possible to merge toml.MetaData structs during import.
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
Previously the TTRPC address was generated as "<GRPC address>.ttrpc".
This change now allows explicit configuration of the TTRPC address, with
the default still being the old format if no value is specified.
As part of this change, a new configuration section is added for TTRPC
listener options.
Signed-off-by: Kevin Parsons <kevpar@microsoft.com>
This adds a singleton `timeout` package that will allow services and user
to configure timeouts in the daemon. When a service wants to use a
timeout, it should declare a const and register it's default value
inside an `init()` function for that package. When the default config
is generated, we can use the `timeout` package to provide the available
timeout keys so that a user knows that they can configure.
These show up in the config as follows:
```toml
[timeouts]
"io.containerd.timeout.shim.cleanup" = 5
"io.containerd.timeout.shim.load" = 5
"io.containerd.timeout.shim.shutdown" = 3
"io.containerd.timeout.task.state" = 2
```
Timeouts in the config are specified in seconds.
Timeouts are very hard to get right and giving this power to the user to
configure things is a huge improvement. Machines can be faster and
slower and depending on the CPU or load of the machine, a timeout may
need to be adjusted.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Closes#1862
This adds a new rpc to the introspection service to provide server
information with a generated UUID that is done on demand and the os and
arch of the server.
ctr output:
```bash
> sudo ctr version
Client:
Version: v1.2.0-802-g57821695.m
Revision: 578216950de9c1c188708369e2a31ac6c494dfee.m
Server:
Version: v1.2.0-802-g57821695.m
Revision: 578216950de9c1c188708369e2a31ac6c494dfee.m
UUID: 92e982a9-f13e-4a2c-9032-e69b27fed454
OS: linux
ARCH: amd64
```
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Fixes#3440
This also returns the task that times out or has an error on the state
call in an UNKNOWN status so that the user can manually kill and remove
the task.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Add three methods for lease service so that the client can use it to
manage the resource by lease, not just gc.root label. With the following
methods, it is easy for client to maintain their own cache system.
```
- AddResource(context.Context, Lease, Resource) error
- RemoveResource(context.Context, Lease, Resource) error
- ListResources(context.Context, Lease) ([]Resource, error)
```
And the resource is to be
```golang
type Resource {
ID string
Type string
}
```
For the snapshots, the Type field will be formatted by
snapshots/%{type}, like snapshots/overlayfs.
fix: #3295
Signed-off-by: Wei Fu <fuweid89@gmail.com>
Extend the Applier interface's Apply method with an optional
options parameter.
For the container image encryption we intend to use the options
parameter to pass image decryption parameters ('dcparameters'),
which are primarily (privatte) keys, in form of a JSON document
under the map key '_dcparameters', and pass them to the Applier's
Apply() method. This helps us to access decryption keys and start
the pipeline with the layer decryption before the layer data are
unzipped and untarred.
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Signed-off-by: Harshal Patil <harshal.patil@in.ibm.com>
megacheck, gosimple and unused has been deprecated and subsumed by
staticcheck. And staticcheck also has been upgraded. we need to update
code for the linter issue.
close: #2945
Signed-off-by: Wei Fu <fuweid89@gmail.com>
This changeset modifies the metadata store to allow one to set a
"content sharing policy" that defines how blobs are shared between
namespaces in the content store.
The default mode "shared" will make blobs available in all namespaces
once it is pulled into any namespace. The blob will be pulled into
the namespace if a writer is opened with the "Expected" digest that
is already present in the backend.
The alternative mode, "isolated" requires that clients prove they have
access to the content by providing all of the content to the ingest
before the blob is added to the namespace.
Both modes share backing data, while "shared" will reduce total
bandwidth across namespaces, at the cost of allowing access to any
blob just by knowing its digest.
Note: Most functional codes and changelog of this commit originate from
Stephen J Day <stephen.day@docker.com>, see
40455aade8Fixes#1713Fixes#2865
Signed-off-by: Eric Lin <linxiulei@gmail.com>
support checkpoint without committing a checkpoint dir into a
checkpoint image and restore without untar image into checkpoint
directory. support for both v1 and v2 runtime
Signed-off-by: Ace-Tang <aceapril@126.com>
Signed-off-by: John Howard <jhoward@microsoft.com>
Allows containerd.exe to run as a Windows service. eg
Register: `.\containerd.exe --register-service`
Start: `net start containerd`
...
Stop: `net stop containerd`
Unregister: `.\containerd.exe --unregister-service`
When running as a service, logs will go to the Windows application
event log.
Since there is no real action the user can do, these can safely be
informative that the underlying filesystem does not support a snapshot
plugin at boot.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
The github.com/containerd/containerd/services/server has a lot of
dependencies, like content, snapshots services implementation and
docker-metrics.
For the client side, it uses the config struct from server package
to start up the containerd in background. It will import a lot of
useless packages which might be conflict with existing vendor's package.
It makes integration easier with single config package.
Signed-off-by: Wei Fu <fuweid89@gmail.com>
The local store could end up in a state where the writer is
closed but the reference is locked after a commit on an
existing object.
Cleans up Commit logic to always close the writer even after
an error occurs, guaranteeing the reference is unlocked after commit.
Adds a test to the content test suite to verify this behavior.
Updates the content store interface definitions to clarify the behavior.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Implements the Windows lcow differ/snapshotter responsible for managing
the creation and lifetime of lcow containers on Windows.
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
Some images like `criu` will have extra libs that it requires. This
adds lib support via LD_LIBRARY_PATH and InstallOpts
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
This adds a way for users to programatically install containerd binary
dependencies.
With runtime v2 and new shim's being built, it will be a challenge to
get those onto machines. Users would have to find the link, download,
place it in their path, yada yada yada.
With this functionality of a managed `/opt` directory, containerd can
use existing image and distribution infra. to get binarys, shims, etc
onto the system.
Configuration:
*default:* `/opt/containerd`
*containerd config:*
```toml
[plugins.opt]
path = "/opt/mypath"
```
Usage:
*code:*
```go
image, err := client.Pull(ctx, "docker.io/crosbymichael/runc:latest")
client.Install(ctx, image)
```
*ctr:*
```bash
ctr content fetch docker.io/crosbymichael/runc:latest
ctr install docker.io/crosbymichael/runc:latest
```
You can manage versions and see what is running via standard image
commands.
Images:
These images MUST be small and only contain binaries.
```Dockerfile
FROM scratch
Add runc /bin/runc
```
Containerd will only extract files in `/bin` of the image.
Later on, we can add support for `/lib`.
The code adds a service to manage an `/opt/containerd` directory and
provide that path to callers via the introspection service.
How to Test:
Delete runc from your system.
```bash
> sudo ctr run --rm docker.io/library/redis:alpine redis
ctr: OCI runtime create failed: unable to retrieve OCI runtime error (open /run/containerd/io.containerd.runtime.v1.linux/default/redis/log.json: no such file or directory): exec: "runc": executable file not found in $PATH: unknown
> sudo ctr content fetch docker.io/crosbymichael/runc:latest
> sudo ctr install docker.io/crosbymichael/runc:latest
> sudo ctr run --rm docker.io/library/redis:alpine redis
1:C 01 Aug 15:59:52.864 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 01 Aug 15:59:52.864 # Redis version=4.0.10, bits=64, commit=00000000, modified=0, pid=1, just started
1:C 01 Aug 15:59:52.864 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
1:M 01 Aug 15:59:52.866 # You requested maxclients of 10000 requiring at least 10032 max file descriptors.
1:M 01 Aug 15:59:52.866 # Server can't set maximum open files to 10032 because of OS error: Operation not permitted.
1:M 01 Aug 15:59:52.866 # Current maximum open files is 1024. maxclients has been reduced to 992 to compensate for low ulimit. If you need higher maxclients increase 'ulimit -n'.
1:M 01 Aug 15:59:52.870 * Running mode=standalone, port=6379.
1:M 01 Aug 15:59:52.870 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
1:M 01 Aug 15:59:52.870 # Server initialized
1:M 01 Aug 15:59:52.870 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
1:M 01 Aug 15:59:52.870 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
1:M 01 Aug 15:59:52.870 * Ready to accept connections
^C1:signal-handler (1533139193) Received SIGINT scheduling shutdown...
1:M 01 Aug 15:59:53.472 # User requested shutdown...
1:M 01 Aug 15:59:53.472 * Saving the final RDB snapshot before exiting.
1:M 01 Aug 15:59:53.484 * DB saved on disk
1:M 01 Aug 15:59:53.484 # Redis is now ready to exit, bye bye...
```
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Removes the start dependency on V1 runtimes in the TasksService for:
// +build windows_v2. For unix and windows (v1) this code remains to load all
v1 runtimes as expected.
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
This change allows implementations to resolve the location of the actual data
using OCI descriptor fields such as MediaType.
No OCI descriptor field is written to the store.
No change on gRPC API.
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
