Commit Graph

326 Commits

Author SHA1 Message Date
Derek McGowan
66aa1d3ef6 Add snapshot walk implementations
Temporarily remove zfs and aufs until interface update

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-10-24 11:11:22 -07:00
Lantao Liu
aaccfcbe2b Fix containerd config dump.
Signed-off-by: Lantao Liu <lantaol@google.com>
2019-10-23 22:38:18 -07:00
Lantao Liu
ffcb1cc9be Fix delete error code on the containerd daemon side.
Signed-off-by: Lantao Liu <lantaol@google.com>
2019-10-09 00:28:51 -07:00
Derek McGowan
0b224ac7d6 Update metadata interfaces for containers and leases
Add more thorough dirty checking across all types which
may be deleted and hold references.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-09-23 15:27:39 -07:00
Wei Fu
3d1fdacccf config_test: sort result before check
config imports is in random order after LoadConfig call so that the
testing check should be done after sort.Strings(out.Imports).

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2019-09-05 16:30:28 +08:00
Michael Crosby
f76eefd272 Merge pull request #3574 from mxpv/cfg
Support config imports
2019-09-04 16:34:11 -04:00
Derek McGowan
a4482d9a6f Fix proxy plugin config validation
Proxy plugins are keyed only on the identifier, the type
is specified within the proxy plugin configuration which
maps to the full URI. The proxy plugin configuration is
not passed to the plugin for configuration like other
plugins.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-09-03 17:31:14 -07:00
Maksym Pavlenko
24b9e2c1a0 Merge configs section by section
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-08-23 15:49:02 -07:00
Maksym Pavlenko
8ebffecbc3 Use map for stream processors
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-08-23 15:31:37 -07:00
Maksym Pavlenko
19cd0a4f12 Append slices when importing config files
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-08-23 10:12:19 -07:00
Maksym Pavlenko
bca0857530 Fix toml plugin decoding
Do not rely on toml metadata when decoding plugin's configs as it's not possible to merge toml.MetaData structs during import.

Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-08-22 17:22:25 -07:00
Maksym Pavlenko
a1e3779cad Support config imports #3289
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-08-22 15:50:58 -07:00
Kevin Parsons
d7e1b25384 Allow explicit configuration of TTRPC address
Previously the TTRPC address was generated as "<GRPC address>.ttrpc".
This change now allows explicit configuration of the TTRPC address, with
the default still being the old format if no value is specified.

As part of this change, a new configuration section is added for TTRPC
listener options.

Signed-off-by: Kevin Parsons <kevpar@microsoft.com>
2019-08-22 00:56:27 -07:00
Michael Crosby
2e8ea9fd6b Allow timeouts to be configured in config
This adds a singleton `timeout` package that will allow services and user
to configure timeouts in the daemon.  When a service wants to use a
timeout, it should declare a const and register it's default value
inside an `init()` function for that package.  When the default config
is generated, we can use the `timeout` package to provide the available
timeout keys so that a user knows that they can configure.

These show up in the config as follows:

```toml
[timeouts]
  "io.containerd.timeout.shim.cleanup" = 5
  "io.containerd.timeout.shim.load" = 5
  "io.containerd.timeout.shim.shutdown" = 3
  "io.containerd.timeout.task.state" = 2

```

Timeouts in the config are specified in seconds.

Timeouts are very hard to get right and giving this power to the user to
configure things is a huge improvement.  Machines can be faster and
slower and depending on the CPU or load of the machine, a timeout may
need to be adjusted.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-08-13 17:36:32 +00:00
Michael Crosby
f867401c69 Use fds and pass Payloads over diff api
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-08-07 18:35:55 +00:00
Michael Crosby
366823727f Add server config for stream processors
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-08-07 18:35:55 +00:00
Michael Crosby
9f0bc2b53f Add UUID and server info to introspection
Closes #1862

This adds a new rpc to the introspection service to provide server
information with a generated UUID that is done on demand and the os and
arch of the server.

ctr output:

```bash
> sudo ctr version

Client:
  Version:  v1.2.0-802-g57821695.m
  Revision: 578216950de9c1c188708369e2a31ac6c494dfee.m

Server:
  Version:  v1.2.0-802-g57821695.m
  Revision: 578216950de9c1c188708369e2a31ac6c494dfee.m
  UUID: 92e982a9-f13e-4a2c-9032-e69b27fed454
  OS: linux
  ARCH: amd64
```

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-08-06 14:40:48 +00:00
Lantao Liu
bb99688914 Fix containerd on windows.
Signed-off-by: Lantao Liu <lantaol@google.com>
2019-08-02 23:18:35 -07:00
Michael Crosby
eb4b3e8772 Fast path getting pid from task
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-07-26 17:48:00 +00:00
Michael Crosby
101d4b78eb Add timeout to task state calls
Fixes #3440

This also returns the task that times out or has an error on the state
call in an UNKNOWN status so that the user can manually kill and remove
the task.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-07-23 19:58:41 +00:00
Maksym Pavlenko
ef7f46eb7b Fix linter errors
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-07-14 20:49:40 -07:00
Michael Crosby
9547d269a1 Add v2 server config support with plugin URIs
Closes #3210

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-06-10 17:54:20 +00:00
Michael Crosby
d4e7efbb93 Merge pull request #3304 from fuweid/me-update-lease
leases: support resource management
2019-06-03 16:43:34 -04:00
Michael Crosby
2ec2089b05 Merge pull request #3123 from stefanberger/extend_apply_call_with_options_parameter
Extend Applier's Apply() method with an optional options parameter
2019-06-03 11:13:40 -04:00
Wei Fu
8a388d6238 leases: support resource management
Add three methods for lease service so that the client can use it to
manage the resource by lease, not just gc.root label. With the following
methods, it is easy for client to maintain their own cache system.

```
 - AddResource(context.Context, Lease, Resource) error
 - RemoveResource(context.Context, Lease, Resource) error
 - ListResources(context.Context, Lease) ([]Resource, error)
```

And the resource is to be

```golang
type Resource {
  ID   string
  Type string
}
```

For the snapshots, the Type field will be formatted by
snapshots/%{type}, like snapshots/overlayfs.

fix: #3295

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2019-05-29 11:00:32 +08:00
Sebastiaan van Stijn
2ed8e60fa1 bump google.golang.org/grpc v1.20.1
full diff: https://github.com/grpc/grpc-go/compare/v1.12.2...v1.20.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-05-05 12:39:23 -07:00
Davor Kapsa
cfc36388b3 Remove redundant error checks
Signed-off-by: Davor Kapsa <davor.kapsa@gmail.com>
2019-04-30 21:28:51 +02:00
Michael Crosby
8722ec03c3 Merge pull request #3213 from jcordasc/small-fixes
Correct import path and PusherFunc signature
2019-04-16 15:32:51 -04:00
Jared Cordasco
a7af3c13a5 Correct import path in services/server package
Signed-off-by: Jared Cordasco <jcordasc@coglib.com>
2019-04-11 23:07:46 -05:00
Michael Crosby
97d247cd0f Rename hrpc to tcpServer
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-04-11 15:06:06 -04:00
Derek McGowan
5f4c977ba0 Merge pull request #3162 from crosbymichael/tcpservice
Add tcp service for grpc listeners
2019-04-11 11:55:00 -07:00
Derek McGowan
acca107732 Merge pull request #3204 from crosbymichael/fix-forward
Fix API forward events for shims
2019-04-11 11:46:46 -07:00
Phil Estes
5703f415c6 Merge pull request #3197 from Random-Liu/add-required-plugins
Add support for required plugins.
2019-04-11 12:02:16 +02:00
Lantao Liu
4b3b99ea11 Add support for required plugins.
Signed-off-by: Lantao Liu <lantaol@google.com>
2019-04-10 11:31:59 -07:00
Michael Crosby
4ba756edda Fix API forward events for shims
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-04-10 13:34:34 -04:00
Michael Crosby
b911ae3428 Add tcp service for grpc listeners
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-04-10 11:12:50 -04:00
Michael Crosby
a6f587e4c4 Use ttrpc to publish runtime v2 events
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-04-09 14:38:50 -04:00
Michael Crosby
a8a805cad3 Add ttrpc server to containerd
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-04-09 13:08:23 -04:00
Stefan Berger
baf3403439 Extend Applier's Apply() method with an optional options parameter
Extend the Applier interface's Apply method with an optional
options parameter.

For the container image encryption we intend to use the options
parameter to pass image decryption parameters ('dcparameters'),
which are primarily (privatte) keys, in form of a JSON document
under the map key '_dcparameters', and pass them to the Applier's
Apply() method. This helps us to access decryption keys and start
the pipeline with the layer decryption before the layer data are
unzipped and untarred.

Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Signed-off-by: Harshal Patil <harshal.patil@in.ibm.com>
2019-04-02 18:19:48 -04:00
Michael Crosby
559dfa59a1 Add configurable plugin path
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-04-01 16:54:27 -04:00
Michael Crosby
9b882c44f8 Merge pull request #3000 from stefanberger/descriptor_annotations.pr
Add missing annotations map to Descriptor for gRPC transfer
2019-03-22 14:05:44 -04:00
John Howard
6034c1950a Windows:Create root/state with ACL
Signed-off-by: John Howard <jhoward@microsoft.com>
2019-03-21 18:47:34 -07:00
Akihiro Suda
f3ff95ab72 decouple api.ContentServer implementation package from bbolt dependency
For moby/buildkit#886

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2019-03-19 16:09:45 +09:00
Stefan Berger
0b711d616a Copy annotations around where necessary
Make sure that the newly added annotations are copied around appropriately.

Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
2019-03-06 12:26:23 -05:00
Michael Crosby
84a24711e8 Add runc.v2 multi-shim
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-02-21 11:09:46 -05:00
Wei Fu
132ee9b826 fix: linter issue
megacheck, gosimple and unused has been deprecated and subsumed by
staticcheck. And staticcheck also has been upgraded. we need to update
code for the linter issue.

close: #2945

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2019-01-23 22:54:51 +08:00
Derek McGowan
8706a355dd Merge pull request #2889 from linxiulei/isolated_content
metadata: define content sharing policy
2019-01-14 13:15:38 -08:00
Eric Lin
4247f2684d metadata: define content sharing policy
This changeset modifies the metadata store to allow one to set a
"content sharing policy" that defines how blobs are shared between
namespaces in the content store.

The default mode "shared" will make blobs available in all namespaces
once it is pulled into any namespace.  The blob will be pulled into
the namespace if a writer is opened with the "Expected" digest that
is already present in the backend.

The alternative mode, "isolated" requires that clients prove they have
access to the content by providing all of the content to the ingest
before the blob is added to the namespace.

Both modes share backing data, while "shared" will reduce total
bandwidth across namespaces, at the cost of allowing access to any
blob just by knowing its digest.

Note: Most functional codes and changelog of this commit originate from
Stephen J Day <stephen.day@docker.com>, see
40455aade8

Fixes #1713 Fixes #2865

Signed-off-by: Eric Lin <linxiulei@gmail.com>
2018-12-21 15:02:21 +08:00
Phil Estes
06e04bc5a9 Merge pull request #2830 from Ace-Tang/support_cr_without_image
cr: support checkpoint/restore without image
2018-12-20 13:24:37 -05:00
zhangyue
6ccb0d0629 fix: remove dead code
Signed-off-by: zhangyue <zy675793960@yeah.net>
2018-12-13 09:55:08 +08:00