Commit Graph

992 Commits

Author SHA1 Message Date
Michael Crosby
51671ef734
Merge pull request #3630 from AkihiroSuda/ctr-newdockerauthorized
ctr: use NewDockerAuthorizer instead of deprecated NewAuthorizer
2019-09-09 12:47:10 -04:00
Akihiro Suda
c1fc21e92e ctr: use NewDockerAuthorizer instead of deprecated NewAuthorizer
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2019-09-09 02:38:07 +09:00
Michael Crosby
fa11147e5f Add --env-file to ctr
Closes #3517

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-09-06 16:25:02 -04:00
Michael Crosby
f76eefd272
Merge pull request #3574 from mxpv/cfg
Support config imports
2019-09-04 16:34:11 -04:00
Derek McGowan
01f7265892
Support v1 configurations for config dump
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-09-03 17:20:44 -07:00
Derek McGowan
b039c39186
Merge pull request #3564 from tiborvass/move-cgroups-dep-to-namespaces-pkg
runtime/opts: move WithNamespaceCgroupDeletion from containerd to its own package
2019-09-03 10:38:53 -07:00
Lantao Liu
dd24d76a13 Fix potential containerd panic during graceful shutdown.
Signed-off-by: Lantao Liu <lantaol@google.com>
2019-09-01 22:11:20 -07:00
Maksym Pavlenko
db3a711738 Add 'containerd config dump' subcommand
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-08-30 10:55:32 -07:00
Michael Crosby
779701b29c Add --seccomp flag to ctr
This enables testing of containers with the default seccomp profile

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-08-29 13:02:21 -04:00
Tibor Vass
6624a70d92 runtime/opts: move WithNamespaceCgroupDeletion from containerd to its own package
The cgroup dependency brings in quite a lot only for WithNamespaceCgroupDeletion,
which is a namespaces.DeleteOpt.

Signed-off-by: Tibor Vass <tibor@docker.com>
2019-08-27 19:02:55 +00:00
Michael Crosby
df84e546b5
Merge pull request #3577 from dmcgowan/change-default-manifest-download
[carry #3127] Update pull default to skip all platform manifests
2019-08-26 15:04:15 -04:00
Derek McGowan
a40c3830df
Add option to pull all metadata
Add flags to pull and fetch to grab all metadata.
Add fetch option to pull only metadata.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-08-26 11:05:21 -07:00
fesu
5cab0d3f3f update the max if cur will be greater than max for setting the limit
Signed-off-by: Su Fei <sofat1989@126.com>
2019-08-26 15:09:01 +08:00
Yu Yi
aae2d0d754
delete unnecessary checks and fix a test
Signed-off-by: Yu Yi <yiyu@google.com>
2019-08-23 14:02:56 -07:00
Yu Yi
9e183f5e52
add cli option to download all manifests
- Add `all-manifests` option to both `ctr content fetch` and `ctr
  images pull`. By default it is false.
- This option ties to `AppendDistributionSourceLabel` in client.

Signed-off-by: Yu Yi <yiyu@google.com>
2019-08-23 14:02:56 -07:00
Kevin Parsons
d7e1b25384 Allow explicit configuration of TTRPC address
Previously the TTRPC address was generated as "<GRPC address>.ttrpc".
This change now allows explicit configuration of the TTRPC address, with
the default still being the old format if no value is specified.

As part of this change, a new configuration section is added for TTRPC
listener options.

Signed-off-by: Kevin Parsons <kevpar@microsoft.com>
2019-08-22 00:56:27 -07:00
Michael Crosby
08061c7c3c
Merge pull request #3540 from crosbymichael/shim-hang
Use non-blocking send and retry for exit events
2019-08-20 09:31:21 -04:00
Phil Estes
c537c8968c
Merge pull request #3550 from kevpar/remove-eventlog
Remove Windows EventLog logging hook
2019-08-19 15:04:42 -04:00
Kevin Parsons
5023d96ad2 Remove Windows EventLog logging hook
EventLog is very old and provides a poor experience. We have supported
ETW for logging for a while, which is much better. We have also
observed an issue where EventLog keeps containerd.exe open, preventing
containerd from being upgraded to a new version. Due to all of this,
it makes sense to remove the old EventLog hook in favor of using ETW
logging on Windows as the primary diagnostic experience.

Signed-off-by: Kevin Parsons <kevpar@microsoft.com>
2019-08-19 10:24:37 -07:00
Michael Crosby
bee4c1a8a2 Add retry and non-blocking send for exit events
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-08-16 13:55:05 +00:00
Michael Crosby
0d27d8f4f2 Unifi reaper logic into package
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-08-16 13:55:05 +00:00
Michael Crosby
2e8ea9fd6b Allow timeouts to be configured in config
This adds a singleton `timeout` package that will allow services and user
to configure timeouts in the daemon.  When a service wants to use a
timeout, it should declare a const and register it's default value
inside an `init()` function for that package.  When the default config
is generated, we can use the `timeout` package to provide the available
timeout keys so that a user knows that they can configure.

These show up in the config as follows:

```toml
[timeouts]
  "io.containerd.timeout.shim.cleanup" = 5
  "io.containerd.timeout.shim.load" = 5
  "io.containerd.timeout.shim.shutdown" = 3
  "io.containerd.timeout.task.state" = 2

```

Timeouts in the config are specified in seconds.

Timeouts are very hard to get right and giving this power to the user to
configure things is a huge improvement.  Machines can be faster and
slower and depending on the CPU or load of the machine, a timeout may
need to be adjusted.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-08-13 17:36:32 +00:00
Michael Crosby
d085d9b464 Remove encryption code from containerd core
We are separating out the encryption code and have designed a few new
interfaces and APIs for processing content streams.  This keep the core
clean of encryption code but enables not only encryption but support of
multiple content types ( custom media types ).

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-08-09 15:01:16 +00:00
Phil Estes
03d934adc4
Merge pull request #3474 from crosbymichael/uuid
Add UUID and server info to introspection
2019-08-06 15:58:05 -04:00
Akihiro Suda
225cc7d5bd
Merge pull request #3494 from jterry75/remove_v2
Completely remove Windows v2 in-tree shim
2019-08-07 02:19:12 +09:00
Michael Crosby
9f0bc2b53f Add UUID and server info to introspection
Closes #1862

This adds a new rpc to the introspection service to provide server
information with a generated UUID that is done on demand and the os and
arch of the server.

ctr output:

```bash
> sudo ctr version

Client:
  Version:  v1.2.0-802-g57821695.m
  Revision: 578216950de9c1c188708369e2a31ac6c494dfee.m

Server:
  Version:  v1.2.0-802-g57821695.m
  Revision: 578216950de9c1c188708369e2a31ac6c494dfee.m
  UUID: 92e982a9-f13e-4a2c-9032-e69b27fed454
  OS: linux
  ARCH: amd64
```

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-08-06 14:40:48 +00:00
Justin Terry (VM)
4b5dfaee13 Completely remove Windows v2 in-tree shim
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2019-08-05 16:49:56 -07:00
Lantao Liu
a5940da62c Remove ctr cri load and update cri-tools to v1.15.0
Signed-off-by: Lantao Liu <lantaol@google.com>
2019-08-05 15:22:36 -07:00
Derek McGowan
adad947b77
Merge pull request #3460 from lumjjb/ctrrecipients
Specify protocols in ctr encrypt recipients
2019-08-01 15:37:40 -07:00
Michael Crosby
f543f2fbcd Call CloseIO when stdin closes in ctr
Fixes #2439

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-07-26 19:52:03 +00:00
Brandon Lum
8cd480c233 Specify protocols in ctr encrypt recipients
Signed-off-by: Brandon Lum <lumjjb@gmail.com>
2019-07-26 13:20:22 -04:00
Phil Estes
29930e9185
Merge pull request #3455 from dmcgowan/fix-default-import-compression
Add option to compress blobs on import
2019-07-25 16:45:11 -04:00
Derek McGowan
02826345cf
Add option to compress blobs on import
Change the default back to leave uncompressed and add
option to do the compression.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-07-25 12:05:38 -07:00
Phil Estes
7ff23438f6
Merge pull request #3452 from lumjjb/keybundles
Create CryptoConfig constructors to replace dcparameters
2019-07-25 14:57:59 -04:00
Michael Crosby
2190c0e5f7
Merge pull request #3450 from kevpar/windows-alloc-console
Allocate a conhost during Windows service startup
2019-07-25 14:55:22 -04:00
Phil Estes
15b59f9684
Merge pull request #3441 from YLonely/multiple-task-delete
Update ctr to support deleting multiple tasks at a time
2019-07-25 14:29:31 -04:00
Brandon Lum
3d1fa69694 Implemented constructors for both encryption and decryption
Signed-off-by: Brandon Lum <lumjjb@gmail.com>
2019-07-24 22:19:39 -04:00
BoWen Yan
392668d007 ctr:Support deleting multiple tasks at a time
Signed-off-by: BoWen Yan <loneybw@gmail.com>
2019-07-25 09:59:15 +08:00
Kevin Parsons
1c7eab1979 Allocate a conhost during Windows service startup
Creating a console for containerd causes it to be inherited by any child
processes, which gives us performance and reliability improvements. See
comment in code for more information.

Another option considered here would be to invoke each child process
with the DETACHED_PROCESS flag. This would save us the containerd
console allocation. The difficulty of this approach would be ensuring
that all process invocation points have had this flag added, and that
any future invocations also use the flag.

Signed-off-by: Kevin Parsons <kevpar@microsoft.com>
2019-07-24 17:50:24 -07:00
Phil Estes
fdab4f4789
Merge pull request #3446 from stefanberger/cleanup
Wrap creation of CryptoConfig in constructors
2019-07-24 14:46:13 -04:00
Stefan Berger
364de4c35d Wrap creation of CryptoConfig in constructors
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
2019-07-23 19:53:23 -04:00
Michael Crosby
f055bdb0aa Remove windows v1 runtime
Closes #3094

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-07-23 18:54:37 +00:00
Derek McGowan
f7761411b8
Merge pull request #3424 from crosbymichael/devices
Add linux device options and --devices flag to ctr
2019-07-22 16:39:02 -07:00
Phil Estes
49fdb9e1b4
Merge pull request #3430 from lumjjb/encgcfix
Made fixes and optimizations to encryption GC
2019-07-22 19:23:47 -04:00
Michael Crosby
1945e26ac3 Add concurrent execs in stress tests
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-07-19 19:12:44 +00:00
Michael Crosby
ffa86b25f9 Add new shims for checking binary sizes
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-07-19 17:49:11 +00:00
Brandon Lum
c6d437fd70 Corrected lease implementation
Signed-off-by: Brandon Lum <lumjjb@gmail.com>
2019-07-18 18:17:20 -04:00
Michael Crosby
725d3ad8cb Add --device flag to ctr
Closes #3066

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-07-18 18:51:05 +00:00
Wei Fu
283d5d9005
Merge pull request #3418 from stevvooe/backout-error-changes
errors: use errdefs errors in client and commands
2019-07-17 21:32:52 -04:00
Stephen Day
804ae89be6
errors: use errdefs errors in client and commands
This change moves from specific, global errors to the errdefs errors.
This makes it easy to handle certain classes of errors while still
adding context to the failure.

Signed-off-by: Stephen Day <stephen.day@getcruise.com>
2019-07-17 13:42:28 -07:00
Derek McGowan
dde436e65b Crypto library movement and changes to content helper interfaces
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-07-17 15:21:29 -04:00
Stefan Berger
bf8804c743 Implemented image encryption/decryption libraries and ctr commands
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Signed-off-by: Brandon Lum <lumjjb@gmail.com>
2019-07-17 15:19:58 -04:00
Phil Estes
d6be45e151
Merge pull request #3416 from crosbymichael/hard-code-err
Replace hard coded error messages
2019-07-15 17:14:59 -04:00
Michael Crosby
61d930ad5b Move specific errors to their respective packages
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-07-15 20:21:11 +00:00
Phil Estes
82826df9bd
Small refactor due to CI linter changes
Without the open variable in use, no reason to define it or use it as the
for loop control.

Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
2019-07-15 10:43:31 -07:00
Michael Crosby
f3e148b1cc
Merge pull request #3411 from mxpv/linter
Switch from gometalinter to golangci-lint
2019-07-15 10:58:59 -04:00
Maksym Pavlenko
ef7f46eb7b Fix linter errors
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-07-14 20:49:40 -07:00
Fahed Dorgaa
abc152d14c fix name in containers file
Signed-off-by: Fahed Dorgaa <fahed.dorgaa@gmail.com>
2019-07-14 21:49:07 +02:00
Peng Wang
4988424fc0 * fix: view snapshot is deleted before diff
Signed-off-by: Peng Wang <wang_peng168@163.com>
2019-07-14 07:06:04 -04:00
Fahed Dorgaa
db95af43f3 centralize harded-code message
Signed-off-by: Fahed Dorgaa <fahed.dorgaa@gmail.com>
2019-07-13 15:07:23 +02:00
Maksym Pavlenko
1918ee4d11 Respect default snapshotter label
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-07-10 12:16:43 -07:00
Maksym Pavlenko
4e2fc81edd Add aliases to namespaces subcommand
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-07-10 12:10:14 -07:00
Michael Crosby
6601b406b7 Refactor runtime code for code sharing
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-07-08 11:47:53 -04:00
Eli Uriegas
af93e4009b ctr: Add images tag subcommand
Adds a subcommand for tagging images managed by containerd.

Usage:

  ctr images tag docker.io/library/alpine:latest docker.io/library/alpine:new

Comes with a --force flag to allow you to gracefully overwrite image references
that already exist.

Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
2019-07-06 15:44:17 +00:00
Michael Crosby
876c8890ae
Merge pull request #3384 from mxpv/exec-log-uri
Support --log-uri in exec subcommand
2019-07-02 14:34:13 -04:00
Maksym Pavlenko
46af8ccd2d Check exec log flags compatibility
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-07-02 09:53:11 -07:00
Eric Hotinger
9fda4a5fe6 Bumps the version of go-winio.
This also refactors the lcow and windows
snapshotters to use go-winio's utility functions for checking the
filesystem type.

Signed-off-by: Eric Hotinger <ehotinger@gmail.com>
2019-07-01 14:51:06 -07:00
Maksym Pavlenko
6b59b425e2 Support --log-uri for exec subcommand
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-07-01 13:32:37 -07:00
Michael Crosby
844c58102e Add cgroup delete opt for ns deletion
Closes #3305

This adds an opt and a `--cgroup,-c` flag to `ctr namespaces rm` to
remove the cgroup that is commonly created with runtimes.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-07-01 15:01:02 -04:00
Shukui Yang
ec78305c49 Close the inherited socket fd
containerd-shim has dup the fd 3, and it don't need fd 3 any more.

Signed-off-by: Shukui Yang <keloyangsk@gmail.com>
2019-06-20 19:35:05 +08:00
Michael Crosby
31afff2944 Fix backwards compat with v2 containerd configs
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-06-11 23:44:39 +00:00
Michael Crosby
9547d269a1 Add v2 server config support with plugin URIs
Closes #3210

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-06-10 17:54:20 +00:00
Phil Estes
1c5b384b30
Merge pull request #3313 from kevpar/service-restart-delay
Change restart delay for Windows service to 15s
2019-05-31 15:30:59 +01:00
Kevin Parsons
cee19d1e6e Change restart delay for Windows service to 15s
Previously we waited for 60 seconds after the service faults to restart
it. However, there isn't much benefit to waiting this long. We expect
15 seconds to be a more reasonable delay.

Signed-off-by: Kevin Parsons <kevpar@microsoft.com>
2019-05-29 16:41:27 -07:00
Wei Fu
290fc9eeab
Merge pull request #3301 from dmcgowan/add-run-platform
Fix run with specified platform
2019-05-24 03:22:35 +08:00
Derek McGowan
a274dbe822
Fix run with specified platform
Adds the platform flag to the run command and resolves
the image based on that platform.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-05-23 11:59:33 -07:00
Derek McGowan
25daa7355c
Merge pull request #3192 from thaJeztah/bump_grpc_1.19.1
bump google.golang.org/grpc v1.20.1
2019-05-22 11:58:52 -07:00
Phil Estes
b99a66c267
Merge pull request #3288 from Ace-Tang/easy-shim
ctr: make ctr shim command easy to use
2019-05-17 15:40:39 +02:00
Wei Fu
e61f7f4913
Merge pull request #3172 from dmcgowan/export-docker-compatibility
Update image export to support Docker format
2019-05-17 14:30:54 +08:00
Ace-Tang
6907062863 ctr: make ctr shim command easy to use
make ctr shim command easy to use for user, shim socket is generated
through sha256, and it can not get directly, change socket flag to id
command, generated socket in code.
It also avoid fail to connect shim v2, since shim v2 have multiple
containers, `ctr shim --socket state` should specify container id, or
get error `rpc error: code = NotFound desc = container not created: not
found`

Signed-off-by: Ace-Tang <aceapril@126.com>
2019-05-17 11:56:08 +08:00
Wei Fu
fbb80b9510 containerd-shim: redirect output into stdout fifo
Redirect is used to make sure that containerd still can read the log of
shim after restart.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2019-05-12 23:03:28 +08:00
Michael Crosby
67b45aef49 Add WithoutRefreshed metadata
Closes #2566

This provides faster lookups and lists for ctr commands.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-05-10 15:16:14 +00:00
Michael Crosby
bd27bef4ad Move checkpoint and restore commands to new files
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-05-10 14:20:19 +00:00
Sebastiaan van Stijn
2ed8e60fa1
bump google.golang.org/grpc v1.20.1
full diff: https://github.com/grpc/grpc-go/compare/v1.12.2...v1.20.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-05-05 12:39:23 -07:00
Phil Estes
d71c7ada27
Merge pull request #3259 from BenTheElder/no-unpack
ctr images import: add --no-unpack option
2019-05-05 12:29:45 -07:00
Benjamin Elder
cb7c780af2 ctr images import: add --no-unpack option
Signed-off-by: Benjamin Elder <bentheelder@google.com>
2019-05-01 17:09:30 -07:00
Davor Kapsa
cfc36388b3 Remove redundant error checks
Signed-off-by: Davor Kapsa <davor.kapsa@gmail.com>
2019-04-30 21:28:51 +02:00
Kevin Parsons
0376dd4605 Don't write dumped stacks to file for ETW capture state
Signed-off-by: Kevin Parsons <kevpar@microsoft.com>
2019-04-19 10:11:24 -07:00
John Howard
7718d060eb Write stack dump to os.TempDir() as well
Signed-off-by: John Howard <jhoward@microsoft.com>
2019-04-18 09:26:11 -07:00
Kevin Parsons
78f3dc433c Allow dumping stacks via ETW capture state
Signed-off-by: Kevin Parsons <kevpar@microsoft.com>
2019-04-15 13:35:35 -07:00
Derek McGowan
5f4c977ba0
Merge pull request #3162 from crosbymichael/tcpservice
Add tcp service for grpc listeners
2019-04-11 11:55:00 -07:00
Phil Estes
5703f415c6
Merge pull request #3197 from Random-Liu/add-required-plugins
Add support for required plugins.
2019-04-11 12:02:16 +02:00
Lantao Liu
4b3b99ea11 Add support for required plugins.
Signed-off-by: Lantao Liu <lantaol@google.com>
2019-04-10 11:31:59 -07:00
Michael Crosby
b911ae3428 Add tcp service for grpc listeners
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-04-10 11:12:50 -04:00
Michael Crosby
a8a805cad3 Add ttrpc server to containerd
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-04-09 13:08:23 -04:00
Derek McGowan
4754d2aeee
Update image export to support Docker format
Add manifest.json file which is used by Docker
to import images.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-04-04 15:23:06 -07:00
Lantao Liu
74eb0dc812 Return event publish errors.
Signed-off-by: Lantao Liu <lantaol@google.com>
2019-04-04 14:55:38 -07:00
Michael Crosby
e7b6fea572
Merge pull request #3022 from mxpv/devmapper
Integrate device mapper snapshotter
2019-03-29 00:06:11 -04:00
Phil Estes
9ab4c8cbcc
Merge pull request #3108 from alculquicondor/fix/import
Allow to import an image for the default platform only.
2019-03-19 13:29:57 -04:00
Aldo Culquicondor
9a8727cf09 Allow to import an image for the default platform only.
Add `all-platforms` option to `ctr images import`.

Signed-off-by: Aldo Culquicondor <acondor@google.com>
2019-03-19 11:25:57 -04:00