Commit Graph

1153 Commits

Author SHA1 Message Date
Eli Uriegas
036db34f37
build: Fix manpage generation
Seems to be that docs/man/ctr.1.md and docs/man/containerd.1.md were
removed in #3637 and were not updated correctly in the Makefile, leading
to build failures like:

    + make man

    make: *** No rule to make target `man/ctr.1', needed by `man'.  Stop.

Changes the gen-manpages command to be specific on which manpages are to
be generated.

Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
2019-10-08 18:22:23 +00:00
fahed dorgaa
c0984941e3 adding go version to client description
Signed-off-by: fahed dorgaa <fahed.dorgaa@gmail.com>

gofmt version.go

Signed-off-by: fahed dorgaa <fahed.dorgaa@gmail.com>

add a Godoc

Signed-off-by: fahed dorgaa <fahed.dorgaa@gmail.com>

go fmt version.go

Signed-off-by: fahed dorgaa <fahed.dorgaa@gmail.com>

add sapces to comment

Signed-off-by: fahed dorgaa <fahed.dorgaa@gmail.com>
2019-10-06 13:38:51 +02:00
Phil Estes
309c9c25ba
Merge pull request #3706 from mrueg/enable-autocmpl
ctr: Enable shell autocompletion
2019-10-03 09:15:51 -04:00
Michael Crosby
8ff5827e98 Update cri and cgroup packages
This change includes a cri master bump and a cgroup bump for windows support
with cgroup stats and reusing the cgroup metric types.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-10-01 10:19:56 -04:00
Manuel Rüger
b5fa55b0c2 ctr: Enable shell autocompletion
Signed-off-by: Manuel Rüger <manuel@rueg.eu>
2019-10-01 09:43:21 +02:00
yuxiaobo
a0ae24b984 Word spelling correction
Signed-off-by: yuxiaobo <yuxiaobogo@163.com>
2019-09-25 16:49:54 +08:00
Eli Uriegas
2a636f8c34
gen-manpages: Create man directory if it does not exist
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
2019-09-23 20:31:26 +00:00
Michael Crosby
fd6299be98 Fix density spec generation
Fixes #3580

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-09-17 16:21:25 -04:00
Lajos Papp
19ecd49ed0 implement ctr -connect-timeout
Signed-off-by: Lajos Papp <lalyos@yahoo.com>
2019-09-16 13:56:53 +02:00
Michael Crosby
5a656cacb4 Move manpage gen to separate binary
This moves the man page generation to a separate binary

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-09-12 14:19:00 -04:00
Michael Crosby
f3a5b8c0a9 Add command to generate man pages
The climan package has a command that can be registered with any urfav
cli app to generate man pages.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-09-11 15:31:02 -04:00
Michael Crosby
51671ef734
Merge pull request #3630 from AkihiroSuda/ctr-newdockerauthorized
ctr: use NewDockerAuthorizer instead of deprecated NewAuthorizer
2019-09-09 12:47:10 -04:00
Akihiro Suda
c1fc21e92e ctr: use NewDockerAuthorizer instead of deprecated NewAuthorizer
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2019-09-09 02:38:07 +09:00
Michael Crosby
fa11147e5f Add --env-file to ctr
Closes #3517

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-09-06 16:25:02 -04:00
Michael Crosby
f76eefd272
Merge pull request #3574 from mxpv/cfg
Support config imports
2019-09-04 16:34:11 -04:00
Derek McGowan
01f7265892
Support v1 configurations for config dump
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-09-03 17:20:44 -07:00
Derek McGowan
b039c39186
Merge pull request #3564 from tiborvass/move-cgroups-dep-to-namespaces-pkg
runtime/opts: move WithNamespaceCgroupDeletion from containerd to its own package
2019-09-03 10:38:53 -07:00
Lantao Liu
dd24d76a13 Fix potential containerd panic during graceful shutdown.
Signed-off-by: Lantao Liu <lantaol@google.com>
2019-09-01 22:11:20 -07:00
Maksym Pavlenko
db3a711738 Add 'containerd config dump' subcommand
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-08-30 10:55:32 -07:00
Michael Crosby
779701b29c Add --seccomp flag to ctr
This enables testing of containers with the default seccomp profile

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-08-29 13:02:21 -04:00
Tibor Vass
6624a70d92 runtime/opts: move WithNamespaceCgroupDeletion from containerd to its own package
The cgroup dependency brings in quite a lot only for WithNamespaceCgroupDeletion,
which is a namespaces.DeleteOpt.

Signed-off-by: Tibor Vass <tibor@docker.com>
2019-08-27 19:02:55 +00:00
Michael Crosby
df84e546b5
Merge pull request #3577 from dmcgowan/change-default-manifest-download
[carry #3127] Update pull default to skip all platform manifests
2019-08-26 15:04:15 -04:00
Derek McGowan
a40c3830df
Add option to pull all metadata
Add flags to pull and fetch to grab all metadata.
Add fetch option to pull only metadata.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-08-26 11:05:21 -07:00
fesu
5cab0d3f3f update the max if cur will be greater than max for setting the limit
Signed-off-by: Su Fei <sofat1989@126.com>
2019-08-26 15:09:01 +08:00
Yu Yi
aae2d0d754
delete unnecessary checks and fix a test
Signed-off-by: Yu Yi <yiyu@google.com>
2019-08-23 14:02:56 -07:00
Yu Yi
9e183f5e52
add cli option to download all manifests
- Add `all-manifests` option to both `ctr content fetch` and `ctr
  images pull`. By default it is false.
- This option ties to `AppendDistributionSourceLabel` in client.

Signed-off-by: Yu Yi <yiyu@google.com>
2019-08-23 14:02:56 -07:00
Kevin Parsons
d7e1b25384 Allow explicit configuration of TTRPC address
Previously the TTRPC address was generated as "<GRPC address>.ttrpc".
This change now allows explicit configuration of the TTRPC address, with
the default still being the old format if no value is specified.

As part of this change, a new configuration section is added for TTRPC
listener options.

Signed-off-by: Kevin Parsons <kevpar@microsoft.com>
2019-08-22 00:56:27 -07:00
Michael Crosby
08061c7c3c
Merge pull request #3540 from crosbymichael/shim-hang
Use non-blocking send and retry for exit events
2019-08-20 09:31:21 -04:00
Phil Estes
c537c8968c
Merge pull request #3550 from kevpar/remove-eventlog
Remove Windows EventLog logging hook
2019-08-19 15:04:42 -04:00
Kevin Parsons
5023d96ad2 Remove Windows EventLog logging hook
EventLog is very old and provides a poor experience. We have supported
ETW for logging for a while, which is much better. We have also
observed an issue where EventLog keeps containerd.exe open, preventing
containerd from being upgraded to a new version. Due to all of this,
it makes sense to remove the old EventLog hook in favor of using ETW
logging on Windows as the primary diagnostic experience.

Signed-off-by: Kevin Parsons <kevpar@microsoft.com>
2019-08-19 10:24:37 -07:00
Michael Crosby
bee4c1a8a2 Add retry and non-blocking send for exit events
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-08-16 13:55:05 +00:00
Michael Crosby
0d27d8f4f2 Unifi reaper logic into package
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-08-16 13:55:05 +00:00
Michael Crosby
2e8ea9fd6b Allow timeouts to be configured in config
This adds a singleton `timeout` package that will allow services and user
to configure timeouts in the daemon.  When a service wants to use a
timeout, it should declare a const and register it's default value
inside an `init()` function for that package.  When the default config
is generated, we can use the `timeout` package to provide the available
timeout keys so that a user knows that they can configure.

These show up in the config as follows:

```toml
[timeouts]
  "io.containerd.timeout.shim.cleanup" = 5
  "io.containerd.timeout.shim.load" = 5
  "io.containerd.timeout.shim.shutdown" = 3
  "io.containerd.timeout.task.state" = 2

```

Timeouts in the config are specified in seconds.

Timeouts are very hard to get right and giving this power to the user to
configure things is a huge improvement.  Machines can be faster and
slower and depending on the CPU or load of the machine, a timeout may
need to be adjusted.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-08-13 17:36:32 +00:00
Michael Crosby
d085d9b464 Remove encryption code from containerd core
We are separating out the encryption code and have designed a few new
interfaces and APIs for processing content streams.  This keep the core
clean of encryption code but enables not only encryption but support of
multiple content types ( custom media types ).

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-08-09 15:01:16 +00:00
Phil Estes
03d934adc4
Merge pull request #3474 from crosbymichael/uuid
Add UUID and server info to introspection
2019-08-06 15:58:05 -04:00
Akihiro Suda
225cc7d5bd
Merge pull request #3494 from jterry75/remove_v2
Completely remove Windows v2 in-tree shim
2019-08-07 02:19:12 +09:00
Michael Crosby
9f0bc2b53f Add UUID and server info to introspection
Closes #1862

This adds a new rpc to the introspection service to provide server
information with a generated UUID that is done on demand and the os and
arch of the server.

ctr output:

```bash
> sudo ctr version

Client:
  Version:  v1.2.0-802-g57821695.m
  Revision: 578216950de9c1c188708369e2a31ac6c494dfee.m

Server:
  Version:  v1.2.0-802-g57821695.m
  Revision: 578216950de9c1c188708369e2a31ac6c494dfee.m
  UUID: 92e982a9-f13e-4a2c-9032-e69b27fed454
  OS: linux
  ARCH: amd64
```

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-08-06 14:40:48 +00:00
Justin Terry (VM)
4b5dfaee13 Completely remove Windows v2 in-tree shim
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2019-08-05 16:49:56 -07:00
Lantao Liu
a5940da62c Remove ctr cri load and update cri-tools to v1.15.0
Signed-off-by: Lantao Liu <lantaol@google.com>
2019-08-05 15:22:36 -07:00
Derek McGowan
adad947b77
Merge pull request #3460 from lumjjb/ctrrecipients
Specify protocols in ctr encrypt recipients
2019-08-01 15:37:40 -07:00
Michael Crosby
f543f2fbcd Call CloseIO when stdin closes in ctr
Fixes #2439

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-07-26 19:52:03 +00:00
Brandon Lum
8cd480c233 Specify protocols in ctr encrypt recipients
Signed-off-by: Brandon Lum <lumjjb@gmail.com>
2019-07-26 13:20:22 -04:00
Phil Estes
29930e9185
Merge pull request #3455 from dmcgowan/fix-default-import-compression
Add option to compress blobs on import
2019-07-25 16:45:11 -04:00
Derek McGowan
02826345cf
Add option to compress blobs on import
Change the default back to leave uncompressed and add
option to do the compression.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-07-25 12:05:38 -07:00
Phil Estes
7ff23438f6
Merge pull request #3452 from lumjjb/keybundles
Create CryptoConfig constructors to replace dcparameters
2019-07-25 14:57:59 -04:00
Michael Crosby
2190c0e5f7
Merge pull request #3450 from kevpar/windows-alloc-console
Allocate a conhost during Windows service startup
2019-07-25 14:55:22 -04:00
Phil Estes
15b59f9684
Merge pull request #3441 from YLonely/multiple-task-delete
Update ctr to support deleting multiple tasks at a time
2019-07-25 14:29:31 -04:00
Brandon Lum
3d1fa69694 Implemented constructors for both encryption and decryption
Signed-off-by: Brandon Lum <lumjjb@gmail.com>
2019-07-24 22:19:39 -04:00
BoWen Yan
392668d007 ctr:Support deleting multiple tasks at a time
Signed-off-by: BoWen Yan <loneybw@gmail.com>
2019-07-25 09:59:15 +08:00
Kevin Parsons
1c7eab1979 Allocate a conhost during Windows service startup
Creating a console for containerd causes it to be inherited by any child
processes, which gives us performance and reliability improvements. See
comment in code for more information.

Another option considered here would be to invoke each child process
with the DETACHED_PROCESS flag. This would save us the containerd
console allocation. The difficulty of this approach would be ensuring
that all process invocation points have had this flag added, and that
any future invocations also use the flag.

Signed-off-by: Kevin Parsons <kevpar@microsoft.com>
2019-07-24 17:50:24 -07:00
Phil Estes
fdab4f4789
Merge pull request #3446 from stefanberger/cleanup
Wrap creation of CryptoConfig in constructors
2019-07-24 14:46:13 -04:00
Stefan Berger
364de4c35d Wrap creation of CryptoConfig in constructors
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
2019-07-23 19:53:23 -04:00
Michael Crosby
f055bdb0aa Remove windows v1 runtime
Closes #3094

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-07-23 18:54:37 +00:00
Derek McGowan
f7761411b8
Merge pull request #3424 from crosbymichael/devices
Add linux device options and --devices flag to ctr
2019-07-22 16:39:02 -07:00
Phil Estes
49fdb9e1b4
Merge pull request #3430 from lumjjb/encgcfix
Made fixes and optimizations to encryption GC
2019-07-22 19:23:47 -04:00
Michael Crosby
1945e26ac3 Add concurrent execs in stress tests
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-07-19 19:12:44 +00:00
Michael Crosby
ffa86b25f9 Add new shims for checking binary sizes
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-07-19 17:49:11 +00:00
Brandon Lum
c6d437fd70 Corrected lease implementation
Signed-off-by: Brandon Lum <lumjjb@gmail.com>
2019-07-18 18:17:20 -04:00
Michael Crosby
725d3ad8cb Add --device flag to ctr
Closes #3066

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-07-18 18:51:05 +00:00
Wei Fu
283d5d9005
Merge pull request #3418 from stevvooe/backout-error-changes
errors: use errdefs errors in client and commands
2019-07-17 21:32:52 -04:00
Stephen Day
804ae89be6
errors: use errdefs errors in client and commands
This change moves from specific, global errors to the errdefs errors.
This makes it easy to handle certain classes of errors while still
adding context to the failure.

Signed-off-by: Stephen Day <stephen.day@getcruise.com>
2019-07-17 13:42:28 -07:00
Derek McGowan
dde436e65b Crypto library movement and changes to content helper interfaces
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-07-17 15:21:29 -04:00
Stefan Berger
bf8804c743 Implemented image encryption/decryption libraries and ctr commands
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Signed-off-by: Brandon Lum <lumjjb@gmail.com>
2019-07-17 15:19:58 -04:00
Phil Estes
d6be45e151
Merge pull request #3416 from crosbymichael/hard-code-err
Replace hard coded error messages
2019-07-15 17:14:59 -04:00
Michael Crosby
61d930ad5b Move specific errors to their respective packages
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-07-15 20:21:11 +00:00
Phil Estes
82826df9bd
Small refactor due to CI linter changes
Without the open variable in use, no reason to define it or use it as the
for loop control.

Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
2019-07-15 10:43:31 -07:00
Michael Crosby
f3e148b1cc
Merge pull request #3411 from mxpv/linter
Switch from gometalinter to golangci-lint
2019-07-15 10:58:59 -04:00
Maksym Pavlenko
ef7f46eb7b Fix linter errors
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-07-14 20:49:40 -07:00
Fahed Dorgaa
abc152d14c fix name in containers file
Signed-off-by: Fahed Dorgaa <fahed.dorgaa@gmail.com>
2019-07-14 21:49:07 +02:00
Peng Wang
4988424fc0 * fix: view snapshot is deleted before diff
Signed-off-by: Peng Wang <wang_peng168@163.com>
2019-07-14 07:06:04 -04:00
Fahed Dorgaa
db95af43f3 centralize harded-code message
Signed-off-by: Fahed Dorgaa <fahed.dorgaa@gmail.com>
2019-07-13 15:07:23 +02:00
Maksym Pavlenko
1918ee4d11 Respect default snapshotter label
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-07-10 12:16:43 -07:00
Maksym Pavlenko
4e2fc81edd Add aliases to namespaces subcommand
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-07-10 12:10:14 -07:00
Michael Crosby
6601b406b7 Refactor runtime code for code sharing
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-07-08 11:47:53 -04:00
Eli Uriegas
af93e4009b ctr: Add images tag subcommand
Adds a subcommand for tagging images managed by containerd.

Usage:

  ctr images tag docker.io/library/alpine:latest docker.io/library/alpine:new

Comes with a --force flag to allow you to gracefully overwrite image references
that already exist.

Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
2019-07-06 15:44:17 +00:00
Michael Crosby
876c8890ae
Merge pull request #3384 from mxpv/exec-log-uri
Support --log-uri in exec subcommand
2019-07-02 14:34:13 -04:00
Maksym Pavlenko
46af8ccd2d Check exec log flags compatibility
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-07-02 09:53:11 -07:00
Eric Hotinger
9fda4a5fe6 Bumps the version of go-winio.
This also refactors the lcow and windows
snapshotters to use go-winio's utility functions for checking the
filesystem type.

Signed-off-by: Eric Hotinger <ehotinger@gmail.com>
2019-07-01 14:51:06 -07:00
Maksym Pavlenko
6b59b425e2 Support --log-uri for exec subcommand
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-07-01 13:32:37 -07:00
Michael Crosby
844c58102e Add cgroup delete opt for ns deletion
Closes #3305

This adds an opt and a `--cgroup,-c` flag to `ctr namespaces rm` to
remove the cgroup that is commonly created with runtimes.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-07-01 15:01:02 -04:00
Shukui Yang
ec78305c49 Close the inherited socket fd
containerd-shim has dup the fd 3, and it don't need fd 3 any more.

Signed-off-by: Shukui Yang <keloyangsk@gmail.com>
2019-06-20 19:35:05 +08:00
Michael Crosby
31afff2944 Fix backwards compat with v2 containerd configs
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-06-11 23:44:39 +00:00
Michael Crosby
9547d269a1 Add v2 server config support with plugin URIs
Closes #3210

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-06-10 17:54:20 +00:00
Phil Estes
1c5b384b30
Merge pull request #3313 from kevpar/service-restart-delay
Change restart delay for Windows service to 15s
2019-05-31 15:30:59 +01:00
Kevin Parsons
cee19d1e6e Change restart delay for Windows service to 15s
Previously we waited for 60 seconds after the service faults to restart
it. However, there isn't much benefit to waiting this long. We expect
15 seconds to be a more reasonable delay.

Signed-off-by: Kevin Parsons <kevpar@microsoft.com>
2019-05-29 16:41:27 -07:00
Wei Fu
290fc9eeab
Merge pull request #3301 from dmcgowan/add-run-platform
Fix run with specified platform
2019-05-24 03:22:35 +08:00
Derek McGowan
a274dbe822
Fix run with specified platform
Adds the platform flag to the run command and resolves
the image based on that platform.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-05-23 11:59:33 -07:00
Derek McGowan
25daa7355c
Merge pull request #3192 from thaJeztah/bump_grpc_1.19.1
bump google.golang.org/grpc v1.20.1
2019-05-22 11:58:52 -07:00
Phil Estes
b99a66c267
Merge pull request #3288 from Ace-Tang/easy-shim
ctr: make ctr shim command easy to use
2019-05-17 15:40:39 +02:00
Wei Fu
e61f7f4913
Merge pull request #3172 from dmcgowan/export-docker-compatibility
Update image export to support Docker format
2019-05-17 14:30:54 +08:00
Ace-Tang
6907062863 ctr: make ctr shim command easy to use
make ctr shim command easy to use for user, shim socket is generated
through sha256, and it can not get directly, change socket flag to id
command, generated socket in code.
It also avoid fail to connect shim v2, since shim v2 have multiple
containers, `ctr shim --socket state` should specify container id, or
get error `rpc error: code = NotFound desc = container not created: not
found`

Signed-off-by: Ace-Tang <aceapril@126.com>
2019-05-17 11:56:08 +08:00
Wei Fu
fbb80b9510 containerd-shim: redirect output into stdout fifo
Redirect is used to make sure that containerd still can read the log of
shim after restart.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2019-05-12 23:03:28 +08:00
Michael Crosby
67b45aef49 Add WithoutRefreshed metadata
Closes #2566

This provides faster lookups and lists for ctr commands.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-05-10 15:16:14 +00:00
Michael Crosby
bd27bef4ad Move checkpoint and restore commands to new files
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-05-10 14:20:19 +00:00
Sebastiaan van Stijn
2ed8e60fa1
bump google.golang.org/grpc v1.20.1
full diff: https://github.com/grpc/grpc-go/compare/v1.12.2...v1.20.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-05-05 12:39:23 -07:00
Phil Estes
d71c7ada27
Merge pull request #3259 from BenTheElder/no-unpack
ctr images import: add --no-unpack option
2019-05-05 12:29:45 -07:00
Benjamin Elder
cb7c780af2 ctr images import: add --no-unpack option
Signed-off-by: Benjamin Elder <bentheelder@google.com>
2019-05-01 17:09:30 -07:00
Davor Kapsa
cfc36388b3 Remove redundant error checks
Signed-off-by: Davor Kapsa <davor.kapsa@gmail.com>
2019-04-30 21:28:51 +02:00
Kevin Parsons
0376dd4605 Don't write dumped stacks to file for ETW capture state
Signed-off-by: Kevin Parsons <kevpar@microsoft.com>
2019-04-19 10:11:24 -07:00
John Howard
7718d060eb Write stack dump to os.TempDir() as well
Signed-off-by: John Howard <jhoward@microsoft.com>
2019-04-18 09:26:11 -07:00
Kevin Parsons
78f3dc433c Allow dumping stacks via ETW capture state
Signed-off-by: Kevin Parsons <kevpar@microsoft.com>
2019-04-15 13:35:35 -07:00
Derek McGowan
5f4c977ba0
Merge pull request #3162 from crosbymichael/tcpservice
Add tcp service for grpc listeners
2019-04-11 11:55:00 -07:00
Phil Estes
5703f415c6
Merge pull request #3197 from Random-Liu/add-required-plugins
Add support for required plugins.
2019-04-11 12:02:16 +02:00
Lantao Liu
4b3b99ea11 Add support for required plugins.
Signed-off-by: Lantao Liu <lantaol@google.com>
2019-04-10 11:31:59 -07:00
Michael Crosby
b911ae3428 Add tcp service for grpc listeners
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-04-10 11:12:50 -04:00
Michael Crosby
a8a805cad3 Add ttrpc server to containerd
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-04-09 13:08:23 -04:00
Derek McGowan
4754d2aeee
Update image export to support Docker format
Add manifest.json file which is used by Docker
to import images.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-04-04 15:23:06 -07:00
Lantao Liu
74eb0dc812 Return event publish errors.
Signed-off-by: Lantao Liu <lantaol@google.com>
2019-04-04 14:55:38 -07:00
Michael Crosby
e7b6fea572
Merge pull request #3022 from mxpv/devmapper
Integrate device mapper snapshotter
2019-03-29 00:06:11 -04:00
Phil Estes
9ab4c8cbcc
Merge pull request #3108 from alculquicondor/fix/import
Allow to import an image for the default platform only.
2019-03-19 13:29:57 -04:00
Aldo Culquicondor
9a8727cf09 Allow to import an image for the default platform only.
Add `all-platforms` option to `ctr images import`.

Signed-off-by: Aldo Culquicondor <acondor@google.com>
2019-03-19 11:25:57 -04:00
Phil Estes
b90eeadafa
Merge pull request #3102 from Ace-Tang/arg_usage
ctr: fix image cmd ArgsUsage
2019-03-18 08:37:18 -07:00
Ace-Tang
14a050688d ctr: fix image cmd ArgsUsage
ctr image list/check 's ArgsUsage should be filter, not ref

Signed-off-by: Ace-Tang <aceapril@126.com>
2019-03-18 21:11:34 +08:00
Phil Estes
9ed2c0aa02
Merge pull request #3085 from crosbymichael/shim-logs
Shim pluggable logging
2019-03-13 12:23:06 -07:00
Michael Crosby
e6ae9cc64f Shim pluggable logging
Closes #603

This adds logging facilities at the shim level to provide minimal I/O
overhead and pluggable logging options.  Log handling is done within the
shim so that all I/O, cpu, and memory can be charged to the container.

A sample logging driver setting up logging for a container the systemd
journal looks like this:

```go
package main

import (
	"bufio"
	"context"
	"fmt"
	"io"
	"sync"

	"github.com/containerd/containerd/runtime/v2/logging"
	"github.com/coreos/go-systemd/journal"
)

func main() {
	logging.Run(log)
}

func log(ctx context.Context, config *logging.Config, ready func() error) error {
	// construct any log metadata for the container
	vars := map[string]string{
		"SYSLOG_IDENTIFIER": fmt.Sprintf("%s:%s", config.Namespace, config.ID),
	}
	var wg sync.WaitGroup
	wg.Add(2)
	// forward both stdout and stderr to the journal
	go copy(&wg, config.Stdout, journal.PriInfo, vars)
	go copy(&wg, config.Stderr, journal.PriErr, vars)

	// signal that we are ready and setup for the container to be started
	if err := ready(); err != nil {
		return err
	}
	wg.Wait()
	return nil
}

func copy(wg *sync.WaitGroup, r io.Reader, pri journal.Priority, vars map[string]string) {
	defer wg.Done()
	s := bufio.NewScanner(r)
	for s.Scan() {
		if s.Err() != nil {
			return
		}
		journal.Send(s.Text(), pri, vars)
	}
}
```

A `logging` package has been created to assist log developers create
logging plugins for containerd.

This uses a URI based approach for logging drivers that can be expanded
in the future.

Supported URI scheme's are:

* binary
* fifo
* file

You can pass the log url via ctr on the command line:

```bash
> ctr run --rm --runtime io.containerd.runc.v2 --log-uri binary://shim-journald docker.io/library/redis:alpine redis
```

```bash
> journalctl -f -t default:redis

-- Logs begin at Tue 2018-12-11 16:29:51 EST. --
Mar 08 16:08:22 deathstar default:redis[120760]: 1:C 08 Mar 2019 21:08:22.703 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
Mar 08 16:08:22 deathstar default:redis[120760]: 1:M 08 Mar 2019 21:08:22.704 # You requested maxclients of 10000 requiring at least 10032 max file descriptors.
Mar 08 16:08:22 deathstar default:redis[120760]: 1:M 08 Mar 2019 21:08:22.704 # Server can't set maximum open files to 10032 because of OS error: Operation not permitted.
Mar 08 16:08:22 deathstar default:redis[120760]: 1:M 08 Mar 2019 21:08:22.704 # Current maximum open files is 1024. maxclients has been reduced to 992 to compensate for low ulimit. If you need higher maxclients increase 'ulimit -n'.
Mar 08 16:08:22 deathstar default:redis[120760]: 1:M 08 Mar 2019 21:08:22.705 * Running mode=standalone, port=6379.
Mar 08 16:08:22 deathstar default:redis[120760]: 1:M 08 Mar 2019 21:08:22.705 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
Mar 08 16:08:22 deathstar default:redis[120760]: 1:M 08 Mar 2019 21:08:22.705 # Server initialized
Mar 08 16:08:22 deathstar default:redis[120760]: 1:M 08 Mar 2019 21:08:22.705 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
Mar 08 16:08:22 deathstar default:redis[120760]: 1:M 08 Mar 2019 21:08:22.705 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
Mar 08 16:08:22 deathstar default:redis[120760]: 1:M 08 Mar 2019 21:08:22.705 * Ready to accept connections
Mar 08 16:08:50 deathstar default:redis[120760]: 1:signal-handler (1552079330) Received SIGINT scheduling shutdown...
Mar 08 16:08:50 deathstar default:redis[120760]: 1:M 08 Mar 2019 21:08:50.405 # User requested shutdown...
Mar 08 16:08:50 deathstar default:redis[120760]: 1:M 08 Mar 2019 21:08:50.406 * Saving the final RDB snapshot before exiting.
Mar 08 16:08:50 deathstar default:redis[120760]: 1:M 08 Mar 2019 21:08:50.452 * DB saved on disk
Mar 08 16:08:50 deathstar default:redis[120760]: 1:M 08 Mar 2019 21:08:50.453 # Redis is now ready to exit, bye bye...
```

The following client side Opts are added:

```go
// LogURI provides the raw logging URI
func LogURI(uri *url.URL) Creator { }
// BinaryIO forwards contianer STDOUT|STDERR directly to a logging binary
func BinaryIO(binary string, args map[string]string) Creator {}
```

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-03-12 12:18:28 -04:00
Wei Fu
506b815483 remotes: add distribution labels to blob data
We can use cross repository push feature to reuse the existing blobs in
the same registry. Before make push fast, we know where the blob comes
from.

Use the `containerd.io/distribution.source. = [,]` as label format. For
example, the blob is downloaded by the docker.io/library/busybox:latest
and the label will be

    containerd.io/distribution.source.docker.io = library/busybox

If the blob is shared by different repos in the same registry, the repo
name will be appended, like:

    containerd.io/distribution.source.docker.io = library/busybox,x/y

NOTE:
1. no need to apply for legacy docker image schema1.
2. the concurrent fetch actions might miss some repo names in label, but
it is ok.
3. it is optional. no need to add label if the engine only uses images
not push.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2019-03-12 13:42:54 +08:00
Maksym Pavlenko
1e893b19ce
devmapper: add no_devmapper build tag
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-03-07 10:53:19 -08:00
Wei Fu
6424a36032 ctr/commands/images/push: don't show progress if it is debug mode
If user sets debug mode, the command push should only show the debug log
information. If the stdout is with flush by the progress status, it is
hard to see the debug log.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2019-03-05 22:21:55 +08:00
Michael Crosby
dea27b1b64
Merge pull request #3036 from jhowardmsft/jjh/etw
Windows: Enable ETW logging
2019-02-21 20:12:35 -05:00
Maksym Pavlenko
2218275ec9
devmapper: register plugin
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-02-21 16:25:55 -08:00
John Howard
e0d7ed12a5 Windows: Enable ETW logging
Signed-off-by: John Howard <jhoward@microsoft.com>
2019-02-21 14:16:44 -08:00
Michael Crosby
84a24711e8 Add runc.v2 multi-shim
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-02-21 11:09:46 -05:00
Michael Crosby
6bcbf88f82 Move runc shim code into common package
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-02-21 10:47:41 -05:00
Veeraiah Chowdary Nuvvula
f6ac73d71e fixing typo and added a missing comment.
Signed-off-by: Veeraiah Chowdary Nuvvula <venuvvul@microsoft.com>
2019-02-18 14:13:02 -08:00
John Howard
88f19a90a8 Windows: Update stackdump event name
Signed-off-by: John Howard <jhoward@microsoft.com>
2019-02-15 15:33:29 -08:00
Justin Terry (VM)
00895670e7 Stop sending stderr with TTY on Windows
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2019-02-11 23:11:40 -08:00
John Howard
59ea134ce1 OCI Modifiers for Windows
Signed-off-by: John Howard <jhoward@microsoft.com>

Needed for the containerd work on Windows and integrating the
oci package from containerd into moby.

No longer sets defaults for
 - .Process.ConsoleSize
 - .Windows.IgnoreFlushesDuringBoot
 - .Windows.Network.AllowUnqualifiedDNSQuery

Adds helper functions and tests for
 - WithWindowsIgnoreFlushesDuringBoot
 - WithWindowNetworksAllowUnqualifiedDNSQuery

Updates `ctr run` on Windows to use the new helper functions,
ConsoleSize is already handled.
2019-02-06 10:44:36 -08:00
Wei Fu
132ee9b826 fix: linter issue
megacheck, gosimple and unused has been deprecated and subsumed by
staticcheck. And staticcheck also has been upgraded. we need to update
code for the linter issue.

close: #2945

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2019-01-23 22:54:51 +08:00
Phil Estes
df97414a77
Merge pull request #2622 from kadisi/export_bug
fix ctr image export not found error
2019-01-16 12:59:34 -05:00
Phil Estes
06e04bc5a9
Merge pull request #2830 from Ace-Tang/support_cr_without_image
cr: support checkpoint/restore without image
2018-12-20 13:24:37 -05:00
kadisi
40267382c5 fix ctr image export not found error
Signed-off-by: Jie Zhang <iamkadisi@163.com>
2018-12-17 08:52:28 +08:00
Justin Cormack
8be05eb237
Fix freebsd build
This brings freebsd in line with Darwin, ie it builds, but some parts may not yet
be fully functional. There is now a WIP `runc` port for FreeBSD at
https://github.com/clovertrail/runc/tree/1501-SupportOnFreeBSD so should be able
to test further.

Signed-off-by: Justin Cormack <justin@specialbusservice.com>
2018-12-16 14:27:42 +00:00
Wei Fu
603e97a9ec ctr/content: make editor flag is required
The $EDITOR is not clear for user. If the env doesn't set the value for
$EDITOR, the users don't know how to use this subcommand when they get
error like this:

```
sh: 1: /tmp/edit-605573012: Permission denied
```

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2018-12-11 18:55:09 +08:00
Justin Terry (VM)
7ac221e8d7 Adding ctr memory and cpu flags
Adds ctr run --memory-limit for all platforms.
Adds ctr run --cpu-count for Windows platforms.

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-12-10 13:31:59 -08:00
Michael Crosby
4a8acf54b1
Merge pull request #2793 from lifubang/runexecflag
fix container cmd args may parsed as ctr args
2018-12-04 11:05:51 -05:00
Lifubang
2d3a4f99ec should no defer when detach
Signed-off-by: Lifubang <lifubang@acmcoder.com>
2018-12-03 19:18:57 +08:00
Michael Crosby
09a5b1f8af
Merge pull request #2839 from jhowardmsft/jjh/service
Windows: Allow running as a service
2018-11-30 11:11:16 -05:00
Ace-Tang
6593399e9f cr: support checkpoint/restore without image
support checkpoint without committing a checkpoint dir into a
checkpoint image and restore without untar image into checkpoint
directory. support for both v1 and v2 runtime

Signed-off-by: Ace-Tang <aceapril@126.com>
2018-11-29 10:19:39 +08:00
Derek McGowan
82c57a3a38
Merge pull request #2828 from jterry75/ctr_isolation
Move ctr run --isolation to Windows only
2018-11-28 10:36:50 -08:00
Justin Terry (VM)
b4a642f6a5 Add dumpStacks support for containerd on Windows
Resolves: #1763

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-11-27 21:02:32 -08:00
John Howard
40d898a820 Windows: Allow running as a service
Signed-off-by: John Howard <jhoward@microsoft.com>

Allows containerd.exe to run as a Windows service. eg

Register: `.\containerd.exe --register-service`

Start: `net start containerd`
...
Stop: `net stop containerd`

Unregister: `.\containerd.exe --unregister-service`

When running as a service, logs will go to the Windows application
event log.
2018-11-27 17:28:04 -08:00
Michael Crosby
3eae8b9c3f
Merge pull request #2631 from masters-of-cats/shim-io-redirect
Use named pipes for shim logs
2018-11-27 10:44:00 -05:00
Justin Terry (VM)
f90e5d564a Move ctr run --isolation to Windows only
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-11-23 17:11:59 -08:00
Ace-Tang
461222dba8 fix: should get runtime name from container info
when use `ctr t checkpoint`, runtime name should get from container
info, but not passed by flag, since task has already running with
specified runtime

Signed-off-by: Ace-Tang <aceapril@126.com>
2018-11-23 15:39:43 +08:00
Derek McGowan
32aa0cd79b
Merge pull request #2632 from ehazlett/checkpoint-restore
Refactor checkpoint and restore to client
2018-11-20 16:31:08 -08:00
Lifubang
2101b1362e add ctr delete --exec-id to debug DeleteProcess
Signed-off-by: Lifubang <lifubang@acmcoder.com>
2018-11-20 19:53:06 +08:00
Phil Estes
55baf50b5a
Merge pull request #2798 from lifubang/execd
add flag -d for ctr t exec to run a service in container
2018-11-20 08:34:29 +08:00
Julia Nedialkova
1d4105cacf Use named pipes for shim logs
Relating to issue [#2606](https://github.com/containerd/containerd/issues/2606)

Co-authored-by: Oliver Stenbom <ostenbom@pivotal.io>
Co-authored-by: Georgi Sabev <georgethebeatle@gmail.com>
Co-authored-by: Giuseppe Capizzi <gcapizzi@pivotal.io>
Co-authored-by: Danail Branekov <danailster@gmail.com>

Signed-off-by: Oliver Stenbom <ostenbom@pivotal.io>
Signed-off-by: Georgi Sabev <georgethebeatle@gmail.com>
Signed-off-by: Giuseppe Capizzi <gcapizzi@pivotal.io>
Signed-off-by: Danail Branekov <danailster@gmail.com>
2018-11-16 16:11:43 +02:00
Lifubang
01f5aa3878 add flag -d for ctr t exec to run a service in container
Signed-off-by: Lifubang <lifubang@acmcoder.com>
2018-11-16 14:29:45 +08:00
Lifubang
51ad4d3020 change args usage for ctr c create
Signed-off-by: Lifubang <lifubang@acmcoder.com>
2018-11-16 14:09:39 +08:00
Lifubang
bd2a21985c fix container cmd args may parsed as ctr args
Signed-off-by: Lifubang <lifubang@acmcoder.com>
2018-11-15 18:32:04 +08:00
Evan Hazlett
6f2f4e4343 checkpoint: add copts to checkpoint; save snapshotter to annotation
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
2018-11-12 11:48:16 +00:00
Evan Hazlett
40caece8dc update tests
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
2018-11-12 11:47:17 +00:00
Evan Hazlett
4fdf720b84 move rw to opt; make snapshot opt; move to NewContainerOpts
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
2018-11-12 11:47:17 +00:00
Evan Hazlett
147208061c add image name and runtime name media types; remove task operation on checkpoint
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
2018-11-12 11:47:17 +00:00
Evan Hazlett
ce0673fd7d Restore take image
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
2018-11-12 11:47:17 +00:00
Evan Hazlett
0e4d9da755 remove task handling from Restore
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
2018-11-12 11:47:16 +00:00
Evan Hazlett
2d3db08daf refactor spec and snapshot restore into opts
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
2018-11-12 11:45:22 +00:00
Evan Hazlett
45c700a955 refactor checkpoint and restore to client
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
2018-11-12 11:45:22 +00:00
Akihiro Suda
dd0539b095 ctr: add --cgroup for oci.WithCgroup
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-11-06 17:39:30 +09:00
Michael Crosby
2a10bc7b44
Merge pull request #2759 from fuweid/bugfix_use_right_flag_value
bugfix: use skip-verify flag not insecure in ctr
2018-11-05 14:20:56 -05:00
Justin Terry (VM)
354dab0caa Forward ctr.exe --debug as io.containerd.runhcs.v1 options
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-11-05 09:08:48 -08:00
Justin Terry (VM)
8329a491b9 Build Windows V1 and V2 runtimes always
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-11-02 11:59:03 -07:00
Wei Fu
a02f7cea86 bugfix: use skip-verify flag in ctr instead of insecure
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2018-11-02 23:51:37 +08:00
Ace-Tang
c3cac72b92 ctr: fix potential panic in metric
Signed-off-by: Ace-Tang <aceapril@126.com>
2018-10-23 14:01:50 +08:00
Phil Estes
0dd3211a06
Remove containerd-release tool from main repo
The release-tool is now located in the `containerd/project` repo.

Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
2018-10-19 12:13:38 -04:00
Wei Fu
06616dab00 enhance: split config from server package
The github.com/containerd/containerd/services/server has a lot of
dependencies, like content, snapshots services implementation and
docker-metrics.

For the client side, it uses the config struct from server package
to start up the containerd in background. It will import a lot of
useless packages which might be conflict with existing vendor's package.

It makes integration easier with single config package.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2018-10-17 16:34:39 +08:00
Michael Crosby
f1dfe86ef4
Merge pull request #2688 from crosbymichael/ctrdecodespec
decode spec in `ctr c info`
2018-10-16 09:38:43 -04:00
Michael Crosby
9fe52f66d6
Merge pull request #2721 from Ace-Tang/master
ctr: add some metric item
2018-10-16 09:38:30 -04:00
Ace-Tang
7ea75ef4a9 ctr: add some metric item
add memory limit, pid info into metric subcommand, since moby also
show them. As blkio read/write IO need more calculation,not add them.

Signed-off-by: Ace-Tang <aceapril@126.com>
2018-10-16 16:54:06 +08:00
Michael Crosby
e86a0689fb Fix stress test for image config opt requirements
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-10-15 18:58:16 -04:00
Phil Estes
1a5f9a3434
Merge pull request #2710 from fuweid/enhance_support_duration_for_cpu_profile_collection
enhance: support specific duration for profile collection
2018-10-11 11:00:25 -04:00
Wei Fu
a2a23d91ca enhance: support specific duration for profile collection
The default duration of cpu profile collection in net/http/pprof is 30
seconds. User should have chance to set the specific duration for the
collection.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2018-10-10 22:57:54 +08:00
Phil Estes
43acab8100
Merge pull request #2690 from dmcgowan/resolver-updates
Update Docker resolver to pass in Authorizer interface
2018-10-03 09:58:42 +02:00
Derek McGowan
a6198b7692
Update resolver code
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-09-28 14:32:23 -07:00
Samuel Karp
607888ce29 ctr: make kill use stop-signal by default
The OCI image specification includes a `StopSignal` field in the image
configuration, denoting the system call signal to be sent to the
container to exit.  This commit adds a new `WithImageStopSignal` container
option that can be used for storing the `StopSignal` field as a label on
the container.  This commit also adjusts `ctr run` to call
`WithImageStopSignal` and `ctr tasks kill` to send the signal stored in
that label by default.

Signed-off-by: Samuel Karp <skarp@amazon.com>
2018-09-27 15:53:38 -07:00
Samuel Karp
b392a3a8e4 signals: move ParseSignal to containerd package
Signed-off-by: Samuel Karp <skarp@amazon.com>
2018-09-27 15:53:38 -07:00
Lifubang
cb0eed833d decode Spec value in command 'ctr c info <containerid>'
Signed-off-by: Lifubang <lifubang@aliyun.com>

fix some errors

use typeurl.UnmarshalAny instead json.Unmarshal to interface{}

Signed-off-by: Lifubang <lifubang@aliyun.com>
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-09-27 10:44:54 -04:00
Michael Crosby
6496078ef8
Merge pull request #2669 from estesp/fix-withuser-comment
Remove a TODO from the code comments that is complete
2018-09-21 14:17:23 -04:00
Justin Terry (VM)
547bb94e4b Fix ctr run for Windows containers
1. Fixes bugs in ctr run that were introduced by 1d9b969
2. Adds support for the --isolated flag that runs Windows HyperV
cotainers instead of process isolated containers on Windows.

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-09-20 14:28:36 -07:00
Phil Estes
41615e8ded
Remove a TODO from the code comments that is complete
WithUser... helpers do support non-snapshot rootfs now.

Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
2018-09-20 09:07:36 -04:00
Derek McGowan
9faeea1e5e
Merge pull request #2649 from estesp/nonewpriv-flag
Add flag to ctr for running with "NoNewPrivileges: false"
2018-09-19 11:17:05 -07:00
Michael Crosby
1ad49689d3 Reduce publish connection timeout
Fixes #2662

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-09-19 11:21:40 -04:00
Derek McGowan
a62be324b7
Unify docker and oci importer
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-09-17 14:41:43 -07:00
Derek McGowan
bce20b75da
Simplify docker importer
Support any layout and rely on manifest.json to reference blobs

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-09-14 14:21:27 -07:00
Derek McGowan
9e6db71954
Add docker importer
Update ctr to support all formats by default

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-09-14 14:21:26 -07:00
Derek McGowan
f57c5cdefb
Refactor image importer
Allow customization of reference creation.
Add option for digest references.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-09-14 14:21:26 -07:00
Phil Estes
c28ce39cea
Add flag to ctr for running with NoNewPrivileges: false
Add flag and With-helper to set NoNewPrivileges to false since it is on
by default in the default UNIX spec for containerd, but off by default
in Docker and CRI plugin use. This allows for easy testing with it off
for comparison.

Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
2018-09-14 11:03:58 -04:00
Michael Crosby
60d13d63c4 Add optional install path
This allows users to consume the install functionality but also install
to other areas instead of the managed `/opt` dir.

```bash
> ctr install --path /usr/local
```

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-09-11 10:36:00 -04:00
Michael Crosby
1597270d04
Merge pull request #2579 from lifubang/ctrrun
fix when --config provided, don't need Image/RootFS
2018-09-10 13:09:08 -04:00
ruicao
1f5ab28216 Typo fix: outputing -> outputting
Signed-off-by: ruicao <ruicao@alauda.io>
2018-09-07 12:47:18 +08:00
Lifubang
48fe63511a code optimization after review
Signed-off-by: Lifubang <lifubang@acmcoder.com>
2018-09-05 23:36:31 +08:00
Justin Terry (VM)
c818a6b13d Merges the oci package for Linux and Windows
On Windows we need to be able to create both Linux and Windows OCI spec
files by default to support WCOW and LCOW scenarios. This merges the
compile time differences into runtime differences between the two based
on the spec and platform the user sets.

It maintains the old behavior with Default specs resulting in the
platform default the binary is compiled for.

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-08-30 09:46:04 -07:00
Michael Crosby
d89ba5ee08
Merge pull request #2591 from dmcgowan/update-release-script
Update release script
2018-08-29 08:26:41 -04:00
Derek McGowan
69e7c77e6a
Add option to add links to changelog
Allows creating links in changelog, similar to what Github does
for markdown but works for dependencies as well.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-08-28 17:59:40 -07:00
Derek McGowan
f76a5ec83a
Update template
Fix spacing on releases.
Add tag prefix line

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-08-28 16:12:56 -07:00
Justin Terry (VM)
0110b3c0bc Introduce the Windows lcow diff/snaphotter
Implements the Windows lcow differ/snapshotter responsible for managing
the creation and lifetime of lcow containers on Windows.

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-08-28 13:51:04 -07:00
Michael Crosby
6b00aaaf20
Merge pull request #2573 from lifubang/relativepathforrootfs
support relative rootfs path in ctr
2018-08-27 10:25:25 -04:00
Phil Estes
68a5db67ad
Merge pull request #2582 from lifubang/startd
add -detach flag for 'ctr t start'
2018-08-27 22:23:31 +08:00
Lifubang
66f6dd8b3b add -detach flag for 'ctr t start'
Signed-off-by: Lifubang <lifubang@aliyun.com>
2018-08-27 18:43:56 +08:00
Derek McGowan
9edcfcc1cb
Add platform match comparer interface
Adds a new platform interface for matching and comparing platforms.
This new interface allows both filtering and ordering of platforms
to support running multiple platform and choosing the best platform.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-08-27 00:11:46 -07:00
Lifubang
1d9b96988f fix when --config provided, don't need Image/RootFS
Signed-off-by: Lifubang <lifubang@aliyun.com>
2018-08-27 11:18:25 +08:00
Lifubang
dc6ed04ff5 support relative rootfs path in ctr
Signed-off-by: Lifubang <lifubang@aliyun.com>
2018-08-24 07:56:37 +08:00
Phil Estes
830363acac
Merge pull request #2544 from kadisi/process_args
set args value of process if args is nil
2018-08-22 15:31:30 -04:00
Michael Crosby
4f644dbfd5 Copy io into and out of console
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-08-22 14:04:17 -04:00
Justin Terry (VM)
019b0c34de Introduce containerd-shim-runhcs-v1 on Windows
Implements the containerd-shim-runhcs-v1 shim on Windows for the runtime
v2 shim API.

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-08-22 08:15:43 -07:00
kadisi
7dae56671d set args value of process if args is nil
Signed-off-by: kadisi <iamkadisi@163.com>
2018-08-21 09:44:50 +08:00
Michael Crosby
b67ea850af Add opt for default unix device permissions
These opts either inherit the parent cgroup device.list or append the
default unix devices like /dev/null /dev/random so that the container
has access.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-08-15 11:52:43 -04:00
Michael Crosby
3be457d7d6 Move content.Fetch configuration to struct
This makes it easier for callers to call this function and populate the
config without relying on specific flags across commands.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-08-09 10:26:31 -04:00
Phil Estes
e8c80f37e2
Merge pull request #2530 from dmcgowan/update-release-tool
Update release tool
2018-08-08 10:35:24 -04:00
Michael Crosby
6ba4ddfdda Add shim log pipe for log forwarding to the daemon
A fifo on unix or named pipe on Windows will be provided to the shim.
It can be located inside the `cwd` of the shim named "log".
The shims can use the existing `github.com/containerd/containerd/log` package to log debug messages.
Messages will automatically be output in the containerd's daemon logs with the correct fiels and runtime set.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-08-07 15:31:00 -04:00
Derek McGowan
aeb322d87d
Update release tool
Allow inclusion of sub-project changes
Order contributors by number of contributions
Add mailmap

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-08-06 17:51:10 -07:00
Phil Estes
2783a19b10
Merge pull request #2518 from crosbymichael/install
Add install support for binary images
2018-08-03 08:45:02 -04:00
Akihiro Suda
74b036491a
Merge pull request #2524 from samuelkarp/time.Hour
replace 3600 seconds with 1 hour
2018-08-03 12:41:20 +09:00
Michael Crosby
5a47c5ec1d Add lib support as an option
Some images like `criu` will have extra libs that it requires.  This
adds lib support via LD_LIBRARY_PATH and InstallOpts

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-08-02 17:11:29 -04:00
Evan Hazlett
1537f31381 Add install support for binary images
This adds a way for users to programatically install containerd binary
dependencies.

With runtime v2 and new shim's being built, it will be a challenge to
get those onto machines.  Users would have to find the link, download,
place it in their path, yada yada yada.

With this functionality of a managed `/opt` directory, containerd can
use existing image and distribution infra. to get binarys, shims, etc
onto the system.

Configuration:

*default:* `/opt/containerd`

*containerd config:*
```toml
[plugins.opt]
	path = "/opt/mypath"

```

Usage:

*code:*

```go
image, err := client.Pull(ctx, "docker.io/crosbymichael/runc:latest")
client.Install(ctx, image)
```

*ctr:*

```bash
ctr content fetch docker.io/crosbymichael/runc:latest
ctr install docker.io/crosbymichael/runc:latest
```

You can manage versions and see what is running via standard image
commands.

Images:

These images MUST be small and only contain binaries.

```Dockerfile
FROM scratch
Add runc /bin/runc
```

Containerd will only extract files in `/bin` of the image.

Later on, we can add support for `/lib`.

The code adds a service to manage an `/opt/containerd` directory and
provide that path to callers via the introspection service.

How to Test:

Delete runc from your system.

```bash
> sudo ctr run --rm  docker.io/library/redis:alpine redis
ctr: OCI runtime create failed: unable to retrieve OCI runtime error (open /run/containerd/io.containerd.runtime.v1.linux/default/redis/log.json: no such file or directory): exec: "runc": executable file not found in $PATH: unknown

> sudo ctr content fetch docker.io/crosbymichael/runc:latest
> sudo ctr  install docker.io/crosbymichael/runc:latest

> sudo ctr run --rm  docker.io/library/redis:alpine redis
1:C 01 Aug 15:59:52.864 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 01 Aug 15:59:52.864 # Redis version=4.0.10, bits=64, commit=00000000, modified=0, pid=1, just started
1:C 01 Aug 15:59:52.864 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
1:M 01 Aug 15:59:52.866 # You requested maxclients of 10000 requiring at least 10032 max file descriptors.
1:M 01 Aug 15:59:52.866 # Server can't set maximum open files to 10032 because of OS error: Operation not permitted.
1:M 01 Aug 15:59:52.866 # Current maximum open files is 1024. maxclients has been reduced to 992 to compensate for low ulimit. If you need higher maxclients increase 'ulimit -n'.
1:M 01 Aug 15:59:52.870 * Running mode=standalone, port=6379.
1:M 01 Aug 15:59:52.870 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
1:M 01 Aug 15:59:52.870 # Server initialized
1:M 01 Aug 15:59:52.870 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
1:M 01 Aug 15:59:52.870 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
1:M 01 Aug 15:59:52.870 * Ready to accept connections
^C1:signal-handler (1533139193) Received SIGINT scheduling shutdown...
1:M 01 Aug 15:59:53.472 # User requested shutdown...
1:M 01 Aug 15:59:53.472 * Saving the final RDB snapshot before exiting.
1:M 01 Aug 15:59:53.484 * DB saved on disk
1:M 01 Aug 15:59:53.484 # Redis is now ready to exit, bye bye...
```

Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-08-02 17:11:29 -04:00
Samuel Karp
9665a2650a *: replace 3600 seconds with 1 hour
Signed-off-by: Samuel Karp <skarp@amazon.com>
2018-08-02 11:40:03 -07:00
Justin Terry (VM)
9ff702b9a1 Fix a typo in runc-v1 shim
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-08-02 11:16:31 -07:00
Justin Terry (VM)
790c3a3663 Remove extra allocation in NewTask
Reorders the code so that it doesnt overwrite the previous allocation
when creating a NewTask via ctr.exe

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-08-01 13:22:18 -07:00
Phil Estes
ed0e73422d
Merge pull request #2468 from dmcgowan/set-platform-on-unpack
Update client Image to have configurable platform
2018-07-31 11:56:09 -04:00
Phil Estes
4249f44d81
Merge pull request #2493 from dmcgowan/sync-lease-removal
Add sync option to lease removal
2018-07-31 11:31:55 -04:00
Stephen Day
920dc79b4d
Merge pull request #2504 from samuelkarp/ctr-t-metrics
ctr: add new metrics subcommand
2018-07-30 13:07:19 -07:00
Derek McGowan
2ebfba575c
Merge pull request #2479 from stevvooe/with-file-combinator
oci: introduce WithSpecFromFile combinator
2018-07-27 15:56:53 -07:00
Samuel Karp
9a34bb046a metrics: add optional json output
Signed-off-by: Samuel Karp <skarp@amazon.com>
2018-07-27 15:21:08 -07:00
Stephen Day
2a1bd7414b
oci: introduce WithSpecFromFile combinator
We introduce a WithSpecFromFile option combinator to allow creation
simpler creation of OCI specs from a file name. Often used as the first
option in a `SpecOpts` slice, it simplifies choosing between a local
file and the built-in default.

The code in `ctr run` has been updated to use the new option, with out
changing the order of operations or functionality present there.

Signed-off-by: Stephen Day <stephen.day@getcruise.com>
2018-07-27 14:25:42 -07:00
Derek McGowan
362405f7b5
Merge pull request #2495 from jterry75/runtime_v2_windows
Adds runtime v2 support for Windows shim's
2018-07-27 11:24:34 -07:00
Samuel Karp
2c87d120df ctr: add new metrics subcommand
Signed-off-by: Samuel Karp <skarp@amazon.com>
2018-07-27 09:37:57 -07:00
Justin Terry (VM)
d3e0c163f8 Adds runtime v2 support for Windows shim's
Implements the various requirements for the runtime v2 code to abstract
away the unix/linux code into the appropriate platform level
abstractions to use the runtime v2 on Windows as well.

Adds support in the Makefile.windows to actually build the runtime v2
code for Windows by setting a shell environment BUILD_WINDOWS_V2=1
before calling make. (Note this disables the compilation of the Windows
runtime v1)

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-07-25 14:09:26 -07:00
Michael Crosby
26e2dd6754
Merge pull request #2425 from avagin/docker-fixes
task: allow to specify namespaces which are restored externally
2018-07-25 10:26:08 -04:00
Sebastiaan van Stijn
da73b98b63
Set default log formatting to use RFC3339Nano with fixed width
This patch changes the logs format to use a fixed-width timestamp,
matching the format that's used in dockerd.

Before:

    $ containerd
    INFO[0000] starting containerd                           revision=a88b6319614de846458750ff882723479ca7b1a1 version=v1.1.0-202-ga88b6319
    INFO[0000] loading plugin "io.containerd.content.v1.content"...  type=io.containerd.content.v1
    INFO[0000] loading plugin "io.containerd.snapshotter.v1.btrfs"...  type=io.containerd.snapshotter.v1
    WARN[0000] failed to load plugin io.containerd.snapshotter.v1.btrfs  error="path /var/lib/containerd/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter"

After:

    $ containerd
    INFO[2018-07-24T08:11:07.397856489Z] starting containerd                           revision=c3195155cacb361cd3549c4d78901b20aa19579a version=v1.1.0-203-gc3195155
    INFO[2018-07-24T08:11:07.399264587Z] loading plugin "io.containerd.content.v1.content"...  type=io.containerd.content.v1
    INFO[2018-07-24T08:11:07.399343959Z] loading plugin "io.containerd.snapshotter.v1.btrfs"...  type=io.containerd.snapshotter.v1
    WARN[2018-07-24T08:11:07.399474423Z] failed to load plugin io.containerd.snapshotter.v1.btrfs  error="path /var/lib/containerd/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter"

Or, when running as child-process of dockerd:

Before:

    root@9637fcd85ea4:/go/src/github.com/docker/docker# dockerd --debug
    DEBU[2018-07-24T08:15:16.946312436Z] Listener created for HTTP on unix (/var/run/docker.sock)
    INFO[2018-07-24T08:15:16.947086499Z] libcontainerd: started new docker-containerd process  pid=231
    INFO[2018-07-24T08:15:16.947137166Z] parsed scheme: "unix"                         module=grpc
    INFO[2018-07-24T08:15:16.947235001Z] scheme "unix" not registered, fallback to default scheme  module=grpc
    INFO[2018-07-24T08:15:16.947463403Z] ccResolverWrapper: sending new addresses to cc: [{unix:///var/run/docker/containerd/docker-containerd.sock 0  <nil>}]  module=grpc
    INFO[2018-07-24T08:15:16.947505954Z] ClientConn switching balancer to "pick_first"  module=grpc
    INFO[2018-07-24T08:15:16.947717368Z] pickfirstBalancer: HandleSubConnStateChange: 0xc420507ab0, CONNECTING  module=grpc
    INFO[0000] starting containerd                           revision=d64c661f1d51c48782c9cec8fda7604785f93587 version=v1.1.1
    DEBU[0000] changing OOM score to -500
    INFO[0000] loading plugin "io.containerd.content.v1.content"...  type=io.containerd.content.v1
    INFO[0000] loading plugin "io.containerd.snapshotter.v1.btrfs"...  type=io.containerd.snapshotter.v1
    WARN[0000] failed to load plugin io.containerd.snapshotter.v1.btrfs  error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter"

After:

    DEBU[2018-07-24T08:21:33.441741970Z] Listener created for HTTP on unix (/var/run/docker.sock)
    INFO[2018-07-24T08:21:33.442428017Z] libcontainerd: started new docker-containerd process  pid=232
    INFO[2018-07-24T08:21:33.442510827Z] parsed scheme: "unix"                         module=grpc
    INFO[2018-07-24T08:21:33.442598812Z] scheme "unix" not registered, fallback to default scheme  module=grpc
    INFO[2018-07-24T08:21:33.442681006Z] ccResolverWrapper: sending new addresses to cc: [{unix:///var/run/docker/containerd/docker-containerd.sock 0  <nil>}]  module=grpc
    INFO[2018-07-24T08:21:33.442770353Z] ClientConn switching balancer to "pick_first"  module=grpc
    INFO[2018-07-24T08:21:33.442871502Z] pickfirstBalancer: HandleSubConnStateChange: 0xc42018bc30, CONNECTING  module=grpc
    INFO[2018-07-24T08:21:33.457963804Z] starting containerd                           revision=597dd082e37f8bc6b6265ca05839d7a300861911 version=597dd082
    DEBU[2018-07-24T08:21:33.458113301Z] changing OOM score to -500
    INFO[2018-07-24T08:21:33.458474842Z] loading plugin "io.containerd.content.v1.content"...  type=io.containerd.content.v1
    INFO[2018-07-24T08:21:33.458911054Z] loading plugin "io.containerd.snapshotter.v1.btrfs"...  type=io.containerd.snapshotter.v1
    WARN[2018-07-24T08:21:33.459366268Z] failed to load plugin io.containerd.snapshotter.v1.btrfs  error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter"

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-07-24 10:23:38 +02:00
Michael Crosby
0d52c71c80
Merge pull request #2474 from dmcgowan/lease-expiration
Improved lease management
2018-07-20 16:54:17 -04:00
Andrei Vagin
fc2fcf6b2a task: WithExit() doesn't have to overwrite existing options
Signed-off-by: Andrei Vagin <avagin@openvz.org>
2018-07-20 13:19:05 -07:00
Michael Crosby
17ab11a236 Fixes for runtimev2 and checkpoint restore
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-07-20 12:09:29 -04:00
Derek McGowan
94e132fd07
Add sync option on lease removal
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-07-19 14:37:55 -07:00
Michael Crosby
17ae673b5c Set runtime for stress tests
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-07-19 12:06:39 -04:00
Derek McGowan
94cfce62ba
Merge pull request #2434 from crosbymichael/shimv2
Runtime v2 (shim API)
2018-07-18 13:14:10 -07:00
Derek McGowan
00a99c0472
Add leases subcommand in ctr
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-07-18 10:43:37 -07:00
Akihiro Suda
02579c8c3f
Merge pull request #2415 from dmcgowan/proxy-plugins
Add support for proxy plugins
2018-07-18 11:17:24 +09:00
Derek McGowan
3a916a0f67
Update client Image to have configurable platform
Separate Fetch and Pull commands in client to distinguish
between platform specific and non-platform specific operations.
`ctr images pull` with all platforms will now unpack all platforms.
`ctr content fetch` now supports platform flags.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-07-17 15:43:03 -07:00
Stephen Day
dfde5ec316
Merge pull request #2418 from crosbymichael/hook-root
Handle abs path for rootfs in oci hook
2018-07-17 12:17:28 -07:00
Michael Crosby
ed697290da
Merge pull request #2463 from crosbymichael/temp-clean
Don't prevent boot on temp cleanup
2018-07-17 10:27:21 -04:00
Michael Crosby
da1b5470cd Runtime v2
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-07-17 10:21:29 -04:00
Michael Crosby
0105959c3d Don't prevent boot on temp cleanup
Fixes #2462
Fixes #2455

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-07-13 16:07:17 -04:00
Derek McGowan
cce0a46c8a
Seed random on ctr and containerd startup
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-07-12 17:51:55 -07:00
Kenfe-Mickaël Laventure
5900361791
Merge pull request #2420 from sudeeshjohn/master
Fixing the formatting directives error during compilation
2018-07-05 08:29:07 -07:00
sudeesh john
5a4f007e48 Fix the formatting directives error during compilation
Signed-off-by: Sudeesh John <sudeesh@linux.vnet.ibm.com>
2018-07-05 20:40:21 +05:30
Kenfe-Mickaël Laventure
39b6ba826a
Merge pull request #2431 from masters-of-cats/plugins-list-subcommand
Introduce plugins/list subcommand
2018-07-02 13:38:39 -07:00
Michael Crosby
6a83168157 Update ttrpc to 94dde388801693c54f88a6596f713b51a8
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-07-02 15:24:15 -04:00
Danail Branekov
3cf3881fa4 Introduce plugins/list subcommand
`Ctr` interface follows the pattern `ctr <command> <subcommand>` except
for the `plugins` command which does not have subcommands. This feels
unnatural to certain users and they would expect that they can list
containerd plugins via `ctr plugins list`.

This commit implements their expectation so that `plugins` becomes a
command "group" and its `list` subcommand actually lists the plugins.

Signed-off-by: Danail Branekov <danailster@gmail.com>
2018-07-02 10:20:20 +03:00
Michael Crosby
08150bfe76 Update ttrpc for containerd repo
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-06-28 09:53:40 -04:00