Commit Graph

1087 Commits

Author SHA1 Message Date
Michael Crosby
f3e148b1cc Merge pull request #3411 from mxpv/linter
Switch from gometalinter to golangci-lint
2019-07-15 10:58:59 -04:00
Maksym Pavlenko
ef7f46eb7b Fix linter errors
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-07-14 20:49:40 -07:00
Fahed Dorgaa
abc152d14c fix name in containers file
Signed-off-by: Fahed Dorgaa <fahed.dorgaa@gmail.com>
2019-07-14 21:49:07 +02:00
Peng Wang
4988424fc0 * fix: view snapshot is deleted before diff
Signed-off-by: Peng Wang <wang_peng168@163.com>
2019-07-14 07:06:04 -04:00
Fahed Dorgaa
db95af43f3 centralize harded-code message
Signed-off-by: Fahed Dorgaa <fahed.dorgaa@gmail.com>
2019-07-13 15:07:23 +02:00
Maksym Pavlenko
1918ee4d11 Respect default snapshotter label
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-07-10 12:16:43 -07:00
Maksym Pavlenko
4e2fc81edd Add aliases to namespaces subcommand
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-07-10 12:10:14 -07:00
Michael Crosby
6601b406b7 Refactor runtime code for code sharing
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-07-08 11:47:53 -04:00
Eli Uriegas
af93e4009b ctr: Add images tag subcommand
Adds a subcommand for tagging images managed by containerd.

Usage:

  ctr images tag docker.io/library/alpine:latest docker.io/library/alpine:new

Comes with a --force flag to allow you to gracefully overwrite image references
that already exist.

Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
2019-07-06 15:44:17 +00:00
Michael Crosby
876c8890ae Merge pull request #3384 from mxpv/exec-log-uri
Support --log-uri in exec subcommand
2019-07-02 14:34:13 -04:00
Maksym Pavlenko
46af8ccd2d Check exec log flags compatibility
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-07-02 09:53:11 -07:00
Eric Hotinger
9fda4a5fe6 Bumps the version of go-winio.
This also refactors the lcow and windows
snapshotters to use go-winio's utility functions for checking the
filesystem type.

Signed-off-by: Eric Hotinger <ehotinger@gmail.com>
2019-07-01 14:51:06 -07:00
Maksym Pavlenko
6b59b425e2 Support --log-uri for exec subcommand
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-07-01 13:32:37 -07:00
Michael Crosby
844c58102e Add cgroup delete opt for ns deletion
Closes #3305

This adds an opt and a `--cgroup,-c` flag to `ctr namespaces rm` to
remove the cgroup that is commonly created with runtimes.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-07-01 15:01:02 -04:00
Shukui Yang
ec78305c49 Close the inherited socket fd
containerd-shim has dup the fd 3, and it don't need fd 3 any more.

Signed-off-by: Shukui Yang <keloyangsk@gmail.com>
2019-06-20 19:35:05 +08:00
Michael Crosby
31afff2944 Fix backwards compat with v2 containerd configs
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-06-11 23:44:39 +00:00
Michael Crosby
9547d269a1 Add v2 server config support with plugin URIs
Closes #3210

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-06-10 17:54:20 +00:00
Phil Estes
1c5b384b30 Merge pull request #3313 from kevpar/service-restart-delay
Change restart delay for Windows service to 15s
2019-05-31 15:30:59 +01:00
Kevin Parsons
cee19d1e6e Change restart delay for Windows service to 15s
Previously we waited for 60 seconds after the service faults to restart
it. However, there isn't much benefit to waiting this long. We expect
15 seconds to be a more reasonable delay.

Signed-off-by: Kevin Parsons <kevpar@microsoft.com>
2019-05-29 16:41:27 -07:00
Wei Fu
290fc9eeab Merge pull request #3301 from dmcgowan/add-run-platform
Fix run with specified platform
2019-05-24 03:22:35 +08:00
Derek McGowan
a274dbe822 Fix run with specified platform
Adds the platform flag to the run command and resolves
the image based on that platform.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-05-23 11:59:33 -07:00
Derek McGowan
25daa7355c Merge pull request #3192 from thaJeztah/bump_grpc_1.19.1
bump google.golang.org/grpc v1.20.1
2019-05-22 11:58:52 -07:00
Phil Estes
b99a66c267 Merge pull request #3288 from Ace-Tang/easy-shim
ctr: make ctr shim command easy to use
2019-05-17 15:40:39 +02:00
Wei Fu
e61f7f4913 Merge pull request #3172 from dmcgowan/export-docker-compatibility
Update image export to support Docker format
2019-05-17 14:30:54 +08:00
Ace-Tang
6907062863 ctr: make ctr shim command easy to use
make ctr shim command easy to use for user, shim socket is generated
through sha256, and it can not get directly, change socket flag to id
command, generated socket in code.
It also avoid fail to connect shim v2, since shim v2 have multiple
containers, `ctr shim --socket state` should specify container id, or
get error `rpc error: code = NotFound desc = container not created: not
found`

Signed-off-by: Ace-Tang <aceapril@126.com>
2019-05-17 11:56:08 +08:00
Wei Fu
fbb80b9510 containerd-shim: redirect output into stdout fifo
Redirect is used to make sure that containerd still can read the log of
shim after restart.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2019-05-12 23:03:28 +08:00
Michael Crosby
67b45aef49 Add WithoutRefreshed metadata
Closes #2566

This provides faster lookups and lists for ctr commands.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-05-10 15:16:14 +00:00
Michael Crosby
bd27bef4ad Move checkpoint and restore commands to new files
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-05-10 14:20:19 +00:00
Sebastiaan van Stijn
2ed8e60fa1 bump google.golang.org/grpc v1.20.1
full diff: https://github.com/grpc/grpc-go/compare/v1.12.2...v1.20.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-05-05 12:39:23 -07:00
Phil Estes
d71c7ada27 Merge pull request #3259 from BenTheElder/no-unpack
ctr images import: add --no-unpack option
2019-05-05 12:29:45 -07:00
Benjamin Elder
cb7c780af2 ctr images import: add --no-unpack option
Signed-off-by: Benjamin Elder <bentheelder@google.com>
2019-05-01 17:09:30 -07:00
Davor Kapsa
cfc36388b3 Remove redundant error checks
Signed-off-by: Davor Kapsa <davor.kapsa@gmail.com>
2019-04-30 21:28:51 +02:00
Kevin Parsons
0376dd4605 Don't write dumped stacks to file for ETW capture state
Signed-off-by: Kevin Parsons <kevpar@microsoft.com>
2019-04-19 10:11:24 -07:00
John Howard
7718d060eb Write stack dump to os.TempDir() as well
Signed-off-by: John Howard <jhoward@microsoft.com>
2019-04-18 09:26:11 -07:00
Kevin Parsons
78f3dc433c Allow dumping stacks via ETW capture state
Signed-off-by: Kevin Parsons <kevpar@microsoft.com>
2019-04-15 13:35:35 -07:00
Derek McGowan
5f4c977ba0 Merge pull request #3162 from crosbymichael/tcpservice
Add tcp service for grpc listeners
2019-04-11 11:55:00 -07:00
Phil Estes
5703f415c6 Merge pull request #3197 from Random-Liu/add-required-plugins
Add support for required plugins.
2019-04-11 12:02:16 +02:00
Lantao Liu
4b3b99ea11 Add support for required plugins.
Signed-off-by: Lantao Liu <lantaol@google.com>
2019-04-10 11:31:59 -07:00
Michael Crosby
b911ae3428 Add tcp service for grpc listeners
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-04-10 11:12:50 -04:00
Michael Crosby
a8a805cad3 Add ttrpc server to containerd
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-04-09 13:08:23 -04:00
Derek McGowan
4754d2aeee Update image export to support Docker format
Add manifest.json file which is used by Docker
to import images.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-04-04 15:23:06 -07:00
Lantao Liu
74eb0dc812 Return event publish errors.
Signed-off-by: Lantao Liu <lantaol@google.com>
2019-04-04 14:55:38 -07:00
Michael Crosby
e7b6fea572 Merge pull request #3022 from mxpv/devmapper
Integrate device mapper snapshotter
2019-03-29 00:06:11 -04:00
Phil Estes
9ab4c8cbcc Merge pull request #3108 from alculquicondor/fix/import
Allow to import an image for the default platform only.
2019-03-19 13:29:57 -04:00
Aldo Culquicondor
9a8727cf09 Allow to import an image for the default platform only.
Add `all-platforms` option to `ctr images import`.

Signed-off-by: Aldo Culquicondor <acondor@google.com>
2019-03-19 11:25:57 -04:00
Phil Estes
b90eeadafa Merge pull request #3102 from Ace-Tang/arg_usage
ctr: fix image cmd ArgsUsage
2019-03-18 08:37:18 -07:00
Ace-Tang
14a050688d ctr: fix image cmd ArgsUsage
ctr image list/check 's ArgsUsage should be filter, not ref

Signed-off-by: Ace-Tang <aceapril@126.com>
2019-03-18 21:11:34 +08:00
Phil Estes
9ed2c0aa02 Merge pull request #3085 from crosbymichael/shim-logs
Shim pluggable logging
2019-03-13 12:23:06 -07:00
Michael Crosby
e6ae9cc64f Shim pluggable logging
Closes #603

This adds logging facilities at the shim level to provide minimal I/O
overhead and pluggable logging options.  Log handling is done within the
shim so that all I/O, cpu, and memory can be charged to the container.

A sample logging driver setting up logging for a container the systemd
journal looks like this:

```go
package main

import (
	"bufio"
	"context"
	"fmt"
	"io"
	"sync"

	"github.com/containerd/containerd/runtime/v2/logging"
	"github.com/coreos/go-systemd/journal"
)

func main() {
	logging.Run(log)
}

func log(ctx context.Context, config *logging.Config, ready func() error) error {
	// construct any log metadata for the container
	vars := map[string]string{
		"SYSLOG_IDENTIFIER": fmt.Sprintf("%s:%s", config.Namespace, config.ID),
	}
	var wg sync.WaitGroup
	wg.Add(2)
	// forward both stdout and stderr to the journal
	go copy(&wg, config.Stdout, journal.PriInfo, vars)
	go copy(&wg, config.Stderr, journal.PriErr, vars)

	// signal that we are ready and setup for the container to be started
	if err := ready(); err != nil {
		return err
	}
	wg.Wait()
	return nil
}

func copy(wg *sync.WaitGroup, r io.Reader, pri journal.Priority, vars map[string]string) {
	defer wg.Done()
	s := bufio.NewScanner(r)
	for s.Scan() {
		if s.Err() != nil {
			return
		}
		journal.Send(s.Text(), pri, vars)
	}
}
```

A `logging` package has been created to assist log developers create
logging plugins for containerd.

This uses a URI based approach for logging drivers that can be expanded
in the future.

Supported URI scheme's are:

* binary
* fifo
* file

You can pass the log url via ctr on the command line:

```bash
> ctr run --rm --runtime io.containerd.runc.v2 --log-uri binary://shim-journald docker.io/library/redis:alpine redis
```

```bash
> journalctl -f -t default:redis

-- Logs begin at Tue 2018-12-11 16:29:51 EST. --
Mar 08 16:08:22 deathstar default:redis[120760]: 1:C 08 Mar 2019 21:08:22.703 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
Mar 08 16:08:22 deathstar default:redis[120760]: 1:M 08 Mar 2019 21:08:22.704 # You requested maxclients of 10000 requiring at least 10032 max file descriptors.
Mar 08 16:08:22 deathstar default:redis[120760]: 1:M 08 Mar 2019 21:08:22.704 # Server can't set maximum open files to 10032 because of OS error: Operation not permitted.
Mar 08 16:08:22 deathstar default:redis[120760]: 1:M 08 Mar 2019 21:08:22.704 # Current maximum open files is 1024. maxclients has been reduced to 992 to compensate for low ulimit. If you need higher maxclients increase 'ulimit -n'.
Mar 08 16:08:22 deathstar default:redis[120760]: 1:M 08 Mar 2019 21:08:22.705 * Running mode=standalone, port=6379.
Mar 08 16:08:22 deathstar default:redis[120760]: 1:M 08 Mar 2019 21:08:22.705 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
Mar 08 16:08:22 deathstar default:redis[120760]: 1:M 08 Mar 2019 21:08:22.705 # Server initialized
Mar 08 16:08:22 deathstar default:redis[120760]: 1:M 08 Mar 2019 21:08:22.705 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
Mar 08 16:08:22 deathstar default:redis[120760]: 1:M 08 Mar 2019 21:08:22.705 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
Mar 08 16:08:22 deathstar default:redis[120760]: 1:M 08 Mar 2019 21:08:22.705 * Ready to accept connections
Mar 08 16:08:50 deathstar default:redis[120760]: 1:signal-handler (1552079330) Received SIGINT scheduling shutdown...
Mar 08 16:08:50 deathstar default:redis[120760]: 1:M 08 Mar 2019 21:08:50.405 # User requested shutdown...
Mar 08 16:08:50 deathstar default:redis[120760]: 1:M 08 Mar 2019 21:08:50.406 * Saving the final RDB snapshot before exiting.
Mar 08 16:08:50 deathstar default:redis[120760]: 1:M 08 Mar 2019 21:08:50.452 * DB saved on disk
Mar 08 16:08:50 deathstar default:redis[120760]: 1:M 08 Mar 2019 21:08:50.453 # Redis is now ready to exit, bye bye...
```

The following client side Opts are added:

```go
// LogURI provides the raw logging URI
func LogURI(uri *url.URL) Creator { }
// BinaryIO forwards contianer STDOUT|STDERR directly to a logging binary
func BinaryIO(binary string, args map[string]string) Creator {}
```

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-03-12 12:18:28 -04:00
Wei Fu
506b815483 remotes: add distribution labels to blob data
We can use cross repository push feature to reuse the existing blobs in
the same registry. Before make push fast, we know where the blob comes
from.

Use the `containerd.io/distribution.source. = [,]` as label format. For
example, the blob is downloaded by the docker.io/library/busybox:latest
and the label will be

    containerd.io/distribution.source.docker.io = library/busybox

If the blob is shared by different repos in the same registry, the repo
name will be appended, like:

    containerd.io/distribution.source.docker.io = library/busybox,x/y

NOTE:
1. no need to apply for legacy docker image schema1.
2. the concurrent fetch actions might miss some repo names in label, but
it is ok.
3. it is optional. no need to add label if the engine only uses images
not push.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2019-03-12 13:42:54 +08:00