Commit Graph

9017 Commits

Author SHA1 Message Date
Phil Estes
bd4f468c62
Merge pull request #5186 from cpuguy83/fix_docker_cert_loading
Fix docker style cert loading.
2021-03-15 10:36:42 -04:00
Phil Estes
5461fa3a75
Merge pull request #5196 from Iceber/fix-rootfs
rootfs: fix the error handling of the snapshotter.Commit
2021-03-15 09:54:47 -04:00
Wei Fu
d895118c7c runtime/v2/runc: fix leaking socket path
When runC shimv2 starts, the StartShim interface will re-exec itself as
long-running process, which will read the `address` during initializing.

```happycase
Process

containerd-shim-runc-v1/v2 start             containerd-shim-runc-v1/v2

	initializing socket

	reexec containerd-shim-runc-v1/v2

	write address into file

						initializing

							read address

	write back to containerd daemon

						serving

						...

						remove address in Shutdown call
```

However, there is no synchronization after reexec. Then the data race is
like:

```leaking-case
Process

containerd-shim-runc-v1/v2 start             containerd-shim-runc-v1/v2

	initializing socket

	reexec containerd-shim-runc-v1/v2

						initializing

							read address

	write address into file

	write back to containerd daemon

						serving

						...

						fail to remove address
						because of empty address
```

The `address` should be writen into file first before reexec.

And if shutdown the whole service before cleanup temporary
resource (like socket file), the Shutdown caller will receive `ttrpc: closed`
sometime, which depends on go runtime scheduler. Then it also causes leaking
socket files.

Since the shimV2-Delete binary API must be called to cleanup shim temporary
resource and shimV2-runC-v1 doesn't support grouping multi containers in one,
it is safe to remove the socket file in the binary call for shimV2-runC-v1.
But for the shimV2-runC-v2 shim, we still cleanup socket in Shutdown.
Hopefully we can find a way to cleanup socket in shimV2-Delete binary
call.

Fix: #5173

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2021-03-15 18:32:00 +08:00
pacoxu
a76cefd124 plugin status should be skip, not error
Signed-off-by: pacoxu <paco.xu@daocloud.io>
2021-03-15 18:04:37 +08:00
Shiming Zhang
766e7953ab Change dgst to digest in debug
Signed-off-by: Shiming Zhang <wzshiming@foxmail.com>
2021-03-15 16:46:19 +08:00
Iceber Gu
4e8b2f309a
rootfs: fix the error handling of the createInitLayer
Signed-off-by: Iceber Gu <wei.cai-nat@daocloud.io>
2021-03-15 15:48:48 +08:00
Fu, Wei
6b410ba41f
Merge pull request #5197 from Iceber/fix-ctr-command
cmd/ctr: fix export command
2021-03-15 15:41:14 +08:00
Fu, Wei
06e6f45c31
Merge pull request #5198 from Iceber/fix-usage
cmd/ctr: use e.g. in the command usage
2021-03-15 15:37:39 +08:00
Iceber Gu
d3ad7f3908
cmd/ctr: use e.g. in the command usage
Signed-off-by: Iceber Gu <wei.cai-nat@daocloud.io>
2021-03-15 13:48:25 +08:00
Iceber Gu
231bbdc379
cmd/ctr: fix export command
Signed-off-by: Iceber Gu <wei.cai-nat@daocloud.io>
2021-03-15 13:22:56 +08:00
Akihiro Suda
ecb881e5e6
add imgcrypt stream processors to the default config
Enable the following config by default:

```toml
version = 2

[plugins."io.containerd.grpc.v1.cri".image_decryption]
  key_model = "node"

[stream_processors]
  [stream_processors."io.containerd.ocicrypt.decoder.v1.tar.gzip"]
    accepts = ["application/vnd.oci.image.layer.v1.tar+gzip+encrypted"]
    returns = "application/vnd.oci.image.layer.v1.tar+gzip"
    path = "ctd-decoder"
    args = ["--decryption-keys-path", "/etc/containerd/ocicrypt/keys"]
    env = ["OCICRYPT_KEYPROVIDER_CONFIG=/etc/containerd/ocicrypt/ocicrypt_keyprovider.conf"]
  [stream_processors."io.containerd.ocicrypt.decoder.v1.tar"]
    accepts = ["application/vnd.oci.image.layer.v1.tar+encrypted"]
    returns = "application/vnd.oci.image.layer.v1.tar"
    path = "ctd-decoder"
    args = ["--decryption-keys-path", "/etc/containerd/ocicrypt/keys"]
    env = ["OCICRYPT_KEYPROVIDER_CONFIG=/etc/containerd/ocicrypt/ocicrypt_keyprovider.conf"]
```

Fix issue 5128

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-03-15 13:27:16 +09:00
Akihiro Suda
ac2726e12c
cmd/containerd: deduplicate config*.go
`config_linux.go` and `config_windows.go` are identical.

`config_unsupported.go` is also almost identical but enables debug logs by default.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-03-15 13:27:16 +09:00
Akihiro Suda
9a7ca39cbd
defaults: add DefaultConfigDir
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-03-15 13:27:15 +09:00
Madhav Jivrajani
8f863afd3a Use net.IP.IsLoopback() to match loopback addresses
- changed the `MatchLocalhost` function in remotes/docker/registry.go
	- Make use of SplitHostPort to split host and port number
- Added additional tests for modified code in remotes/docker/registry_test.go
- Note: this does not handle mathcing of IP addresses in octal, decimal or hex format or a mix of these.

Fixes: #5129

Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
2021-03-15 09:57:01 +05:30
Wei Fu
eabd9b98b6 runtime: ignore file-already-closed error if dead shim
fix: #5130

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2021-03-15 12:18:26 +08:00
Maksym Pavlenko
e231b955dd
Merge pull request #5154 from estesp/zero-len
Better error when handling a descriptor of size zero
2021-03-14 15:40:39 -07:00
Maksym Pavlenko
5b7f2657d7
Merge pull request #5184 from thaJeztah/fixate_buildtags
Prevent runc inheriting BUILDTAGS from containerd
2021-03-13 22:32:01 -08:00
Phil Estes
f6a9942733
Better error when handling a descriptor of size zero
Signed-off-by: Phil Estes <estesp@amazon.com>
2021-03-13 22:48:55 -05:00
Phil Estes
92009ad7a3
Merge pull request #5164 from errordeveloper/master
Improve unexpected response error handling
2021-03-13 22:05:53 -05:00
Phil Estes
6f94b156fa
Merge pull request #5189 from TBBle/reduce-load-on-ratelimited-docker.io
Reduce load on ratelimited docker.io
2021-03-13 22:05:15 -05:00
Phil Estes
42266dadcf
Merge pull request #5182 from thaJeztah/bump_go_runc
go.mod: github.com/containerd/go-runc v0.0.0-20201020171139-16b287bc67d0
2021-03-13 22:01:56 -05:00
Sebastiaan van Stijn
bd2c0898aa
go.mod: github.com/containerd/go-runc v0.0.0-20201020171139-16b287bc67d0
full diff: 7016d3ce23...16b287bc67

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-13 20:12:12 +01:00
Derek McGowan
1432839310
Merge pull request #5185 from thaJeztah/update_imgcrypt
go.mod: update containerd/imgcrypt, containerd/aufs, containerd/zfs
2021-03-13 09:55:14 -08:00
Sebastiaan van Stijn
24a8b460e7
go.mod: github.com/containerd/zfs v0.0.0-20210313052028-2233970b74b1
full diff: 11e8f1707f...2233970b74

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-13 10:45:23 +01:00
Sebastiaan van Stijn
f0b6bcb775
go.mod: github.com/containerd/aufs v0.0.0-20210313051948-ffdde1490233
full diff: 76a6863f29...ffdde14902

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-13 10:41:46 +01:00
Paul "TBBle" Hampson
5cfc4a80d4 Enable some image-pull tests on Windows
Now that they are using a multi-arch image, they should work on Windows
like they work elsewhere.

This also means non-AMD64 platforms do this test with their native
platform version, not the linux/amd64 platform version.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2021-03-13 13:11:02 +11:00
Paul "TBBle" Hampson
b375f90128 Use k8s.gcr.io/pause:3.4.1 in pull-only tests
This reduces the need to pull random images from docker.io, and should
greatly reduce the tendancy to hit their hourly rate-limit during
integration test runs.

TestImagePullSomePlatforms uses k8s.gcr.io/pause:3.2 so that it does not
see the content pulled by TestImagePullAllPlatforms. This image is
multi-arch, but not multi-os.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2021-03-13 13:08:12 +11:00
Paul "TBBle" Hampson
bcc02002a2 go mod tidy after containerd moved to hcsshim v0.8.15
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2021-03-13 12:38:36 +11:00
Brian Goff
b0b6d9aa03 Add support for using a host registry dir in cri
This will be used instead of the cri registry config in the main config
toml.

---

Also pulls in changes from containerd/cri@d0b4eecbb3

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-03-12 22:42:22 +00:00
Brian Goff
1fd99e24a2 Fix docker style cert loading.
The certs dir parsing was skipping over files instead of reading them,
as such the certs would never load.

It was also stating the file name rather than the full path for cert
pairs.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-03-12 22:42:20 +00:00
Sebastiaan van Stijn
fcab1fe537
go.mod: github.com/containerd/imgcrypt 7ed62a527887
full diff: 0ae5c75f59...7ed62a5278

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-12 19:16:37 +01:00
Sebastiaan van Stijn
55a9bbc033
Prevent runc inheriting BUILDTAGS from containerd
Both runc and containerd use BUILDTAGS to customize go build-tags.

When building containerd with custom build-tags, runc inherited
those, causing the default to be overwritten, e.g.;

    make BUILDTAGS=no_btrfs cri-cni-release
    (in script/setup/install-runc)

    HEAD is now at 12644e61 VERSION: release 1.0.0~rc93
    make[1]: Entering directory '/tmp/tmp.ZJzc2KtI0A/runc'
    go build -trimpath "-mod=vendor" "-buildmode=pie"  -tags "no_btrfs" -ldflags "-X main.gitCommit="12644e614e25b05da6fd08a38ffa0cfe1903fdec" -X main.version=1.0.0-rc93 " -o runc .
                                                   ^^^^^^^^^^^^^^^^

This patch brings back the BUILDTAGS make-var in the runc-install
script, which fixates the buildtags to our defaults.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-12 18:37:18 +01:00
Derek McGowan
8cf669ce34
Fix unsupported files exporting functions for apparmor and seccomp
Signed-off-by: Derek McGowan <derek@mcg.dev>
2021-03-12 08:47:05 -08:00
Derek McGowan
35eeb24a17
Fix exported comments enforcer in CI
Add comments where missing and fix incorrect comments

Signed-off-by: Derek McGowan <derek@mcg.dev>
2021-03-12 08:47:05 -08:00
Derek McGowan
9d5c1165a6
Merge pull request #5178 from estesp/go-1.16.2
Update to Go 1.16.2
2021-03-12 08:23:18 -08:00
Derek McGowan
031775ee5e
Merge pull request #5179 from dims/document-the-runtime-public-method
Document the Runtime public method
2021-03-12 08:20:16 -08:00
Davanum Srinivas
b446ed1ca8
Document the Runtime public method
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2021-03-12 10:29:04 -05:00
Phil Estes
a2f5a9d43a
Merge pull request #5101 from dims/drop-github.com/Microsoft/hcsshim/test
Move *_test.go in root directory to integration/client
2021-03-12 10:07:54 -05:00
Phil Estes
8cbef0fade
Update to Go 1.16.2
Signed-off-by: Phil Estes <estesp@amazon.com>
2021-03-12 09:59:01 -05:00
Kazuyoshi Kato
437b7f71ba ci: log the status of GitHub Actions' VM at the end
To investigate issues like #4969, it would be helpful to understand
the status of the VM at the end.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2021-03-11 16:52:57 -08:00
Davanum Srinivas
6a4aa1e2e7
Separate go module for client tests
Will help us drop dependency to github.com/Microsoft/hcsshim/test in the
main go.mod

Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2021-03-11 19:27:45 -05:00
Derek McGowan
d8208e2e37
Merge pull request #5170 from dims/switch-zuul-merge-mode 2021-03-11 14:52:35 -08:00
Davanum Srinivas
481105834b
Switch from merge-resolve(default) to merge mode for containerd-build-arm64
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2021-03-11 16:53:15 -05:00
Mike Brown
2205da64a6 clean up install-cni and install-critools
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
2021-03-11 10:21:32 -06:00
Mike Brown
1b1bb48709 adds a new install-deps in the makefile; and some debug out
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
2021-03-11 10:21:32 -06:00
Mike Brown
ce8e8e8907
Merge pull request #5036 from thaJeztah/split_runc_binary
Separate runc binary version from libcontainer version, and remove obsolete build-tags
2021-03-11 09:24:30 -06:00
Akihiro Suda
381cbdd39b
Merge pull request #5056 from estesp/go-1.16
Update to Go 1.16.1
2021-03-11 23:53:08 +09:00
Ilya Dmitrichenko
d1b7784357
Use ErrUnexpectedStatus more consistently
Signed-off-by: Ilya Dmitrichenko <errordeveloper@gmail.com>
2021-03-11 14:37:59 +00:00
Ilya Dmitrichenko
a11d785784
Include URL and method in ErrUnexpectedStatus
This should help with debugging expected responses.

Signed-off-by: Ilya Dmitrichenko <errordeveloper@gmail.com>
2021-03-11 14:28:46 +00:00
Phil Estes
8634cd9dfd
Merge pull request #5157 from AkihiroSuda/deflake-TestRestartMonitor
deflake TestRestartMonitor
2021-03-11 09:04:34 -05:00