Commit Graph

11497 Commits

Author SHA1 Message Date
Derek McGowan
5a66c2ae5c
Merge pull request #4881 from AkihiroSuda/converter
add Image content converter
2021-01-21 22:24:54 -08:00
Akihiro Suda
5ca3ac65c4
add Image content converter
Go example:
```go
opts := []converter.Opt{
  // convert Docker media types to OCI ones
  converter.WithDocker2OCI(true),
  // convert tar.gz layers to uncompressed tar layers
  converter.WithLayerConvertFunc(uncompress.LayerConvertFunc),
}
srcRef := "example.com/foo:orig"
dstRef := "example.com/foo:converted"
dstImg, err = converter.Convert(ctx, client, dstRef, srcRef, opts...)
fmt.Println(dstImg.Target)
```

ctr example: `ctr images convert --oci --uncompress example.com/foo:orig example.com/foo:converted`

Go test: `go test -exec sudo -test.root -test.run TestConvert`

The implementation is from https://github.com/containerd/stargz-snapshotter/pull/224,
but eStargz-specific functions are not included in this PR.

eStargz converter can be specified by importing `estargz` package and using `WithLayerConvertFunc(estargz.LayerConvertFunc)` option.

This converter interface will be potentially useful for converting zstd and ocicrypt layers as well.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-01-22 13:33:19 +09:00
Derek McGowan
9b9de47eb9
Merge pull request #4824 from dcantah/dcantah/reuse-scratch
Add scratch space re-use functionality to LCOW snapshotter
2021-01-21 17:21:31 -08:00
Derek McGowan
392b2857aa
Merge pull request #4848 from katiewasnothere/ctr_image_chainid
Add a new flag to ctr "images pull" to print the image's chainID
2021-01-21 17:11:53 -08:00
Sebastiaan van Stijn
e1445dff12
profiles: seccomp: update to Linux 5.11 syscall list
These syscalls (some of which have been in Linux for a while but were
missing from the profile) fall into a few buckets:

 * close_range(2), epoll_wait2(2) are just extensions of existing "safe
   for everyone" syscalls.

 * The mountv2 API syscalls (fs*(2), move_mount(2), open_tree(2)) are
   all equivalent to aspects of mount(2) and thus go into the
   CAP_SYS_ADMIN category.

 * process_madvise(2) is similar to the other process_*(2) syscalls and
   thus goes in the CAP_SYS_PTRACE category.

Co-authored-by: Aleksa Sarai <asarai@suse.de>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-01-21 21:30:25 +01:00
Maksim An
ddb5e1651a Enhance logging driver and ctr tasks to support windows
Signed-off-by: Maksim An <maksiman@microsoft.com>
2021-01-21 12:17:32 -08:00
Derek McGowan
83f8d6126b
Merge pull request #4925 from dmcgowan/prepare-1.5.0-beta.0
Add release notes for v1.5.0-beta.0
2021-01-21 09:21:53 -08:00
Derek McGowan
c4bff3d17c
Merge pull request #4943 from AkihiroSuda/platforms-literallyonly
platforms: add subarchless version of Only()
2021-01-20 17:27:43 -08:00
Derek McGowan
7d6d526462
Add release notes for v1.5.0-beta.0
Signed-off-by: Derek McGowan <derek@mcg.dev>
2021-01-20 10:45:16 -08:00
Phil Estes
c35b4cfed5
Merge pull request #4955 from adisky/fix-doc
Clean Up Doc and fix some broken links
2021-01-20 08:46:17 -05:00
Aditi Sharma
d09bf18862 Clean Up Doc and fix some broken links
Signed-off-by: Aditi Sharma <adi.sky17@gmail.com>
2021-01-20 15:45:43 +05:30
Phil Estes
1230bd6303
Merge pull request #4952 from crosbymichael/label-etc-files
[cri] label etc files for selinux containers
2021-01-19 16:21:35 -05:00
Maksym Pavlenko
9c3f171391
Merge pull request #4953 from ImJasonH/cpuinfo
Derive cpuinfo as needed, instead of at init-time
2021-01-19 13:16:30 -08:00
Jason Hall
363f2c392c Derive cpuinfo as needed, instead of at init-time
This changes platforms.Parse to hit /proc to look up CPU info only when
it's needed, instead of in init(). This makes the package a bit easier
for other packages to consume, especially clients that don't call
platforms.Parse or need to lookup CPU info.

Signed-off-by: Jason Hall <jasonhall@redhat.com>
2021-01-19 14:10:39 -05:00
Michael Crosby
a731039238 [cri] label etc files for selinux containers
Signed-off-by: Michael Crosby <michael@thepasture.io>
2021-01-19 13:42:09 -05:00
Hu Shuai
1d26595cf3 Add unit test for func in remotes/docker/pusher.go
Signed-off-by: Hu Shuai <hus.fnst@cn.fujitsu.com>
2021-01-19 16:24:27 +08:00
Daniel Canter
ff1451cab8 Scratch size customization and UVM scratch creation for WCOW snapshotter
* Currently we rely on making the UVMs sandbox.vhdx in the shim itself instead of this being
made by the snapshotter itself. This change adds a label that affects whether to create the UVMs
scratch layer in the snapshotter itself.

* Adds container scratch size customization. Before adding the computestorage calls
(vendored in with https://github.com/containerd/containerd/pull/4859) there was no way to make a containers
or UVMs scratch size less than the default (20 for containers and 10 for the UVM).

Signed-off-by: Daniel Canter <dcanter@microsoft.com>
2021-01-18 07:33:52 -08:00
Akihiro Suda
e22ce0fa03
platforms: add subarchless version of Only()
`OnlyStrict()` returns a match comparer for a single platform.

Unlike `Only()`, `OnlyStrict()` does not match sub platforms.
So, "arm/vN" will not match "arm/vM" where M < N, and "amd64" will not also match "386".

`OnlyStrict()` matches non-canonical forms. So, "arm64" matches "arm/64/v8".

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-01-15 14:58:14 +09:00
Wei Fu
66fec3bbbf
Merge pull request #4932 from tianon/arm64-fallback
Allow arm64 to fallback to arm (v8, v7, v6, v5)
2021-01-14 20:01:49 +08:00
Maksym Pavlenko
cb76f534f6
Merge pull request #4865 from yukiisbored/yuki_is_bored/openbsd-support
Bring OpenBSD support
2021-01-13 17:39:14 -08:00
Daniel Canter
3e5acb9d91 Add scratch space re-use functionality to LCOW snapshotter
Currently we would create a new disk and mount this into the LCOW UVM for every container but there
are certain scenarios where we'd rather just mount a single disk and then have every container share this one
storage space instead of every container having it's own xGB of space to play around with.

This is accomplished by just making a symlink to the disk that we'd like to share and then
using ref counting later on down the stack in hcsshim if we see that we've already mounted this
disk.

Signed-off-by: Daniel Canter <dcanter@microsoft.com>
2021-01-13 15:20:46 -08:00
Tianon Gravi
4073aaa7a9 Allow arm64 to fallback to arm (v8, v7, v6, v5)
This isn't supported by *all* arm64 chips, but it is common enough that I think it's worth an explicit fallback.  I think it will be more common for images to have arm64 support without arm support, but even if a user has an arm64 chip that does not support arm32, having it fail to run the arm32 image is an acceptable compromise (because it's non-trivial to detect arm32 support without running a binary, AFAIK).

Also, before this change the failure would've simply been "no such image" instead of "failed to run" so I think it's pretty reasonable to allow it to try the additional 32bit set of images just in case one of them actually does work (like it will on many popular chips like 64bit Raspberry Pis and AWS Graviton).

Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
2021-01-13 09:19:08 -08:00
Maksym Pavlenko
124fc14a46
Merge pull request #4926 from dims/update-k8s-to-latest-v1.20.x
Update k8s to latest v1.20.x
2021-01-12 17:45:26 -08:00
Kathryn Baldauf
14df541a40 Add a new flag to images pull for printing out the pulled image's chainID
Signed-off-by: Kathryn Baldauf <kabaldau@microsoft.com>
2021-01-12 15:42:40 -08:00
Mike Brown
32506bd454
Merge pull request #4934 from dims/bump-golang-1.15.6
Bump Golang 1.15.6
2021-01-12 16:07:24 -06:00
Davanum Srinivas
b72534967e
Bump Golang 1.15.6
Changes: https://golang.org/doc/devel/release.html#go1.15

Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2021-01-12 16:44:36 -05:00
Davanum Srinivas
378c119245
Bump k8s deps to v1.20.1
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2021-01-12 16:44:20 -05:00
Michael Crosby
abc0041a0e
Merge pull request #4918 from liusdu/sig_bus
signal: do not print message when dealing with SIG_PIPE
2021-01-12 15:52:57 -05:00
Michael Crosby
38604a76ca
Merge pull request #4933 from TBBle/fix-resolver-header-map-panic
Ensure request.header is never nil
2021-01-12 15:50:17 -05:00
Derek McGowan
7dffdfa560
Move documentation and helper directories out of root
Signed-off-by: Derek McGowan <derek@mcg.dev>
2021-01-12 12:19:50 -08:00
Paul "TBBle" Hampson
785ac3ab71 Ensure request.header is never nil
Header.Clone() will return `nil` if called on a nil object.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2021-01-13 05:46:50 +11:00
Mike Brown
550b4949cb
Merge pull request #4700 from mikebrow/cri-security-profile-update
CRI security profile update for CRI graduation
2021-01-12 12:21:56 -06:00
Phil Estes
3b6a386803
Merge pull request #4528 from tianon/platforms-compare
Add amd64->386 fallback
2021-01-12 11:38:36 -05:00
Phil Estes
e62d03b61a
Merge pull request #4578 from thaJeztah/use_moby_sys
mount: replace mountinfo handling with moby/sys/mountinfo
2021-01-12 11:33:28 -05:00
Phil Estes
9db6aa625d
Merge pull request #4302 from songjiayang/update-docker-resolver
Refactor dockerResolver with resolveDockerBase helper
2021-01-12 11:15:06 -05:00
Liu Hua
86277395cf signal: do not print message when dealing with SIG_PIPE
If we print message when SIG_PIPE occuers in signal handler.
There is a loop {print->SIG_PIPE->print->SIG_PIPE...}, which consume
a lot of cpu time. So do not print message in this situaiton.

Signed-off-by: Liu Hua <weldonliu@tencent.com>
2021-01-12 12:34:53 +08:00
Maksym Pavlenko
a4f4a43110
Merge pull request #4916 from stefanberger/streamproc_env_vars
Allow passing environent variables to StreamProcessors
2021-01-11 16:34:12 -08:00
Tianon Gravi
5fa5f15de5 Add amd64->386 fallback
Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
2021-01-11 16:18:19 -08:00
Derek McGowan
6bf5650450
Merge pull request #4923 from fuweid/fix-wrong-context
runtime/v2: should use defer ctx to cleanup
2021-01-11 15:34:47 -08:00
Stefan Berger
1917ca5f79 Allow passing environent variables to StreamProcessors
Add support for an 'env' field to the StreamProcessor configuration
and append the environment variables found there to the os.Environ()
array.
The env field takes environment variables in the form of key=value.

Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
2021-01-11 13:15:12 -05:00
Maksym Pavlenko
04df60d106
Merge pull request #4858 from samuelkarp/freebsd-native-snapshotter
Support the native snapshotter on FreeBSD
2021-01-11 09:52:56 -08:00
Wei Fu
846cb963cc runtime/v2: should use defer ctx to cleanup
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2021-01-11 23:22:38 +08:00
Phil Estes
75c2646229
Merge pull request #4902 from mxpv/losetup
[Carry] mount: handle loopback mount
2021-01-08 16:18:37 -05:00
Maksym Pavlenko
d7ec6e9e89
Merge pull request #4917 from AkihiroSuda/demote-introspection-log-level
introspection: demote log level
2021-01-08 12:32:11 -08:00
Sebastiaan van Stijn
2374178c9b
pkg/cri/server: optimizations in unmountRecursive()
Use a PrefixFilter() to get only the mounts we're interested in,
which removes the need to manually filter mounts from the mountinfo
results.

Additional optimizations can be made, as:

> ... there's a little known fact that `umount(MNT_DETACH)` is actually
> recursive in Linux, IOW this function can be replaced with
> `unix.Umount(target, unix.MNT_DETACH)` (or `mount.UnmountAll(target, unix.MNT_DETACH)`
>  (provided that target itself is a mount point).

e8fb2c392f (r535450446)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-01-08 17:32:01 +01:00
Sebastiaan van Stijn
7572919201
mount: remove remaining uses of mount.Self()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-01-08 17:31:59 +01:00
Sebastiaan van Stijn
474abe5723
mount: remove unused mount.PID()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-01-08 17:31:57 +01:00
Sebastiaan van Stijn
f6a0ec7c0a
mount.CleanupTempMounts() use github.com/moby/sys/mountinfo
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-01-08 17:31:55 +01:00
Sebastiaan van Stijn
43578a9d6e
mount: mount.PID(), mount.Self(): change signature to return pointers
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-01-08 17:31:53 +01:00
Sebastiaan van Stijn
fb464b94d3
mount: replace mountinfo handling with moby/sys/mountinfo
Trying to reduce duplicated effort in maintaining a mountinfo
parser, this patch replaces the local implementation with the
implementation in github.com/moby/sys, which is actively maintained
and contains various optimizations.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-01-08 17:31:50 +01:00