Commit Graph

12100 Commits

Author SHA1 Message Date
Akhil Mohan
85df1cc6b8
doc: update CI signal info in README
add the new testgrid dashboard information into the release doc
and readme so that it can act as CI signal for a containerd release

Signed-off-by: Akhil Mohan <akhilerm@gmail.com>
2023-04-12 02:04:04 -07:00
Derek McGowan
c5a43b0007
Merge pull request #8366 from mxpv/stats
[sbserver] Backport CRI stats patches to sandboxed CRI
2023-04-10 13:38:30 -07:00
Derek McGowan
bee8267c75
Merge pull request #8370 from everpeace/follow-up-8136
Follow Up for #8136
2023-04-10 09:42:10 -07:00
Shingo Omura
dc2fc987ca
capture desc variable in range variable just in case that it run in parallel mode
Signed-off-by: Shingo Omura <everpeace@gmail.com>
2023-04-10 20:59:11 +09:00
Shingo Omura
05bb52b273
Use t.TempDir instead of os.MkdirTemp
Signed-off-by: Shingo Omura <everpeace@gmail.com>
2023-04-10 20:58:36 +09:00
Fu Wei
5885db62c8
Merge pull request #8136 from everpeace/fix-additiona-gids-to-read-image-user
[CRI] fix additionalGids: it should fallback to imageConfig.User when securityContext.RunAsUser,RunAsUsername are empty
2023-04-09 14:59:07 +08:00
Samuel Karp
3313231040
Merge pull request #8365 from estesp/setup-go-v4 2023-04-08 23:03:29 -07:00
Maksym Pavlenko
79cb4b0000 [sbserver] handle missing cpu stats
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2023-04-07 15:59:40 -07:00
Maksym Pavlenko
464a4977a6 [sbserver] Refactor usageNanoCores be to used for all OSes
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2023-04-07 15:56:23 -07:00
Phil Estes
aee3587651
Update to setup-go@v4 action
The latest setup-go action caches the Go pkg cache and may have several
minute-per-run speed improvement on CI runs which have to fill the
pkg cache.

Signed-off-by: Phil Estes <estesp@amazon.com>
2023-04-07 15:26:40 -04:00
Kazuyoshi Kato
7cd72cce99
Merge pull request #8043 from gabriel-samfira/wcow_mount_layers_rebased
Mount snapshots on Windows
2023-04-06 16:34:05 -07:00
Derek McGowan
2503bef581
Merge pull request #8351 from rumpl/oci-readonly-mounts
oci: Use WithReadonlyTempMount when adding users/groups
2023-04-05 13:18:18 -07:00
Phil Estes
3c6ddee128
Merge pull request #8352 from thaJeztah/bump_go1.20.3
update go to go1.20.3, go1.19.8
2023-04-05 11:17:49 -04:00
Sebastiaan van Stijn
0fb2d91322
update go to go1.20.3, go1.19.8
go1.20.3 (released 2023-04-04) includes security fixes to the go/parser,
html/template, mime/multipart, net/http, and net/textproto packages, as well
as bug fixes to the compiler, the linker, the runtime, and the time package.
See the Go 1.20.3 milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.20.3+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.20.2...go1.20.3

go1.19.8 (released 2023-04-04) includes security fixes to the go/parser,
html/template, mime/multipart, net/http, and net/textproto packages, as well as
bug fixes to the linker, the runtime, and the time package. See the Go 1.19.8
milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.19.8+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.19.7...go1.19.8

Further details from the announcement on the mailing list:

We have just released Go versions 1.20.3 and 1.19.8, minor point releases.
These minor releases include 4 security fixes following the security policy:

- go/parser: infinite loop in parsing

  Calling any of the Parse functions on Go source code which contains `//line`
  directives with very large line numbers can cause an infinite loop due to
  integer overflow.
  Thanks to Philippe Antoine (Catena cyber) for reporting this issue.
  This is CVE-2023-24537 and Go issue https://go.dev/issue/59180.

- html/template: backticks not treated as string delimiters

  Templates did not properly consider backticks (`) as Javascript string
  delimiters, and as such did not escape them as expected. Backticks are
  used, since ES6, for JS template literals. If a template contained a Go
  template action within a Javascript template literal, the contents of the
  action could be used to terminate the literal, injecting arbitrary Javascript
  code into the Go template.

  As ES6 template literals are rather complex, and themselves can do string
  interpolation, we've decided to simply disallow Go template actions from being
  used inside of them (e.g. "var a = {{.}}"), since there is no obviously safe
  way to allow this behavior. This takes the same approach as
  github.com/google/safehtml. Template.Parse will now return an Error when it
  encounters templates like this, with a currently unexported ErrorCode with a
  value of 12. This ErrorCode will be exported in the next major release.

  Users who rely on this behavior can re-enable it using the GODEBUG flag
  jstmpllitinterp=1, with the caveat that backticks will now be escaped. This
  should be used with caution.

  Thanks to Sohom Datta, Manipal Institute of Technology, for reporting this issue.

  This is CVE-2023-24538 and Go issue https://go.dev/issue/59234.

- net/http, net/textproto: denial of service from excessive memory allocation

  HTTP and MIME header parsing could allocate large amounts of memory, even when
  parsing small inputs.

  Certain unusual patterns of input data could cause the common function used to
  parse HTTP and MIME headers to allocate substantially more memory than
  required to hold the parsed headers. An attacker can exploit this behavior to
  cause an HTTP server to allocate large amounts of memory from a small request,
  potentially leading to memory exhaustion and a denial of service.
  Header parsing now correctly allocates only the memory required to hold parsed
  headers.

  Thanks to Jakob Ackermann (@das7pad) for discovering this issue.

  This is CVE-2023-24534 and Go issue https://go.dev/issue/58975.

- net/http, net/textproto, mime/multipart: denial of service from excessive resource consumption

  Multipart form parsing can consume large amounts of CPU and memory when
  processing form inputs containing very large numbers of parts. This stems from
  several causes:

  mime/multipart.Reader.ReadForm limits the total memory a parsed multipart form
  can consume. ReadForm could undercount the amount of memory consumed, leading
  it to accept larger inputs than intended. Limiting total memory does not
  account for increased pressure on the garbage collector from large numbers of
  small allocations in forms with many parts. ReadForm could allocate a large
  number of short-lived buffers, further increasing pressure on the garbage
  collector. The combination of these factors can permit an attacker to cause an
  program that parses multipart forms to consume large amounts of CPU and
  memory, potentially resulting in a denial of service. This affects programs
  that use mime/multipart.Reader.ReadForm, as well as form parsing in the
  net/http package with the Request methods FormFile, FormValue,
  ParseMultipartForm, and PostFormValue.

  ReadForm now does a better job of estimating the memory consumption of parsed
  forms, and performs many fewer short-lived allocations.

  In addition, mime/multipart.Reader now imposes the following limits on the
  size of parsed forms:

  Forms parsed with ReadForm may contain no more than 1000 parts. This limit may
  be adjusted with the environment variable GODEBUG=multipartmaxparts=. Form
  parts parsed with NextPart and NextRawPart may contain no more than 10,000
  header fields. In addition, forms parsed with ReadForm may contain no more
  than 10,000 header fields across all parts. This limit may be adjusted with
  the environment variable GODEBUG=multipartmaxheaders=.

  Thanks to Jakob Ackermann for discovering this issue.

  This is CVE-2023-24536 and Go issue https://go.dev/issue/59153.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-05 16:03:25 +02:00
Djordje Lukic
cab056226f
oci: Use WithReadonlyTempMount when adding users/groups
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2023-04-05 12:09:36 +02:00
Akihiro Suda
a85af0f5ca
Merge pull request #8343 from AkihiroSuda/crun-1.8.3
CI: bump up crun to 1.8.3
2023-04-05 10:52:14 +09:00
Gabriel Adrian Samfira
c9e5c33a18 UnmountAll is a no-op for missing mount points
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-04-04 12:59:52 -07:00
Gabriel Adrian Samfira
8538e7a2ac Improve error messages and remove check
* Improve error messages
  * remove a check for the existance of unmount target. We probably
    should not mask that the target was missing.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-04-04 12:07:34 -07:00
Gabriel Adrian Samfira
ba74cdf150 Make ReadOnly() available on all platforms
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-04-04 02:04:56 -07:00
Gabriel Adrian Samfira
7bb2756bc4 Increase integration test tmieout to 20m
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-04-04 00:29:14 -07:00
Gabriel Adrian Samfira
1279ad880c Remove bind code path in mount()
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-04-03 23:18:44 -07:00
Samuel Karp
cbb2d08af0
Merge pull request #8342 from davidhsingyuchen/exit-status-typo 2023-04-03 11:04:12 -07:00
Gabriel Adrian Samfira
6a5b4c9c24 Remove "bind" code path from diff
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-04-03 08:11:35 -07:00
Akihiro Suda
76690706f4
CI: bump up crun to 1.8.3
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-04-03 14:30:15 +09:00
Gabriel Adrian Samfira
d373ebc4de Properly mount base layers
As opposed to a writable layer derived from a base layer, the volume
path of a base layer, once activated and prepared will not be a WCIFS
volume, but the actual path on disk to the snapshot. We cannot directly
mount this folder, as that would mean a client may gain access and
potentially damage important metadata files that would render the layer
unusabble.

For base layers we need to mount the Files folder which must exist in
any valid base windows-layer.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-04-02 08:35:34 -07:00
Gabriel Adrian Samfira
ca5605b4a2 Skip parent layer options on bind mounts
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-04-01 08:49:24 -07:00
Gabriel Adrian Samfira
7f82dd91f4 Add ReadOnly() function
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-04-01 08:43:14 -07:00
Hsing-Yu (David) Chen
157fe6a13a docs: fix typo in comment of ExitStatus.Exit(Code|Time)
Signed-off-by: Hsing-Yu (David) Chen <davidhsingyuchen@gmail.com>
2023-04-01 00:48:26 -07:00
Gabriel Adrian Samfira
4012c1b853 Remove escalated privileges
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-03-31 06:17:35 -07:00
Gabriel Adrian Samfira
54f8abe553 Use DefaultSnapshotter
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-03-31 06:17:35 -07:00
Gabriel Adrian Samfira
47dd3dcffb use t.Fatal if we cannot enable process privileges
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-03-31 06:17:35 -07:00
Gabriel Adrian Samfira
e31bef15fa Update continuity
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-03-31 06:17:32 -07:00
Gabriel Adrian Samfira
95687a9324 Fix go.mod, simplify boolean logic, add logging
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-03-31 06:16:56 -07:00
Gabriel Adrian Samfira
7a36efd75e Ignore ERROR_NOT_FOUND error when removing mount
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-03-31 06:16:55 -07:00
Gabriel Adrian Samfira
db32798592 Update continuity, go-winio and hcsshim
Update dependencies and remove the local bindfilter files. Those have
been moved to go-winio.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-03-31 06:16:52 -07:00
Gabriel Adrian Samfira
00efd3e6d8 Remove unused function
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-03-31 06:15:19 -07:00
Gabriel Adrian Samfira
dc980b14a0 Grant needed privileges for snapshotter tests
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-03-31 06:15:19 -07:00
Gabriel Adrian Samfira
feb637f92d Fix layer comparison and enable read-only checks
fstest.CheckDirectoryEqual checks if any files in the diff matches a
list of known metadataFiles. This only happens if we specify the initial
layer as the first parameter and the mutated layer as the second.

This also enables the read-only view checks, as the bind filter allows
us to mount a layer as ro.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-03-31 06:15:18 -07:00
Gabriel Adrian Samfira
36dc2782c4 Use bind filer for mounts
The bind filter supports bind-like mounts and volume mounts. It also
allows us to have read-only mounts.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2023-03-31 06:15:18 -07:00
Paul "TBBle" Hampson
d591bb0421 Enable TestSnapshotterClient on Windows
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2023-03-31 06:15:18 -07:00
Paul "TBBle" Hampson
7b36becd2d Run Windows snapshotter through the test suite
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2023-03-31 06:15:18 -07:00
Paul "TBBle" Hampson
ac30eabbdc Fix misspelling of 'Native' as 'Naive'
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2023-03-31 06:15:18 -07:00
Paul "TBBle" Hampson
639c5799a2 Add paired 'mount' log for 'unmount'
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2023-03-31 06:15:18 -07:00
Paul "TBBle" Hampson
8395e3a89a Don't use all-upper-case filenames in snapshot tests
NTFS, when presented with an all-caps filename, assumes you are just
being loud for no reason, and instead stores an all-lower-case filename.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2023-03-31 06:15:18 -07:00
Paul "TBBle" Hampson
909730decb Skip tests that do not apply to WCOW on Windows
Filesystem permissions and ownership are not modifiable via an image
mount.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2023-03-31 06:15:18 -07:00
Paul "TBBle" Hampson
469c13997a Ensure mounts are unmounted before leaving the test
This is necessary on Windows, as it's not possible to delete a snapshot
while it is still mounted, even if the mount-point has been deleted.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2023-03-31 06:15:17 -07:00
Paul "TBBle" Hampson
84cc3e496b Unify testutil.Unmount on Windows and Unix
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2023-03-31 06:15:17 -07:00
Paul "TBBle" Hampson
474a257b16 Implement Windows mounting for bind and windows-layer mounts
Using symlinks for bind mounts means we are not protecting an RO-mounted
layer against modification. Windows doesn't currently appear to offer a
better approach though, as we cannot create arbitrary empty WCOW scratch
layers at this time.

For windows-layer mounts, Unmount does not have access to the mounts
used to create it. So we store the relevant data in an Alternate Data
Stream on the mountpoint in order to be able to Unmount later.

Based on approach in https://github.com/containerd/containerd/pull/2366,
with sign-offs recorded as 'Based-on-work-by' trailers below.

This also partially-reverts some changes made in #6034 as they are not
needed with this mounting implmentation, which no longer needs to be
handled specially by the caller compared to non-Windows mounts.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
Based-on-work-by: Michael Crosby <crosbymichael@gmail.com>
Based-on-work-by: Darren Stahl <darst@microsoft.com>
2023-03-31 06:15:17 -07:00
Paul "TBBle" Hampson
34b07d3e2d Implement WCOW parentless active snapshots and view snapshots
The WCOW layer support does not support creating sandboxes with no
parent.  Instead, parentless scratch layers must be layed out as a
directory containing only a directory named 'Files', and all data stored
inside 'Files'. At commit-time, this will be converted in-place into a
read-only layer suitable for use as a parent layer.

The WCOW layer support also does not deal with making read-only layers,
i.e. layers that are prepared to be parent layers, visible in a
read-only manner. A bind-mount or junction point cannot be made
read-only, so a view must instead be a small sandbox layer that we can
mount via WCOW, and discard later, to protect the layer against
accidental or deliberate modification.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2023-03-31 06:15:17 -07:00
Fu Wei
988ee8ffef
Merge pull request #8208 from Iceber/fix_runtime_path
fix the task setting the runtime path
2023-03-31 12:38:08 +08:00