There are many Kubernetes clusters running on ARM64. Enable ARM64 runner
is to commit to support ARM64 platform officially.
Signed-off-by: Wei Fu <fuweid89@gmail.com>
This is effectively a revert of 2ac9968401, which
switched from os/exec to the golang.org/x/sys/execabs package to mitigate
security issues (mainly on Windows) with lookups resolving to binaries in the
current directory.
from the go1.19 release notes https://go.dev/doc/go1.19#os-exec-path
> ## PATH lookups
>
> Command and LookPath no longer allow results from a PATH search to be found
> relative to the current directory. This removes a common source of security
> problems but may also break existing programs that depend on using, say,
> exec.Command("prog") to run a binary named prog (or, on Windows, prog.exe) in
> the current directory. See the os/exec package documentation for information
> about how best to update such programs.
>
> On Windows, Command and LookPath now respect the NoDefaultCurrentDirectoryInExePath
> environment variable, making it possible to disable the default implicit search
> of “.” in PATH lookups on Windows systems.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
LOOP_CONFIGURE is a new ioctl that is a lot faster than
the LOOP_SET_FD+LOOP_SET_STATUS64 calls
Signed-off-by: Alexandru Matei <alexandru.matei@uipath.com>
It's followup for #5890.
The containerd-shim process depends on the mount package to init rootfs
for container. For the container enable user namespace, the mount
package needs to fork child process to get the brand-new user namespace.
However, there are two reapers in one process (described by the
following list) and there are race-condition cases.
1. mount package
2. sys.Reaper as global one which watch all the SIGCHLD.
=== [kill(2)][kill] the wrong process ===
Currently, we use pipe to ensure that child process is alive. However,
the pide file descriptor can be hold by other process, which the child
process cannot exit by self. We should use [kill(2)][kill] to ensure the
child process. But we might kill the wrong process if the child process
might be reaped by containerd-shim and the PID might be reused by other
process.
=== [waitid(2)][waitid] on the wrong child process ===
```
containerd-shim process:
Goroutine 1(GetUsernsFD): Goroutine 2(Reaper)
1. Ready to wait for child process X
2. Received SIGCHLD from X
3. Reaped the zombie child process X
(X has been reused by other child process)
4. Wait on process X
The goroutine 1 will be stuck until the process X has been terminated.
```
=== open `/proc/X/ns/user` on the wrong child process ===
There is also pid-reused risk between opening `/proc/$pid/ns/user` and
writing `/proc/$pid/u[g]id_map`.
```
containerd-shim process:
Goroutine 1(GetUsernsFD): Goroutine 2(Reaper)
1. Fork child process X
2. Write /proc/X/uid_map,gid_map
3. Received SIGCHLD from X
4. Reaped the zombie child process X
(X has been reused by other process)
5. Open /proc/X/ns/user file as usernsFD
The usernsFD links to the wrong X!!!
```
In order to fix the race-condition, we should use [CLONE_PIDFD][clone2] (Since
Linux v5.2).
When we fork child process `X`, the kernel will return a process file
descriptor `X_PIDFD` referencing to child process `X`. With the pidfd, we can
use [pidfd_send_signal(2)][pidfd_send_signal] (Since Linux v5.1)
to send signal(0) to ensure the child process `X` is alive. If the `X` has
terminated and its PID has been recycled for another process. The
pidfd_send_signal fails with the error ESRCH.
Therefore, we can open `/proc/X/{ns/user,uid_map,gid_map}` file
descriptors as first and then use pidfd_send_signal to check the process
is still alive. If so, we can ensure the file descriptors are valid and
reference to the child process `X`. Even if the `X` PID has been reused
after pidfd_send_signal call, the file descriptors are still valid.
```code
X, pidfd = clone2(CLONE_PIDFD)
usernsFD = open /proc/X/ns/user
uidmapFD = open /proc/X/uid_map
gidmapFD = open /proc/X/gid_map
pidfd_send_signal pidfd, signal(0)
return err if no such process
== When we arrive here, we can ensure usernsFD/uidmapFD/gidmapFD are correct
== even if X has been reused after pidfd_send_signal call.
update uid/gid mapping by uidmapFD/gidmapFD
return usernsFD
```
And the [waitid(2)][waitid] also supports pidfd type (Since Linux 5.4).
We can use pidfd type waitid to ensure we are waiting for the correct
process. All the PID related race-condition issues can be resolved by
pidfd.
```bash
➜ mount git:(followup-idmapped) pwd
/home/fuwei/go/src/github.com/containerd/containerd/mount
➜ mount git:(followup-idmapped) sudo go test -test.root -run TestGetUsernsFD -count=1000 -failfast -p 100 ./...
PASS
ok github.com/containerd/containerd/mount 3.446s
```
[kill]: <https://man7.org/linux/man-pages/man2/kill.2.html>
[clone2]: <https://man7.org/linux/man-pages/man2/clone.2.html>
[pidfd_send_signal]: <https://man7.org/linux/man-pages/man2/pidfd_send_signal.2.html>
[waitid]: <https://man7.org/linux/man-pages/man2/waitid.2.html>
Signed-off-by: Wei Fu <fuweid89@gmail.com>
It says: The prefix path **must be absolute, have all symlinks resolved, and cleaned**. But those requirements are violated in lots of places.
What happens when it is given a non-canonicalized path is that `mountinfo.GetMounts` will not find mounts.
The trivial case is:
```
$ mkdir a && ln -s a b && mkdir b/c b/d && mount --bind b/c b/d && cat /proc/mounts | grep -- '[ab]/d'
/dev/sdd3 /home/user/a/d ext4 rw,noatime,discard 0 0
```
We asked to bind-mount b/c to b/d, but ended up with mount in a/d.
So, mount table always contains canonicalized mount points, and it is an error to look for non-canonicalized paths in it.
Signed-off-by: Marat Radchenko <marat@slonopotamus.org>
This patch introduces idmapped mounts support for
container rootfs.
The idmapped mounts support was merged in Linux kernel 5.12
torvalds/linux@7d6beb7.
This functionality allows to address chown overhead for containers that
use user namespace.
The changes are based on experimental patchset published by
Mauricio Vásquez #4734.
Current version reiplements support of idmapped mounts using Golang.
Performance measurement results:
Image idmapped mount recursive chown
BusyBox 00.135 04.964
Ubuntu 00.171 15.713
Fedora 00.143 38.799
Signed-off-by: Mauricio Vásquez <mauricio@kinvolk.io>
Signed-off-by: Artem Kuzin <artem.kuzin@huawei.com>
Signed-off-by: Alexey Perevalov <alexey.perevalov@huawei.com>
Signed-off-by: Ilya Hanov <ilya.hanov@huawei-partners.com>
Helpers to convert from containerd's [Mount] to its protobuf structure for
[Mount] and vice-versa appear three times. It seems sane to just expose
this facility in /mount.
Signed-off-by: Danny Canter <danny@dcantah.dev>
* Improve error messages
* remove a check for the existance of unmount target. We probably
should not mask that the target was missing.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
As opposed to a writable layer derived from a base layer, the volume
path of a base layer, once activated and prepared will not be a WCIFS
volume, but the actual path on disk to the snapshot. We cannot directly
mount this folder, as that would mean a client may gain access and
potentially damage important metadata files that would render the layer
unusabble.
For base layers we need to mount the Files folder which must exist in
any valid base windows-layer.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Update dependencies and remove the local bindfilter files. Those have
been moved to go-winio.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
The bind filter supports bind-like mounts and volume mounts. It also
allows us to have read-only mounts.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Using symlinks for bind mounts means we are not protecting an RO-mounted
layer against modification. Windows doesn't currently appear to offer a
better approach though, as we cannot create arbitrary empty WCOW scratch
layers at this time.
For windows-layer mounts, Unmount does not have access to the mounts
used to create it. So we store the relevant data in an Alternate Data
Stream on the mountpoint in order to be able to Unmount later.
Based on approach in https://github.com/containerd/containerd/pull/2366,
with sign-offs recorded as 'Based-on-work-by' trailers below.
This also partially-reverts some changes made in #6034 as they are not
needed with this mounting implmentation, which no longer needs to be
handled specially by the caller compared to non-Windows mounts.
Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
Based-on-work-by: Michael Crosby <crosbymichael@gmail.com>
Based-on-work-by: Darren Stahl <darst@microsoft.com>
This is necessary so we can mount snapshots more than once with overlayfs,
otherwise mounts enter an unknown state.
related: https://github.com/moby/buildkit/pull/1100
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
Co-authored-by: Zou Nengren <zouyee1989@gmail.com>
From golangci-lint:
> SA1019: rand.Read has been deprecated since Go 1.20 because it
>shouldn't be used: For almost all use cases, crypto/rand.Read is more
>appropriate. (staticcheck)
> SA1019: rand.Seed has been deprecated since Go 1.20 and an alternative
>has been available since Go 1.0: Programs that call Seed and then expect
>a specific sequence of results from the global random source (using
>functions such as Int) can be broken when a dependency changes how
>much it consumes from the global random source. To avoid such breakages,
>programs that need a specific result sequence should use
>NewRand(NewSource(seed)) to obtain a random generator that other
>packages cannot access. (staticcheck)
See also:
- https://pkg.go.dev/math/rand@go1.20#Read
- https://pkg.go.dev/math/rand@go1.20#Seed
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
- Add Target to mount.Mount.
- Add UnmountMounts to unmount a list of mounts in reverse order.
- Add UnmountRecursive to unmount deepest mount first for a given target, using
moby/sys/mountinfo.
Signed-off-by: Edgar Lee <edgarhinshunlee@gmail.com>
This change spins up a new goroutine, locks it to a thread, then
unshares CLONE_FS which allows us to `Chdir` from inside the thread
without affecting the rest of the program.
The thread is no longer usable after unshare so it leaves the thread
locked to prevent go from returning the thread to the thread pool.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Use the IoctlRetInt, IoctlSetInt and IoctlLoopSetStatus64 helper
functions defined in the golang.org/x/sys/unix package instead of
manually wrapping these using a locally defined ioctl function.
Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
This error was added in c5843b7615, but no longer
used since a5a9f91832, which implemented Windows
support.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The directory created by `T.TempDir` is automatically removed when the
test and all its subtests complete.
Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
gccgo changes the mangling scheme
b483d0e0a2
The change is available in gcc-11, which is the least version that
implements go1.16.
Signed-off-by: Shengjing Zhu <zhsj@debian.org>
