Some images like `criu` will have extra libs that it requires. This
adds lib support via LD_LIBRARY_PATH and InstallOpts
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
This adds a way for users to programatically install containerd binary
dependencies.
With runtime v2 and new shim's being built, it will be a challenge to
get those onto machines. Users would have to find the link, download,
place it in their path, yada yada yada.
With this functionality of a managed `/opt` directory, containerd can
use existing image and distribution infra. to get binarys, shims, etc
onto the system.
Configuration:
*default:* `/opt/containerd`
*containerd config:*
```toml
[plugins.opt]
path = "/opt/mypath"
```
Usage:
*code:*
```go
image, err := client.Pull(ctx, "docker.io/crosbymichael/runc:latest")
client.Install(ctx, image)
```
*ctr:*
```bash
ctr content fetch docker.io/crosbymichael/runc:latest
ctr install docker.io/crosbymichael/runc:latest
```
You can manage versions and see what is running via standard image
commands.
Images:
These images MUST be small and only contain binaries.
```Dockerfile
FROM scratch
Add runc /bin/runc
```
Containerd will only extract files in `/bin` of the image.
Later on, we can add support for `/lib`.
The code adds a service to manage an `/opt/containerd` directory and
provide that path to callers via the introspection service.
How to Test:
Delete runc from your system.
```bash
> sudo ctr run --rm docker.io/library/redis:alpine redis
ctr: OCI runtime create failed: unable to retrieve OCI runtime error (open /run/containerd/io.containerd.runtime.v1.linux/default/redis/log.json: no such file or directory): exec: "runc": executable file not found in $PATH: unknown
> sudo ctr content fetch docker.io/crosbymichael/runc:latest
> sudo ctr install docker.io/crosbymichael/runc:latest
> sudo ctr run --rm docker.io/library/redis:alpine redis
1:C 01 Aug 15:59:52.864 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 01 Aug 15:59:52.864 # Redis version=4.0.10, bits=64, commit=00000000, modified=0, pid=1, just started
1:C 01 Aug 15:59:52.864 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
1:M 01 Aug 15:59:52.866 # You requested maxclients of 10000 requiring at least 10032 max file descriptors.
1:M 01 Aug 15:59:52.866 # Server can't set maximum open files to 10032 because of OS error: Operation not permitted.
1:M 01 Aug 15:59:52.866 # Current maximum open files is 1024. maxclients has been reduced to 992 to compensate for low ulimit. If you need higher maxclients increase 'ulimit -n'.
1:M 01 Aug 15:59:52.870 * Running mode=standalone, port=6379.
1:M 01 Aug 15:59:52.870 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
1:M 01 Aug 15:59:52.870 # Server initialized
1:M 01 Aug 15:59:52.870 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
1:M 01 Aug 15:59:52.870 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
1:M 01 Aug 15:59:52.870 * Ready to accept connections
^C1:signal-handler (1533139193) Received SIGINT scheduling shutdown...
1:M 01 Aug 15:59:53.472 # User requested shutdown...
1:M 01 Aug 15:59:53.472 * Saving the final RDB snapshot before exiting.
1:M 01 Aug 15:59:53.484 * DB saved on disk
1:M 01 Aug 15:59:53.484 # Redis is now ready to exit, bye bye...
```
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
With `fs.RootPath`, the target file will be the file which the softlink
points to, like:
touch /tmp/zzz
ln -s /tmp/zzz /tmp/xxx
ln /tmp/xxx /tmp/yyy
The `/tmp/yyy` should be same with the `/tmp/xxx`, not `/tmp/zzz`. We
should allow hardlink to softlink file.
Signed-off-by: Wei Fu <fhfuwei@163.com>
This allows non-privileged users to use containerd. This is part of a
larger track of work integrating containerd into Cloudfoundry's garden
with support for rootless.
[#156343575]
Signed-off-by: Claudia Beresford <cberesford@pivotal.io>
The go-tar implementation which is used cannot handle sockets.
There's no good reason to preserve a socket, they are basically useless without
the process which made them.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
Currently directory changes are not added to the list of
included directories, allowing those directories to receive
duplicate entries where there is both a metadata change to the
directory and a change to a file under that directory.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
To avoid buffer bloat in long running processes, we try to use buffer
pools where possible. This is meant to address shim memory usage issues,
but may not be the root cause.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Ensure whiteout is not pointing to the current directory
or parent directory before removing. Protects against invalid
removal of the parent or current directory.
Add whiteout related tar tests using manufactured tar conditions.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
This linter checks for unnecessary type convertions.
Some convertions are whitelisted because their type is different
on 32bit platforms
Signed-off-by: Daniel Nephin <dnephin@gmail.com>
Call chmod on all open files and created directories to
ensure permission is set as expected without changing umask.
Fixes#1608
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Now that golang.org/x/sys/unix provides the Mkdev, Major and Minor
functions for every OS, use them instead of the locally defined version
which uses the Linux specific device major/minor encoding.
This also means that the device number should now be properly encoded on
e.g. Darwin, FreeBSD or Solaris.
Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
Corrects compile on FreeBSD by handling the lack of ENODATA on FreeBSD.
Since continuity project has already handled this, using their const is
simpler than separating a few extra files in containerd/containerd.
Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
The syscall package is locked down and the comment in [1] advises to
switch code to use the corresponding package from golang.org/x/sys. Do
so and replace usage of package syscall with package
golang.org/x/sys/{unix,windows} where applicable.
[1] https://github.com/golang/go/blob/master/src/syscall/syscall.go#L21-L24
This will also allow to get updates and fixes for syscall wrappers
without having to use a new go version.
Errno, Signal and SysProcAttr aren't changed as they haven't been
implemented in x/sys/. Stat_t from syscall is used if standard library
packages (e.g. os) require it. syscall.ENOTSUP, syscall.SIGKILL and
syscall.SIGTERM are used for cross-platform files.
Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
This moves the RootPath function out of the archive package and into the
fs package for external use.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Fixes bug for resolving symlinks which allowed fully resolving
an existing symlink to a path, causing some symlinks to get
overridden as symlinks to self.
Updates logic to split name into parent path, resolve the parent
path, then safely join back with the base name.
Uses the split code to ensure parent directories are created in
all cases.
Replaces `rootJoin` with filepath.Join to the root, which already
correctly cleans relative symlinks to the root.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Replace cases where a tar specified name is joined to a directory
with root path to bound name to path.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Ensure symlinks cannot be used to breakout of unpack directory.
Evaluate absolute symlinks as scoped to unpack directory.
Allow symlinks which point outside the root to be created.
Scope all resolution of symlinks to the unpack directory.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
This mainly fixes Linux vs generic Unix differences, with some
differences between Darwin and Freebsd (which are close bit not
identical). Should make fixing for other Unix platforms easier.
Note there are not yet `runc` equivalents for these platforms;
my current use case is image manipulation for the `moby` tool.
However there is interest in OCI runtime ports for both platforms.
Current status is that MacOS can build and run `ctr`, `dist`
and `containerd` and some operations are supported. FreeBSD 11
still needs some more fixes to continuity for extended attributes.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Previously hardlinking to an unmodified file or linking to a file
which was touched by not detected as modified caused a new file
to be created on unpack. This new file and the original source file
were not linked since no link record was created in the tar. This
change addresses this by adding links for all hardlinks to a file
when it is detected as changed. These links will be written after
the source file is written and may occur out of order in regard to
file name.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Touching the file before hardlinking was covering up a bug in
how hard links are handled in tar. Without the touch the hardlink
should still be included and both files should hardlink after
the tar is applied.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Test deleting multiple files, directories, and creating
the equivalent of an opaque directory.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Removed unused requires root test function and updated
tar requires function to use lookup method.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Archive package handles generating and applying diff tar streams
based on the OCI diff tar specification.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
