Commit Graph

288 Commits

Author SHA1 Message Date
Kevin Parsons
225d9b120c Fix LCOW layer ordering
Due to an error in the OCI specf for layerFolders, the runhcs shim was
passing the layers for LCOW in reverse order. This fixes the ordering
by simply removing the code which reversed the layers for LCOW.

Signed-off-by: Kevin Parsons <kevpar@microsoft.com>
2019-02-07 17:27:01 -08:00
Michael Crosby
2dacef07ca Add shim skeleton code
This allows runtime authors to quickly bootstrap new shim
implementations.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-02-06 15:03:02 -05:00
Justin Terry (VM)
dfcc5942f1 Fix deadlock in Windows runhcs shim exec
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2019-02-01 10:58:01 -08:00
Justin Terry (VM)
6ed293ba94 Fix bug in shim path lookup
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2019-01-28 14:04:14 -08:00
Justin Terry (VM)
d63099c4a6 Allow matching shim path side by side with containerd
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2019-01-28 12:49:14 -08:00
Justin Terry (VM)
764afa0d18 Include extension for shim binary format on Windows
Use full name including extension for shim binary format on Windows in order to
match any stat path faster without a fallback.

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2019-01-28 12:42:08 -08:00
Michael Crosby
85aa8ad361 Move task events to runc v2 shim
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-01-25 14:15:43 -05:00
Phil Estes
f63d28984c
Merge pull request #2939 from jterry75/bug_publishstart
Implement the Runtime v2 Shim async task model for runhcs
2019-01-25 12:39:48 -05:00
Michael Crosby
f444696989 Return out of windows signal handler
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-01-23 16:06:49 -05:00
Lantao Liu
26ab393e7d Use context.Background for O_NONBLOCK OpenFifo.
Signed-off-by: Lantao Liu <lantaol@google.com>
2019-01-23 10:18:54 -08:00
Wei Fu
132ee9b826 fix: linter issue
megacheck, gosimple and unused has been deprecated and subsumed by
staticcheck. And staticcheck also has been upgraded. we need to update
code for the linter issue.

close: #2945

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2019-01-23 22:54:51 +08:00
Michael Crosby
35582cb7a3
Merge pull request #2899 from fuweid/proposal-add-Add-method-in-PlatformRuntime
runtime: add Add/Delete method in PlatformRuntime interface
2019-01-22 13:48:39 -05:00
Justin Terry (VM)
6468619d73 Implement the Runtime v2 Shim async task model for runhcs
Changes the requirement of a Runtime v2 shim in order to avoid race conditions
between shim and shim client sending async events. Places a requirement of what
events and what order a shim must comply to.

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2019-01-17 14:56:37 -08:00
John Howard
4ef9bf5c84 Windows: Publish exit status correctly in TaskExit
Signed-off-by: John Howard <jhoward@microsoft.com>

Before this change, the shim was only publishing a non-zero exit status
(exit code) in the case that the process.Wait() call failed. This
grabs the exit status correctly when process.Wait() succeeds too.
2019-01-15 11:12:27 -08:00
Phil Estes
33b56e253e
Merge pull request #2927 from jterry75/bug_io_relay_close
Fix issue in runhcs shim CloseIO
2019-01-15 09:59:09 -05:00
John Howard
e30bba53ff
Merge pull request #2926 from jterry75/bug_argon_mount
Fix runhcs shim bug in Create with "len(Rootfs) == 0"
2019-01-14 15:26:32 -08:00
Justin Terry (VM)
c5a8c9fc12 Fix issue in runhcs shim CloseIO
The call was closing all upstream IO when a shim.CloseIO call was made rather
than just the Stdin as it is supposed to.

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2019-01-14 13:31:28 -08:00
Justin Terry (VM)
31616e7945 Fix runhcs shim bug in Create with "len(Rootfs) == 0"
Rootfs length can be set to zero if the upstream caller fully manages storage
and mounts on their own. In this case just treat the bundle as a fully complete
OCI spec and run it without doing any storage work in the shim.

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2019-01-14 13:24:15 -08:00
Justin Terry (VM)
bcd4cc51c8 Fixes a bug in runhcs shim Exec.Pid
When an exec occurs the pid was not properly updated on the in memory state
value causing many queries to see a 0.

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2019-01-14 13:18:02 -08:00
Michael Crosby
fcf5462d49
Merge pull request #2914 from jterry75/runhcs_runtime_opts
Update runhcs options to include CRI Sandbox support
2019-01-07 11:54:41 -05:00
Justin Terry (VM)
bc76e01a87 Update runhcs options to include CRI Sandbox support
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2019-01-04 14:37:10 -08:00
Justin Terry (VM)
dee0945e18 Fix spurious ttrpc client shutdown error log on success
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2019-01-03 13:44:29 -08:00
Wei Fu
568b5be936 runtime: add Add/Delete method in PlatformRuntime interface
The two new method Add/Delete can allow custom plugin to add or migrate
existing task into major Runtime plugin.

close: #2888

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2018-12-29 13:56:38 +08:00
Phil Estes
47b328aab7
Merge pull request #2897 from crosbymichael/atomic-delete
Ensure bundle removal is atomic
2018-12-21 08:27:43 -05:00
Michael Crosby
36e4dc603e Ensure bundle removal is atomic
This makes bundle removal atomic by first renaming the bundle and
working directories to a hidden path before removing the underlying
directories.

Closes #2567
Closes #2327

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-12-20 13:45:18 -05:00
Phil Estes
06e04bc5a9
Merge pull request #2830 from Ace-Tang/support_cr_without_image
cr: support checkpoint/restore without image
2018-12-20 13:24:37 -05:00
Justin Terry (VM)
0ec6526fd3 Update the delete docs for Runtime V2 on Windows
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-12-17 13:26:57 -08:00
Justin Cormack
8be05eb237
Fix freebsd build
This brings freebsd in line with Darwin, ie it builds, but some parts may not yet
be fully functional. There is now a WIP `runc` port for FreeBSD at
https://github.com/clovertrail/runc/tree/1501-SupportOnFreeBSD so should be able
to test further.

Signed-off-by: Justin Cormack <justin@specialbusservice.com>
2018-12-16 14:27:42 +00:00
Michael Crosby
3ae8e8a30b Add shim config for shim binary options
Fixes #2855

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-12-05 13:14:26 -05:00
Ace-Tang
6593399e9f cr: support checkpoint/restore without image
support checkpoint without committing a checkpoint dir into a
checkpoint image and restore without untar image into checkpoint
directory. support for both v1 and v2 runtime

Signed-off-by: Ace-Tang <aceapril@126.com>
2018-11-29 10:19:39 +08:00
Justin Terry (VM)
09bf314bfd Cache shim v2 exec.LookPath results
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-11-27 15:11:04 -08:00
Ace-Tang
fd16bf6d46 runtimev2: add image-path and work-path for c/r
add ImagePath and WorkPath for checkpoint process, add CriuImagePath
and CriuWorkPath for create process in runtime v2 protobuf

Signed-off-by: Ace-Tang <aceapril@126.com>
2018-11-24 23:08:25 +08:00
Phil Estes
181a522142
Merge pull request #2807 from lifubang/shimlockwhenstdinclose
fix pipe in broken may cause shim lock forever
2018-11-20 22:38:22 +08:00
Lifubang
e76a8879eb fix pipe in broken may cause shim lock forever for runtime v1
Signed-off-by: Lifubang <lifubang@acmcoder.com>
2018-11-19 09:25:43 +08:00
Lifubang
b3438f7a6f fix pipe in broken may cause shim lock forever for runtime v2
Signed-off-by: Lifubang <lifubang@acmcoder.com>
2018-11-19 09:02:49 +08:00
Michael Crosby
956e49a36b Revert v2 dropped events
Partial revert of #2748

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-11-16 11:34:15 -05:00
Justin Terry (VM)
8e25ca6bf0 Revendor github.com/sirupsen/logrus to v1.0.3
logrus v1.0.3 was the first release that include the change in
terminal_windows.go that stops exec'ing "cmd ver" to obtain the version
information and rather uses the x/sys/crypto/terminal.IsTerminal on the
console fd. On Windows this is a significant performance difference to
avoid the additional process activation of the "cmd ver" for each
invocation of the shim/runhcs executables.

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-11-13 09:27:29 -08:00
Ace-Tang
c4feaa75cf fix: fix failed to get container-shim relation with io.containerd.runc.v1
add '-id' flag when start container with io.containerd.runc.v1 shim, or user
can not get container-shim relation from 'ps -ef',like

```
/usr/bin/containerd-shim-runc-v1 -namespace default -address
/run/containerd/containerd.sock -publish-binary /usr/bin/containerd
```

Signed-off-by: Ace-Tang <aceapril@126.com>
2018-11-09 11:01:35 +08:00
Wei Fu
38d7d59e8a enhance: update v1/v2 runtime
1. avoid dead lock during kill, fetch allProcesses before handle events
2. use argu's ctx instead of context.Backgroud() in openlog

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2018-11-06 22:48:43 +08:00
Justin Terry (VM)
a33ad40245 Implement io.containerd.runhcs.v1 shim log opts
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-11-05 09:08:48 -08:00
Justin Terry (VM)
ec3dbd155e Add io.containerd.runhcs.v1 shim proto options
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-11-05 09:08:48 -08:00
Michael Crosby
232a063496 Increase reaper buffer size and non-blocking send
Fixes #2709

This increases the buffer size for process exit subscribers. It also
implements a non-blocking send on the subscriber channel.  It is better
to drop an exit even than it is to block a shim for one slow subscriber.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-10-29 16:46:58 -04:00
Ace-Tang
7b1b16b741 runtime-v2: add validation for runtime name
add validation for runtime name, if runtime name is invalid,
containerd will got panic.

Signed-off-by: Ace-Tang <aceapril@126.com>
2018-10-19 15:19:45 +08:00
Wei Fu
accadd7118 fixtypo: misspell in runtime package
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2018-10-18 23:21:05 +08:00
Justin Terry (VM)
3f1d9b2c4f Revendor github.com/Microsoft/hcsshim
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-10-15 13:38:24 -07:00
Justin Terry (VM)
ab2031236a Add blocking buffered writes to shim
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-10-02 11:10:43 -07:00
Justin Terry (VM)
beb1f432be Review fixes
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-10-02 11:10:43 -07:00
Justin Terry (VM)
2ddbb2db05 Handle shim delete workdir on Windows
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-10-02 11:10:43 -07:00
Justin Terry (VM)
b8945d35f5 Decrease shim timeout on pipe not found
On Windows because of the way the log pipe is forwarded to the shim there is a
condition where the pipe listener may not yet be active when a client tries to
connect. To handle this case we allow polling on the file and rety on pipe not
found. This limits the pipe not found retry to 5 seconds but leaves the connect
timeout alone as if there is a listener we want to connect to it normally.

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-10-02 11:10:43 -07:00
Justin Terry (VM)
ddbeb3f7c7 Adds Windows shim reconnect logs support
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-10-02 11:10:37 -07:00
Justin Terry (VM)
81eb40fabf Adds containerd-shim-runhcs verbose logging support
Revendors to Microsoft/hcsshim v0.7.5 that added support for logging all
runhcs.exe commands via Windows named pipes. This now launches all runhcs.exe
commands and forwards debug logging to the containerd-shim-runhcs log when
with --debug.

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-09-27 07:46:09 -07:00
Justin Terry (VM)
772644e978 Fixes containerd-shim-runhcs State on exec id
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-09-27 07:46:09 -07:00
Justin Terry (VM)
83437ef646 Fixes containerd-shim-runhcs Delete on exec id
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-09-27 07:46:09 -07:00
Justin Terry (VM)
84aa0bfde6 Forward containerd debug to shim invocation
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-09-27 07:46:01 -07:00
Justin Terry (VM)
547bb94e4b Fix ctr run for Windows containers
1. Fixes bugs in ctr run that were introduced by 1d9b969
2. Adds support for the --isolated flag that runs Windows HyperV
cotainers instead of process isolated containers on Windows.

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-09-20 14:28:36 -07:00
Justin Terry (VM)
7768ab1b5e Update runhcs-shim to use go-bindings
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-09-20 10:40:33 -07:00
John Howard
2586f3fbb9 boltdb/bolt --> go.etcd.io/bbolt
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-09-12 15:23:57 -07:00
Michael Crosby
c48cafea40
Merge pull request #2619 from nashasha1/fix/typo-in-runtime
Fix some typo in runtime and snapshots
2018-09-10 09:32:34 -04:00
Xiaodong Zhang
e6d787172c Fix some typo in runtime and snapshots
Signed-off-by: Xiaodong Zhang <a4012017@sina.com>
2018-09-08 08:31:42 +08:00
Justin Terry (VM)
ef910311e8 Add a Windows section for Linux oci on LCOW
When creating a default OCI spec on Windows that is targeting the LCOW
platform it needs to contain a Windows section as well. This adds the
Windows section by default. It also protects against this case for all
OCI creation that doesnt use the OCI package in the runhcs-shim.

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-09-06 11:05:45 -07:00
Tom Godkin
b5ccc66c2c Do not kill all on task delete by default
- Still KillAll if the task uses the hosts pid namespace
 - Test for both host pid namespace and normal cases

Co-authored-by: Oliver Stenbom <ostenbom@pivotal.io>
Co-authored-by: Georgi Sabev <georgethebeatle@gmail.com>
Signed-off-by: Oliver Stenbom <ostenbom@pivotal.io>
2018-08-30 15:58:33 +01:00
Michael Crosby
2205e8d67a Improve shim locking
Only lock around shim state and not on actions

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-08-29 11:22:02 -04:00
Michael Crosby
d50e25360c Add context cancel for epoll
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-08-29 10:44:47 -04:00
Justin Terry (VM)
0110b3c0bc Introduce the Windows lcow diff/snaphotter
Implements the Windows lcow differ/snapshotter responsible for managing
the creation and lifetime of lcow containers on Windows.

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-08-28 13:51:04 -07:00
Derek McGowan
ce1161f806
Merge pull request #2551 from crosbymichael/stdin-block
Don't block on STDIN open
2018-08-28 10:24:05 -07:00
Derek McGowan
be42d777ff
Merge pull request #2575 from crosbymichael/workdir
Remove and create workdir if state dir does not exist
2018-08-27 16:19:25 -07:00
Michael Crosby
ac78a5b615 Remove and create workdir if state dir does not exist
This is the case where the work dir could still exist if a machine
reboots, reseting the state dir.  On container creation, we should just
clear out the work dir.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-08-27 18:18:50 -04:00
Justin Terry (VM)
e88ec1f1a6 Fix incorrect ID usage in Windows runtime v2
Sometimes the wrong ID was being used because its not correct to assume
that ExecID is always set. The assumption was that for API's where it is
not an exec ID == ExecID but thats not true. ExecID == "" if it is not
an exec. This uses the correct ID in all cases.

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-08-27 11:08:12 -07:00
Michael Crosby
bc1ff51411 Don't block on STDIN open
This was found testing other runtime shims that are faster than runc(no
containerization).  This is a race that can cause the shim to block
forever.  It's not an issue for out/err because we open both sides of
the pipe, but for stdin, it expects the client to have it opened.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-08-27 10:44:53 -04:00
Justin Terry (VM)
019b0c34de Introduce containerd-shim-runhcs-v1 on Windows
Implements the containerd-shim-runhcs-v1 shim on Windows for the runtime
v2 shim API.

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-08-22 08:15:43 -07:00
Justin Terry (VM)
8362d9aa48 switch shim log Windows client/server direction
Switches the client/server direction of the shim-log pipe on Windows so
that the shim is the listener. This allows the containerd client to
reconnect as needed to the log streams.

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-08-08 12:58:45 -07:00
Michael Crosby
6ba4ddfdda Add shim log pipe for log forwarding to the daemon
A fifo on unix or named pipe on Windows will be provided to the shim.
It can be located inside the `cwd` of the shim named "log".
The shims can use the existing `github.com/containerd/containerd/log` package to log debug messages.
Messages will automatically be output in the containerd's daemon logs with the correct fiels and runtime set.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-08-07 15:31:00 -04:00
Justin Terry (VM)
dcb905701c Adds retry support to Windows AnonDialer
Adds retry support to AnonDialer if the pipe does not exist. This will
retry up to the timeout for the pipe to exist and connect. This solves
the race between the containerd-shim-* start command and the
reinvocation.

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-08-01 13:22:25 -07:00
Justin Terry (VM)
9f13b74f4a Runtime v2 absolute shim path to executable
Fixes an issue where the runtime v2 was not using an absolute path to
the executable but setting the .Dir field on the exec.Cmd. This causes
the executable to need to be relative to .Dir but no shim is actually
copied to the bundle directory that its work dir is set to.

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-07-31 15:07:55 -07:00
Michael Crosby
23fbdbaf13 Cleanup workdirs on manager load
This cleans up persistent work dirs on TaskManager boot.  These dirs can
be left behind in a machine reboot.  The state in /run will not exist
but the work dir in the root does, we should cleanup work dirs when
tasks are not loaded.

This also improves error handling that would prevent the task manager
from loading when a single task fails to load or cleanup.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-07-30 10:21:04 -04:00
Derek McGowan
362405f7b5
Merge pull request #2495 from jterry75/runtime_v2_windows
Adds runtime v2 support for Windows shim's
2018-07-27 11:24:34 -07:00
Justin Terry (VM)
af1b6a026e Review feedback.
1. Moves the log message for each socket to the appropriate _unix and
_windows.go
2. Replaces all reference to Abstract Socket for Windows.
3. Adds support for ctrl+c on Windows to exit a shim.

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-07-27 09:49:33 -07:00
Michael Crosby
13549f7a07 Abstract to SocketAddress
This updates some methods for cross platform use as well as unifying
_unix files.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-07-27 12:39:53 -04:00
Michael Crosby
9d72b4543b Handle windows signals
Since windows does not require a signal handler, we just block on the
channel forever so that it does not exit.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-07-26 11:24:20 -04:00
Michael Crosby
f15a1170d3 Add windows publisher
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-07-26 11:21:50 -04:00
Justin Terry (VM)
d3e0c163f8 Adds runtime v2 support for Windows shim's
Implements the various requirements for the runtime v2 code to abstract
away the unix/linux code into the appropriate platform level
abstractions to use the runtime v2 on Windows as well.

Adds support in the Makefile.windows to actually build the runtime v2
code for Windows by setting a shell environment BUILD_WINDOWS_V2=1
before calling make. (Note this disables the compilation of the Windows
runtime v1)

Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2018-07-25 14:09:26 -07:00
Michael Crosby
710df57854 Fast path bundle cleanup on load
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-07-25 14:37:56 -04:00
Michael Crosby
17ab11a236 Fixes for runtimev2 and checkpoint restore
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-07-20 12:09:29 -04:00
Michael Crosby
d53a96fbe4 Add comments for oci protos
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-07-18 15:12:18 -04:00
Michael Crosby
026b7d922f Add README.md to runtime v2
This readme specifies shim api and authoring docs.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-07-18 12:08:11 -04:00
Michael Crosby
fd31052d9c Add oom events to shim
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-07-17 16:52:05 -04:00
Michael Crosby
7e49c601a8 Add shim start for shim creation
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-07-17 10:21:29 -04:00
Michael Crosby
da1b5470cd Runtime v2
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-07-17 10:21:29 -04:00