Commit Graph

391 Commits

Author SHA1 Message Date
Michael Crosby
2ed8d12bb0
Merge pull request #3845 from fahedouch/v2_shim_test
v2 runtime shim test
2020-04-13 12:26:05 -04:00
Maksym Pavlenko
0caa233158 Rework shim logger shutdown process
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2020-04-07 12:42:04 -07:00
Michael Crosby
649f2aac66 add -v to shim binaries
Request came from a slack message that shims do not output their versions making
it hard for users and operators to know what version of a shim they have on the
system.  This adds a `-v` flag to the shims so that users can see if a shim is
in sync with containerd or what versions of shims that they are running.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2020-03-17 13:23:06 -04:00
Maksym Pavlenko
2532bdf43f
Merge pull request #4100 from lifubang/publisher
fix dial error when clean up a dead shim
2020-03-14 15:19:48 -07:00
lifubang
488d6194f2 fix dial error when clean up a dead shim
Signed-off-by: lifubang <lifubang@acmcoder.com>
2020-03-12 10:57:55 +08:00
Kir Kolyshkin
6e638ad27a Nit: fix use of bufio.Scanner.Err
The Err() method should be called after the Scan() loop, not inside it.

Found by: git grep -A3 -F '.Scan()'

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-03-11 19:36:21 -07:00
Tobias Klauser
a9bd451ab4 Avoid duplicate imports of github.com/gogo/protobuf/types
Re-use the import aliased as `ptypes`.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2020-03-10 09:41:03 +01:00
zyu
e3ab8bda60 Avoid allocating slice for finding Process
Signed-off-by: zyu <yuzhihong@gmail.com>
2020-03-06 09:51:26 -08:00
Ted Yu
a687d3a36d Check error return from json.Unmarshal
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2020-03-05 13:38:08 -05:00
Maksym Pavlenko
4d242818bf
Merge pull request #4053 from AkihiroSuda/vendor-grpc-20200225
vendor protobuf & grpc (GoGoProtoPackageIsVersion3)
2020-02-27 11:59:59 -08:00
Phil Estes
669f516b0e
Merge pull request #4062 from tedyu/start-shim-defer
Use named error return for service#StartShim
2020-02-27 13:23:31 -05:00
Ted Yu
f8ade8debd Use named error return for service#StartShim
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2020-02-27 06:18:05 -08:00
Derek McGowan
06b284026d
Merge pull request #4063 from tedyu/namespace-path
fix killall when use pidnamespace
2020-02-26 23:08:31 -08:00
Ted Yu
4105135e36 fix killall when use pidnamespace
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2020-02-26 20:56:49 -08:00
Phil Estes
ebec675a8d
Merge pull request #3802 from vladimiroff/unify-dialers
Unify dialer implementations
2020-02-26 16:54:22 -05:00
Kiril Vladimiroff
4dd75be2b9
Unify dialer implementations
Instead of having several dialer implementations, leave only one in
`pkg/dialer` and call it from `pkg/ttrpcutil`, `runtime/v(1|2)/shim`
which had their own

Closes #3471.

Signed-off-by: Kiril Vladimiroff <kiril@vladimiroff.org>
2020-02-26 23:29:04 +02:00
Akihiro Suda
8e448bb279 vendor protobuf & grpc
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-02-26 10:57:05 +09:00
Wei Fu
18e581dd91 bugfix: cleanup dangling shim by brand new context
When there is timeout or cancel for create container, killShim will fail
because of canceled context. The shim will be dangling and unmanageable.

Need to use new context to do cleanup.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2020-02-21 16:49:58 +08:00
Li Yuxuan
84464b801f v2: Cancel shim log ctx when ttrpc is closed
The background context aovids shim blocking when the ctx is cancelled
unexpectedly during shim start. But if the shim exits unexpectedly
before opening the pipe, the fd will never be closed.
`onCloseWithShimLog` makes sure that the shim log fd is closed properly
once the shim disconnects.

Signed-off-by: Li Yuxuan <liyuxuan04@baidu.com>
2020-02-20 23:20:10 +08:00
fahedouch
486d33631e test runtime v2 CPU settings
Signed-off-by: fahedouch <fahed.dorgaa@gmail.com>
2020-01-14 18:23:54 +01:00
Seth Pellegrino
66508589d3 fix: eventfd leak for v2 runtime with v1 cgroups
There's no OOM monitoring for the v2 cgroups yet, so it seems unlikely
that there was a leak in that case.

Signed-off-by: Seth Pellegrino <spellegrino@newrelic.com>
2020-01-13 10:49:11 -08:00
Seth Pellegrino
9456040acb fix: eventfd leak
Only start watching the cgroup for OOMs when the first process starts
instead of on every process.

Signed-off-by: Seth Pellegrino <spellegrino@newrelic.com>
2020-01-13 10:39:54 -08:00
Li Yuxuan
1fb1d93212 v2: Fix missing ns when openShimLog on windows
Related to
https://github.com/containerd/containerd/pull/3921#discussion_r363046745

Signed-off-by: Li Yuxuan <liyuxuan04@baidu.com>
2020-01-05 19:42:33 +08:00
Li Yuxuan
d82fa43193 v2: Call shim.Delete at first when create is failed
If the context is cancelled during `shim.Create()`, such as the client
disconnects unexpectedly. The created shim will never be deleted.
What's more, if the context is cancelled during `openShimLog()`, the
fifo will be closed and block the shim output.

Signed-off-by: Li Yuxuan <liyuxuan04@baidu.com>
2019-12-28 00:02:11 +08:00
Erik Sipsma
fbd46d7094
runtime v2: Close platform in runc shim's Shutdown method.
Previously, the platform was closed as part of the Delete method when the
process was an init for a task and there were no more tasks after its deletion.
This can create problems if another task is created within the shim right after
the delete runs, which results in the platform being closed but the shim
continuing to run.

This change moves closing the platform to the Shutdown method after the shim's
context is canceled, which ensures the platform is only closed once the shim
is sure its done servicing containers.

Signed-off-by: Erik Sipsma <sipsma@amazon.com>
2019-12-19 09:47:40 -05:00
Akihiro Suda
b02e20f12e cgroup2: enable controllers automatically
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2019-12-12 02:56:51 +09:00
Akihiro Suda
8f870c233f support cgroup2
* only shim v2 runc v2 ("io.containerd.runc.v2") is supported
* only PID metrics is implemented. Others should be implemented in separate PRs.
* lots of code duplication in v1 metrics and v2 metrics. Dedupe should be separate PR.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2019-12-12 02:56:51 +09:00
Michael Crosby
f8cca26f3c Handle large output in v2 shim with TTY
Reized the I/O buffers to align with the size of the kernel buffers with fifos
and move the close aspect of the console to key off of the stdin closing.

Fixes #3738

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-10-11 15:42:05 -04:00
Lantao Liu
ffcb1cc9be Fix delete error code on the containerd daemon side.
Signed-off-by: Lantao Liu <lantaol@google.com>
2019-10-09 00:28:51 -07:00
Lantao Liu
06be794cb2 Fix shim delete error code.
Signed-off-by: Lantao Liu <lantaol@google.com>
2019-10-07 23:21:57 -07:00
Derek McGowan
0b224ac7d6
Update metadata interfaces for containers and leases
Add more thorough dirty checking across all types which
may be deleted and hold references.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-09-23 15:27:39 -07:00
Kathryn Baldauf
b4211d94e2 fail on file not found for shim reconnect on containerd restart
Signed-off-by: Kathryn Baldauf <kabaldau@microsoft.com>
2019-09-17 14:49:29 -07:00
Derek McGowan
b039c39186
Merge pull request #3564 from tiborvass/move-cgroups-dep-to-namespaces-pkg
runtime/opts: move WithNamespaceCgroupDeletion from containerd to its own package
2019-09-03 10:38:53 -07:00
Kathryn Baldauf
2d8a65b1b2 Export shim publisher functions
- Our out of tree shim would like to publish events with ttrpc. These
functions should be exposed so our shim doesn't need to reimplement
publisher logic.

Signed-off-by: Kathryn Baldauf <kabaldau@microsoft.com>
2019-08-27 17:15:15 -07:00
Tibor Vass
6624a70d92 runtime/opts: move WithNamespaceCgroupDeletion from containerd to its own package
The cgroup dependency brings in quite a lot only for WithNamespaceCgroupDeletion,
which is a namespaces.DeleteOpt.

Signed-off-by: Tibor Vass <tibor@docker.com>
2019-08-27 19:02:55 +00:00
chentanjun
8266a3c5e7 fix-up spelling mistake
Signed-off-by: chentanjun <2799194073@qq.com>
2019-08-27 13:45:41 +08:00
Michael Crosby
6cf031e1e4 Pass ttrpc address to shim via env
Because of the way go handles flags, passing a flag that is not defined
will cause an error. In our case, if we kept this as a flag, then
third-party shims would break when they see this new flag.  To fix this,
I moved this new configuration option to an env var.  We should use env
vars from here on out to avoid breaking shim compat.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-08-22 20:37:49 +00:00
Kevin Parsons
d7e1b25384 Allow explicit configuration of TTRPC address
Previously the TTRPC address was generated as "<GRPC address>.ttrpc".
This change now allows explicit configuration of the TTRPC address, with
the default still being the old format if no value is specified.

As part of this change, a new configuration section is added for TTRPC
listener options.

Signed-off-by: Kevin Parsons <kevpar@microsoft.com>
2019-08-22 00:56:27 -07:00
Phil Estes
640860a042
Merge pull request #3559 from fuweid/avoid-read-config
runtime: only check killall for init process
2019-08-20 13:08:55 -04:00
Michael Crosby
08061c7c3c
Merge pull request #3540 from crosbymichael/shim-hang
Use non-blocking send and retry for exit events
2019-08-20 09:31:21 -04:00
Wei Fu
1073868e5e runtime: only check killall for init process
When containerd-shim does reaper, the most processes are not init
process. Since json.Decode consumes more CPU resource, we should check
killall option for init process only.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2019-08-20 19:18:34 +08:00
Phil Estes
fc9335d75c
Merge pull request #3459 from crosbymichael/timeout-config
Allow timeouts to be configured in config
2019-08-19 13:16:43 -04:00
Li Yuxuan
04caf1fc4e Ignore fifo error when using v2 multi-container shim
When using a multi-container shim, the fifo of the 2nd to Nth container
will not be opened when the ctx is done. This will cause an
`ErrReadClosed` that can be ignored.

Signed-off-by: Li Yuxuan <liyuxuan04@baidu.com>
2019-08-17 09:40:08 +08:00
Michael Crosby
0d27d8f4f2 Unifi reaper logic into package
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-08-16 13:55:05 +00:00
Shukui Yang
bb4c92c773 Fix shim hung
shim.Reap and shim.Default.Wait may deadlock, use Monitor.Notify
to fix this issue.

Signed-off-by: Shukui Yang <keloyangsk@gmail.com>
2019-08-16 13:55:05 +00:00
Michael Crosby
2e8ea9fd6b Allow timeouts to be configured in config
This adds a singleton `timeout` package that will allow services and user
to configure timeouts in the daemon.  When a service wants to use a
timeout, it should declare a const and register it's default value
inside an `init()` function for that package.  When the default config
is generated, we can use the `timeout` package to provide the available
timeout keys so that a user knows that they can configure.

These show up in the config as follows:

```toml
[timeouts]
  "io.containerd.timeout.shim.cleanup" = 5
  "io.containerd.timeout.shim.load" = 5
  "io.containerd.timeout.shim.shutdown" = 3
  "io.containerd.timeout.task.state" = 2

```

Timeouts in the config are specified in seconds.

Timeouts are very hard to get right and giving this power to the user to
configure things is a huge improvement.  Machines can be faster and
slower and depending on the CPU or load of the machine, a timeout may
need to be adjusted.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-08-13 17:36:32 +00:00
Akihiro Suda
225cc7d5bd
Merge pull request #3494 from jterry75/remove_v2
Completely remove Windows v2 in-tree shim
2019-08-07 02:19:12 +09:00
Li Yuxuan
08483d18ad v2: Close ttrpc connection when Delete()
This avoids potential socket leak when the connected v2 shim of runtime
serving multiple containers.

Signed-off-by: Li Yuxuan <liyuxuan04@baidu.com>
2019-08-06 20:35:59 +08:00
Justin Terry (VM)
4b5dfaee13 Completely remove Windows v2 in-tree shim
Signed-off-by: Justin Terry (VM) <juterry@microsoft.com>
2019-08-05 16:49:56 -07:00
Derek McGowan
ac1cb6d5d4
Merge pull request #3467 from kevpar/dial-pipe-err
Improve error return from AnonDialer on Windows
2019-08-01 15:41:54 -07:00