Commit Graph

67 Commits

Author SHA1 Message Date
Kazuyoshi Kato
2ee3ce510c Use insecure.NewCredentials instead of grpc.WithInsecure
grpc.WithInsecure is being deprecated.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2021-12-16 23:10:06 +00:00
Phil Estes
330961c2d5
Merge pull request #6358 from jonyhy96/feat-error
refactor: functions for error log and error return
2021-12-14 10:16:54 -05:00
Phil Estes
6e9e759553
Merge pull request #6225 from jonyhy96/feat-blot-open-timeout
feat: add timeout for bolt open
2021-12-13 11:22:28 -05:00
haoyun
dd26d3d092 feat: support custom timeout for blot open
Signed-off-by: haoyun <yun.hao@daocloud.io>
2021-12-13 17:02:37 +08:00
haoyun
c0d07094be feat: Errorf usage
Signed-off-by: haoyun <yun.hao@daocloud.io>
2021-12-13 14:31:53 +08:00
Justin Terry
63895de455 Add support for TMP override on toml
When running containerd as a service it may be hard to
override the TMP location of the process. This is especially
true on Windows when running containerd in SCM. This change
allows you to set the 'temp' location in the config.toml when
the service starts up that overrides its TEMP/TMP/TMPDIR usage.

This is helpful on Linux as well but it primarily solves the
performance issue on Windows when running containerd across
volumes. IE: If you configure your data/root paths on a volume
other than the SystemDrive the snapshotter does a temporary unpack
on the SystemDrive and then has to copy contents of that data
to the snapshot folder on the destination volume. By alinging the
tmp with the destination it is a simple move operation instead of
a copy operation.

Signed-off-by: Justin Terry <jlterry@amazon.com>
2021-11-16 10:43:48 -08:00
Brian Goff
130a9c7ddb Ensure namespace is proxied to grpc/ttrpc plugins
Before this change we only ever had the grpc incoming medata set so when
we make a request to a shim or a grpc plugin the namespace is not sent
over the RPC.

I need this for github.com/cpuguy83/systemdshim, and I am sure there are
other use-cases where this would be needed.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-10-16 00:09:50 +00:00
Derek McGowan
18d483b236
Update cgroups to v1.0.2
Signed-off-by: Derek McGowan <derek@mcg.dev>
2021-10-08 17:48:33 -07:00
Michael Crosby
f1054dbbde
fix integration client vendor
Signed-off-by: Michael Crosby <michael@thepasture.io>
2021-10-08 16:20:21 +00:00
Michael Crosby
e48bbe8394 add runc shim support for sched core
In linux 5.14 and hopefully some backports, core scheduling allows processes to
be co scheduled within the same domain on SMT enabled systems.

The containerd impl sets the core sched domain when launching a shim. This
allows a clean way for each shim(container/pod) to be in its own domain and any
additional containers, (v2 pods) be be launched with the same domain as well as
any exec'd process added to the container.

kernel docs: https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/core-scheduling.html

Signed-off-by: Michael Crosby <michael@thepasture.io>
2021-10-08 16:18:09 +00:00
Derek McGowan
63b7e5771e
Merge pull request #5973 from Juneezee/deprecate-ioutil
refactor: move from io/ioutil to io and os package
2021-10-01 10:52:06 -07:00
Brian Goff
084387e0b4 Move tracing to plugin
This just makes the implementation a little cleaner.
It also makes the trace exporter pluggable.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-09-21 21:19:46 +00:00
Eng Zer Jun
50da673592
refactor: move from io/ioutil to io and os package
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-09-21 09:50:38 +08:00
Fu Wei
e1ad779107
Merge pull request #5817 from dmcgowan/shim-plugins
Add support for shim plugins
2021-09-12 18:18:20 +08:00
Akihiro Suda
d3aa7ee9f0
Run go fmt with Go 1.17
The new `go fmt` adds `//go:build` lines (https://golang.org/doc/go1.17#tools).

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-08-22 09:31:50 +09:00
Derek McGowan
8d135d2842
Add support for shim plugins
Refactor shim v2 to load and register plugins.
Update init shim interface to not require task service implementation on
returned service, but register as plugin if it is.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2021-08-17 11:06:09 -07:00
Michael Crosby
218db0f9af
Merge pull request #5835 from dmcgowan/plugin-events-cleanup
Move plugin context events into separate plugin
2021-08-07 21:47:11 -04:00
Derek McGowan
0a0621bb47
Move plugin context events into separate plugin
Signed-off-by: Derek McGowan <derek@mcg.dev>
2021-08-05 22:59:20 -07:00
Derek McGowan
6f027e38a8
Remove redundant build tags
Remove build tags which are already implied by the name of the file.
Ensures build tags are used consistently

Signed-off-by: Derek McGowan <derek@mcg.dev>
2021-08-05 22:27:46 -07:00
Alakesh Haloi
3597ac859d [otel-tracing] Initial opentelemetry support
Add basic intiialization of opentelemetry including minimum support to
be able to read open telemetry config from config.toml and initialize
exporter. Tracer is initialized and ready to be be used for creating
spans, sub spans etc. With no opentelemetry configuration enabled in
config file, this patch is a no-op.

Basic config stub to be added to use opentelemetry is to add following
in config.toml. We use otlp exporter with default port 4317.

[otel]
  exporter_name = "otlp"
  exporter_endpoint = "0.0.0.1:4317"

otel-collector binary needs to run listening at the same port.

Signed-off-by: Alakesh Haloi <alakeshh@amazon.com>
2021-08-04 14:25:01 -07:00
zwtop
63fe34add6 grpc config add options tcp_tls_ca
Signed-off-by: zwtop <wang.zhan@smartx.com>
2021-06-30 10:58:32 +08:00
ktock
fdb76f55d8 Fix backword-compatibility issue of non-versioned config file
According to the doc about `config.toml` of containerd:

```
If no version number is specified inside the config file then it is assumed to
be a version 1 config and parsed as such.
```

However, it's not true recently.
This will break the backward-compatibility in some environment.
This commit fixes this issue.

Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2021-04-15 10:00:58 +09:00
Davanum Srinivas
9ad087947d
Switch all our tests to version 2
Also warn when someone uses version 1

Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2021-04-12 13:18:33 -04:00
Maksym Pavlenko
ddd4298a10 Migrate current TOML code to github.com/pelletier/go-toml
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2021-03-25 13:13:33 -07:00
Stefan Berger
1917ca5f79 Allow passing environent variables to StreamProcessors
Add support for an 'env' field to the StreamProcessor configuration
and append the environment variables found there to the os.Environ()
array.
The env field takes environment variables in the form of key=value.

Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
2021-01-11 13:15:12 -05:00
Vlad Ungureanu
f12b68cc86 Allow configuration of different log formats: text, json
Signed-off-by: Vlad Ungureanu <vladu@palantir.com>
2020-12-04 13:12:41 -05:00
Sebastiaan van Stijn
f2edc6f164
vendor: update gotest.tools v3.0.2
full diff: https://github.com/gotestyourself/gotest.tools/compare/v2.3.0...v3.0.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-28 17:47:20 +01:00
Akihiro Suda
8e448bb279 vendor protobuf & grpc
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-02-26 10:57:05 +09:00
Akihiro Suda
8f870c233f support cgroup2
* only shim v2 runc v2 ("io.containerd.runc.v2") is supported
* only PID metrics is implemented. Others should be implemented in separate PRs.
* lots of code duplication in v1 metrics and v2 metrics. Dedupe should be separate PR.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2019-12-12 02:56:51 +09:00
Michael Crosby
f9d8f0e56e
Merge pull request #3794 from fahedouch/server_test
add tests to server_test
2019-11-04 10:37:26 -05:00
fahedouch
7c016b655b
add tests to server_test
Signed-off-by: fahedouch <fahed.dorgaa@gmail.com>
2019-11-04 09:15:05 -05:00
Lantao Liu
aaccfcbe2b Fix containerd config dump.
Signed-off-by: Lantao Liu <lantaol@google.com>
2019-10-23 22:38:18 -07:00
Wei Fu
3d1fdacccf config_test: sort result before check
config imports is in random order after LoadConfig call so that the
testing check should be done after sort.Strings(out.Imports).

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2019-09-05 16:30:28 +08:00
Michael Crosby
f76eefd272
Merge pull request #3574 from mxpv/cfg
Support config imports
2019-09-04 16:34:11 -04:00
Derek McGowan
a4482d9a6f
Fix proxy plugin config validation
Proxy plugins are keyed only on the identifier, the type
is specified within the proxy plugin configuration which
maps to the full URI. The proxy plugin configuration is
not passed to the plugin for configuration like other
plugins.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-09-03 17:31:14 -07:00
Maksym Pavlenko
24b9e2c1a0 Merge configs section by section
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-08-23 15:49:02 -07:00
Maksym Pavlenko
8ebffecbc3 Use map for stream processors
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-08-23 15:31:37 -07:00
Maksym Pavlenko
19cd0a4f12 Append slices when importing config files
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-08-23 10:12:19 -07:00
Maksym Pavlenko
bca0857530 Fix toml plugin decoding
Do not rely on toml metadata when decoding plugin's configs as it's not possible to merge toml.MetaData structs during import.

Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-08-22 17:22:25 -07:00
Maksym Pavlenko
a1e3779cad Support config imports #3289
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-08-22 15:50:58 -07:00
Kevin Parsons
d7e1b25384 Allow explicit configuration of TTRPC address
Previously the TTRPC address was generated as "<GRPC address>.ttrpc".
This change now allows explicit configuration of the TTRPC address, with
the default still being the old format if no value is specified.

As part of this change, a new configuration section is added for TTRPC
listener options.

Signed-off-by: Kevin Parsons <kevpar@microsoft.com>
2019-08-22 00:56:27 -07:00
Michael Crosby
2e8ea9fd6b Allow timeouts to be configured in config
This adds a singleton `timeout` package that will allow services and user
to configure timeouts in the daemon.  When a service wants to use a
timeout, it should declare a const and register it's default value
inside an `init()` function for that package.  When the default config
is generated, we can use the `timeout` package to provide the available
timeout keys so that a user knows that they can configure.

These show up in the config as follows:

```toml
[timeouts]
  "io.containerd.timeout.shim.cleanup" = 5
  "io.containerd.timeout.shim.load" = 5
  "io.containerd.timeout.shim.shutdown" = 3
  "io.containerd.timeout.task.state" = 2

```

Timeouts in the config are specified in seconds.

Timeouts are very hard to get right and giving this power to the user to
configure things is a huge improvement.  Machines can be faster and
slower and depending on the CPU or load of the machine, a timeout may
need to be adjusted.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-08-13 17:36:32 +00:00
Michael Crosby
366823727f Add server config for stream processors
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-08-07 18:35:55 +00:00
Michael Crosby
9547d269a1 Add v2 server config support with plugin URIs
Closes #3210

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-06-10 17:54:20 +00:00
Sebastiaan van Stijn
2ed8e60fa1
bump google.golang.org/grpc v1.20.1
full diff: https://github.com/grpc/grpc-go/compare/v1.12.2...v1.20.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-05-05 12:39:23 -07:00
Davor Kapsa
cfc36388b3 Remove redundant error checks
Signed-off-by: Davor Kapsa <davor.kapsa@gmail.com>
2019-04-30 21:28:51 +02:00
Michael Crosby
8722ec03c3
Merge pull request #3213 from jcordasc/small-fixes
Correct import path and PusherFunc signature
2019-04-16 15:32:51 -04:00
Jared Cordasco
a7af3c13a5 Correct import path in services/server package
Signed-off-by: Jared Cordasco <jcordasc@coglib.com>
2019-04-11 23:07:46 -05:00
Michael Crosby
97d247cd0f Rename hrpc to tcpServer
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-04-11 15:06:06 -04:00
Derek McGowan
5f4c977ba0
Merge pull request #3162 from crosbymichael/tcpservice
Add tcp service for grpc listeners
2019-04-11 11:55:00 -07:00