Commit Graph

447 Commits

Author SHA1 Message Date
Kazuyoshi Kato
6596a70861 Use github.com/containerd/cgroups/v3 to remove gogo
Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2022-11-14 21:07:48 +00:00
Sebastiaan van Stijn
eaedadbed0
replace strings.Split(N) for strings.Cut() or alternatives
Go 1.18 and up now provides a strings.Cut() which is better suited for
splitting key/value pairs (and similar constructs), and performs better:

```go
func BenchmarkSplit(b *testing.B) {
        b.ReportAllocs()
        data := []string{"12hello=world", "12hello=", "12=hello", "12hello"}
        for i := 0; i < b.N; i++ {
                for _, s := range data {
                        _ = strings.SplitN(s, "=", 2)[0]
                }
        }
}

func BenchmarkCut(b *testing.B) {
        b.ReportAllocs()
        data := []string{"12hello=world", "12hello=", "12=hello", "12hello"}
        for i := 0; i < b.N; i++ {
                for _, s := range data {
                        _, _, _ = strings.Cut(s, "=")
                }
        }
}
```

    BenchmarkSplit
    BenchmarkSplit-10            8244206               128.0 ns/op           128 B/op          4 allocs/op
    BenchmarkCut
    BenchmarkCut-10             54411998                21.80 ns/op            0 B/op          0 allocs/op

While looking at occurrences of `strings.Split()`, I also updated some for alternatives,
or added some constraints; for cases where an specific number of items is expected, I used `strings.SplitN()`
with a suitable limit. This prevents (theoretical) unlimited splits.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-07 10:02:25 +01:00
Sebastiaan van Stijn
f90219d472
services/server/config: TestMergeConfigs(): use correctly formatted values
This updates the test to:

- Use correctly formatted values for RequiredPlugins and DisabledPlugins (values
  are expected to have a `io.containerd.` prefix). While not needed for the test
  to pass (no validation is performed), it's good to have these values in the
  correct format (in case we want to add validation at this stage).
- Set a `Version` for both (as version 1 / no version was deprecated)

The `Version` field in this test was used to verify the "integer override"
behavior; setting "Version: 2" for both would no longer cover that case. As there
are only 2 integer fields in the config (Version and OOMScore) and OOMScore was
already used in the test, I added separate test-cases for that.

Looking at the test, we should consider what we want the behaviour to be if the
override file does not specify a version (implicitly: version 1), or if the version
is different from the original one; do we want mergeConfig() to produce an error
when merging a v2 config with a v1 config (or v3 with v2)?

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-07 10:01:28 +01:00
Maksym Pavlenko
b7d0d12715 Cleanup sandbox interfaces
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-10-25 12:31:32 -04:00
Sebastiaan van Stijn
3ebeb6d79b
linting: address gosec G112/G114
GOGC=75 golangci-lint run
    services/server/server.go:320:27: G114: Use of net/http serve function that has no support for setting timeouts (gosec)
        return trapClosedConnErr(http.Serve(l, m))
                                 ^
    services/server/server.go:340:27: G114: Use of net/http serve function that has no support for setting timeouts (gosec)
        return trapClosedConnErr(http.Serve(l, m))
                                 ^
    cmd/containerd-stress/main.go:238:13: G114: Use of net/http serve function that has no support for setting timeouts (gosec)
            if err := http.ListenAndServe(c.Metrics, metrics.Handler()); err != nil {
                      ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-12 14:40:11 +02:00
Akihiro Suda
70fbedc217
archive: add WithSourceDateEpoch() for whiteouts
This makes diff archives to be reproducible.

The value is expected to be passed from CLI applications via the $SOUCE_DATE_EPOCH env var.

See https://reproducible-builds.org/docs/source-date-epoch/
for the $SOURCE_DATE_EPOCH specification.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2022-10-08 08:45:03 +09:00
Daniel Canter
4333e6a6d6 Swap to net.ErrClosed checks for services
In Go 1.16 `net.ErrClosed` was exported, removing the need to check the
exact text of "use of closed network connection". The stdlib's net listeners
are all setup for this to be a reality, but on Windows containerd uses the
the go-winio projects named pipe implementation as the listener for services.
Before version 0.6.0 this project returned a different error named
`ErrPipeListenerClosed` for using a closed pipe, where this error was just
an `errors.New` with the same text as `net.ErrClosed`, so checking against
`net.ErrClosed` wasn't possible.

Starting in 0.6.0 go-winio has that error assigned to `net.ErrClosed` directly
so this *should* be alright to finally change.

Signed-off-by: Daniel Canter <dcanter@microsoft.com>
2022-09-28 18:20:12 -07:00
zounengren
344b25995b delete redundent import alias and and type conversion
Signed-off-by: zounengren <zouyee1989@gmail.com>
2022-08-31 21:36:24 +08:00
Maksym Pavlenko
ff65fc2d0e Make TaskList generic
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-08-10 14:02:53 -07:00
Maksym Pavlenko
aa3303b697 Update sandbox protobuf to match CRI
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-07-29 16:08:07 -07:00
Derek McGowan
99e210c50a
Move metadata plugin registration to seperate package
Signed-off-by: Derek McGowan <derek@mcg.dev>
2022-06-22 17:38:41 -07:00
Derek McGowan
4ac1031e0c
Add snapshotter key to snapshot events
Consumers of snapshot events require the snapshotter
field to refer to the related snapshot.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2022-06-20 18:25:53 -07:00
Oleg Zhurakivskyy
8245e35eb8 Make test path a constant
Signed-off-by: Oleg Zhurakivskyy <oleg.zhurakivskyy@intel.com>
2022-06-13 23:28:17 +03:00
Akihiro Suda
739cb4c99a
config: improve config v1 deprecation message
The previous wording was causing confusion.
Fix moby/moby issue 43628

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2022-05-23 10:10:07 +09:00
AllenZMC
eaec6530d7 fix some confusing typos
Signed-off-by: AllenZMC <zhongming.chang@daocloud.io>
2022-05-17 23:53:36 +08:00
Mike Brown
6b35307594
Merge pull request #5490 from askervin/5Bu_blockio
Support for cgroups blockio
2022-04-29 10:07:56 -05:00
Antti Kervinen
10576c298e cri: support blockio class in pod and container annotations
This patch adds support for a container annotation and two separate
pod annotations for controlling the blockio class of containers.

The container annotation can be used by a CRI client:
  "io.kubernetes.cri.blockio-class"

Pod annotations specify the blockio class in the K8s pod spec level:
  "blockio.resources.beta.kubernetes.io/pod"
  (pod-wide default for all containers within)

  "blockio.resources.beta.kubernetes.io/container.<container_name>"
  (container-specific overrides)

Correspondingly, this patch adds support for --blockio-class and
--blockio-config-file to ctr, too.

This implementation follows the resource class annotation pattern
introduced in RDT and merged in commit 893701220.

Signed-off-by: Antti Kervinen <antti.kervinen@intel.com>
2022-04-29 11:44:09 +03:00
Kazuyoshi Kato
1a095e18ba Rename Size_ to Size
Previouslty "Size" was reserved by protoc-gen-gogoctrd and user-generated
"Size" was automatically renamed to "Size_" to avoid conflicts.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2022-04-22 15:31:53 +00:00
Kazuyoshi Kato
e3db7de8f5 Remove gogo/protobuf and adjust types
This commit migrates containerd/protobuf from github.com/gogo/protobuf
to google.golang.org/protobuf and adjust types. Proto-generated structs
cannot be passed as values.

Fixes #6564.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2022-04-22 15:31:53 +00:00
Kazuyoshi Kato
dfa6e8763e diff: hide types.Any from clients
This commit hides types.Any from the diff package's interface. Clients
(incl. imgcrypt) shouldn't aware about gogo/protobuf.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2022-04-21 13:43:20 +00:00
Kazuyoshi Kato
aa1526defc Use proto.Marshal instead of calling Marshal() on objects
With google.golang.org/protobuf, proto-generated objects only have
ProtoReflect(). They don't have Marshal() anymore (see #6564).

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2022-04-20 17:14:24 +00:00
Kazuyoshi Kato
95dde4959d Embed "Unimplemented" structs as proto recommended
Embedding these structs will be mandatory after migratring off from
gogo/protobuf (see #6564).

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2022-04-20 17:14:23 +00:00
Kazuyoshi Kato
237ef0de9b Remove all gogoproto extensions
This commit removes the following gogoproto extensions;

- gogoproto.nullable
- gogoproto.customename
- gogoproto.unmarshaller_all
- gogoproto.stringer_all
- gogoproto.sizer_all
- gogoproto.marshaler_all
- gogoproto.goproto_unregonized_all
- gogoproto.goproto_stringer_all
- gogoproto.goproto_getters_all

None of them are supported by Google's toolchain (see #6564).

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2022-04-20 07:23:28 +00:00
Maksym Pavlenko
e2fd25f3d8 Move runtime v2 proto
Move runtime v2 protos to api/runtime package.

Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-04-19 17:59:33 -07:00
Kazuyoshi Kato
88c0c7201e Consolidate gogo/protobuf dependencies under our own protobuf package
This would make gogo/protobuf migration easier.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2022-04-19 15:53:36 +00:00
Kazuyoshi Kato
80b825ca2c Remove gogoproto.stdtime
This commit removes gogoproto.stdtime, since it is not supported by
Google's official toolchain
(see https://github.com/containerd/containerd/issues/6564).

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2022-04-19 13:39:30 +00:00
Derek McGowan
be60973a30
Merge pull request #6703 from mxpv/s
Sandbox API
2022-04-18 20:55:06 -07:00
Derek McGowan
fe8da6dcaf
Move lease manager plugin to separate package
Create lease plugin type to separate lease manager from services plugin.
This allows other service plugins to depend on the lease manager.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2022-04-15 11:08:47 -07:00
Fu Wei
e5f4c29a2b
Merge pull request #6792 from Junnplus/typo 2022-04-11 11:08:39 +08:00
Ye Sijun
6d69aca561 make consistent for checkpoint path
Signed-off-by: Ye Sijun <junnplus@gmail.com>
2022-04-09 17:28:17 +08:00
Maksym Pavlenko
85a49e4ee7 [sandbox] Cleanup interfaces
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-04-08 13:34:50 -07:00
Maksym Pavlenko
de49745723 [sandbox] Fix CI
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-04-08 13:34:48 -07:00
Maksym Pavlenko
b7a36950f6 [Sandbox] Add Wait and PID
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-04-08 13:33:48 -07:00
Maksym Pavlenko
0d165e6544 Restore sandboxes on daemon restart
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-04-08 13:33:48 -07:00
Maksym Pavlenko
35195737e7 [sandbox] Register shim plugin after #6301
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-04-08 13:33:48 -07:00
Maksym Pavlenko
982de8a5d5 Launch sandboxed containers from task service
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-04-08 13:33:47 -07:00
Maksym Pavlenko
00f7a6bf2b [sandbox] Address PR review comments
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-04-08 13:33:47 -07:00
Maksym Pavlenko
6343fe3ea2 [sandbox] Implement sandbox controller
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-04-08 13:33:47 -07:00
Maksym Pavlenko
df234edeb4 [sandbox] Add controller service
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-04-08 13:33:47 -07:00
Maksym Pavlenko
a433beb79f [sandbox] Update vendor
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-04-08 13:33:47 -07:00
Maksym Pavlenko
0e77f758a4 [sandbox] Implement store service
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-04-08 13:33:47 -07:00
Derek McGowan
ffddd4446c
Merge pull request #6761 from kzys/bbolt-freelist
Disable writing freelist to make the file robust against data corruptions
2022-04-05 21:36:49 -07:00
Maksym Pavlenko
871b6b6a9f Use testify
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-04-01 18:17:58 -07:00
Kazuyoshi Kato
6da3183105 Disable writing freelist to make the file robust against data corruptions
A bbolt database has a freelist to track all pages that are available
for allocation. However writing the list takes some time and reading
the list sometimes panics.

This commit sets NoFreelistSync true to skipping the freelist entirely,
following what etcd does.

https://github.com/etcd-io/etcd/blob/v3.5.2/server/mvcc/backend/config_linux.go#L31

Fixes #4838.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2022-04-01 21:06:42 +00:00
Phil Estes
3633cae64b
Merge pull request #6706 from kzys/typeurl-upgrade
Use typeurl.Any instead of github.com/gogo/protobuf/types.Any
2022-03-25 10:38:46 -04:00
Kazuyoshi Kato
96b16b447d Use typeurl.Any instead of github.com/gogo/protobuf/types.Any
This commit upgrades github.com/containerd/typeurl to use typeurl.Any.
The interface hides gogo/protobuf/types.Any from containerd's Go client.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2022-03-24 20:50:07 +00:00
Sebastiaan van Stijn
c091d48cb9
Use cgroups.AddProc() for cgroups v1
All occurrences only passed a PID, so we can use this utility to make
the code more symmetrical with their cgroups v2 counterparts.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-23 15:25:09 +01:00
Kazuyoshi Kato
067611fdea Remove enumvalue_customname, goproto_enum_prefix and enum_customname
This commit removes gogoproto.enumvalue_customname,
gogoproto.goproto_enum_prefix and gogoproto.enum_customname.

All of them make proto-generated Go code more idiomatic, but we already
don't use these enums in our external-surfacing types and they are anyway
not supported by Google's official toolchain (see #6564).

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2022-03-21 19:48:16 +00:00
Kazuyoshi Kato
3eeeb9429a Remove gogoproto.customtype
gogoproto.customtype is used to have go-digest.Digest instead of string.
While it is convinient, protoc-gen-go doesn't support the extension
and that blocks #6564.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2022-03-18 23:14:44 +00:00
Fu Wei
79d7df70d6
Merge pull request #6681 from Juneezee/test/t.TempDir 2022-03-16 14:54:16 +08:00