Commit Graph

192 Commits

Author SHA1 Message Date
Maksym Pavlenko
871b6b6a9f Use testify
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-04-01 18:17:58 -07:00
Kazuyoshi Kato
96b16b447d Use typeurl.Any instead of github.com/gogo/protobuf/types.Any
This commit upgrades github.com/containerd/typeurl to use typeurl.Any.
The interface hides gogo/protobuf/types.Any from containerd's Go client.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2022-03-24 20:50:07 +00:00
Fu Wei
79d7df70d6
Merge pull request #6681 from Juneezee/test/t.TempDir 2022-03-16 14:54:16 +08:00
Phil Estes
58bae86d8e
Merge pull request #6660 from henry118/shared-ns
Add shared content label to namespaces
2022-03-15 13:57:52 -07:00
Eng Zer Jun
18ec2761c0
test: use T.TempDir to create temporary test directory
The directory created by `T.TempDir` is automatically removed when the
test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-03-15 14:03:50 +08:00
Henry Wang
b8bf504e94 Enable gosec linter for golangci-lint
`gosec` linter is able to identify issues described in #6584

e.g.

$ git revert 54e95e6b88
[gosec dfc8ca1ec] Revert "fix Implicit memory aliasing in for loop"
 2 files changed, 2 deletions(-)

$ make check
+ proto-fmt
+ check
GOGC=75 golangci-lint run
containerstore.go:192:54: G601: Implicit memory aliasing in for loop. (gosec)
		containers = append(containers, containerFromProto(&container))
		                                                   ^
image_store.go:132:42: G601: Implicit memory aliasing in for loop. (gosec)
		images = append(images, imageFromProto(&image))
		                                       ^
make: *** [check] Error 1

I also disabled following two settings which prevent the linter to show a complete list of issues.

* max-issues-per-linter (default 50)
* max-same-issues (default 3)

Furthermore enabling gosec revealed many other issues. For now I blacklisted the ones except G601.

Will create separate tasks to address them one by one moving next.

Signed-off-by: Henry Wang <henwang@amazon.com>
2022-03-14 22:50:54 +00:00
Henry Wang
2e080bf491 Add shared content label to namespaces
Signed-off-by: Henry Wang <henwang@amazon.com>
2022-03-11 23:37:02 -08:00
Kazuyoshi Kato
f048a25938 docs: add doc-comments on GC-related methods
Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2022-01-24 14:26:14 -08:00
Cody Roseborough
8dd36423b6 Revert "Add shared content label to namespaces"
This reverts commit e692a01926.

Signed-off-by: Cody Roseborough <cdr@amazon.com>
2022-01-12 16:38:06 -08:00
haoyun
bbe46b8c43 feat: replace github.com/pkg/errors to errors
Signed-off-by: haoyun <yun.hao@daocloud.io>
Co-authored-by: zounengren <zouyee1989@gmail.com>
2022-01-07 10:27:03 +08:00
Kazuyoshi Kato
44b28b61ff medatada: make namespaces' deletion error less cryptic
The error message was unnecessary cryptic. `snapshot-[name]` notation
was only used here and hard to understand.

Instead it should say `snapshots on "..." snapshotter`.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2021-12-13 09:28:24 -08:00
Derek McGowan
63b7e5771e
Merge pull request #5973 from Juneezee/deprecate-ioutil
refactor: move from io/ioutil to io and os package
2021-10-01 10:52:06 -07:00
Derek McGowan
b9cf0d75a9
Fix panic in metadata content writer on copy error
The `createAndCopy` function is only called when `nw.w` is nil
in order to create a new writer and prepare it. The current code
is attempting to close `nw.w` when there is a copy error. The
correct behavior would be to close the new writer and not touch `nw.w`.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2021-09-23 10:29:52 -07:00
Eng Zer Jun
50da673592
refactor: move from io/ioutil to io and os package
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-09-21 09:50:38 +08:00
Cody Roseborough
e692a01926 Add shared content label to namespaces
Adds shared content labels to namespaces allowing content to be shared
between namespaces if that namespace is specifically tagged as being
sharable by adding the `containerd.io/namespace/sharable` label to the
namespace.

Signed-off-by: Cody Roseborough <cdr@amazon.com>
2021-07-28 18:49:32 +00:00
Mike Brown
014748bc04 fix invalid validation error checking
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
2021-06-03 15:58:11 -05:00
Iceber Gu
e37ddafab4 metadata: modify NewLeaseManager to return leases.Manager
Signed-off-by: Iceber Gu <wei.cai-nat@daocloud.io>
2021-05-07 21:26:17 +08:00
Kazuyoshi Kato
cb1580937a metadata: improve deleting a non-empty namespace's error message
Deleting a non-empty namespace fails with

> namespace must be empty: failed precondition

This change improves the error message by listing the types of
the objects in the namespace that prevent deletion.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2021-04-15 15:49:44 -07:00
Maksym Pavlenko
bb6c0c2de7 Add more bolt utils
This PR extracts out a few more bolt utils from the codebase for easier management.

Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2021-04-11 19:32:46 -07:00
Kazuyoshi Kato
e74ace9ad8 content: support filters on local.store#Walk()
While Walk() has been taking filter strings, it was not using the parameter.
This change actually makes the filtering work.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2020-11-11 11:17:48 -08:00
Hajime Tazaki
f4741fb8c5 fix make test failure of missing sha256 package
Fixes: c50ff694 ("refactor(native): separate init from implementation")

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
2020-09-10 18:50:49 +09:00
Sebastiaan van Stijn
dc92ad6520
Replace errors.Cause() with errors.Is()
Dependencies may be switching to use the new `%w` formatting
option to wrap errors; switching to use `errors.Is()` makes
sure that we are still able to unwrap the error and detect the
underlying cause.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-05-08 14:36:45 +02:00
Derek McGowan
123af61c0b
Add Cleanup to snapshot API
Cleanup is an optional method a snapshotter may implement.
Cleanup can be used to cleanup resources after a snapshot
has been removed. This function allows a snapshotter to defer
longer resource cleanup until after snapshot removals are
completed. Adding this to the API allows proxy snapshotters
to leverage this enhancement.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2020-01-07 14:59:20 -08:00
ktock
493a36de95 Move label filter to snapshots package
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2019-12-25 12:13:08 +09:00
Derek McGowan
526c0db693
Support target snapshot references on prepare
Allows backend snapshots to bring existing snapshots into
a namespace without requiring clients to fully snapshots
when the target reference is known. Backend snapshots must
explicitly implement this functionality, it is equivalent
to sharing across namespaces and is up to the backend to
use the label when it is given or ignore it.

This enables remote snapshot functionality for a backend to
query for a target snapshot before a client has performed
any work to create that snapshot.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-11-25 10:07:25 -08:00
Derek McGowan
d1261b5087
Update snapshot parent filter property to always exist
The parent property may be empty but always exists.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-11-25 10:07:25 -08:00
Phil Estes
76aa4e546e
Use common identifiers package with less restrictive regex
Regular expressions in the namespaces package are redundant with the
pre-existing validator in the "identifiers" package; replace this custom
usage with the validator in "identifiers"

Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
2019-11-04 21:22:01 -05:00
Derek McGowan
66aa1d3ef6
Add snapshot walk implementations
Temporarily remove zfs and aufs until interface update

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-10-24 11:11:22 -07:00
bpopovschi
e8c14c07c6
Added filters to snapshots API
Signed-off-by: bpopovschi <zyqsempai@mail.ru>
2019-10-24 11:11:22 -07:00
yuxiaobo
62c2eea78d fix-up typo
Signed-off-by: yuxiaobo <yuxiaobogo@163.com>
2019-09-26 13:02:03 +08:00
yuxiaobo
5ea9363624 Perfect documentation
Signed-off-by: yuxiaobo <yuxiaobogo@163.com>
2019-09-25 16:59:31 +08:00
Derek McGowan
0b224ac7d6
Update metadata interfaces for containers and leases
Add more thorough dirty checking across all types which
may be deleted and hold references.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-09-23 15:27:39 -07:00
Guangming Wang
6d0e355c6d
cleanup code typos in leases.go
Signed-off-by: Guangming Wang <guangming.wang@daocloud.io>
2019-08-17 00:27:08 +08:00
Maksym Pavlenko
98ead46d1a Call Update of snapshotter plugin
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2019-07-26 15:56:07 -07:00
Derek McGowan
dd0a45dfe0
Add flat GC label for leases
Provide a flag which configures a lease to only hold
reference to its given references and ignore label references
during garbage collection rooted from the lease.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-07-18 11:16:46 -07:00
Derek McGowan
63ceaf877d
Add testing log context
Enables showing debug logs in testing output.
For integration tests the client log output will show
in addition to daemon output, with timestamps for better
correlation.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-07-16 10:31:43 -07:00
Edgar Lee
7d21172453 Fix metadata content store to call writer digest after commit
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2019-07-08 07:26:49 -07:00
Michael Crosby
844c58102e Add cgroup delete opt for ns deletion
Closes #3305

This adds an opt and a `--cgroup,-c` flag to `ctr namespaces rm` to
remove the cgroup that is commonly created with runtimes.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-07-01 15:01:02 -04:00
Eric Hotinger
75f183887a Allow opts to flow to the backend snapshotter during snapshot creation.
Signed-off-by: Eric Hotinger <ehotinger@gmail.com>
2019-06-19 11:00:22 -07:00
Wei Fu
8a388d6238 leases: support resource management
Add three methods for lease service so that the client can use it to
manage the resource by lease, not just gc.root label. With the following
methods, it is easy for client to maintain their own cache system.

```
 - AddResource(context.Context, Lease, Resource) error
 - RemoveResource(context.Context, Lease, Resource) error
 - ListResources(context.Context, Lease) ([]Resource, error)
```

And the resource is to be

```golang
type Resource {
  ID   string
  Type string
}
```

For the snapshots, the Type field will be formatted by
snapshots/%{type}, like snapshots/overlayfs.

fix: #3295

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2019-05-29 11:00:32 +08:00
Derek McGowan
41b7b0e0dc
Ensure namespaced writer commit has correct namespace
The namespaced writer Commit method must always have a
namespace in the context as checked by the removeIngestLease
function, resulting in a panic when not provided.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-05-21 11:08:55 -07:00
Davor Kapsa
cfc36388b3 Remove redundant error checks
Signed-off-by: Davor Kapsa <davor.kapsa@gmail.com>
2019-04-30 21:28:51 +02:00
Derek McGowan
68c44f8cc8
Merge pull request #3122 from stefanberger/descriptor_annotations.pr
Extend adaptImage function with annotations case
2019-03-22 14:28:15 -07:00
Stefan Berger
5d50b9c2bb Extend adaptImage function with annotations case
Extend the adaptImage function with a case for handling the annotations
so they can be used in the filter adaptors for fieldpaths.

Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
2019-03-22 14:56:26 -04:00
Michael Crosby
9b882c44f8
Merge pull request #3000 from stefanberger/descriptor_annotations.pr
Add missing annotations map to Descriptor for gRPC transfer
2019-03-22 14:05:44 -04:00
Stefan Berger
09cf2a629b Extend metadata images test with fieldpaths for Annotations
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
2019-03-07 14:20:56 -05:00
Stefan Berger
5124f9ee54 Write the Annotations map into the bolt db
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
2019-03-07 13:55:58 -05:00
Stefan Berger
02cc1485df Prepare boltutil for reading and writing another map
Refactor the code so that another function can also read and write maps
into the bolt db.

Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
2019-03-07 13:55:22 -05:00
Derek McGowan
7cfb99ab9d
Add content gc ref labels from containers, images, and snapshots
Currently the objects which can retain content from labels
are limited. This limitation has required clients to work
around this and and in some cases add outside reference
counting (e.g. buildkit keeping content for snapshots).
Updated the logic to treat content and snapshot labels equally
and simplified the code in the process.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-03-04 14:51:07 -08:00
Wei Fu
4b3e0a8aaa metadata: merge snapshot labels with metadata's labels
fix the no-op issue.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2019-02-18 16:11:12 +08:00