Commit Graph

73 Commits

Author SHA1 Message Date
Wei Fu
6b7e237fc7 chore: use go fix to cleanup old +build buildtag
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2022-12-29 14:25:14 +08:00
Sebastiaan van Stijn
717aa5929d
Remove uses of deprecated go-digest.NewDigestFromHex, go-digest.Digest.Hex
Both of these were deprecated in 55f675811a,
but the format of the GoDoc comments didn't follow the correct format, which
caused them not being picked up by tools as "deprecated".

This patch updates uses in the codebase to use the alternatives.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-08 14:47:05 +01:00
Derek McGowan
bc01f8fc05
Add reader option to local content reader at
Allows optimized copying from a local content file into another file.

Signed-off-by: Derek McGowan <derek@mcg.dev>
2022-09-20 20:50:04 -07:00
Kazuyoshi Kato
9d286baedf crypto/sha256 must be imported
Otherwise, go-digest panics (panic: sha256 not available).

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2022-08-17 21:43:47 +00:00
Maksym Pavlenko
ca3b9b50fe Run gofmt 1.19
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-08-04 18:18:33 -07:00
Kazuyoshi Kato
e9e33f847d Copy fuzzers from github.com/cncf/cncf-fuzzing
This commit copies the fuzzers from the repository except for
containerd_import_structured_fuzzer.go.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2022-07-06 15:21:48 +00:00
Maksym Pavlenko
871b6b6a9f Use testify
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-04-01 18:17:58 -07:00
Eng Zer Jun
18ec2761c0
test: use T.TempDir to create temporary test directory
The directory created by `T.TempDir` is automatically removed when the
test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-03-15 14:03:50 +08:00
Derek McGowan
48c7529de2
Fix incorrect error wrapped when closing ingest file
Signed-off-by: Derek McGowan <derek@mcg.dev>
2022-01-07 12:23:18 -08:00
haoyun
bbe46b8c43 feat: replace github.com/pkg/errors to errors
Signed-off-by: haoyun <yun.hao@daocloud.io>
Co-authored-by: zounengren <zouyee1989@gmail.com>
2022-01-07 10:27:03 +08:00
Tobias Klauser
df21828d27
content/local: use syscall.Timespec.Unix
Use the syscall method instead of repeating the type conversions for
the syscall.Stat_t Atim/Atimespec members. This also allows to drop the
//nolint: unconvert comments.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2022-01-03 12:37:25 +01:00
haoyun
c0d07094be feat: Errorf usage
Signed-off-by: haoyun <yun.hao@daocloud.io>
2021-12-13 14:31:53 +08:00
Shiming Zhang
9dfbcbbc48 content/local: Close the file if Seek fails
Signed-off-by: Shiming Zhang <wzshiming@foxmail.com>
2021-11-23 16:18:13 +08:00
Eng Zer Jun
50da673592
refactor: move from io/ioutil to io and os package
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-09-21 09:50:38 +08:00
Akihiro Suda
d3aa7ee9f0
Run go fmt with Go 1.17
The new `go fmt` adds `//go:build` lines (https://golang.org/doc/go1.17#tools).

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-08-22 09:31:50 +09:00
Kazuyoshi Kato
a3d6edc0ed content: return the error with its timestamp
The current timestamp is probably included in production logs, but
won't be in testing environments such as GitHub Actions.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2021-08-09 16:52:40 -07:00
Sebastiaan van Stijn
63c185da9a
content/local: inline sys.StatATimeAsTime()
The sys.StatATimeAsTime() utility is currently only used in a single place,
but because it's living in the "sys" package, also brings in other dependencies,
such as Microsoft/hcsshim.

This patch inlines the code from sys.StatATimeAsTime(), to remove that dependency.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-22 10:14:29 +02:00
Phil Estes
edfd8d5993
Change Wrapf of non-error to an actual error
Signed-off-by: Phil Estes <estesp@amazon.com>
2021-06-02 14:48:49 -04:00
Shiming Zhang
9fcea1d3f0 Rename atomicWrite to writeToCompletion
Signed-off-by: Shiming Zhang <wzshiming@foxmail.com>
2021-05-27 18:07:45 +08:00
Shiming Zhang
9e576b8891 Optimize backoff
Signed-off-by: Shiming Zhang <wzshiming@foxmail.com>
2021-04-15 23:25:09 +08:00
Maksym Pavlenko
571cc32df6 Add OpenReader to create ReaderAt interface
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2021-03-08 12:04:58 -08:00
Akihiro Suda
20346607b9
Merge pull request #4701 from kzys/content-store-filter 2021-01-25 22:35:45 +09:00
Muhammad Kaisar Arkhan
380b52652c Bring OpenBSD support
Signed-off-by: Muhammad Kaisar Arkhan <hi@yukiisbo.red>
2021-01-04 14:43:48 +01:00
Kazuyoshi Kato
c95d71cf60 content: include the staleness of the lock when tryLock() fails
When multiple clients are pulling the same image, we may have this
lock error. Short-lived locks are probably fine, but long-lived locks
may indicate that containerd has some issues.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2020-12-02 14:08:02 -08:00
Kazuyoshi Kato
e74ace9ad8 content: support filters on local.store#Walk()
While Walk() has been taking filter strings, it was not using the parameter.
This change actually makes the filtering work.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2020-11-11 11:17:48 -08:00
Leonardo Taccari
4c47fe0a2f Add support for NetBSD
Signed-off-by: Leonardo Taccari <leot@NetBSD.org>
2020-09-22 20:03:50 +02:00
Stephen Day
4615111ba2
content/local: validate digest before calculating path
Signed-off-by: Stephen Day <stephen.day@getcruise.com>
2020-04-09 17:37:47 -07:00
Sebastiaan van Stijn
f2edc6f164
vendor: update gotest.tools v3.0.2
full diff: https://github.com/gotestyourself/gotest.tools/compare/v2.3.0...v3.0.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-28 17:47:20 +01:00
Michael Crosby
cf7fb14efa Update timestamp atomic write
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-06-05 19:34:55 +00:00
Tonis Tiigi
5b9bd993a8 differ: fix deadlock on commit error
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-04-25 17:50:17 -07:00
fesu
7045801fd4 delete the ingest when any errors happen during resuming status
Signed-off-by: Su Fei  <fesu@ebay.com>
2019-04-06 11:21:18 +08:00
Wei Fu
132ee9b826 fix: linter issue
megacheck, gosimple and unused has been deprecated and subsumed by
staticcheck. And staticcheck also has been upgraded. we need to update
code for the linter issue.

close: #2945

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2019-01-23 22:54:51 +08:00
Derek McGowan
0f756495a9
Fix writer deadlock in local store
The local store could end up in a state where the writer is
closed but the reference is locked after a commit on an
existing object.
Cleans up Commit logic to always close the writer even after
an error occurs, guaranteeing the reference is unlocked after commit.
Adds a test to the content test suite to verify this behavior.
Updates the content store interface definitions to clarify the behavior.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-10-15 16:33:45 -07:00
Derek McGowan
6875d3df3a
Always check exists on commit error
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-09-14 01:26:03 -07:00
Derek McGowan
c0cb2f2568
Add testcase for commit already exist
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-09-13 17:38:28 -07:00
Michael Wan
92243ff72a bugfix: updatedAt timestamp file may be empty
Signed-off-by: Michael Wan <zirenwan@gmail.com>
2018-08-29 21:49:40 -04:00
Derek McGowan
dfc9991135
Add content ingests to lease and gc
Allow content ingests to be cleaned up during gc.
Use a default expiration on content ingests or make
use of the lease expiration when provided.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-07-25 16:54:14 -07:00
Derek McGowan
1c6929cbd4
Remove use of crypto rand in tests
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-07-12 17:51:55 -07:00
Vincent Demeester
832b05ae67
Update tests to use gotest.tools angel
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-06-08 21:02:01 +02:00
Akihiro Suda
d88de4a34f content: change Writer/ReaderAt to take OCI
This change allows implementations to resolve the location of the actual data
using OCI descriptor fields such as MediaType.

No OCI descriptor field is written to the store.

No change on gRPC API.

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-06-01 11:51:43 +09:00
Michael Crosby
009ba4d797 Move testutils to pkg
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2018-05-22 17:08:38 -04:00
Kunal Kushwaha
b12c3215a0 Licence header added
Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
2018-02-19 10:32:26 +09:00
Daniel Nephin
ef48a0268e Migrate to gotestyourself/assert
Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2018-02-12 12:26:26 -05:00
Stephen J Day
3295bca5bf
content/testsuite: pass context to hold lease
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2018-01-29 16:29:51 -08:00
Daniel Nephin
184bc25629 Add unconvert linter
This linter checks for unnecessary type convertions.

Some convertions are whitelisted because their type is different
on 32bit platforms

Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2018-01-09 17:36:44 -05:00
Phil Estes
00bc24fcea
Merge pull request #1917 from dnephin/less-verbose-tests
Less verbose test suite
2017-12-15 12:23:49 -05:00
Daniel Nephin
49fffe8ec7 Less verbose tests
Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2017-12-14 11:00:40 -05:00
Daniel Nephin
9184908075 Fix image pull after a failure
When resuming from a failed pull writer.Truncate() was not
seeking to the proper position in the file. This caused writes to
happen after the previously written content, instead of at the start
of the file.

Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2017-12-13 18:24:12 -05:00
Kir Kolyshkin
acc6f4ec77 MkdirAll: fix usage
(below is a quote from my runc commit 6f82d4b)

TL;DR: check for IsExist(err) after a failed MkdirAll() is both
redundant and wrong -- so two reasons to remove it.

Quoting MkdirAll documentation:

> MkdirAll creates a directory named path, along with any necessary
> parents, and returns nil, or else returns an error. If path
> is already a directory, MkdirAll does nothing and returns nil.

This means two things:

1. If a directory to be created already exists, no error is
returned.

2. If the error returned is IsExist (EEXIST), it means there exists
a non-directory with the same name as MkdirAll need to use for
directory. Example: we want to MkdirAll("a/b"), but file "a"
(or "a/b") already exists, so MkdirAll fails.

The above is a theory, based on quoted documentation and my UNIX
knowledge.

3. In practice, though, current MkdirAll implementation [1] returns
ENOTDIR in most of cases described in #2, with the exception when
there is a race between MkdirAll and someone else creating the
last component of MkdirAll argument as a file. In this very case
MkdirAll() will indeed return EEXIST.

Because of #1, IsExist check after MkdirAll is not needed.

Because of #2 and #3, ignoring IsExist error is just plain wrong,
as directory we require is not created. It's cleaner to report
the error now.

Note this error is all over the tree, I guess due to copy-paste,
or trying to follow the same usage pattern as for Mkdir(),
or some not quite correct examples on the Internet.

[1] https://github.com/golang/go/blob/f9ed2f75/src/os/path.go

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2017-11-30 10:18:08 -08:00
Daniel Nephin
ee04cfa3f9 Add staticcheck linter
Fix issues with sync.Pool being passed an array and not a pointer.
See https://github.com/dominikh/go-tools/blob/master/cmd/staticcheck/docs/checks/SA6002

Add missing tests for content.Copy

Fix T.Fatal being called in a goroutine

Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2017-11-28 13:05:30 -05:00