Commit Graph

22 Commits

Author SHA1 Message Date
Wei Fu
6b7e237fc7 chore: use go fix to cleanup old +build buildtag
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2022-12-29 14:25:14 +08:00
Jeff Widman
050cd58ce6 Drop deprecated ioutil
`ioutil` has been deprecated by golang. All the code in `ioutil` just
forwards functionality to code in either the `io` or `os` packages.

See https://github.com/golang/go/pull/51961 for more info.

Signed-off-by: Jeff Widman <jeff@jeffwidman.com>
2022-07-23 08:36:20 -07:00
Kazuyoshi Kato
e9e33f847d Copy fuzzers from github.com/cncf/cncf-fuzzing
This commit copies the fuzzers from the repository except for
containerd_import_structured_fuzzer.go.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2022-07-06 15:21:48 +00:00
Kazuyoshi Kato
a42688639c Restore decompression benchmarks
The benchmarks were deleted in #2640 but we could use that to evaluate
zstd further.

Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>
2022-05-18 18:25:11 +00:00
Maksym Pavlenko
2d59a39445 Use t.Setenv instead of os.Setenv
Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-04-01 13:53:17 -07:00
Eng Zer Jun
18ec2761c0
test: use T.TempDir to create temporary test directory
The directory created by `T.TempDir` is automatically removed when the
test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-03-15 14:03:50 +08:00
Derek McGowan
cc5a4c8297
Merge pull request #6177 from dkkb/feature/zstd_with_skippable_frame
compression: support zstd with skippable frame
2021-11-01 10:29:05 -07:00
Da McGrady
6fa9f22fa3
compression: support zstd with skippable frame
As a matter of fact, there are two frame formats defined by Zstandard: Zstandard frames and Skippable frames.
So we should probably support zstd algorithms with skippable frames.
See https://tools.ietf.org/id/draft-kucherawy-dispatch-zstd-00.html#rfc.section.2 for more details.

Signed-off-by: Da McGrady <dabkb@aol.com>
2021-11-01 11:36:46 +08:00
Eng Zer Jun
50da673592
refactor: move from io/ioutil to io and os package
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-09-21 09:50:38 +08:00
Sebastiaan van Stijn
2ac9968401
replace uses of os/exec with golang.org/x/sys/execabs
Go 1.15.7 contained a security fix for CVE-2021-3115, which allowed arbitrary
code to be executed at build time when using cgo on Windows. This issue also
affects Unix users who have “.” listed explicitly in their PATH and are running
“go get” outside of a module or with module mode disabled.

This issue is not limited to the go command itself, and can also affect binaries
that use `os.Command`, `os.LookPath`, etc.

From the related blogpost (ttps://blog.golang.org/path-security):

> Are your own programs affected?
>
> If you use exec.LookPath or exec.Command in your own programs, you only need to
> be concerned if you (or your users) run your program in a directory with untrusted
> contents. If so, then a subprocess could be started using an executable from dot
> instead of from a system directory. (Again, using an executable from dot happens
> always on Windows and only with uncommon PATH settings on Unix.)
>
> If you are concerned, then we’ve published the more restricted variant of os/exec
> as golang.org/x/sys/execabs. You can use it in your program by simply replacing

This patch replaces all uses of `os/exec` with `golang.org/x/sys/execabs`. While
some uses of `os/exec` should not be problematic (e.g. part of tests), it is
probably good to be consistent, in case code gets moved around.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-25 18:11:09 +02:00
Giuseppe Scrivano
30802fac73
compression: add support for the zstd algorithm
zstd is a compression algorithm that has a very fast decoder, while
providing also good compression ratios.  The fast decoder makes it
suitable for container images, as decompressing the tarballs is a very
expensive operation.

https://github.com/opencontainers/image-spec/pull/788 added support
for zstd to the OCI image specs.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2020-12-07 09:56:30 +01:00
dzzg
612628c2f9 fix wrong spells in compression.go
Signed-off-by: zhengguang zhu <zhengguang.zhu@daocloud.io>
2019-07-29 12:47:17 +00:00
Derek McGowan
db358a9fd2
Fix panic when bufio Reader called in 2 goroutines
A panic was seen related to the buffer being reset in
one goroutine while being read in another. In the case
of pigz an early cancellation will cause the reader to
close, resetting the buffer and signaling the process
to shut down, but races since the process must stop
reading before the reset otherwise the a panic may occur.
This fix guarantees that the bufio is always reset and
returned to the pool on the same goroutine that is
doing the read. If a buffer is not fully read the
buffered reader should just be discarded and not
returned back to the pool.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-09-26 18:11:30 -07:00
Maksym Pavlenko
e8fac24e7b
Remove decompression benchmark, use sync.Once for initialization
Benchmark gist: https://git.io/fASMy

Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2018-09-18 17:09:12 -07:00
Maksym Pavlenko
003b27eff4
Move compression benchmark test data to gist
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2018-09-17 13:41:04 -07:00
Maksym Pavlenko
4d7d63f390
Improve layer decompression speed by using pigz
Signed-off-by: Maksym Pavlenko <makpav@amazon.com>
2018-09-17 13:41:04 -07:00
Phil Estes
c55b9636f7
Merge pull request #2263 from alibaba/fix_no_gzip
fixbug: blob for schemav1 could be uncompressed
2018-07-30 18:49:28 -04:00
Derek McGowan
1c6929cbd4
Remove use of crypto rand in tests
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-07-12 17:51:55 -07:00
frank yang
046536cfb1 fixbug: blob for schemav1 could be uncompressed
Signed-off-by: frank yang <yyb196@gmail.com>
2018-06-08 11:50:54 +08:00
Kunal Kushwaha
b12c3215a0 Licence header added
Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
2018-02-19 10:32:26 +09:00
unclejack
7b4d238ee3 archive,cmd,images: clean up some code
Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com>
2017-03-30 11:41:22 +03:00
Akihiro Suda
6089c1525b new package: compression (ported from docker/pkg/archive)
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-03-16 05:29:27 +00:00