Commit Graph

3069 Commits

Author SHA1 Message Date
Michael Crosby
434f0e679f Compile shim without cgo for size and mem usage
This compiles the shim without cgo and statically which ends up to have
lower memory usage and binary size.  buildmode=pie cannot be used
without cgo.

ref: https://github.com/golang/go/issues/17789#issuecomment-258542220

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-11-17 17:32:49 -05:00
Akihiro Suda
7ef4aa5c25 shim: support non-default binary name
The binary name used for executing "containerd publish" was hard-coded
in the shim code, and hence it did not work with customized daemon
binary name. (e.g. `docker-containerd`)

This commit allows specifying custom daemon binary via `containerd-shim
-containerd-binary ...`.
The daemon invokes this command with `os.Executable()` path.

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-11-17 22:21:54 +00:00
Akihiro Suda
2fad7822e3
Merge pull request #1782 from dnephin/move-io-pkg
Move io.go into io package
2017-11-18 07:20:15 +09:00
Daniel Nephin
298dabc6c2 Move io.go into cio package
Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2017-11-17 17:04:45 -05:00
Kenfe-Mickaël Laventure
bc063f263c
Merge pull request #1778 from stevvooe/weak-imports-on-plugins
api: use weak imports for plugin protobufs
2017-11-17 08:48:16 -08:00
Phil Estes
2edc475818
Merge pull request #1776 from stevvooe/decouple-events-grpc
events: decouple events package from grpc
2017-11-17 11:44:46 -05:00
Michael Crosby
a4968710fd
Merge pull request #1777 from jessvalarezo/migrate-client-code
client: migrate client code to containerd
2017-11-17 09:32:34 -05:00
Phil Estes
77cb722e1c
Merge pull request #1775 from dmcgowan/fix-push-different-tags
Update docker pusher check tag
2017-11-17 08:21:08 -05:00
Stephen J Day
50532f231a
api: use weak imports for plugin protobufs
Because of a side-effect import, we have the possibility of pulling in
several unnecessary packages that are used by the plugin and not at
runtime to implement protobuf structures. Setting these imports to
`weak` prevents this from happening, reducing the total import set,
reducing memory usage and binary size.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-11-16 20:18:15 -08:00
Stephen Day
9cb91f8bfa
Merge pull request #1765 from dmcgowan/rm-lease-resources
Add support for removing leases on resource removal
2017-11-16 18:53:53 -08:00
Stephen J Day
2d05c4aa1a
events: decouple events package from grpc
By defining a concrete, non-protobuf type for the events interface, we
can completely decouple it from the grpc packages that are expensive at
runtime. This does requires some allocation cost for converting between
types, but the saving for the size of the shim are worth it.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-11-16 18:49:46 -08:00
Jess Valarezo
8c5dede74e client: move content store client to containerd package
Signed-off-by: Jess Valarezo <valarezo.jessica@gmail.com>
2017-11-16 18:14:21 -08:00
Jess Valarezo
aefaeeacb0 client: move diff client to containerd package
Signed-off-by: Jess Valarezo <valarezo.jessica@gmail.com>
2017-11-16 18:06:14 -08:00
Jess Valarezo
8d23e5ef9c client: move image store client to containerd package
Signed-off-by: Jess Valarezo <valarezo.jessica@gmail.com>
2017-11-16 17:47:54 -08:00
Jess Valarezo
6cd9962c7f client: move namespaces client to containerd package
Signed-off-by: Jess Valarezo <valarezo.jessica@gmail.com>
2017-11-16 17:47:38 -08:00
Derek McGowan
56a35d5cb9
Update docker pusher check tag
Currently pushing a new tag to a manifest which already
exists in the registry skips the tag push because it
only checks that the manifest exists. This updates the
logic to instead check if the tag exists and is at the
same digest.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-11-16 17:29:29 -08:00
Derek McGowan
5c6e789dde
Merge pull request #1769 from stevvooe/split-events-types
api/events: split event types from events service
2017-11-16 16:29:28 -08:00
Jess Valarezo
5b64f7030d client: move snapshot client to containerd package
Signed-off-by: Jess Valarezo <valarezo.jessica@gmail.com>
2017-11-16 15:45:07 -08:00
Stephen J Day
09b5ca1072
api/events: split event types from events service
To avoid importing all of grpc when consuming events, the types of
events have been split in to a separate package. This should allow a
reduction in memory usage in cases where a package is consuming events
but not using the gprc service directly.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-11-16 15:20:46 -08:00
Stephen Day
fc149f0ef9
Merge pull request #1770 from crosbymichael/resume
content, remotes: download resumption support
2017-11-16 13:57:19 -08:00
Michael Crosby
4192f99cf0 Only compare times on non-windows
There is a bug in the windows CI that causes a time difference between
the host and the container.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-11-16 16:25:18 -05:00
Stephen J Day
3e5e2ecc0a content/local: consistent handling of data and locks
The locks now retry on the backend side to prevent clients from having
to round trip on locks that might be momentarily held. This exposed some
timing errors in the updated_at fields for content ingest, so we've had
to move that to a separate file to export the monotonic go runtime
timestamps.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-11-16 16:13:19 -05:00
Stephen J Day
f4fdb940ed vendor: include require package from testify
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-11-16 16:13:13 -05:00
Stephen J Day
682151b166 remotes/docker: implement seekable http requests
To support resumable download, the fetcher for a remote must implement
`io.Seeker`. If implemented the `content.Copy` function will detect the
seeker and begin from where the download was terminated by a previous
attempt.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-11-16 16:13:06 -05:00
Stephen J Day
a9308e174d content/local: ensure that resumption is properly supported
While early PoCs had download resumption working, we didn't have tests
and had not verified the behavior. With this test suite, we now are able
to show that download resumption is properly supported in the content
store. In particular, there was a bug where resuming a download would
not issue the writes to the correct offset in the file. A Seek was added
to ensure we are writing from the current ingest offset.

In this investigation, it was also discovered that using the OS/Disk
created time on files was skewed from the monotonic clock in Go's
runtime. The startedat values are now taken from the Go runtime and
written to a separate file.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-11-16 16:12:59 -05:00
Stephen J Day
368dc17a4c testutil: add Size to DumpDir output
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-11-16 16:12:51 -05:00
Daniel Nephin
4df7075a74 Remove integration-parallel target
Always run in parallel. The old target can be emulated with

TESTFLAGS_PARALLEL=1 make integration

Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2017-11-16 11:15:33 -05:00
Daniel Nephin
05ec01e827 Fix .gitignore for coverage files
Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2017-11-16 11:14:44 -05:00
Phil Estes
7d64713dce
Merge pull request #1768 from stevvooe/consolidate-empty
protobuf: use the gogo/types package for empty
2017-11-16 10:31:17 -05:00
Stephen J Day
c5022ad92d
protobuf: use the gogo/types package for empty
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-11-15 19:08:54 -08:00
Kenfe-Mickaël Laventure
cc7e5934c8
Merge pull request #1760 from dnephin/run-unit-tests-on-windows
Re-enable unit tests on appveyor
2017-11-15 18:53:16 -08:00
Stephen Day
f49b1b7179
Merge pull request #1764 from crosbymichael/exit
Add publish subcommand for publishing events
2017-11-15 16:46:25 -08:00
Derek McGowan
96d2c1e0c2
Merge pull request #1766 from dnephin/fix-error-message
Normalize 'already exists' errors
2017-11-15 14:53:20 -08:00
Daniel Nephin
2e7f7318cc Normalize 'already exists' errors
and fix some error messages where they were wrong or redundant

Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2017-11-15 16:46:27 -05:00
Jess Valarezo
f3a63f52dc
Add support for removing leases on resource removal
Signed-off-by: Jessica Valarezo <valarezo.jessica@gmail.com>
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-11-15 12:09:59 -08:00
Daniel Nephin
a72279e53d Skip some tests on windows where the implementation is missing
Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2017-11-15 14:52:00 -05:00
Daniel Nephin
fe5bb4a0fc Update platform tests to use the defaultOS
fix tests on windows

Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2017-11-15 14:31:50 -05:00
Daniel Nephin
666d946455 Implement diff.compareSysStat on windows
and update diff tests to work with slash paths on windows

Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2017-11-15 14:31:50 -05:00
Daniel Nephin
5025b53704 Re-enable unit tests on appveyor
Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2017-11-15 14:31:48 -05:00
Michael Crosby
a522a6c7ee Add publish subcommand for publishing events
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-11-15 11:30:03 -05:00
Phil Estes
a542cf7dac
Merge pull request #1761 from stevvooe/update-protoc
README, travis: update protobuf compiler version
2017-11-15 09:59:51 -05:00
Stephen Day
19168f98fe
Merge pull request #1754 from AkihiroSuda/fix-snapshot-commit-spec
snapshot: fix specification of Commit
2017-11-14 17:55:38 -08:00
Stephen Day
571c002ef6
Merge pull request #1755 from AkihiroSuda/fix-ctr-snapshot-args
ctr: fix args
2017-11-14 17:55:14 -08:00
Stephen Day
343d2d3c5d
Merge pull request #1758 from dnephin/speed-up-unit-test
Reduce number of blobs in TestWalkBlobs
2017-11-14 17:52:46 -08:00
Stephen J Day
7354a7f001
README, travis: update protobuf compiler version
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-11-14 17:46:50 -08:00
Akihiro Suda
5eff92d1ba ctr: fix args
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-11-15 01:34:12 +00:00
Akihiro Suda
5e8218a63b snapshot: fix specification of Commit
The default implementations remove the key after Commit.
However, the specification had required implementations not to remove
the key.

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-11-15 01:32:19 +00:00
Daniel Nephin
d8cf30e42e Reduce number of blobs in TestWalkBlobs
Reduce runtime from 170s to 3s

Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2017-11-14 17:51:44 -05:00
Stephen Day
c78c156feb
Merge pull request #1728 from AkihiroSuda/snapshot-closable
snapshot: add Close()
2017-11-14 14:41:50 -08:00
Stephen Day
d15cee0358
Merge pull request #1737 from estesp/progress-line-fixup
Handle progress line length properly for formatting
2017-11-14 14:40:32 -08:00