Commit Graph

3064 Commits

Author SHA1 Message Date
Phil Estes
2edc475818
Merge pull request #1776 from stevvooe/decouple-events-grpc
events: decouple events package from grpc
2017-11-17 11:44:46 -05:00
Michael Crosby
a4968710fd
Merge pull request #1777 from jessvalarezo/migrate-client-code
client: migrate client code to containerd
2017-11-17 09:32:34 -05:00
Phil Estes
77cb722e1c
Merge pull request #1775 from dmcgowan/fix-push-different-tags
Update docker pusher check tag
2017-11-17 08:21:08 -05:00
Stephen J Day
50532f231a
api: use weak imports for plugin protobufs
Because of a side-effect import, we have the possibility of pulling in
several unnecessary packages that are used by the plugin and not at
runtime to implement protobuf structures. Setting these imports to
`weak` prevents this from happening, reducing the total import set,
reducing memory usage and binary size.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-11-16 20:18:15 -08:00
Stephen Day
9cb91f8bfa
Merge pull request #1765 from dmcgowan/rm-lease-resources
Add support for removing leases on resource removal
2017-11-16 18:53:53 -08:00
Stephen J Day
2d05c4aa1a
events: decouple events package from grpc
By defining a concrete, non-protobuf type for the events interface, we
can completely decouple it from the grpc packages that are expensive at
runtime. This does requires some allocation cost for converting between
types, but the saving for the size of the shim are worth it.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-11-16 18:49:46 -08:00
Jess Valarezo
8c5dede74e client: move content store client to containerd package
Signed-off-by: Jess Valarezo <valarezo.jessica@gmail.com>
2017-11-16 18:14:21 -08:00
Jess Valarezo
aefaeeacb0 client: move diff client to containerd package
Signed-off-by: Jess Valarezo <valarezo.jessica@gmail.com>
2017-11-16 18:06:14 -08:00
Jess Valarezo
8d23e5ef9c client: move image store client to containerd package
Signed-off-by: Jess Valarezo <valarezo.jessica@gmail.com>
2017-11-16 17:47:54 -08:00
Jess Valarezo
6cd9962c7f client: move namespaces client to containerd package
Signed-off-by: Jess Valarezo <valarezo.jessica@gmail.com>
2017-11-16 17:47:38 -08:00
Derek McGowan
56a35d5cb9
Update docker pusher check tag
Currently pushing a new tag to a manifest which already
exists in the registry skips the tag push because it
only checks that the manifest exists. This updates the
logic to instead check if the tag exists and is at the
same digest.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-11-16 17:29:29 -08:00
Derek McGowan
5c6e789dde
Merge pull request #1769 from stevvooe/split-events-types
api/events: split event types from events service
2017-11-16 16:29:28 -08:00
Jess Valarezo
5b64f7030d client: move snapshot client to containerd package
Signed-off-by: Jess Valarezo <valarezo.jessica@gmail.com>
2017-11-16 15:45:07 -08:00
Stephen J Day
09b5ca1072
api/events: split event types from events service
To avoid importing all of grpc when consuming events, the types of
events have been split in to a separate package. This should allow a
reduction in memory usage in cases where a package is consuming events
but not using the gprc service directly.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-11-16 15:20:46 -08:00
Stephen Day
fc149f0ef9
Merge pull request #1770 from crosbymichael/resume
content, remotes: download resumption support
2017-11-16 13:57:19 -08:00
Michael Crosby
4192f99cf0 Only compare times on non-windows
There is a bug in the windows CI that causes a time difference between
the host and the container.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-11-16 16:25:18 -05:00
Stephen J Day
3e5e2ecc0a content/local: consistent handling of data and locks
The locks now retry on the backend side to prevent clients from having
to round trip on locks that might be momentarily held. This exposed some
timing errors in the updated_at fields for content ingest, so we've had
to move that to a separate file to export the monotonic go runtime
timestamps.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-11-16 16:13:19 -05:00
Stephen J Day
f4fdb940ed vendor: include require package from testify
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-11-16 16:13:13 -05:00
Stephen J Day
682151b166 remotes/docker: implement seekable http requests
To support resumable download, the fetcher for a remote must implement
`io.Seeker`. If implemented the `content.Copy` function will detect the
seeker and begin from where the download was terminated by a previous
attempt.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-11-16 16:13:06 -05:00
Stephen J Day
a9308e174d content/local: ensure that resumption is properly supported
While early PoCs had download resumption working, we didn't have tests
and had not verified the behavior. With this test suite, we now are able
to show that download resumption is properly supported in the content
store. In particular, there was a bug where resuming a download would
not issue the writes to the correct offset in the file. A Seek was added
to ensure we are writing from the current ingest offset.

In this investigation, it was also discovered that using the OS/Disk
created time on files was skewed from the monotonic clock in Go's
runtime. The startedat values are now taken from the Go runtime and
written to a separate file.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-11-16 16:12:59 -05:00
Stephen J Day
368dc17a4c testutil: add Size to DumpDir output
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-11-16 16:12:51 -05:00
Daniel Nephin
4df7075a74 Remove integration-parallel target
Always run in parallel. The old target can be emulated with

TESTFLAGS_PARALLEL=1 make integration

Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2017-11-16 11:15:33 -05:00
Daniel Nephin
05ec01e827 Fix .gitignore for coverage files
Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2017-11-16 11:14:44 -05:00
Phil Estes
7d64713dce
Merge pull request #1768 from stevvooe/consolidate-empty
protobuf: use the gogo/types package for empty
2017-11-16 10:31:17 -05:00
Stephen J Day
c5022ad92d
protobuf: use the gogo/types package for empty
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-11-15 19:08:54 -08:00
Kenfe-Mickaël Laventure
cc7e5934c8
Merge pull request #1760 from dnephin/run-unit-tests-on-windows
Re-enable unit tests on appveyor
2017-11-15 18:53:16 -08:00
Stephen Day
f49b1b7179
Merge pull request #1764 from crosbymichael/exit
Add publish subcommand for publishing events
2017-11-15 16:46:25 -08:00
Derek McGowan
96d2c1e0c2
Merge pull request #1766 from dnephin/fix-error-message
Normalize 'already exists' errors
2017-11-15 14:53:20 -08:00
Daniel Nephin
2e7f7318cc Normalize 'already exists' errors
and fix some error messages where they were wrong or redundant

Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2017-11-15 16:46:27 -05:00
Jess Valarezo
f3a63f52dc
Add support for removing leases on resource removal
Signed-off-by: Jessica Valarezo <valarezo.jessica@gmail.com>
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-11-15 12:09:59 -08:00
Daniel Nephin
a72279e53d Skip some tests on windows where the implementation is missing
Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2017-11-15 14:52:00 -05:00
Daniel Nephin
fe5bb4a0fc Update platform tests to use the defaultOS
fix tests on windows

Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2017-11-15 14:31:50 -05:00
Daniel Nephin
666d946455 Implement diff.compareSysStat on windows
and update diff tests to work with slash paths on windows

Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2017-11-15 14:31:50 -05:00
Daniel Nephin
5025b53704 Re-enable unit tests on appveyor
Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2017-11-15 14:31:48 -05:00
Michael Crosby
a522a6c7ee Add publish subcommand for publishing events
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-11-15 11:30:03 -05:00
Phil Estes
a542cf7dac
Merge pull request #1761 from stevvooe/update-protoc
README, travis: update protobuf compiler version
2017-11-15 09:59:51 -05:00
Stephen Day
19168f98fe
Merge pull request #1754 from AkihiroSuda/fix-snapshot-commit-spec
snapshot: fix specification of Commit
2017-11-14 17:55:38 -08:00
Stephen Day
571c002ef6
Merge pull request #1755 from AkihiroSuda/fix-ctr-snapshot-args
ctr: fix args
2017-11-14 17:55:14 -08:00
Stephen Day
343d2d3c5d
Merge pull request #1758 from dnephin/speed-up-unit-test
Reduce number of blobs in TestWalkBlobs
2017-11-14 17:52:46 -08:00
Stephen J Day
7354a7f001
README, travis: update protobuf compiler version
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-11-14 17:46:50 -08:00
Akihiro Suda
5eff92d1ba ctr: fix args
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-11-15 01:34:12 +00:00
Akihiro Suda
5e8218a63b snapshot: fix specification of Commit
The default implementations remove the key after Commit.
However, the specification had required implementations not to remove
the key.

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-11-15 01:32:19 +00:00
Daniel Nephin
d8cf30e42e Reduce number of blobs in TestWalkBlobs
Reduce runtime from 170s to 3s

Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2017-11-14 17:51:44 -05:00
Stephen Day
c78c156feb
Merge pull request #1728 from AkihiroSuda/snapshot-closable
snapshot: add Close()
2017-11-14 14:41:50 -08:00
Stephen Day
d15cee0358
Merge pull request #1737 from estesp/progress-line-fixup
Handle progress line length properly for formatting
2017-11-14 14:40:32 -08:00
Phil Estes
f435ec0733
Merge pull request #1757 from dnephin/fix-coverage
Fix coverage report
2017-11-14 13:33:11 -05:00
Daniel Nephin
b28e1eee42 Fix coverage report
Previously the coverage.txt file was never populated

Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2017-11-14 12:47:48 -05:00
Phil Estes
37ee054e61
Merge pull request #1739 from crosbymichael/shim-redux
Move shim process code into subpackage
2017-11-14 10:20:28 -05:00
Akihiro Suda
4feb6f228a snapshot: add Close()
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-11-14 02:00:33 +00:00
Stephen Day
f9933e9f96
Merge pull request #1742 from jessvalarezo/ctr-task-kill
ctr: update task kill to take exec-id
2017-11-13 13:53:26 -08:00