Commit Graph

21 Commits

Author SHA1 Message Date
Derek McGowan
0b224ac7d6
Update metadata interfaces for containers and leases
Add more thorough dirty checking across all types which
may be deleted and hold references.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-09-23 15:27:39 -07:00
Eric Lin
4247f2684d metadata: define content sharing policy
This changeset modifies the metadata store to allow one to set a
"content sharing policy" that defines how blobs are shared between
namespaces in the content store.

The default mode "shared" will make blobs available in all namespaces
once it is pulled into any namespace.  The blob will be pulled into
the namespace if a writer is opened with the "Expected" digest that
is already present in the backend.

The alternative mode, "isolated" requires that clients prove they have
access to the content by providing all of the content to the ingest
before the blob is added to the namespace.

Both modes share backing data, while "shared" will reduce total
bandwidth across namespaces, at the cost of allowing access to any
blob just by knowing its digest.

Note: Most functional codes and changelog of this commit originate from
Stephen J Day <stephen.day@docker.com>, see
40455aade8

Fixes #1713 Fixes #2865

Signed-off-by: Eric Lin <linxiulei@gmail.com>
2018-12-21 15:02:21 +08:00
John Howard
2586f3fbb9 boltdb/bolt --> go.etcd.io/bbolt
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-09-12 15:23:57 -07:00
Derek McGowan
dfc9991135
Add content ingests to lease and gc
Allow content ingests to be cleaned up during gc.
Use a default expiration on content ingests or make
use of the lease expiration when provided.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-07-25 16:54:14 -07:00
Derek McGowan
29b72d4ff0
Support lease filters
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-07-18 10:43:37 -07:00
sudeesh john
5a4f007e48 Fix the formatting directives error during compilation
Signed-off-by: Sudeesh John <sudeesh@linux.vnet.ibm.com>
2018-07-05 20:40:21 +05:30
Akihiro Suda
d88de4a34f content: change Writer/ReaderAt to take OCI
This change allows implementations to resolve the location of the actual data
using OCI descriptor fields such as MediaType.

No OCI descriptor field is written to the store.

No change on gRPC API.

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-06-01 11:51:43 +09:00
Derek McGowan
43d0a5cb60
Pass in context to lease done function in client
Allows the client to choose the context to finish the lease.
This allows the client to switch contexts when the main context
used to the create the lease may have been cancelled.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-03-22 14:09:24 -07:00
Stephen Day
b3b95c0a2a
Merge pull request #2154 from dmcgowan/shared-content-ingests
content: shared content across namespaces
2018-03-12 16:11:32 -07:00
Derek McGowan
b3aeba7062
Allow test runners to wrap contexts
Let the test runners choose the namespaces and
wrap the contexts. This allows the test suite to create
multiple contexts without worrying about namespacing
or leasing in the contexts.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-02-22 14:45:10 -08:00
Kunal Kushwaha
b12c3215a0 Licence header added
Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
2018-02-19 10:32:26 +09:00
Stephen J Day
3295bca5bf
content/testsuite: pass context to hold lease
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2018-01-29 16:29:51 -08:00
Derek McGowan
0d8b093361
metadata: add content lease on existing content
When a writer is requested for an object that already
exists, add that object to the provided any lease to prevent
other operations from affecting the current lease's use of
that content.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-01-09 14:33:30 -08:00
Derek McGowan
7b08bcdb65
Add support for label storage in local content store
Allows running tests which require labels on the content store

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-10-31 12:59:52 -07:00
Derek McGowan
8d892a651b
Update metadata plugin initialization
Updates metadata plugin to require content and
snapshotter plugins be loaded and initializes with
those plugins, keeping the metadata database structure
static after initialization. Service plugins now only
require metadata plugin access snapshotter or content
stores through metadata, which was already required
behavior of the services.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-10-04 11:25:46 -07:00
Derek McGowan
2ab70f21ac
Add content and snapshot store references
Update database object to hold reference to the
data stores.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-10-04 11:16:22 -07:00
Derek McGowan
dee8dc2cda
Add support for content labels on commit
Add commit options which allow for setting labels on commit.
Prevents potential race between garbage collector reading labels
after commit and labels getting set.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-08-11 14:15:20 -07:00
Derek McGowan
c4387a159e
Add content test suite run to client
Fix bugs in content deletion and upload status

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-08-11 14:08:34 -07:00
Derek McGowan
8e1591bd8b
Call upload status tests
Fix metadata status not returning correct reference string

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-07-24 10:11:35 -07:00
Derek McGowan
bb3c9c5927
Rename content/fs to content/local
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-07-24 10:11:35 -07:00
Derek McGowan
938f3185bd
Add content test suite
Add content test suite with test for writer.
Update fs and metadata implementations to use test suite.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-07-24 10:11:34 -07:00