Commit Graph

86 Commits

Author SHA1 Message Date
Michael Crosby
d7af92e00c Move Mount into mount pkg
This moves both the Mount type and mountinfo into a single mount
package.

This also opens up the root of the repo to hold the containerd client
implementation.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-05-22 16:41:12 -07:00
Stephen J Day
cb68150a2f
protos: remove need for customname on ID fields
Credit to Aaron Lehmann for the implementation from SwarmKit.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-05-18 18:50:59 -07:00
Michael Crosby
3695ba77bb Merge pull request #849 from dmcgowan/snapshot-service
Add snapshot and diff service
2017-05-16 10:35:12 -07:00
Derek McGowan
098ff94b24
Add snapshot and diff service
Remove rootfs service in place of snapshot service. Adds
diff service for extracting and creating diffs. Diff
creation is not yet implemented. This service allows
pulling or creating images without needing root access to
mount. Additionally in the future this will allow containerd
to ensure extractions happen safely in a chroot if needed.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-05-15 16:50:16 -07:00
Stephen J Day
320dd70935
fs: remove fsutils package
No need to have a util package under a package... er, package.

We should move most of this functionality to continuity.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-05-15 16:33:58 -07:00
Derek McGowan
7027e8862a
overlay: call rollback on commit error
Rollback was not being called when the function was called with
a key which does not exist. This failure to call rollback kept
the database open and caused all new requests to block.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-05-11 15:55:47 -07:00
Stephen J Day
193abed96e
content: unify provider and ingester
The split between provider and ingester was a long standing division
reflecting the client-side use cases. For the most part, we were
differentiating these for the algorithms that operate them, but it made
instantation and use of the types challenging. On the server-side, this
distinction is generally less important. This change unifies these types
and in the process we get a few benefits.

The first is that we now completely access the content store over GRPC.
This was the initial intent and we have now satisfied this goal
completely. There are a few issues around listing content and getting
status, but we resolve these with simple streaming and regexp filters.
More can probably be done to polish this but the result is clean.

Several other content-oriented methods were polished in the process of
unification. We have now properly seperated out the `Abort` method to
cancel ongoing or stalled ingest processes. We have also replaced the
`Active` method with a single status method.

The transition went extremely smoothly. Once the clients were updated to
use the new methods, every thing worked as expected on the first
compile.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-05-10 17:05:53 -07:00
Phil Estes
a1173ce712 Merge pull request #815 from AkihiroSuda/dtype
overlay: abort if backing fs lacks support for d_type
2017-05-10 14:24:00 -05:00
Stephen Day
d553fab079 Merge pull request #822 from AkihiroSuda/fix-vet
fix `make vet` failures, and enable `make vet` on CI
2017-05-10 11:42:24 -07:00
Michael Crosby
07fda4b73a Merge pull request #813 from AkihiroSuda/btrfs-zero-config
btrfs: auto-detect device
2017-05-10 09:53:18 -07:00
Akihiro Suda
8cd218237b mountinfo: refactored
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-05-10 05:10:22 +00:00
Akihiro Suda
837205005f fix make vet failures, and enable make vet on CI
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-05-10 04:36:41 +00:00
Akihiro Suda
825c2ce3cc overlay: abort if backing fs lacks support for d_type
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-05-08 06:54:31 +00:00
Phil Estes
91410874e7
Add pause/unpause implementation
This adds pause and unpause to containerd's execution service and the
same commands to the `ctr` client.

Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
2017-05-06 16:05:37 -04:00
Akihiro Suda
d5707d3ac7 btrfs: auto-detect device
No longer need to set `plugins.snapshot-btrfs.device` manually

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-05-06 17:29:37 +00:00
Stephen J Day
1eb515c038
vendor: move to use containerd/btrfs
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-05-03 18:53:57 -07:00
Michael Crosby
317aeea0fd Merge pull request #731 from stevvooe/snapshot-usage
snapshot: add Usage method to Snapshotter
2017-04-28 14:36:34 -07:00
Stephen J Day
66c504d1bb
snapshot: add Usage method to Snapshotter
To allow the querying of usage for snapshots, we define a new method on
the snapshotter to query the resources in use by a single snapshot.
Conversely, it can be said that if the snapshot was deleted, the
reported amount of usage would be recovered.

There are few problems with this model in the implementation of btrfs
that need to be worked out. In btrfs, it is hard to resolve the amount
of data usage with the use of quotas but these may report valuables that
are incompatible with the model.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-04-26 17:13:52 -07:00
Vincent Batts
7d018f2306
snapshot: the examples need context
the api for these functions have since added `context.Context`

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2017-04-25 12:18:03 -04:00
Kenfe-Mickael Laventure
c5843b7615 Initial windows runtime work
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-04-07 09:20:44 -07:00
Phil Estes
e5c8c5634a Merge pull request #696 from crosbymichael/exec
execution: add exec for additional processes
2017-04-06 13:53:46 -04:00
Derek McGowan
63f01733b0
Update snapshot mkdir to check for directory exist
When starting up a snapshot driver on subsequent runs, the
mkdir call will return an exist error, this can be safely
ignored.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-04-05 17:04:44 -07:00
Derek McGowan
8a73673b5b
Update naive driver to use storage package
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-04-05 15:35:34 -07:00
Michael Crosby
7715ddcefa Add exec APIs
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-04-05 11:51:56 -07:00
Qiang Huang
a0dda70b2d Use MkdirAll when create snapshots directory
Otherwise we'll get:
```
mkdir /var/lib/containerd/snapshot/overlay/snapshots: file exists
```
when we already have snapshots there.

Signed-off-by: Qiang Huang <h.huangqiang@huawei.com>
2017-04-05 00:52:46 -04:00
Qiang Huang
6132bec05a Add Kill API for shim service
Signed-off-by: Qiang Huang <h.huangqiang@huawei.com>
2017-04-05 08:26:35 +08:00
Derek McGowan
1c2f5fe203
Update documentation to storage package
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-04-04 14:58:30 -07:00
Derek McGowan
9ffbfccdaf
Refactor overlay and btrfs to pass lint
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-04-04 14:58:24 -07:00
Derek McGowan
b319ba7c5a
Move boltdb implementation to storage package
Removes storage interface and replaces with storage functions.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-04-04 14:58:18 -07:00
Michael Crosby
4f2b443a27 Rewrite imports for new github org
This rewrites the Go imports after switching to the new github org.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-04-03 14:05:44 -07:00
Michael Crosby
71e8d765df set +x on overlay fs dirs
We need to set +x on the overlay dirs or after dropping from root to a
non-root user an eperm will happen on exec or other file access

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-03-24 12:55:35 -07:00
Derek McGowan
6997c5c32d Update btrfs undo logic
Ensure undo failures are logged in a consistent way.
Remove created subvolume on commit failure on create.
Ensure that removed value get logged on delete failure.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-03-22 10:58:45 -07:00
Derek McGowan
55ef482c5a Add storage testsuite
Added error variables and error check methods to snapshots

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-03-21 23:16:39 -07:00
Derek McGowan
8383519d96
Move parent bucket to same level as snapshot bucket
Adds a zero byte separator on composite key.
Stores child key in back reference to allow following.
Add parent bucket to get/create bucket functions.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-03-21 23:15:36 -07:00
Derek McGowan
397499c288
Add root flag to boltdb tests
Needed for CI to run root coverage

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-03-21 23:15:36 -07:00
Derek McGowan
73f467a32b
Use enumerated kind in boltdb proto message
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-03-21 23:15:36 -07:00
Derek McGowan
912746b016
Update btrfs driver to use snapshot storage
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-03-21 23:15:36 -07:00
Derek McGowan
61b524aff2
Update overlay snapshot driver to use metastore
Update tests to use boltdb.
Update test suite to pass context.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-03-21 23:15:35 -07:00
Derek McGowan
63ea9908c0
Add bolt db metastore implementation
Add metastore benchmark suite to test metastore performance.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-03-21 23:15:35 -07:00
Derek McGowan
17e7e70fe2
Add snapshot metadata store interface
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-03-21 23:15:35 -07:00
Kunal Kushwaha
8f13b160b1 Testcase for multiple Prepare/View on same key.
Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
2017-03-08 11:16:12 +09:00
Michael Crosby
27a99400e8 Merge pull request #610 from dmcgowan/snapshot-move-testsuite
snapshot: move testsuite to sub-package
2017-03-07 16:45:04 -08:00
Derek McGowan
a4247e2aa9 Add snapshot plugin type
Update existing snapshot drivers to register as plugins.
Load snapshot driver at containerd startup.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-03-07 14:55:36 -08:00
Derek McGowan
66bbf5c449 Move snapshot testsuite to sub package
Prevents imports of snapshot to require importing test packages.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-03-07 09:16:45 -08:00
Akihiro Suda
750cdc951f snapshotter: add more assertion
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-03-06 08:34:43 +00:00
Derek McGowan
2831e9b469 Merge pull request #581 from kunalkushwaha/testsuite-transitive
Testcase to test Transitivity of snapshot layers.
2017-03-02 11:38:49 -08:00
Kunal Kushwaha
8bbce4b80f Testcase for Transitivity of snapshot layers.
Given snapshots A <- B <- C,
B is the parent of C and A is a transitive parent of C

Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
2017-02-28 18:35:03 +09:00
Stephen Day
9743a1531d Merge pull request #575 from dmcgowan/snapshot-test-checks
Update basic snapshotter test
2017-02-27 20:09:22 -08:00
Kunal Kushwaha
d9dd68ef5d Testcases added for to test Stat() Interface.
Two test cases added
- Prepare, Stat on new layer, should return Active layer.
- Prepare & Commit , Stat on new layer, should return Committed layer

Signed-off-by: Kunal Kushwaha <kushwaha_kunal_v7@lab.ntt.co.jp>
2017-02-28 11:19:32 +09:00
Derek McGowan
b8855a8fa6 Update basic snapshotter test
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-27 16:49:02 -08:00