Add doc about remote snapshotter
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
This commit is contained in:
ktock
parent
bf672cccee
commit
c2081369c5
129
docs/remote-snapshotter.md
Normal file
129
docs/remote-snapshotter.md
Normal file
@ -0,0 +1,129 @@
|
||||
# Remote Snapshotter
|
||||
|
||||
Containerd allows snapshotters to reuse snapshots existing somewhere managed by them.
|
||||
|
||||
_Remote Snapshotter_ is a snapshotter that leverages this functionality and reuses snapshots that are stored in a remotely shared place.
|
||||
These remotely shared snapshots are called _remote snapshots_.
|
||||
Remote snapshotter allows containerd to prepare these remote snapshots without pulling layers from registries, which hopefully shorten the time to take for image pull.
|
||||
|
||||
One of the remote snapshotter implementations is [Stargz Snapshotter](https://github.com/containerd/stargz-snapshotter).
|
||||
This enables containerd to lazily pull images from standard-compliant registries leveraging remote snapshotter functionality and stargz images by google/crfs.
|
||||
|
||||
## The containerd client API
|
||||
|
||||
The containerd client's `Pull` API with unpacking-mode allows the underlying snapshotter to query for remote snapshots before fetching content.
|
||||
Remote snapshotter needs to be plugged into containerd in [the same ways as normal snapshotters](/PLUGINS.md).
|
||||
|
||||
```go
|
||||
image, err := client.Pull(ctx, ref,
|
||||
containerd.WithPullUnpack,
|
||||
containerd.WithPullSnapshotter("my-remote-snapshotter"),
|
||||
)
|
||||
```
|
||||
|
||||
## Passing snapshotter-specific information
|
||||
|
||||
Some remote snapshotters requires snapshotter-specific information through `Pull` API.
|
||||
The information will be used in various ways including searching snapshot contents from a remote store.
|
||||
One of the example snapshotters that requires snapshotter-specific information is stargz snapshotter.
|
||||
It requires the image reference name and layer digests, etc. for searching layer contents from registries.
|
||||
|
||||
Snapshotters receive the information through user-defined labels prefixed by `containerd.io/snapshot/`.
|
||||
The containerd client supports two ways to pass these labels to the underlying snapshotter.
|
||||
|
||||
### Using snapshotter's `WithLabels` option
|
||||
|
||||
User-defined labels can be passed down to the underlying snapshotter using snapshotter option `WithLabels`.
|
||||
Specified labels will be passed down every time the containerd client queries a remote snapshot.
|
||||
This is useful if the values of these labels are determined statically regardless of the snapshots.
|
||||
These user-defined labels must be prefixed by `containerd.io/snapshot/`.
|
||||
|
||||
```go
|
||||
import "github.com/containerd/containerd/snapshots"
|
||||
|
||||
image, err := client.Pull(ctx, ref,
|
||||
containerd.WithPullUnpack,
|
||||
containerd.WithPullSnapshotter(
|
||||
"my-remote-snapshotter",
|
||||
snapshots.WithLabels(map[string]string{
|
||||
"containerd.io/snapshot/reference": ref,
|
||||
}),
|
||||
),
|
||||
)
|
||||
```
|
||||
|
||||
### Using the containerd client's `WithImageHandlerWrapper` option
|
||||
|
||||
User-defined labels can also be passed using an image handler wrapper.
|
||||
This is useful when labels vary depending on the snapshot.
|
||||
|
||||
Every time the containerd client queries remote snapshot, it passes `Annotations` appended to the targeting layer descriptor (means the layer descriptor that will be pulled and unpacked for preparing that snapshot) to the underlying snapshotter.
|
||||
These annotations are passed to the snapshotter as user-defined labels.
|
||||
The values of annotations can be dynamically added and modified in the handler wrapper.
|
||||
Note that annotations must be prefixed by `containerd.io/snapshot/`.
|
||||
[CRI plugin](https://github.com/containerd/cri/blob/09d6426f33cac217528158ddc6d254ca7d597a7b/pkg/server/image_pull.go#L127) and [stargz snapshotter](https://github.com/containerd/stargz-snapshotter/blob/875ec333403a885f5b6e5b64c94ec4dc713e0596/cmd/ctr-remote/commands/rpull.go#L97) leverage this method.
|
||||
|
||||
```go
|
||||
import "github.com/ktock/snapshotter/handler"
|
||||
|
||||
if _, err := client.Pull(ctx, ref,
|
||||
containerd.WithPullUnpack,
|
||||
containerd.WithPullSnapshotter("my-remote-snapshotter"),
|
||||
containerd.WithImageHandlerWrapper(handler.Wrapper(ref)),
|
||||
)
|
||||
```
|
||||
|
||||
## Snapshotter APIs for querying remote snapshots
|
||||
|
||||
The containerd client queries remote snapshots to the underlying remote snapshotter using snapshotter APIs.
|
||||
This section describes the high-level overview of how snapshotter APIs are used for remote snapshots functionality, with some piece of pseudo-codes that describe the simplified logic implemented in the containerd client.
|
||||
For more details, see [`unpacker.go`](/unpacker.go) that implements this logic.
|
||||
|
||||
During image pull, the containerd client calls `Prepare` API with the label `containerd.io/snapshot.ref`.
|
||||
This is a containerd-defined label which contains ChainID that targets a committed snapshot that the client is trying to prepare.
|
||||
At this moment, user-defined labels (prefixed by `containerd.io/snapshot/`) will also be merged into the labels option.
|
||||
|
||||
```go
|
||||
// Gets annotations appended to the targetting layer which would contain
|
||||
// snapshotter-specific information passed by the user.
|
||||
labels := snapshots.FilterInheritedLabels(desc.Annotations)
|
||||
if labels == nil {
|
||||
labels = make(map[string]string)
|
||||
}
|
||||
|
||||
// Specifies ChainID of the targeting committed snapshot.
|
||||
labels["containerd.io/snapshot.ref"] = chainID
|
||||
|
||||
// Merges snapshotter options specified by the user which would contain
|
||||
// snapshotter-specific information passed by the user.
|
||||
opts := append(rCtx.SnapshotterOpts, snapshots.WithLabels(labels))
|
||||
|
||||
// Calls `Prepare` API with target indentifier and snapshotter-specific
|
||||
// information.
|
||||
mounts, err = sn.Prepare(ctx, key, parent.String(), opts...)
|
||||
```
|
||||
|
||||
If this snapshotter is a remote snapshotter, that committed snapshot hopefully exists, for example, in a shared remote store.
|
||||
Remote snapshotter must define and enforce policies about whether it will use an existing snapshot.
|
||||
When remote snapshotter allows the user to use that snapshot, it must return `ErrAlreadyExists`.
|
||||
|
||||
If the containerd client gets `ErrAlreadyExists` by `Prepare`, it ensures the existence of that committed snapshot by calling `Stat` with the ChainID.
|
||||
If this snapshot is available, the containerd client skips pulling and unpacking layer that would otherwise be needed for preparing and committing that snapshot.
|
||||
|
||||
```go
|
||||
mounts, err = sn.Prepare(ctx, key, parent.String(), opts...)
|
||||
if err != nil {
|
||||
if errdefs.IsAlreadyExists(err) {
|
||||
// Ensures the layer existence
|
||||
if _, err := sn.Stat(ctx, chainID); err != nil {
|
||||
// Handling error
|
||||
} else {
|
||||
// snapshot found with ChainID
|
||||
// pulling/unpacking will be skipped
|
||||
continue
|
||||
}
|
||||
} else {
|
||||
return err
|
||||
}
|
||||
}
|
||||
```
|
||||
Loading…
Reference in New Issue
Block a user