130 lines
6.1 KiB
Markdown
130 lines
6.1 KiB
Markdown
# Remote Snapshotter
|
|
|
|
Containerd allows snapshotters to reuse snapshots existing somewhere managed by them.
|
|
|
|
_Remote Snapshotter_ is a snapshotter that leverages this functionality and reuses snapshots that are stored in a remotely shared place.
|
|
These remotely shared snapshots are called _remote snapshots_.
|
|
Remote snapshotter allows containerd to prepare these remote snapshots without pulling layers from registries, which hopefully shorten the time to take for image pull.
|
|
|
|
One of the remote snapshotter implementations is [Stargz Snapshotter](https://github.com/containerd/stargz-snapshotter).
|
|
This enables containerd to lazily pull images from standard-compliant registries leveraging remote snapshotter functionality and stargz images by google/crfs.
|
|
|
|
## The containerd client API
|
|
|
|
The containerd client's `Pull` API with unpacking-mode allows the underlying snapshotter to query for remote snapshots before fetching content.
|
|
Remote snapshotter needs to be plugged into containerd in [the same ways as normal snapshotters](/docs/PLUGINS.md).
|
|
|
|
```go
|
|
image, err := client.Pull(ctx, ref,
|
|
containerd.WithPullUnpack,
|
|
containerd.WithPullSnapshotter("my-remote-snapshotter"),
|
|
)
|
|
```
|
|
|
|
## Passing snapshotter-specific information
|
|
|
|
Some remote snapshotters requires snapshotter-specific information through `Pull` API.
|
|
The information will be used in various ways including searching snapshot contents from a remote store.
|
|
One of the example snapshotters that requires snapshotter-specific information is stargz snapshotter.
|
|
It requires the image reference name and layer digests, etc. for searching layer contents from registries.
|
|
|
|
Snapshotters receive the information through user-defined labels prefixed by `containerd.io/snapshot/`.
|
|
The containerd client supports two ways to pass these labels to the underlying snapshotter.
|
|
|
|
### Using snapshotter's `WithLabels` option
|
|
|
|
User-defined labels can be passed down to the underlying snapshotter using snapshotter option `WithLabels`.
|
|
Specified labels will be passed down every time the containerd client queries a remote snapshot.
|
|
This is useful if the values of these labels are determined statically regardless of the snapshots.
|
|
These user-defined labels must be prefixed by `containerd.io/snapshot/`.
|
|
|
|
```go
|
|
import "github.com/containerd/containerd/snapshots"
|
|
|
|
image, err := client.Pull(ctx, ref,
|
|
containerd.WithPullUnpack,
|
|
containerd.WithPullSnapshotter(
|
|
"my-remote-snapshotter",
|
|
snapshots.WithLabels(map[string]string{
|
|
"containerd.io/snapshot/reference": ref,
|
|
}),
|
|
),
|
|
)
|
|
```
|
|
|
|
### Using the containerd client's `WithImageHandlerWrapper` option
|
|
|
|
User-defined labels can also be passed using an image handler wrapper.
|
|
This is useful when labels vary depending on the snapshot.
|
|
|
|
Every time the containerd client queries remote snapshot, it passes `Annotations` appended to the targeting layer descriptor (means the layer descriptor that will be pulled and unpacked for preparing that snapshot) to the underlying snapshotter.
|
|
These annotations are passed to the snapshotter as user-defined labels.
|
|
The values of annotations can be dynamically added and modified in the handler wrapper.
|
|
Note that annotations must be prefixed by `containerd.io/snapshot/`.
|
|
[CRI plugin](https://github.com/containerd/cri/blob/09d6426f33cac217528158ddc6d254ca7d597a7b/pkg/server/image_pull.go#L127) and [stargz snapshotter](https://github.com/containerd/stargz-snapshotter/blob/875ec333403a885f5b6e5b64c94ec4dc713e0596/cmd/ctr-remote/commands/rpull.go#L97) leverage this method.
|
|
|
|
```go
|
|
import "github.com/ktock/snapshotter/handler"
|
|
|
|
if _, err := client.Pull(ctx, ref,
|
|
containerd.WithPullUnpack,
|
|
containerd.WithPullSnapshotter("my-remote-snapshotter"),
|
|
containerd.WithImageHandlerWrapper(handler.Wrapper(ref)),
|
|
)
|
|
```
|
|
|
|
## Snapshotter APIs for querying remote snapshots
|
|
|
|
The containerd client queries remote snapshots to the underlying remote snapshotter using snapshotter APIs.
|
|
This section describes the high-level overview of how snapshotter APIs are used for remote snapshots functionality, with some piece of pseudo-codes that describe the simplified logic implemented in the containerd client.
|
|
For more details, see [`unpacker.go`](/unpacker.go) that implements this logic.
|
|
|
|
During image pull, the containerd client calls `Prepare` API with the label `containerd.io/snapshot.ref`.
|
|
This is a containerd-defined label which contains ChainID that targets a committed snapshot that the client is trying to prepare.
|
|
At this moment, user-defined labels (prefixed by `containerd.io/snapshot/`) will also be merged into the labels option.
|
|
|
|
```go
|
|
// Gets annotations appended to the targeting layer which would contain
|
|
// snapshotter-specific information passed by the user.
|
|
labels := snapshots.FilterInheritedLabels(desc.Annotations)
|
|
if labels == nil {
|
|
labels = make(map[string]string)
|
|
}
|
|
|
|
// Specifies ChainID of the targeting committed snapshot.
|
|
labels["containerd.io/snapshot.ref"] = chainID
|
|
|
|
// Merges snapshotter options specified by the user which would contain
|
|
// snapshotter-specific information passed by the user.
|
|
opts := append(rCtx.SnapshotterOpts, snapshots.WithLabels(labels))
|
|
|
|
// Calls `Prepare` API with target identifier and snapshotter-specific
|
|
// information.
|
|
mounts, err = sn.Prepare(ctx, key, parent.String(), opts...)
|
|
```
|
|
|
|
If this snapshotter is a remote snapshotter, that committed snapshot hopefully exists, for example, in a shared remote store.
|
|
Remote snapshotter must define and enforce policies about whether it will use an existing snapshot.
|
|
When remote snapshotter allows the user to use that snapshot, it must return `ErrAlreadyExists`.
|
|
|
|
If the containerd client gets `ErrAlreadyExists` by `Prepare`, it ensures the existence of that committed snapshot by calling `Stat` with the ChainID.
|
|
If this snapshot is available, the containerd client skips pulling and unpacking layer that would otherwise be needed for preparing and committing that snapshot.
|
|
|
|
```go
|
|
mounts, err = sn.Prepare(ctx, key, parent.String(), opts...)
|
|
if err != nil {
|
|
if errdefs.IsAlreadyExists(err) {
|
|
// Ensures the layer existence
|
|
if _, err := sn.Stat(ctx, chainID); err != nil {
|
|
// Handling error
|
|
} else {
|
|
// snapshot found with ChainID
|
|
// pulling/unpacking will be skipped
|
|
continue
|
|
}
|
|
} else {
|
|
return err
|
|
}
|
|
}
|
|
```
|