Jacob Blain Christen
48894d6f5e
Mirror repository rewrites (v1.1)
...
Support CRI configuration to allow for request-time rewrite rules
applicable only to the repository portion of resource paths when pulling
images. Because the rewrites are applied at request time, images
themselves will not be "rewritten" -- images as stored by CRI (and the
underlying containerd facility) will continue to present as normal.
As an example, if you use the following config for your containerd:
```toml
[plugins]
[plugins."io.containerd.grpc.v1.cri"]
[plugins."io.containerd.grpc.v1.cri".registry]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
endpoint = ["https://registry-1.docker.io/v2 "]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io".rewrite]
"^library/(.*)" = "my-org/$1"
```
And then subsequently invoke `crictl pull alpine:3.13` it will pull
content from `docker.io/my-org/alpine:3.13` but still show up as
`docker.io/library/alpine:3.13` in the `crictl images` listing.
This commit has been reworked from the original implementation. Rewites
are now done when resolving instead of when building the request, so
that auth token scopes stored in the context properly reflect the
rewritten repository path. For the original implementation, see
06c4ea9baec2b278b8172a789bf601168292f645.
Ref: https://github.com/k3s-io/k3s/issues/11191#issuecomment-2455525773
Signed-off-by: Jacob Blain Christen <jacob@rancher.com>
Co-authored-by: Brad Davidson <brad.davidson@rancher.com>
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
2025-03-20 21:34:00 +00:00
Sebastiaan van Stijn
700b90618d
resolver/docker: fix confusing "trying next host" log
...
The resolver uses some log messages to indicate the next host will be used,
however it would print this message even if no next host was available
to try.
This patch changes the log messages to indicate the action taken, which is
either "fetch failed" or if there's no other hosts to try, "trying next host".
While updating, also slightly updated the logs for consistency.
Before this patch:
DEBU[2024-10-23T20:49:29.640581097Z] resolving host=registry-1.docker.io spanID=6e2b9c871009cfd9 traceID=6c65b97879548fce4372faa8375d17e6
DEBU[2024-10-23T20:49:29.640632763Z] do request host=registry-1.docker.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent="docker/27.3.1 go/go1.22.7 git-commit/41ca978 kernel/6.10.11-linuxkit os/linux arch/arm64 containerd-client/1.7.22+unknown storage-driver/overlayfs UpstreamClient(Docker-Client/27.3.1 \\(linux\\))" request.method=HEAD spanID=6e2b9c871009cfd9 traceID=6c65b97879548fce4372faa8375d17e6 url="https://registry-1.docker.io/v2/library/nosuchimage/manifests/latest "
DEBU[2024-10-23T20:49:29.979613013Z] fetch response received host=registry-1.docker.io response.header.content-length=162 response.header.content-type=application/json response.header.date="Wed, 23 Oct 2024 20:49:29 GMT" response.header.docker-distribution-api-version=registry/2.0 response.header.docker-ratelimit-source=94.210.180.92 response.header.strict-transport-security="max-age=31536000" response.header.www-authenticate="Bearer realm=\"https://auth.docker.io/token \",service=\"registry.docker.io\",scope=\"repository:library/nosuchimage:pull\"" response.status="401 Unauthorized" spanID=6e2b9c871009cfd9 traceID=6c65b97879548fce4372faa8375d17e6 url="https://registry-1.docker.io/v2/library/nosuchimage/manifests/latest "
DEBU[2024-10-23T20:49:29.979907138Z] Unauthorized header="Bearer realm=\"https://auth.docker.io/token \",service=\"registry.docker.io\",scope=\"repository:library/nosuchimage:pull\"" host=registry-1.docker.io spanID=6e2b9c871009cfd9 traceID=6c65b97879548fce4372faa8375d17e6
DEBU[2024-10-23T20:49:29.980038430Z] do request host=registry-1.docker.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent="docker/27.3.1 go/go1.22.7 git-commit/41ca978 kernel/6.10.11-linuxkit os/linux arch/arm64 containerd-client/1.7.22+unknown storage-driver/overlayfs UpstreamClient(Docker-Client/27.3.1 \\(linux\\))" request.method=HEAD spanID=6e2b9c871009cfd9 traceID=6c65b97879548fce4372faa8375d17e6 url="https://registry-1.docker.io/v2/library/nosuchimage/manifests/latest "
DEBU[2024-10-23T20:49:30.466825972Z] fetch response received host=registry-1.docker.io response.header.content-length=162 response.header.content-type=application/json response.header.date="Wed, 23 Oct 2024 20:49:30 GMT" response.header.docker-distribution-api-version=registry/2.0 response.header.docker-ratelimit-source=94.210.180.92 response.header.strict-transport-security="max-age=31536000" response.header.www-authenticate="Bearer realm=\"https://auth.docker.io/token \",service=\"registry.docker.io\",scope=\"repository:library/nosuchimage:pull\",error=\"insufficient_scope\"" response.status="401 Unauthorized" spanID=6e2b9c871009cfd9 traceID=6c65b97879548fce4372faa8375d17e6 url="https://registry-1.docker.io/v2/library/nosuchimage/manifests/latest "
DEBU[2024-10-23T20:49:30.467056055Z] Unauthorized header="Bearer realm=\"https://auth.docker.io/token \",service=\"registry.docker.io\",scope=\"repository:library/nosuchimage:pull\",error=\"insufficient_scope\"" host=registry-1.docker.io spanID=6e2b9c871009cfd9 traceID=6c65b97879548fce4372faa8375d17e6
INFO[2024-10-23T20:49:30.467273305Z] trying next host error="pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed" host=registry-1.docker.io spanID=6e2b9c871009cfd9 traceID=6c65b97879548fce4372faa8375d17e6
With this patch:
DEBU[2024-10-23T21:17:53.487428843Z] resolving host=registry-1.docker.io spanID=4a74f71b08243447 traceID=fc1177ef212d33617c883842b9b9b8db
DEBU[2024-10-23T21:17:53.487498968Z] do request host=registry-1.docker.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent="docker/dev go/go1.22.8 git-commit/06c2ba1fa02626e242dc8dfe888f022bcd247c52 kernel/6.10.11-linuxkit os/linux arch/arm64 containerd-client/1.7.22+unknown storage-driver/overlayfs UpstreamClient(Docker-Client/27.3.1 \\(darwin\\))" request.method=HEAD spanID=4a74f71b08243447 traceID=fc1177ef212d33617c883842b9b9b8db url="https://registry-1.docker.io/v2/library/nosuchimage/manifests/latest "
DEBU[2024-10-23T21:17:53.832270052Z] fetch response received host=registry-1.docker.io response.header.content-length=162 response.header.content-type=application/json response.header.date="Wed, 23 Oct 2024 21:17:53 GMT" response.header.docker-distribution-api-version=registry/2.0 response.header.docker-ratelimit-source=94.210.180.92 response.header.strict-transport-security="max-age=31536000" response.header.www-authenticate="Bearer realm=\"https://auth.docker.io/token \",service=\"registry.docker.io\",scope=\"repository:library/nosuchimage:pull\"" response.status="401 Unauthorized" spanID=4a74f71b08243447 traceID=fc1177ef212d33617c883842b9b9b8db url="https://registry-1.docker.io/v2/library/nosuchimage/manifests/latest "
DEBU[2024-10-23T21:17:53.832843177Z] Unauthorized header="Bearer realm=\"https://auth.docker.io/token \",service=\"registry.docker.io\",scope=\"repository:library/nosuchimage:pull\"" host=registry-1.docker.io spanID=4a74f71b08243447 traceID=fc1177ef212d33617c883842b9b9b8db
DEBU[2024-10-23T21:17:53.833364760Z] do request host=registry-1.docker.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent="docker/dev go/go1.22.8 git-commit/06c2ba1fa02626e242dc8dfe888f022bcd247c52 kernel/6.10.11-linuxkit os/linux arch/arm64 containerd-client/1.7.22+unknown storage-driver/overlayfs UpstreamClient(Docker-Client/27.3.1 \\(darwin\\))" request.method=HEAD spanID=4a74f71b08243447 traceID=fc1177ef212d33617c883842b9b9b8db url="https://registry-1.docker.io/v2/library/nosuchimage/manifests/latest "
DEBU[2024-10-23T21:17:54.345483219Z] fetch response received host=registry-1.docker.io response.header.content-length=162 response.header.content-type=application/json response.header.date="Wed, 23 Oct 2024 21:17:54 GMT" response.header.docker-distribution-api-version=registry/2.0 response.header.docker-ratelimit-source=4203339e-74c0-11e4-bea4-0242ac11001b response.header.strict-transport-security="max-age=31536000" response.header.www-authenticate="Bearer realm=\"https://auth.docker.io/token \",service=\"registry.docker.io\",scope=\"repository:library/nosuchimage:pull\",error=\"insufficient_scope\"" response.status="401 Unauthorized" spanID=4a74f71b08243447 traceID=fc1177ef212d33617c883842b9b9b8db url="https://registry-1.docker.io/v2/library/nosuchimage/manifests/latest "
DEBU[2024-10-23T21:17:54.345601386Z] Unauthorized header="Bearer realm=\"https://auth.docker.io/token \",service=\"registry.docker.io\",scope=\"repository:library/nosuchimage:pull\",error=\"insufficient_scope\"" host=registry-1.docker.io spanID=4a74f71b08243447 traceID=fc1177ef212d33617c883842b9b9b8db
INFO[2024-10-23T21:17:54.345801761Z] fetch failed error="pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed" host=registry-1.docker.io spanID=4a74f71b08243447 traceID=fc1177ef212d33617c883842b9b9b8db
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-10-26 14:16:00 +02:00
