Repository navigation
Proxy the OCI referrers API in the container handler - #427
Open
pinguinfuss wants to merge 13 commits into
Open
pinguinfuss wants to merge 13 commits into
pinguinfuss wants to merge 13 commits into
Conversation
GET /v2/{name}/referrers/{digest} used to fall through to a plain 404,
so clients only ever saw referrers stored under the sha256-<hex> tag
schema. Registries that keep SBOMs and signatures behind the referrers
API looked empty through the proxy.
The index is now fetched from the upstream that the repository routes
to and cached like the tag list: fresh within the metadata TTL,
revalidated with If-None-Match, and served stale when the upstream is
down. artifactType is not forwarded, so one cached row covers every
filter and clients filter the full list themselves, which they do when
OCI-Filters-Applied is absent. An upstream 404 is passed through
untouched, so clients still fall back to the tag schema for registries
like GHCR. When the upstream cannot answer and nothing is cached, the
proxy keeps answering 404 as before rather than failing the request.
Closes git-pkgs#144
The second page now answers with an absolute Link pointing back at the upstream host, and the test checks that it comes back rewritten to the proxy. That was missing so far, and it also gives the upstream variable a reason to be declared before the handler closure, which staticcheck complained about.
loadContainerReferrers was a line-for-line copy of loadContainerTags with the types renamed, which dupl flags in CI. The part that opens the cache row and reads the body now lives in loadContainerMetadata next to storeContainerMetadata, and the referrers loader only maps the row onto its struct. The tag and manifest loaders are left as they are.
For foo/manifests/referrers/<digest> the manifest handler and the referrers handler send the very same upstream path, so the test passed even with the referrers case moved in front of the manifest case. The fake upstream now rejects requests that only accept an image index, which is what the referrers handler sends.
A repository can be reached as upstream/<name>/<repo> and, when the default registry is the same host, as plain <repo>. Both end up in the same cache row, so the pagination Link has to be rewritten when the row is served, not when it is stored. The link test now reads the row through both paths; with a store-time rewrite the second path would get the first one's link.
Without If-None-Match the fake upstream just sends the full index again, so the old assertions held either way. The test now counts the 304s the upstream sent and expects exactly one.
The cosign mention was too broad. cosign reaches the referrers API for its newer signature bundles, while the old .sig tags keep going through the normal manifest path. Also rewraps the paragraph, one line had run past the others.
The ETag test now backdates the cached row instead of running with a zero TTL. After the upstream answers 304, a third request has to come from the cache. Before, the test would also have passed if the 304 branch forgot to bump fetchedAt and every request kept revalidating.
None of the fake upstreams left out Content-Type, so the default to the image index type was never exercised. oras-go rejects anything else, so it is worth pinning, both for the first fetch and for the cached copy.
The invalid-request table only had digests that were too short or used an unknown algorithm, so it would not notice if the digest pattern lost its anchors. Two cases now put junk before and after an otherwise valid sha256 digest.
The referrers handler had its own key function, but it did exactly what containerManifestCacheKey already does with four strings. Referrers rows sit in their own oci-referrers ecosystem anyway, so they can't collide with manifests.
A cached referrers response carries the same fields as a cached tag list, and the freshness check was a straight copy too. So the referrers code just uses cachedContainerTags and containerTagsFresh now. Loading, storing and writing stay on their own, since the ecosystem and the Link handling are different.
Now that containerd ns support is on main, referrers requests have to go through registryForRequest too, otherwise a _default mirror would look every subject up on Docker Hub. ns only picks the registry, so it is dropped from the forwarded query and the cache key, and the next-page Link keeps it, the same way tag lists do. This also follows the new rewriteContainerTagsLink signature, which takes the namespace.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
/v2/{name}/referrers/{digest}currently falls through to a plain 404, so clients only ever see the oldsha256-<hex>tags and miss SBOMs and signatures that registries keep behind the real API.This proxies the endpoint to the routed upstream (including
ns) and caches the index like tag lists: fresh for the metadata TTL, revalidated after that, stale if the upstream is down.A few choices worth a look:
artifactTypeisn't forwarded. One cached row serves every filter, and withoutOCI-Filters-Appliedthe client filters itself, which the spec allows.OCI-Subjectis left out, it only matters for pushes.Existing routes are unchanged. Tests are in
internal/handler/container_referrers_test.go.Closes #144.