Skip to content

feat(core): add HubModelCatalogEntry, a typed view of a hub model's catalog metadata - #6530

Closed
evakravi wants to merge 1 commit into
aws:masterfrom
evakravi:feat/hub-model-catalog-entry
Closed

evakravi wants to merge 1 commit into
aws:masterfrom
evakravi:feat/hub-model-catalog-entry

Conversation

@evakravi

Copy link
Copy Markdown
Member

Problem

A hub model's task, license, provider, size, context window, languages, modalities, model type, capabilities, and recipes live in HubContentSearchKeywords as prefixed strings (@task:, @license:, @recipe:finetuning_sft_lora, ...) and in the HubContentDocument JSON. The SDK has no reader for them. jumpstart/search.py, common_utils/recipe_utils.py, and training_session/_utils.py each split the keywords for the one field they use, and the SageMaker agent skills repeat the full parse in four scripts. The hub writes at most 50 keywords per model, so a model with many languages loses its @model-size: and @context-window: keywords while the document still carries the values. No consumer handled that, so a size filter over the keywords silently excluded those models.

Solution

Add sagemaker.core.jumpstart.hub.catalog.HubModelCatalogEntry, a dataclass built from a HubContent resource, a DescribeHubContent response, a ListHubContents summary, or a Search record (from_hub_content, plus catalog_entries for a list). It reads every keyword prefix into a typed field, keeps the hub's spelling, and lets the document fill only what the keyword cap dropped (a keyword is authoritative). HubRecipe parses @recipe: keywords and RecipeCollection entries into type, technique, and strategy, so an entry answers supports_technique("sft") and fine_tuning_techniques() without a second lookup. customization_supported and bedrock_supported read the @capability: keywords. The training guide gains a usage section.

Tests

New sagemaker-core/tests/unit/jumpstart/hub/test_catalog.py, 9 tests: every keyword prefix from a list summary, recipe parsing with and without a strategy, malformed recipe keywords ignored, the document filling fields the cap dropped, a keyword winning over the document for a scalar field, a malformed document giving an empty entry, the name requirement, a HubContent resource with Unassigned() fields, and catalog_entries over a list.

cd sagemaker-core && python -m pytest tests/unit/jumpstart/hub/test_catalog.py -q
9 passed

The module is new, so origin/master has no behavior to compare. black --line-length 100, flake8, and pydocstyle pass on the new files.

Merge Checklist

  • I read the contribution guide.
  • The change is backward compatible.
  • The commits use the repository commit-message format.
  • The change creates no S3 or STS client.
  • The change has regression coverage and adds no dependency.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

…atalog metadata

A hub model's task, license, provider, size, context window, languages,
modalities, model type, capabilities, and recipes live in
`HubContentSearchKeywords` as prefixed strings (`@task:`, `@license:`,
`@recipe:finetuning_sft_lora`, ...) and in the `HubContentDocument` JSON. The
SDK had no reader for them. `jumpstart/search.py`, `recipe_utils.py`, and
`training_session/_utils.py` each split the keywords for the one field they
needed, and the SageMaker agent skills repeat the full parse in four scripts.
The hub writes at most 50 keywords per model, so a model with many languages
loses its `@model-size:` and `@context-window:` keywords while the document
still carries the values. No consumer handled that.

Add `sagemaker.core.jumpstart.hub.catalog.HubModelCatalogEntry`, built from a
`HubContent` resource, a `DescribeHubContent` response, a `ListHubContents`
summary, or a `Search` record. A keyword is authoritative and the document
fills what the cap dropped. `HubRecipe` parses `@recipe:` keywords and
`RecipeCollection` entries into type, technique, and strategy, and the entry
answers `supports_technique("sft")` and `fine_tuning_techniques()` directly.

---
X-AI-Prompt: How do I read a JumpStart hub model's license, size, and supported fine-tuning techniques without parsing the search keywords myself?
X-AI-Tool: claude-code
@evakravi
evakravi deployed to auto-approve October 23, 2026 17:42 — with GitHub Actions Active
@evakravi
evakravi deployed to auto-approve October 23, 2026 17:42 — with GitHub Actions Active
@evakravi
evakravi deployed to auto-approve October 23, 2026 17:46 — with GitHub Actions Active
@evakravi

Copy link
Copy Markdown
Member Author

Superseded by #6533, which combines the four changes into one PR and moves the hub-model metadata accessors onto HubContent itself (a HubContentMixin in sagemaker.core.hub_content) instead of a separate class.

@evakravi evakravi closed this Oct 23, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI code review

This PR adds a new, self-contained HubModelCatalogEntry dataclass (plus HubRecipe and catalog_entries) under sagemaker.core.jumpstart.hub.catalog, giving callers a typed view of a hub model's catalog metadata read from HubContentSearchKeywords and the HubContentDocument, with keywords taking precedence and the document filling fields the 50-keyword cap dropped. The module is additive (no existing behavior changed) and comes with 9 focused unit tests covering keyword parsing, recipe parsing, document fill-in/precedence, malformed input, the resource path, and the list helper. I verified the required HubContent resource import, the shape attributes, and that the test construction matches existing fixtures; the implementation looks correct and well-tested. Only one minor readability note.

Reviewed commit f1a2150. Automated review; verify before acting.

"""Group keyword values by catalog attribute, in hub order."""
values: Dict[str, List[str]] = {}
for keyword in keywords:
lowered = keyword.lower()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Low: This merged.extend(... if v.lower() not in _lower_set(merged)) relies on _lower_set(merged) being re-evaluated for each item while merged is simultaneously grown by extend. It is correct (and even dedups duplicates within document_hits), but the dependence on lazy generator evaluation against the list being mutated is subtle. Consider building the lowered set once and updating it explicitly in a plain loop so the dedup intent is obvious to future maintainers.

This branch was successfully deployed

1 active deployment
auto-approve — f1a2150d Deployed Oct 23, 2026 by evakravi via wait-for-approval #739
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant