Skip to content

fix: handle resource units during training quota allocation - #474

Open
shin0343 wants to merge 1 commit into
aws:mainfrom
shin0343:fix/training-kubernetes-quantities
Open

shin0343 wants to merge 1 commit into
aws:mainfrom
shin0343:fix/training-kubernetes-quantities

Conversation

@shin0343

Copy link
Copy Markdown

What's changing and why?

Handle explicit CPU and memory units during training quota allocation. Previously, CPU 500m raised a conversion error and memory 1024Mi was interpreted as 1024 GiB before clipping. Parse explicit Kubernetes units, preserve exponent notation through quota calculations, and emit fixed-point GiB quantities.

Retain unitless memory's existing GiB meaning, including values emitted by already-installed hyperpod-pytorch-job-template versions. Reject negative/non-finite/overflowing quantities before mutating the job.

Before/After UX

Before: A request of cpu=500m, memory=1024Mi failed or allocated the wrong memory amount. Small memory-only requests could also corrupt the derived CPU value.

After: The request resolves to 0.5 CPU and 1 GiB. Ki/small-memory quantities preserve their value, and existing CLI/template GiB inputs retain their allocations.

How was this change tested?

  • Full unit suite on Python 3.11: 1,286 passed.
  • 39 quantity cases cover installed-template conversion without replacing its resources, legacy/exponent GiB values, explicit units, memory-only derived CPU, defaults, malformed quantities and arithmetic/float overflow.
  • Changed files parse with Python 3.8 grammar; new tests pass Ruff lint/format; changed files pass git-secrets and git diff --check.
  • Command: PYTHONPATH=src python -m pytest --confcutdir=test/unit_tests -o addopts='' -q test/unit_tests using the prepared environment. The flag avoids the parent template-install hook; default coverage options are separated.

Are unit tests added?

Yes, 39 parameterized/public-allocation cases.

Are integration tests added?

No live AWS/EKS tests. Existing and new tests run locally without provisioning infrastructure.

Reviewer Guidelines

  • All automated PR checks pass
  • Failed tests include local run results/screenshots proving they work
  • Changes are documentation-only

Preserve legacy unitless GiB inputs while normalizing explicit Kubernetes units. Keep exponent-form quantities intact through allocation and reject invalid values before mutation.

Signed-off-by: shin0343 <shin0343@mail.hongik.ac.kr>
@shin0343
shin0343 requested a review from a team as a code owner October 25, 2026 16:06
@shin0343

Copy link
Copy Markdown
Author

The current head is 7d6b205067822b486b417e8c8a9d92d7b734cb6b. collab-check passed; wait-for-approval is still waiting before the external-contributor CI jobs can run.

Could a maintainer approve CI for this head? Local validation recorded in the PR includes 1,286 passing unit tests on Python 3.11 and 39 quantity regression cases. No live AWS/EKS validation was performed.

This branch is waiting to be deployed

1 waiting deployment
manual-approval — 7d6b2050 Waiting Oct 25, 2026 by shin0343 via wait-for-approval #827
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant