Repository navigation

feature: add emr-serverless step for SageMaker Pipelines - #5325

Closed

wutimot wants to merge 161 commits into

aws:master-v2from

wutimot:feature/emr-serverless-step

Closed

feature: add emr-serverless step for SageMaker Pipelines#5325
wutimot wants to merge 161 commits into
aws:master-v2from
wutimot:feature/emr-serverless-step

Commits on Jan 29, 2025
- change: update image_uri_configs 01-27-2025 06:18:13 PST
  
  sagemaker-bot
  authored and
  root
  committed
  60732d3
- fix: skip TF tests for unsupported versions (#5007)
  
  erickb336
  authored and
  root
  committed
  eb3a774
- change: update image_uri_configs 01-29-2025 06:18:08 PST
  
  sagemaker-bot
  authored and
  root
  committed
  ebcd26f
- chore: add new images for HF TGI (#5005)
  
  authored and
  root
  committed
  0772ecd
- feat: use jumpstart deployment config image as default optimization image (#4992)
  
  authored and
  root
  committed
  ae03c31
Commits on Feb 1, 2025
- prepare release v2.238.0
  ci
  committed
  401fc81
- update development version to v2.238.1.dev0
  ci
  committed
  71f6d22
- Fix ssh host policy (#4966)
  
  authored and
  root
  committed
  1328e69
- change: Allow telemetry only in supported regions (#5009)
  
  authored and
  root
  committed
  138a2e9
- mpirun protocol - distributed training with @remote decorator (#4998)
  
  authored and
  root
  committed
  caaf47e
- feat: Add support for deepseek recipes (#5011)
  
  erickb336
  authored and
  root
  committed
  4533790
Commits on Feb 14, 2025
- prepare release v2.239.0
  ci
  committed
  75f3295
- update development version to v2.239.1.dev0
  ci
  committed
  352b922
- change: update image_uri_configs 02-04-2025 06:18:00 PST
  
  sagemaker-bot
  authored and
  root
  committed
  8910496
- Create GitHub action to trigger canaries (#5008)
  
  nileshvd
  authored and
  root
  committed
  9cb2415
- Add docstring for image_uris.retrieve
  
  pintaoz
  authored and
  root
  committed
  87e25c0
- fix: fix ValueError when updating a data quality monitoring schedule (#5002)
  
  authored and
  root
  committed
  7f5439f
- Fixed pagination failing while listing collections (#5020)
  
  authored and
  root
  committed
  80d3c02
- Add cleanup logic to model builder integ tests for endpoints (#5022)
  
  sage-maker
  authored and
  root
  committed
  4073bce
- fix: bug in get latest version was getting the max sorted alphabetically (#5014)
  
  authored and
  root
  committed
  a8e225e
- Fix documentation for local mode (#5026)
  
  authored and
  root
  committed
  c7b4b72
- Fix sourcedir.tar.gz filenames in docstrings (#5019)
  
  authored and
  root
  committed
  86dd6ae
- Add type hint for ProcessingOutput (#5030)
  
  authored and
  root
  committed
  2948ae3
- Fix FeatureGroup docstring (#5028)
  
  authored and
  root
  committed
  31c5e0a
- Fix Tensorflow doc link (#5029)
  
  authored and
  root
  committed
  fc0e7d0
- Fix the workshop link for Step Functions (#5034)
  
  authored and
  root
  committed
  b6e15be
- Fix all type hint and docstrings for callable (#5035)
  
  authored and
  root
  committed
  6c5e222
- fix: keep sagemaker_session from being overridden to None (#5021)
  
  authored and
  root
  committed
  0f5054e
Commits on Feb 19, 2025
- prepare release v2.239.1
  ci
  committed
  f7cd6d1
- update development version to v2.239.2.dev0
  ci
  committed
  b0e17cf
- Move RecordSerializer and RecordDeserializer to sagemaker.serializers and sagemaker.deserialzers (#5037)
  
  authored and
  root
  committed
  35ddf9c
- Add framework_version to all TensorFlowModel examples (#5038)
  
  authored and
  root
  committed
  b485f94
- Fix hyperparameter strategy docs (#5045)
  
  sage-maker
  authored and
  root
  committed
  4da15cd
- fix: pass in inference_ami_version to model_based endpoint type (#5043)
  
  authored and
  root
  committed
  01c72c7
- Add warning about not supporting torch.nn.SyncBatchNorm (#5046)
  
  authored and
  root
  committed
  35acb3a
- prepare release v2.239.2
  ci
  committed
  45263fe
- update development version to v2.239.3.dev0
  ci
  committed
  b6ddeee
- change: update image_uri_configs 02-19-2025 06:18:15 PST
  
  sagemaker-bot
  authored and
  root
  committed
  642a2ca
- change: added ap-southeast-7 and mx-central-1 for Jumpstart (#5049)
  
  authored and
  root
  committed
  dc8d350
Commits on Feb 25, 2025
- prepare release v2.239.3
  ci
  committed
  0757e9d
- update development version to v2.239.4.dev0
  ci
  committed
  a9583d5
- change: update image_uri_configs 02-20-2025 06:18:08 PST
  
  sagemaker-bot
  authored and
  root
  committed
  dfad50d
- feat: Add support for TGI Neuronx 0.0.27 and HF PT 2.3.0 image in PySDK (#5050)
  
  authored and
  root
  committed
  122ea28
- Add backward compatbility for RecordSerializer and RecordDeserializer (#5052)
  
  authored and
  root
  committed
  7a8635b
- py_version doc fixes (#5048)
  
  sage-maker
  authored and
  root
  committed
  dc8c305
- change: update image_uri_configs 02-21-2025 06:18:10 PST
  
  sagemaker-bot
  authored and
  root
  committed
  01dff74
- fix: altconfig hubcontent and reenable integ test (#5051)
  
  authored and
  root
  committed
  945db32
- fix: forbid extras in Configs (#5042)
  
  erickb336
  authored and
  root
  committed
  f3dab1e
- Remove main function entrypoint in ModelBuilder dependency manager. (#5058)
  
  authored and
  root
  committed
  7829a7e
- documentation: Removed a line about python version requirements of training script which can misguide users. (#5057)
  
  authored and
  root
  committed
  eec6e15
Commits on Mar 6, 2025
- prepare release v2.240.0
  ci
  committed
  6f09793
- update development version to v2.240.1.dev0
  ci
  committed
  a125d17
- Fix key error in _send_metrics() (#5068)
  
  authored and
  root
  committed
  f5330fc
- fix: Added check for the presence of model package group before creating one (#5063)
  
  authored and
  root
  committed
  9c5b657
- Use sagemaker session's s3_resource in download_folder (#5064)
  
  authored and
  root
  committed
  85c66d7
- Fix error when there is no session to call _create_model_request() (#5062)
  
  authored and
  root
  committed
  28bd3c4
- Ensure Model.is_repack() returns a boolean (#5060)
  
  authored and
  root
  committed
  4a02f6c
- feat: Allow ModelTrainer to accept hyperparameters file (#5059)
  
  erickb336
  authored and
  root
  committed
  b9ff4ad
- feature: support training for JumpStart model references as part of Curated Hub Phase 2 (#5070)
  
  authored and
  root
  committed
  0c73ce0
- feat: Make DistributedConfig Extensible (#5039)
  
  authored and
  root
  committed
  24d8ef1
- Skip tests with deprecated instance type (#5077)
  
  authored and
  root
  committed
  35484d2
Commits on Mar 24, 2025
- Basic PoC
  nargokul
  committed
  8326c23
Commits on Apr 11, 2025
- Add sagemaker_utils module (#1667)
  
  authored
  6a4e165
Commits on Apr 30, 2025
- Cleanup V3 branch (#1674)
  erickb336
  authored
  31345aa
Commits on May 1, 2025
- Move SageMaker Core as a submodule in Python SDK (#1669)
  
  pintaoz-aws
  and
  pintaoz
  authored
  062c289
Commits on May 5, 2025
- Add inital HubContentDocument Utils, Model Classes, and ModelTrainer class method (#1675)
  erickb336
  authored
  69fc0d1
Commits on May 6, 2025
- Refractor ImageRetriever in v3 (#1673)
  
  pintaoz-aws
  and
  pintaoz
  authored
  58359a4
Commits on May 7, 2025
- Sagemaker Train Module and changes to related utils (#1671)
  nargokul
  authored
  a5b2a14
Commits on May 8, 2025
- Cleanup Train Module (#1678)
  erickb336
  authored
  9f70fb2
- Test intelligent defaults in sagemaker-core with utils module (#1676)
  
  rsareddy0329
  and
  Roja Reddy Sareddy
  authored
  40afb77
Commits on May 13, 2025
- Baseline ModelTrainer from live (#1681)
  erickb336
  committed
  59bdfec
Commits on May 15, 2025
- Add serializer and deserializer in Endpoint configuration (#1680)
  
  rsareddy0329
  and
  Roja Reddy Sareddy
  authored
  ce3099a
Commits on May 19, 2025
- Add search_public_hub_models function for jumpstart hub (#1683)
  mollyheamazon
  authored
  0fbac6d
- Add JumpStart Support in ModelTrainer (#1677)
  erickb336
  authored
  b364cb6
- Add workflows in master-v3 branch (#1686)
  
  rsareddy0329
  and
  Roja Reddy Sareddy
  authored
  e002aba
Commits on May 21, 2025
- Delete .github directory
  rsareddy0329
  authored
  9c796b8
- Merge inputs from method with class inputs (#1693)
  erickb336
  authored
  a9a5072
- Support intelligent defaults for ImageRetriever (#1679)
  
  pintaoz-aws
  and
  pintaoz
  authored
  5e6e9e1
Commits on May 27, 2025
- Add submodule workflow (#1692)
  
  rsareddy0329
  and
  Roja Reddy Sareddy
  authored
  7fc508a
- Retrieve image uri from s3 (#1694)
  
  pintaoz-aws
  and
  pintaoz
  authored
  7986335
Commits on Jun 3, 2025
- Fix: next_token causing infinite loop in fetching Clusters (#1699)
  zhaoqizqwang
  authored
  1940fae
Commits on Jun 4, 2025
- Add submodule workflow and Tox config update to test PR workflows (#1697)
  
  rsareddy0329
  and
  Roja Reddy Sareddy
  authored
  ac5eff0
Commits on Jun 9, 2025
- feat: working sagemaker-serve with reliance on v2 lib (#1702)
  sage-maker
  authored
  5d3bec9
Commits on Jun 11, 2025
- feature: working model.py and tests with reliance on v2 (#1704)
  sage-maker
  authored
  2cc2681
Commits on Jun 12, 2025
- feature: model.py uses sagemaker-core Session (#1706)
  sage-maker
  authored
  82329ce
- style: whitespace
  sage-maker
  authored
  3f95c12
Commits on Aug 5, 2025
- change: task 3 remove dependency on predictor (#1725)
  sage-maker
  authored
  d939202
Commits on Aug 12, 2025
- feat: task 4 + 5 use sagemaker core session and change init/build (#1729)
  sage-maker
  authored
  6fa633e
Commits on Nov 18, 2025
- Update submodule-codebuild-ci.yml (#1846)
  nargokul
  authored
  337e40b
Commits on Nov 19, 2025
- Update submodule-codebuild-ci.yml to support PR checks for V3 launch (#1851)
  aviruthen
  authored
  992f30f
Commits on Nov 20, 2025
- Release PySDK V3
  
  authored
  3203e49
- Small bug fix (#5327)
  aviruthen
  authored
  fa30a6d
- Update pyproject.toml and prepare for v3.0.1 release (#5329)
  zhaoqizqwang
  authored
  869f3ee
Commits on Nov 21, 2025
- Update example notebooks (#5330)
  aviruthen
  authored
  76f0d2e
- Update telemetry
  zhaoqizqwang
  committed
  9827b07
- Fix pysdk v3 Telemetry (#5331)
  zhaoqizqwang
  authored
  17116f4
- Prepare for release 3.0.2 (#5333)
  zhaoqizqwang
  authored
  a587f34
Commits on Nov 22, 2025
- Revert v3.0.2 Changelog and prepare SageMaker Core v2.0.1 release (#5334)
  zhaoqizqwang
  authored
  8c4585d
Commits on Nov 23, 2025
- Update GIthub workflows for V3 (#5335)
  zhaoqizqwang
  authored
  e7d024c
Commits on Nov 24, 2025
- Update GIthub actions for V3 (#5337)
  zhaoqizqwang
  authored
  0bc333e
- Set CI health source version explicitly (#5339)
  zhaoqizqwang
  authored
  44f560e
Commits on Nov 25, 2025
- Update inference example notebooks (#5340)
  aviruthen
  authored
  ed6a83b
- Skip optimize test (#5342)
  aviruthen
  authored
  16c2591
- Add training examples (#5341)
  aviruthen
  authored
  290fdc2
- Update readme (#5343)
  aviruthen
  authored
  a50681f
Commits on Nov 26, 2025
- Reroute telemetry to proper bucket (#5345)
  aviruthen
  authored
  41f9aa3
- Fix TuningStep validation error by adding V3 HyperparameterTuner duck typing support (#5344)
  aviruthen
  authored
  a317edd
Commits on Dec 1, 2025
- Update imports in example notebook (#5347)
  zhaoqizqwang
  authored
  1286f17
Commits on Dec 3, 2025
- Fix jumpstart integ test (#5351)
  zhaoqizqwang
  authored
  c05f0e7
- renamed sagemaker training drivers folder (#5356)
  aviruthen
  authored
  d99b596
- Update submodule-codebuild-ci.yml
  mollyheamazon
  authored
  ba1604c
- Update submodule-codebuild-ci.yml
  mollyheamazon
  authored
  b2e8a3d
- Fix: Update token in PR checks submodule-codebuild-ci.yml
  rsareddy0329
  authored
  14e1d3f
- Fix: Update submodule-codebuild-ci.yml for tests to be independent
  mollyheamazon
  authored
  e5c19c9
- feat: model customization v3 release PR (#5358)
  rsareddy0329
  authored
  0443307
Commits on Dec 4, 2025
- Update README for model customization launch (#5365)
  papriwal
  authored
  462bed0
Commits on Dec 5, 2025
- fix: Fix the recipe selection for multiple recipe scenario (#5367)
  
  rsareddy0329
  and
  Roja Reddy Sareddy
  authored
  db12b1d
- Updating evaluator to append timestamp to function name (#5370)
  jam-jee
  authored
  8c07487
- Skip RLVR integ tests (#5373)
  aviruthen
  authored
  d9bfe7c
- fix: Add lambda_type parameter for Nova models and migrate to ModelPackage (#5374)
  
  authored
  4e50064
Commits on Dec 6, 2025
- Cleaned up Model Customization Notebooks (#5368)
  
  nargokul
  and
  rsareddy0329
  authored
  f32f615
Commits on Dec 7, 2025
- fix: Address Hyperparameter issue , validate s3 output path, additional unit tests (#5376)
  
  rsareddy0329
  and
  Roja Reddy Sareddy
  authored
  60574e5
Commits on Dec 8, 2025
- Updating benchmark evaluation for subtasks and datasets (#5378)
  jam-jee
  authored
  6be0fd3
- Fix: Add validation to bedrock reward models - RLAIF Trainer (#5381)
  
  rsareddy0329
  and
  Roja Reddy Sareddy
  authored
  6991648
Commits on Dec 9, 2025
- Fix unit and integ tests (#5377)
  aviruthen
  authored
  2466b8b
Commits on Dec 10, 2025
- Update PR check workflow file (#5397)
  zhaoqizqwang
  authored
  5b203d4
- Separate PR check workflows (#5399)
  zhaoqizqwang
  authored
  36f96be
- Wait timeout exception handling (#5400)
  jam-jee
  authored
  3cc40a3
- Fix: Add allow list for bedrock eval models (#5394)
  
  rsareddy0329
  and
  Roja Reddy Sareddy
  authored
  81b51eb
- cleaning up ai registry notebook (#5403)
  jam-jee
  authored
  8e2778c
- ft: Remove insecure SSL context creation in Triton Python backend (#5372)
  
  pravali96
  and
  Pravali Uppugunduri
  authored
  5e7a3ef
- Fix: Bug fixes for s3 path validation, mlflow app creation (#5402)
  
  rsareddy0329
  and
  Roja Reddy Sareddy
  authored
  cd406fa
- Update CHANGELOG.md and versions (#5404)
  rsareddy0329
  authored
  1ab6d30
Commits on Dec 11, 2025
- Fix: fix sagemaker-serve tests (#5407)
  
  rsareddy0329
  and
  Roja Reddy Sareddy
  authored
  98161ac
- Update readme to remove readthedocs reference temporarily (#5408)
  aviruthen
  authored
  9da502a
Commits on Dec 12, 2025
- add xgboost 3.0-5 to release (#5405)
  
  Yadan-Wei
  and
  Yadan Wei
  authored
  194e74b
Commits on Dec 15, 2025
- Bug fix for hmac key for V3 (#5379)
  
  aviruthen
  and
  zhaoqizqwang
  authored
  fb0d789
- Add issues template, specify V2 or V3 bug (#5414)
  aviruthen
  authored
  fd37adc
Commits on Dec 16, 2025
- Fix get_child_process_ids parsing issue (#5416)
  
  pintaoz-aws
  and
  pintaoz
  authored
  4055fcf
Commits on Dec 17, 2025
- feat: Evaluator handshake to trainer (#5420)
  
  rsareddy0329
  and
  Roja Reddy Sareddy
  authored
  840f3a1
- feat: Adding DataSets format validation (#5415)
  jam-jee
  authored
  2c7c4b5
- Fix _run_shell_cmd() to use list input (#5422)
  
  pintaoz-aws
  and
  pintaoz
  authored
  cd95b19
- Fix failing ai_registry unit tests (#5426)
  aviruthen
  authored
  d089d40
Commits on Dec 18, 2025
- feat: add evaluator tagging for jumpstart models (#5413)
  
  Narrohag
  and
  aviruthen
  authored
  0bec208
- Fixing pip installation of PySDK V3 (#5421)
  aviruthen
  authored
  a881369
- fix: update evaluate_base_model as False, minor change to README (#5425)
  
  rsareddy0329
  and
  Roja Reddy Sareddy
  authored
  8695cca
- Migrate recent commits in master-v2 (#5423)
  
  zhaoqizqwang
  and
  aviruthen
  authored
  1c7faf0
- Add input validation and resource management improvements V3 (#5418)
  aviruthen
  authored
  a74f9ab
Commits on Dec 19, 2025
- Update CHANGELOG.md sagemaker meta (#5431)
  rsareddy0329
  authored
  22d30f5
- Add aws batch (#5409)
  aviruthen
  authored
  f272de0
- Initialize framework and framework version in post_init (#5424)
  aviruthen
  authored
  87c525f
Commits on Dec 20, 2025
- Add validation in conda env name (#5430)
  
  pintaoz-aws
  and
  pintaoz
  authored
  8c9ab9b
- Update files for release (#5434)
  aviruthen
  authored
  36fd82f
Commits on Dec 22, 2025
- Add validation for traversal components in S3 path (#5436)
  
  pintaoz-aws
  and
  pintaoz
  authored
  1cfbc3b
Commits on Dec 23, 2025
- Update OutputDataConfig documentation to avoid customer confusion
  mollyheamazon
  committed
  4686abf
Commits on Jan 5, 2026
- fix: Update telemetry constants to include MLOPS as well (#5438)
  
  rsareddy0329
  and
  Roja Reddy Sareddy
  authored
  7c64af7
Commits on Jan 6, 2026
- fix: Remove tags from ProcessingJob creation in Processor (#5459)
  nargokul
  authored
  4722723
Commits on Jan 7, 2026
- New PR check workflow (#5460)
  aviruthen
  authored
  0fd9af3
Commits on Jan 8, 2026

Commits

Commits on Jan 29, 2025

Commits on Feb 1, 2025

Commits on Feb 14, 2025

Commits on Feb 19, 2025

Commits on Feb 25, 2025

Commits on Mar 6, 2025

Commits on Mar 24, 2025

Commits on Apr 11, 2025

Commits on Apr 30, 2025

Commits on May 1, 2025

Commits on May 5, 2025

Commits on May 6, 2025

Commits on May 7, 2025

Commits on May 8, 2025

Commits on May 13, 2025

Commits on May 15, 2025

Commits on May 19, 2025

Commits on May 21, 2025

Commits on May 27, 2025

Commits on Jun 3, 2025

Commits on Jun 4, 2025

Commits on Jun 9, 2025

Commits on Jun 11, 2025

Commits on Jun 12, 2025

Commits on Aug 5, 2025

Commits on Aug 12, 2025

Commits on Nov 18, 2025

Commits on Nov 19, 2025

Commits on Nov 20, 2025

Commits on Nov 21, 2025

Commits on Nov 22, 2025

Commits on Nov 23, 2025

Commits on Nov 24, 2025

Commits on Nov 25, 2025

Commits on Nov 26, 2025

Commits on Dec 1, 2025

Commits on Dec 3, 2025