Skip to content

Do not delegate CoreML integer mm with a constant weight - #23689

Open
hemanth1999k wants to merge 1 commit into
pytorch:mainfrom
hemanth1999k:coreml-reject-int-mm-constant-weight
Open

hemanth1999k wants to merge 1 commit into
pytorch:mainfrom
hemanth1999k:coreml-reject-int-mm-constant-weight

Conversation

@hemanth1999k

@hemanth1999k hemanth1999k commented Oct 11, 2026 •

Copy link
Copy Markdown

Summary

coremltools 9.0 lowers an mm against a constant weight to linear, which only takes floating point inputs. An int32 mm with a constant weight therefore lowers to Core ML without complaint, then fails when the program loads (error -14 while building the execution plan). An integer mm of two runtime inputs becomes matmul and runs fine.

CoreMLPartitioner now leaves an integer mm whose weight comes from a parameter, buffer, or lifted constant to the portable backend. It also leaves behind any constant-only op that only feeds that weight (for example a transpose of a buffer); otherwise that op would be delegated on its own as a partition with no runtime inputs, which Core ML rejects before iOS 18. Integer mm of two inputs and float mm stay delegated. The analysis is two linear passes over the graph and is cached per graph.

This is a stopgap: apple/coremltools#2719 keeps integer mm on matmul and is on coremltools main but not in a release yet. The new code carries a TODO to remove it once ExecuTorch picks up a coremltools release with that change. Happy to drop this PR if you would rather wait for the coremltools bump.

Fixes #11691

Test plan

Added test_integer_mm_with_constant_weight_falls_back_to_portable to backends/apple/coreml/test/test_coreml_partitioner.py. It covers four models:

  • int32 mm with a constant weight, plus an abs that must stay delegated
  • int32 mm with a transposed buffer as the weight
  • int32 mm of two inputs (stays delegated)
  • float mm with a constant weight (stays delegated)

For each it checks which ops are delegated, runs the program through executorch.runtime.Runtime, and compares with eager.

python -m pytest backends/apple/coreml/test/test_coreml_partitioner.py
python -m pytest backends/apple/coreml/test/test_torch_ops.py

Partitioner tests 13/13 and test_torch_ops.py 12/12 pass with the change; on main only the new test fails. Run on macOS 26.2, executorch 1.5.1 / main, torch 2.14.0, coremltools 9.0. ufmt and flake8 are clean.

cc @nil-is-all

coremltools 9.0 lowers an mm against a constant weight to linear, which
only takes floating point inputs. An int32 mm with a constant weight
therefore lowered fine, then failed to load (error -14 building the
execution plan). An integer mm of two runtime inputs becomes matmul and
runs.

The partitioner now leaves such mm nodes to the portable backend, along
with any constant-only op that only feeds their weight (e.g. a transpose
of a buffer), which would otherwise be delegated as a partition with no
runtime inputs. Integer mm of two inputs and float mm stay delegated.

This is a stopgap until ExecuTorch consumes a coremltools release with
apple/coremltools#2719, which keeps integer mm on matmul.

Fixes pytorch#11691
@pytorch-bot

pytorch-bot Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/23689

Note: Links to docs will display an error until the docs builds have been completed.

⚠️ 16 Awaiting Approval

As of commit 9c74b12 with merge base f2575be (image):

AWAITING APPROVAL - The following workflows need approval before CI can run:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@linux-foundation-easycla

linux-foundation-easycla Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

CLA Signed
The committers listed above are authorized under a signed CLA.

  • ✅ login: hemanth1999k / name: hemanth1999k (9c74b12)

@meta-cla

meta-cla Bot commented Oct 11, 2026

Copy link
Copy Markdown

Hi @hemanth1999k!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks!

@hemanth1999k

Copy link
Copy Markdown
Author

@pytorchbot label "release notes: apple"

@hemanth1999k

Copy link
Copy Markdown
Author

/easycla

@pytorch-bot pytorch-bot Bot added the release notes: apple Changes to the Apple backend delegate label Oct 11, 2026
@meta-cla

meta-cla Bot commented Oct 11, 2026

Copy link
Copy Markdown

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks!

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 11, 2026
@executorch-triage executorch-triage Bot added the community: contribution PRs coming from community (excluding hardware partners) label Oct 11, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. community: contribution PRs coming from community (excluding hardware partners) release notes: apple Changes to the Apple backend delegate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CoreML int32 mm with fixed arg fails to load at runtime

1 participant