Repository navigation
Do not delegate CoreML integer mm with a constant weight - #23689
hemanth1999k wants to merge 1 commit into
Conversation
coremltools 9.0 lowers an mm against a constant weight to linear, which only takes floating point inputs. An int32 mm with a constant weight therefore lowered fine, then failed to load (error -14 building the execution plan). An integer mm of two runtime inputs becomes matmul and runs. The partitioner now leaves such mm nodes to the portable backend, along with any constant-only op that only feeds their weight (e.g. a transpose of a buffer), which would otherwise be delegated as a partition with no runtime inputs. Integer mm of two inputs and float mm stay delegated. This is a stopgap until ExecuTorch consumes a coremltools release with apple/coremltools#2719, which keeps integer mm on matmul. Fixes pytorch#11691
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/23689
Note: Links to docs will display an error until the docs builds have been completed.
|
|
|
|
Hi @hemanth1999k! Thank you for your pull request and welcome to our community. Action RequiredIn order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you. ProcessIn order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA. Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks! |
|
@pytorchbot label "release notes: apple" |
|
/easycla |
|
Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks! |
Summary
coremltools 9.0 lowers an
mmagainst a constant weight tolinear, which only takes floating point inputs. An int32mmwith a constant weight therefore lowers to Core ML without complaint, then fails when the program loads (error -14 while building the execution plan). An integermmof two runtime inputs becomesmatmuland runs fine.CoreMLPartitionernow leaves an integermmwhose weight comes from a parameter, buffer, or lifted constant to the portable backend. It also leaves behind any constant-only op that only feeds that weight (for example a transpose of a buffer); otherwise that op would be delegated on its own as a partition with no runtime inputs, which Core ML rejects before iOS 18. Integermmof two inputs and floatmmstay delegated. The analysis is two linear passes over the graph and is cached per graph.This is a stopgap: apple/coremltools#2719 keeps integer
mmonmatmuland is on coremltools main but not in a release yet. The new code carries a TODO to remove it once ExecuTorch picks up a coremltools release with that change. Happy to drop this PR if you would rather wait for the coremltools bump.Fixes #11691
Test plan
Added
test_integer_mm_with_constant_weight_falls_back_to_portabletobackends/apple/coreml/test/test_coreml_partitioner.py. It covers four models:mmwith a constant weight, plus anabsthat must stay delegatedmmwith a transposed buffer as the weightmmof two inputs (stays delegated)mmwith a constant weight (stays delegated)For each it checks which ops are delegated, runs the program through
executorch.runtime.Runtime, and compares with eager.Partitioner tests 13/13 and
test_torch_ops.py12/12 pass with the change; on main only the new test fails. Run on macOS 26.2, executorch 1.5.1 / main, torch 2.14.0, coremltools 9.0.ufmtandflake8are clean.cc @nil-is-all