[cuda] Score autotune picks with one event tick of tolerance, on longer kernels

[Gasoonjia]

SaturatedCpuExportTest (test_cuda_graph_timing.py) and Int4AutotunePickTest score each autotune decision by its regret: the pick's reference time divided by the best candidate's. Event timing is quantized: cudaEventElapsedTime steps by 0.512 µs on the A10G CI GPU, and hipEventElapsedTime by about 1 µs. A reading is the difference of two quantized timestamps, so it can fall on either side of the true time by about a tick. On a kernel of a few microseconds, that is a large ratio. The A10G job failed on (https://github.com/pytorch/executorch/actions/runs/37835437673/job/113511485001):

AssertionError: 1.2000000727595779 not less than or equal to 1.2 :
Pick(kind='inductor', name='triton_poi_fused__unsafe_view_add_transpose_copy_view_copy_9', candidates=2, regret=1.2000000727595779)

That is exactly fp32(6 × 0.512 µs) / fp32(5 × 0.512 µs).

Changes

1. One tick of tolerance: _regret() credits the best candidate one tick, pick / (best + 1 tick), floored at 1.0, with margins for the float32 event times.
   • A difference the timer can't resolve no longer counts as a worse pick.
   • Anything larger is charged in proportion, so there's no jump at one tick and the limits stay meaningful at every scale.
   • No thresholds change.
2. Longer kernels, so one tick is a small fraction of them:
   • SaturatedCpuExportTest's model now uses widths (512, 1536, 3072) and 4096 tokens (was (256, 768, 1536) and 64).
   • Int4AutotunePickTest's smallest weight is now 8192×2048 (was 2048×2048).
   • summarize() now also prints the fastest kernel's reference time, so CI shows how far above one tick the shortest kernel runs.

Tests

RegretTest runs on CPU and covers both the CUDA and the HIP tick:

• one tick apart scores 1.0, checked up to 31250-tick kernels;
• two ticks apart is charged, checked up to ~1 s;
• larger gaps score pick / (best + 1).

Test plan

CI: unittest-cuda / unittest-cuda-quantized-gemm should show RegretTest, SaturatedCpuExportTest and Int4AutotunePickTest passing, the "fastest kernel" line in their output, and how long the two pick tests now take.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/23635

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ You can merge normally! (1 Unrelated Failure), 4 Unclassified Failures

As of commit 09a91f7 with merge base f2575be ():

UNCLASSIFIED FAILURES - DrCI could not classify the following jobs because the workflow did not run on the merge base. The failures may be pre-existing on trunk or introduced by this PR:

• Test CUDA Builds / test-executorch-cuda-build-13.4 / linux-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• Test CUDA Builds / unittest-cuda / linux-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  ##[error]received SIGINT
• Test CUDA Builds / unittest-cuda-runtime / linux-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1
• Test ROCm AOTI / unittest-rocm-gfx950-rocm7.2 / linux-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  backends/cuda/tests/test_int4_quantized_gemm.py::Int4AutotunePickTest::test_compile_time_picks_are_the_fastest_under_cpu_load

FLAKY - The following job failed but was likely due to flakiness present on trunk:

• Test CUDA Builds / check-all-cuda-builds (gh) (detected as infra flaky with no log or failing log classifier)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.