[ET-VK][test] Move operator benchmark binaries under ops/benchmark

[SS-JIA]

Stack from ghstack (oldest at bottom):

• [ET-VK][test] Run operator benchmarks from one etvk_bench binary #23598
• -> [ET-VK][test] Move operator benchmark binaries under ops/benchmark #23597
• [ET-VK][gemm] Nest linear and matmul sources by scheme #23493
• [ET-VK][convolution] Nest convolution sources by route #23492
• [ET-VK][binary] Nest binary operator sources #23491
• [ET-VK][codegen] Support nested shader directories #23490

Ulterior Motive

Restructure the Vulkan operator test+benchmark binaries around a single etvk_benchmark entry point with registered, per-operator test case sets, so related cases (such as a model's bottleneck shapes) can be grouped and run in isolation.

Rationale

What: Move the test+benchmark binaries, their shared utilities, and the test-only ops and shaders from test/custom_ops/ into test/ops/benchmark/, with one subdirectory per operator under cases/, and split the monolithic utils.{h,cpp} by topic.

Why: The flat custom_ops/ directory mixes benchmark binaries, framework code, test-only ops, and correctness-only gtests, and utils.{h,cpp} (3.5k lines) mixes value specs, test cases, execution, and reporting. Grouping the binaries by operator gives the upcoming case-set registration a home for each operator's cases and reference code.

Details

• Layout under test/ops/benchmark/
  • framework/: test framework utilities, conv2d_utils, cm_utils, and test_coopmat_probe.
  • test_ops/impl/ and test_ops/glsl/: test-only ops and shaders, formerly impl/ and glsl/.
  • cases/: one directory per operator, conv2d/ (direct, depthwise, pointwise), conv1d/, q8csw_conv2d/, q4gsw_linear/ (including test_fpa_q4gsw_linear and test_coopmat_linear_bench), q8csw_linear/, mm/, sdpa/, embedding_q4gsw/, choose_qparams_per_row/, add/; cases/q8ta/ collects the q8ta operators: conv2d/, conv2d_transposed/, linear/, binary/, unary/, clone/, pixel_shuffle/, qdq/.
• utils.{h,cpp} split
  • config: global output and debugging options.
  • labels.h: test case naming helpers.
  • value_spec: ValueSpec, data generation, half-precision conversion, and print_valuespec_data.
  • test_case: ReferenceKey, the shader filter defaults, and TestCase.
  • results: BenchmarkResult, TestResult, and table printing.
  • runner: graph setup, RepeatedGraphExecutor, and execute_test_case(s).
  • weight_utils: weight canvases and weight sum helpers.
  • utils.h includes all of the above, so binaries are unchanged.
  • Moving code into new files exposes it to CLANGTIDY, so its findings are fixed: braces around single-statement bodies, explicit on the single-argument BenchmarkResult, TestResult, and TestCase constructors, const& for the generator, reference, FLOP calculator, and op_name parameters, utils::safe_downcast<int> for one narrowing, (void) for an unused parameter, and NOLINTNEXTLINE on enum switches that list only some enumerators (each has a default) and on the using namespace vkcompute that binaries rely on. utils.h no longer includes <iostream>.
• Build
  • test/ops/benchmark/{BUCK,targets.bzl} define the binaries, prototyping_utils, operator_implementations, and custom_ops_shaderlib. Binary target names are unchanged; only the package path changes.
  • The CMake project and build_and_run.sh move with the binaries and take source paths from the new subdirectories.
• The correctness-only gtests (utils_test, q8ta_conv2d_route_test, q8ta_conv2d_stream_plan_test) stay in test/custom_ops/; utils_test now depends on prototyping_utils in its new package.
• Binaries include framework headers by full path (executorch/backends/vulkan/test/ops/benchmark/framework/utils.h).
• CI references move to the new package: the GPU-host and Eureka Skycastle workflows, and the OSS vulkan.yml and pull.yml workflows.
• Moved files are recorded as renames; the new BUCK/targets.bzl and the split framework files are recorded as copies.

Binary sources, test cases, and test-only ops are otherwise unchanged.

Differential Revision: D123690842

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/23597

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit f7868bd with merge base bdb489e ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.