[kz930]

What changes were proposed in this PR?

The runner discovers the operators that implement the trait, configures each
from its schema, runs it through the engine and through its generated script,
and compares. An operator that cannot be run is reported as a row with the
reason rather than passed over, since an operator missing from the report is
a fact the report has to carry.

OperatorBehaviorSpec is what turns that into tests: one per operator per
configuration its schema offers, named so a single operator can be run alone.

Every operator reads one canonical table, and it now also carries the values a type reaches only at its edges. That means a double at 1e7 and past it, and one under 1e-3, which Java writes in E notation. It means timestamps in 1600, 2262, 2500 and 9999, outside pandas' nanosecond range. And it means text of digits with leading zeros, which a reader must not take for a number. Without them, a difference that only shows at those values is never asked about.

Bullet and Gauge are compared by the figures their script writes, now that it writes every one rather than the first.

A whole workflow is compared too. WorkflowParitySpec, in amber's integration tests, runs each plan in the engine and through its translated script, and compares every leaf. Rows compare as a set unless the leaf's operator declares an order, since the engine runs most operators on several workers. The plans are the ones aimed at how fragments go wrong together: a chain, a holed integer column, a cast timestamp, a Union, both branches of a Split, a column named like a placeholder, and File Scan reading the paths its upstream names. TranslatedPlanRunSpec keeps the two the engine has nothing to compare with: two charts in one plan, and a chart that cannot be drawn.

amber's tests take the compiling service's and workflow-operator's classes and test resources, not the modules themselves. The compiling service brings dropwizard 4, which would evict the dropwizard 1 amber is built on.

Runs whose reason is that both paths raise are no longer skipped. They sit in their own list, variantsBothRefuse, run like any other, and pass only when the engine and the script both refuse, since a script that answers where the engine refuses is as wrong as a different answer. Only a run named there may pass on two errors, so a broken environment cannot turn the suite green. The one exception found is the dummy classifier on an empty table: it fits on nothing into a model that cannot predict, so it stays in variantsNotRun with that reason. File Scan reads the canonical table's own JSON, which both paths refuse as an archive and as a typed line, so those variants are listed there too. Its archives and typed lines are compared on real files, in the scenarios #8364 and #8363 add. A large binary is written to object storage, which the harness does not give the executor, so that variant is in variantsNotRun.

Any related issues, documentation, discussions?

Part of #8325, 23 of 27; that issue lists the set in order. The edge values pass only once #8341, #8704, #8357, #8709, #8710 and #8597 are in. Each of those fixes how one path reads or writes them. All of them come earlier in the set or merge before it. An empty table runs on every operator once #8766 is in, which hands a port that carried no rows its declared columns.

Closes #8425, the task this change is the whole of.

The workflow-level comparison answers the review that asked for plans run end to end. It calls runTranslated from #8358.

How was this PR tested?

The tests in this change cover it. The whole set is exercised together once the last piece lands: every operator run through the engine and through its generated script, and the two answers compared.

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Claude Opus 5, Claude Opus 5.5)

[github-actions]

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

• Contributors with relevant context: @aicam
  You can notify them by mentioning @aicam in a comment.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.72%. Comparing base (d102e26) to head (40811f3).
⚠️ Report is 101 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #8362      +/-   ##
============================================
- Coverage     92.72%   92.72%   -0.01%     
- Complexity     4946     4950       +4     
============================================
  Files          1242     1243       +1     
  Lines         52671    52623      -48     
  Branches       6519     6506      -13     
============================================
- Hits          48838    48793      -45     
  Misses         2214     2214              
+ Partials       1619     1616       -3     

Flag	Coverage Δ		*Carryforward flag
access-control-service	77.38% <ø> (ø)
agent-service	99.16% <ø> (ø)		Carriedforward from 62598dd
amber	88.47% <ø> (+0.01%)	⬆️	Carriedforward from 62598dd
computing-unit-managing-service	60.48% <ø> (+0.07%)	⬆️
config-service	87.37% <ø> (ø)
file-service	81.53% <ø> (ø)		Carriedforward from 62598dd
frontend	96.52% <ø> (-0.01%)	⬇️	Carriedforward from 62598dd
notebook-migration-service	83.73% <ø> (ø)
pyamber	98.56% <ø> (ø)		Carriedforward from 62598dd
workflow-compiling-service	74.09% <ø> (ø)		Carriedforward from 62598dd

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[carloea2]

The transform verification runner looks good.

[carloea2]

Reviewed alongside the related export and verification PRs. These findings are based on code inspection and focused Python checks, not a full Scala suite run.

[kz930]

@carloea2 the finding is fixed: a connected plan is now translated and run end to end. Would you take another look when you have a moment?

[carloea2]

The translated chain and branch tests now run through the real translator. Looks good.

[carloea2]

Visualization verification compares only one Plotly figure. Bullet and Gauge can produce one figure per input row, but standalone writes only the first figure. A wrong second figure can pass. Please compare every produced figure and add a multirow test.

[carloea2]

The only full plan Union test gives both inputs the same file. Wiring the first input twice and dropping the second still passes. Please use distinct input data and assert a value unique to each port.

[carloea2]

The full plan tests only create links from output port 0. A Split or If workflow can wire output port 1 incorrectly and every connected plan test still passes. Please add a connected plan that uses both output ports and verify each branch separately.

[kz930]

All three fixed: each union port now reads its own file and the assertion is on the union's own frame, a new plan wires both of a split's output ports and reads each branch separately, and the Plotly comparison now reads every figure a run drew rather than the first. That last one lands in #8359 and #8509, where compare.py and the comparator live.

[carloea2]

The connected plan checks now cover the separate output routes and distinct source inputs. Looks good.

[kz930]

@carloea2 Since your approval the canonical table carries edge values (doubles Java writes in E notation, years outside pandas' range, digit strings), and Bullet and Gauge are compared by every figure they write.

[carloea2]

Read the runner, routing rules and whole-plan tests. With the exact prerequisite implementations supplied locally, 41 tests pass: 29 canonical-fixture checks, all 8 translated plans, and 4 executions through the actual runner. Those exercise native and generated Limit and Union variants, plus real Python UDF and exported Pie and Contour variants, including nulls, empty input and error pages. Formatting passes.

sbt -batch 'WorkflowCompilingService/testOnly *CanonicalFixtureSpec *TranslatedPlanRunSpec *RunnerExecutionReviewSpec -- -oDF' 'scalafmtCheckAll'

No new runner-specific finding. The branch alone cannot compile yet, and the other routing assertions still need their operator export prerequisites. This is not merge ready until those and CI are cleared.

[kz930]

@carloea2 the end-to-end layer now also runs each plan in the Texera engine and compares every leaf with the script's output (WorkflowParitySpec). Would you take another look when you have a moment?