[XiaoHongbo-Hope]

Purpose

LeRobot temporal windows load auxiliary frames alongside each requested sample. These auxiliary rows currently convert control fields such as index and timestamp to tensors even when only their State/Action or image features are used.

Convert only the requested window features for auxiliary rows. Base samples retain all fields, explicitly requested control-field windows still work, and all rows undergo the existing metadata validation.

Three paired prototype runs on #10701 with public lerobot/pusht: 4,996 → 5,478 samples/s (+9.6%), with average PSS essentially unchanged at 3.20 GiB. Local warm-cache CPU decoding, batch 32, 8 workers, 8-frame windows, uint8, no transforms/model; 10 warmup + 500 measured batches. Same-run official Lance achieved 6,091 samples/s. These are diagnostic prototype measurements, not a benchmark of this PR head.

Tests

• Related LeRobot/video tests: 81 passed, 89 subtests passed; 29 optional-dependency skips.
• Covers omitted auxiliary conversions, explicitly requested timestamp windows, and rejection of invalid auxiliary-frame metadata.
• Real-data prototype validation: all tensor bytes match the baseline across 16 batches, including images and State/Action.
• Flake8 and git diff --check passed.

[JingsongLi]

+1