Simplifies the storage internals, refuses data directories it cannot load safely, and refreshes the benchmark numbers from Release builds.

[koculu]

Summary

Simplifies the storage internals, refuses data directories it cannot load safely, and refreshes the benchmark numbers from Release builds. Also adds a query benchmark script and an option to benchmark ingestion without indexes.

Changes

Engine

• Dimension and rollup indexes share one base class that decides whether to reuse, repair, or rebuild an index and tracks resume and completion.
• Metadata and the joint-rollup journal share one file format: two files written in turn, each with a generation number and a SHA-256 checksum. Metadata is now meta/meta-a.bin and meta/meta-b.bin.
• Startup refuses a directory that holds data but has no valid metadata. Before, such a directory silently started over with a new visitor secret and new salts.
• Duplicate code removed: the user-agent cache, raw slice splitting, page-mark rollups, lazy decoding, retention cutoffs, granularity indexing, and strict UTF-8.

Dashboard

• Startup opens the analytics engine and fails with the exception if the engine can't load, instead of failing on the first request.
• The ingestion benchmark has a Maintain dimension and rollup indexes toggle. Reports of a run without indexes reopen without indexes.

Benchmarks and docs

• yarn benchmark:queries replays the documented 5M-event query benchmark in headless Chrome. Results go to artifacts/benchmarks/ by default.
• Query and ingestion numbers in the docs and README are now from Release runs. The ingestion section also shows a run without indexes.
• internal-docs/storage-format.md describes the new metadata file format.

Housekeeping

• Version 1.0.2; PureStack packages 1.1.15.

Breaking changes

Data directories written by earlier builds fail to open with "holds data but no valid metadata". Delete them and regenerate the data. Storage format versions stay at 2.

Testing

• Full .NET suite passed (892/892) after the engine changes. Benchmark and dashboard option tests pass after the toggle.
• Dashboard UI type check and Biome lint pass.