Skip to content

0.6.0: shared_context(threads=) - #10

Merged
davidackerman merged 4 commits into
mainfrom
shared-threads
Oct 23, 2026
Merged

davidackerman merged 4 commits into
mainfrom
shared-threads

Conversation

@davidackerman

@davidackerman davidackerman commented Oct 23, 2026 •

Copy link
Copy Markdown
Member

0.6.0: one new argument, wa.shared_context(cache_bytes, threads=), at chunkmirage's request. No placement change and no unit table change. Nothing changes for a caller that doesn't pass it.

  • threads=None (the default) keeps 0.5's limit of thread_budget() on the decoding and file I/O pools.
  • threads=N limits both pools to N.
  • threads=0 writes no limit, so tensorstore sizes the pools itself, at every CPU on the node.

Each (cache_bytes, threads) pair gets one watched context per process. A pickled handle on one comes back on the same one: recipe_of gives ("shared", (cache_bytes, threads)). A shared context at the default limit still pickles as ("shared", cache_bytes), so the other process sizes it by its own job.

Why: chunkmirage's server keeps one context per cache budget, shared by every store it opens, at its own --threads. Under bsub -n 1 the default cap gives it one thread per pool. It can't use capped_context here, because that needs a count and returns a new context on every call. Its draft PR #53 builds and watches its own contexts in about 12 lines. Once this ships, those lines become wa.shared_context(cache_bytes, threads=0).

This change also updates:

  • spec Behaviour 7, the API listing and the test list;
  • the README;
  • the CHANGELOG, as 0.6.0 (unreleased).

Also documented in spec Behaviour 7 and the capped_context_spec docstring: what a pool of one costs. c-elegans measured it on /nrs on 2026-10-23, from one process, alternating the limit op by op:

  • One-chunk blocks: identical at limits 1 and 4.
  • 8-chunk blocks in a two-slot job: about 2x faster at 4. Reads took 157 ms at 1 against 73 ms at 4, and writes 312 ms against 158 ms.
  • 8-chunk blocks in a one-slot job: only 7 to 19% faster at 4. Reads took 147 ms against 119 ms, and writes 325 ms against 302 ms.

So most of the gain is a core the worker wasn't given.

185 tests pass locally (1 opt-in stress test skipped), and ruff passes.

https://claude.ai/code/session_0141wgF8mkGbjGfBFKfKhp64

… own (0.6.0)

chunkmirage's server keeps one context per cache budget for every store it opens, at
its own --threads, and under `bsub -n 1` the default cap gives it one thread per pool.
threads=None keeps 0.5's thread_budget() limit, N limits both pools to N, 0 writes no
limit. Each (cache_bytes, threads) is one watched context per process, and a pickled
handle comes back on the same one.

Claude-Session: https://claude.ai/code/session_0141wgF8mkGbjGfBFKfKhp64
… blocks

c-elegans measured on /nrs, op by op from one process in a two-slot job: one-chunk
blocks are the same at limits 1 and 4, 8-chunk blocks read and write about twice as
fast at 4. A one-slot worker is unmeasured.

Claude-Session: https://claude.ai/code/session_0141wgF8mkGbjGfBFKfKhp64
@davidackerman
davidackerman marked this pull request as ready for review October 23, 2026 16:45
c-elegans' bsub -n 1 run: 8-chunk blocks read 147 against 119 ms and write 325 against
302 ms at limits 1 and 4, against 2x in a two-slot job.

Claude-Session: https://claude.ai/code/session_0141wgF8mkGbjGfBFKfKhp64
@davidackerman
davidackerman merged commit 1bb9391 into main Oct 23, 2026
4 checks passed
@davidackerman
davidackerman deleted the shared-threads branch October 23, 2026 16:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant