Repository navigation
0.7.0: time out document reads and opens - #11
Merged
Merged
Conversation
A server that accepts a connection and never answers held wa.open and wa.metadata forever: Retry bounded data reads only (cellmap-flow's finding). Every document read or written and every store open now waits under the calling function's retries, as an operation of no bytes, with tensorstore's request retries inside each attempt. wa.metadata, wa.detect and wa.open_tensorstore take retries= as wa.open already did. Listings and the creating open are left alone. Claude-Session: https://claude.ai/code/session_0141wgF8mkGbjGfBFKfKhp64
No live consumer reads funlib attributes from cellmap-analyze or cellmap-flow outputs: cellmap-analyze 0.6.0 writes OME only, cellmap-flow reads back its own, dacapo is fed none of them, cellmap-data and fibsem-tools read OME only. Notes cellmap-data's translation-as-corner reading. Claude-Session: https://claude.ai/code/session_0141wgF8mkGbjGfBFKfKhp64
Claude-Session: https://claude.ai/code/session_0141wgF8mkGbjGfBFKfKhp64
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
0.7.0: store opens time out. No placement change and no unit table change.
The bug (cellmap-flow's finding).
Retrybounded data reads only. Suppose an http server accepts the connection and never answers. Thenwa.openandwa.metadatawaited on the metadata fetch forever. Reproduced locally: still waiting after 40 s.The fix.
Retryas data reads do. Each is treated as an operation of no bytes, so by default an attempt waits 5 s times its number, for up to ten attempts.retries.wa.metadata,wa.detectandwa.open_tensorstoregainretries=;wa.open,wa.open_pyramidandwa.createalready took it.TimeoutErrorwithin seconds. The error names the URL and the attempts.requestsprobe in front of its picker's dtype poll.Also in this PR: spec open question 5 is resolved, with no
funlib_attrs. No live consumer reads funlib attributes from cellmap-analyze or cellmap-flow outputs:Open question 11 now points at an OME-shaped array attribute. The spec also notes that cellmap-data reads an OME translation as voxel 0's corner, which puts it half a voxel off.
186 tests pass locally (1 opt-in stress test skipped), and ruff passes.
https://claude.ai/code/session_0141wgF8mkGbjGfBFKfKhp64