Repository navigation
fix(cli): list Team projects in one request and transfer files in parallel - #1703
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8be86d884d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
8be86d8 to
7181e12
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7181e1299f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7fa5a9a7ef
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1fcdce374e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…allel Team `bm cloud push`/`pull` were slow for two reasons (#1604): - Listing walked the project with one Depth: 1 PROPFIND per directory, serially. `list_project_files` now sends one `Depth: infinity` PROPFIND and reads each file's project-relative path from its href. - Files moved one at a time. They now move through a task group of at most TRANSFER_CONCURRENCY (8) workers. The first failure cancels the rest and is reported as one WebdavError that names each failed file and how many files were transferred. The client does not fall back to the per-directory walk. If the service answers one level deep, the client stops and says the service does not support recursive listings yet. If the service refuses with DAV:propfind-finite-depth, the client says the project is too large for one listing. Conflict detection, create-only writes, --on-conflict, the additive semantics and rate-limit retries are unchanged. Requires basicmachines-co/basic-memory-cloud#2400 to be deployed first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01APFUk2bjEwMptMQqpRhjea Signed-off-by: phernandez <paul@basicmachines.co>
Push and pull now run eight transfers at once, and each one held its whole file in memory: download_file read response.content and push read the file with read_bytes(). Peak memory grew with the number of files in flight. Downloads now stream straight into the staged temp file beside the destination. Uploads read the file in 1 MiB chunks and declare Content-Length up front, so the service can still refuse an oversized file before reading it. The rate-limit retry helper streams responses and opens a fresh body for each attempt. Error bodies are still read in full because the error message quotes them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01APFUk2bjEwMptMQqpRhjea Signed-off-by: phernandez <paul@basicmachines.co>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01APFUk2bjEwMptMQqpRhjea Signed-off-by: phernandez <paul@basicmachines.co>
With keep-both, a conflict copy's name can also be a new file in the plan. Both transfers are create-only, so with concurrent workers the one that finished first won the name and the conflict copy could be refused. Transfers are now grouped by destination, and each group runs in plan order on one worker, so the conflict copy claims the name as it did when transfers ran serially. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01APFUk2bjEwMptMQqpRhjea Signed-off-by: phernandez <paul@basicmachines.co>
…bodies An upload retry re-opened the source path, so a file replaced during a Retry-After wait could be sent under the original Content-Length and mtime. The file is now opened once; each attempt rewinds that handle, and size and mtime come from fstat on it. A streamed 3xx response was left unread, so describing the failure raised ResponseNotRead instead of naming the status. Every non-2xx response is now read before it is handed back. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01APFUk2bjEwMptMQqpRhjea Signed-off-by: phernandez <paul@basicmachines.co>
1fcdce3 to
df4b089
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: df4b0899d0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
1 participant
Fixes #1604.
Deploy order: this depends on basicmachines-co/basic-memory-cloud#2400 being deployed first. Without it, the service answers the new
Depth: infinitylisting one level deep. When a project has subdirectories, the client detects that and stops with an explicit error. It does not fall back to the per-directory walk.Why
Team
bm cloud push/pulltook minutes on real projects. A 680-file project needed 103 serial PROPFINDs at about 0.9 s each just to list, and then each file moved one at a time. Per the agreed plan on the issue, Team members keep push/pull, the WebDAV path gets faster, and no code is deleted.Change
list_project_filessends onePROPFINDwithDepth: infinityfor the project. Each file's project-relative path comes from its percent-encoded href. The display name is now only a consistency check: a mismatch raises.DAV:propfind-finite-depth(the cloud's cap) becomes a clear error naming the cause.asyncio.TaskGroupof at mostTRANSFER_CONCURRENCY = 8workers that share one iterator, so the task count stays bounded for any plan size.WebdavErrornaming each failed file and the count transferred before the stop.OSErrorstill propagates as itself, chained to the group._check_plan,--on-conflict, create-only writes (If-None-Match: *, exclusive local create), never-delete semantics, the push-side re-list, and the fix(cli): wait out rate limits during a WebDAV transfer #1532 rate-limit retries.Verification
just fast-checkis clean (ruff, format, ty).BASIC_MEMORY_TESTMON_SELECT_FLAGS="--import-mode=importlib" just fast-test tests/cli/cloud/passed (184).Depth: infinityis sent, and nested paths are parsed from hrefs, including percent-encoded ones and a directory named like its parent;bm cloud pullagainst a one-level server exits 1 with the explicit message after sending oneDepth: infinityrequest.CLI help text is unchanged, so
just man-regenis not needed. CHANGELOG is untouched.🤖 Generated with Claude Code
https://claude.ai/code/session_01APFUk2bjEwMptMQqpRhjea