Skip to content

fs: write files in one thread pool round trip - #65489

Open
codebytere wants to merge 1 commit into
nodejs:mainfrom
codebytere:perf/fs-writefile-one-roundtrip
Open

fs: write files in one thread pool round trip#65489
codebytere wants to merge 1 commit into
nodejs:mainfrom
codebytere:perf/fs-writefile-one-roundtrip

Conversation

@codebytere

Copy link
Copy Markdown
Member

Makes fs.writeFile(path, data) and fsPromises.writeFile(path, data) do open + write + close as one thread pool request (WriteFileJob, next to the ReadFileJob from #65327) instead of one request per step.

benchmark/fs/writefile-promises.js (buf, duration=2), 20 runs, this branch vs main; fs.writeFile() measured the same way with an equivalent callback loop:

concurrent=1 concurrent=10
fsPromises.writeFile, 2 B +108 % +51 %
fsPromises.writeFile, 1 KiB +108 % +51 %
fsPromises.writeFile, 64 KiB +76 % +30 %
fs.writeFile, 1 KiB +26 % +38 %

(1 MiB stays on the chunked FileHandle path and is unchanged.)

fs.writeFile() took three libuv round trips (open, write, close), each its own request with its own queue wait, completion callback and JS/C++ crossing, and fsPromises.writeFile() did the same through a FileHandle. For the small files applications write most (settings, state, caches) the round trips are the cost, and each one occupies a pool slot that concurrent fs, dns.lookup() and crypto work is also queueing for.

WriteFileJob (an AsyncWrap + ThreadPoolWork) opens, writes the whole buffer (looping on short writes) and closes as one pool task, keeping the buffer alive until it is done. fs.writeFile() uses it for path arguments without flush; fsPromises.writeFile() additionally keeps data above one write chunk (512 KiB) and iterables on the FileHandle path, so large writes stay abortable between chunks as before. File descriptors, FileHandles, flush: true and an active VFS keep their existing paths. Open failures report syscall open with the path and write failures write, permission errors arrive through the callback/promise, an abort signalled while the write is in flight is still reported as an AbortError, and the job is an FSREQCALLBACK async resource that emits the write fs trace event.

Refs: #65327

Tests: new test-fs-writefile-one-roundtrip.js (flags a/wx, mode, failing syscall and path, every ArrayBufferView kind, a multi-megabyte buffer, both APIs); tests that used fs.writeFile() as a proxy for open/close trace events or injected FileHandle faults into path-based writes are adjusted to keep testing what they test; test-fs-*write*, *append*, test-fs-promises*, permission and async-hooks fs tests pass.


Disclosure: the code, test, measurements and this description were written by Claude Code, directed and reviewed by @codebytere.

@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. fs Issues and PRs related to the fs subsystem / file system. needs-ci PRs that need a full CI run. labels Aug 22, 2026
@codecov

codecov Bot commented Aug 22, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 85.89744% with 22 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.14%. Comparing base (21f0f27) to head (4c3fcd6).
⚠️ Report is 36 commits behind head on main.

Files with missing lines Patch % Lines
src/node_file.cc 82.52% 4 Missing and 14 partials ⚠️
lib/internal/fs/promises.js 88.57% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #65489      +/-   ##
==========================================
+ Coverage   90.12%   90.14%   +0.02%     
==========================================
  Files         752      751       -1     
  Lines      252315   252677     +362     
  Branches    47444    47540      +96     
==========================================
+ Hits       227395   227781     +386     
+ Misses      16217    16186      -31     
- Partials     8703     8710       +7     
Files with missing lines Coverage Δ
lib/fs.js 98.35% <100.00%> (-0.05%) ⬇️
lib/internal/fs/promises.js 92.42% <88.57%> (-0.11%) ⬇️
src/node_file.cc 74.33% <82.52%> (+0.36%) ⬆️

... and 62 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

fs.writeFile(path, data) took three libuv thread pool round trips
(open, write, close), each its own request with its own queue wait,
completion callback and JS/C++ crossing, and fs.promises.writeFile()
did the same through a FileHandle. For the small files applications
write most, the round trips are the cost, and each occupies a pool slot
that concurrent fs, dns.lookup() and crypto work is also queueing for.

Add WriteFileJob next to ReadFileJob: an AsyncWrap + ThreadPoolWork
that opens, writes the whole buffer (looping on short writes) and
closes as one pool task, keeping the buffer alive until it is done.
fs.writeFile() uses it for path arguments without flush;
fs.promises.writeFile() additionally keeps data above one write chunk
(and iterables) on the FileHandle path, so large writes stay abortable
between chunks as before. File descriptors, FileHandles, flush: true
and an active VFS keep their existing paths.

Behavior is otherwise kept: open failures report syscall 'open' with
the path, write failures 'write'; permission errors are delivered
through the callback/promise; an abort signalled while the write is in
flight is still reported as an AbortError; the job is an FSREQCALLBACK
resource for async_hooks and emits the 'write' fs trace event.

Tests that used fs.writeFile() as a proxy for open/close trace events,
or injected FileHandle faults for path-based writes, are adjusted to
keep testing what they test.

Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
@codebytere
codebytere force-pushed the perf/fs-writefile-one-roundtrip branch from 9fb687f to 4c3fcd6 Compare August 22, 2026 18:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++ Issues and PRs that require attention from people who are familiar with C++. fs Issues and PRs related to the fs subsystem / file system. needs-ci PRs that need a full CI run.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants