Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
737c7c6
stream: keep long-lived stream/iter objects in fast mode
jasnell Oct 4, 2026
1bdd59b
stream: make endSync() optional in pipeToSync()
jasnell Oct 4, 2026
c72ebe0
stream: create stream/iter iterator results with a constructor
jasnell Oct 4, 2026
57dbd6f
stream: avoid per-write allocations in stream/iter writers
jasnell Oct 4, 2026
bc6aa22
stream: construct stream/iter byte view snapshots and batch entries
jasnell Oct 4, 2026
115ddac
stream: construct stream/iter wait and merge records
jasnell Oct 4, 2026
4492420
stream: construct the options passed to stream/iter transforms
jasnell Oct 4, 2026
cdae3e1
stream: do not miss 'drain' in fromWritable()
jasnell Oct 4, 2026
c3803d6
stream: share the once option for stream/iter abort listeners
jasnell Oct 4, 2026
15c62f0
stream: make stream/iter from() cancellation waits cheaper
jasnell Oct 4, 2026
17ae0b0
stream: yield bounded batches directly in stream/iter from()
jasnell Oct 4, 2026
d0da1f7
stream: avoid batch entries for single-chunk pipeTo() writes
jasnell Oct 4, 2026
773f425
stream: wait without an async function in stream/iter from()
jasnell Oct 4, 2026
d2d8139
stream: normalize async sources without a generator in from()
jasnell Oct 4, 2026
fdc4c02
stream: normalize sync sources without an async generator in from()
jasnell Oct 4, 2026
2fbcad3
stream: run stream/iter transforms without async generators
jasnell Oct 4, 2026
406ac2f
stream: read stream/iter sources with one abort listener
jasnell Oct 4, 2026
978ca82
stream: remove async generator layers from stream/iter pull()
jasnell Oct 4, 2026
15fba76
stream: avoid quadratic batching in zlib/iter transforms
jasnell Oct 4, 2026
a9fc11f
stream: use a RingBuffer for stream/iter merge()'s ready queue
jasnell Oct 4, 2026
883571e
stream: use a RingBuffer for stream/iter broadcast pending reads
jasnell Oct 4, 2026
ee5aa7f
stream: use a RingBuffer for the stream/iter operation queue
jasnell Oct 4, 2026
d510b07
stream: append to arrays with indexed stores in stream/iter hot paths
jasnell Oct 4, 2026
f7be69d
stream: read stream/iter push() readables without normalizing
jasnell Oct 4, 2026
8e2dba6
stream: avoid allocations on every stream/iter push() write
jasnell Oct 4, 2026
e58a927
stream: snapshot common stream/iter byte views in fewer fields
jasnell Oct 4, 2026
639d13a
stream: resolve stream/iter push() return() with its value
jasnell Oct 4, 2026
2b4f9ca
doc: document stream/iter iterator results and from() identity
jasnell Oct 4, 2026
6b43ce4
stream: check stream/iter pipe writes more cheaply
jasnell Oct 4, 2026
c84710a
stream: read sync sources synchronously in stream/iter pipeTo()
jasnell Oct 4, 2026
e552389
stream: read async sources more cheaply in stream/iter pipeTo()
jasnell Oct 4, 2026
468fed1
stream: do not retain a promise per read in stream/iter share()
jasnell Oct 7, 2026
564ad60
stream: return stream/iter from() results unchanged from from()
jasnell Oct 7, 2026
626786e
stream: create stream/iter from() values without async generators
jasnell Oct 7, 2026
9178f77
stream: create stream/iter transform signals and options lazily
jasnell Oct 7, 2026
d200eda
stream: handle stream/iter pipeline aborts in one place
jasnell Oct 7, 2026
0ccea7a
stream: keep stream/iter pipeTo() fast paths with a signal
jasnell Oct 7, 2026
a9e4a40
stream: read stream/iter consumer sources without a layer per signal
jasnell Oct 7, 2026
2909ce6
stream: read stream/iter from() async sources in one reaction
jasnell Oct 7, 2026
c3a0e47
stream: check stream/iter byte views by their byteLength alone
jasnell Oct 7, 2026
bd5e9e6
stream: write to stream/iter writers with only write() inline
jasnell Oct 7, 2026
0675a04
stream: encode small stream/iter strings into a pool
jasnell Oct 7, 2026
e8619cf
stream: read stream/iter share() consumers without a chain per read
jasnell Oct 7, 2026
c7fb80b
stream: apply stream/iter stateful transforms without async generators
jasnell Oct 7, 2026
7e718b6
stream: read stream/iter from() sync sources without a generator
jasnell Oct 7, 2026
d6e4869
stream: read stream/iter share() sync sources at once
jasnell Oct 7, 2026
3b21d30
stream: pump stream/iter Broadcast.from() sources with fewer layers
jasnell Oct 7, 2026
7942cc6
stream: create stream/iter from() value streams as class instances
jasnell Oct 7, 2026
7526d3d
stream: normalize stream/iter fromSync() inputs without generators
jasnell Oct 7, 2026
ec89237
stream: read a single stream/iter merge() source without a generator
jasnell Oct 7, 2026
6e43cb1
stream: do not keep stream/iter from() sync sources busy for a tick
jasnell Oct 7, 2026
8af1485
stream: create stream/iter pending promises without a result object
jasnell Oct 7, 2026
6108877
stream: keep stream/iter broadcast waiters in reused lists
jasnell Oct 7, 2026
0fb5beb
stream: create the stream/iter share() read reaction once per consumer
jasnell Oct 7, 2026
79f7b0c
stream: read stream/iter from() sources in pipelines without a promis…
jasnell Oct 7, 2026
a4627d9
stream: buffer single-chunk stream/iter push() writes without a batch…
jasnell Oct 7, 2026
be801cd
stream: buffer single-chunk stream/iter broadcast() and share() batch…
jasnell Oct 7, 2026
00f64c6
stream: release only stream/iter reads that can be cancelled while pe…
jasnell Oct 7, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 32 additions & 4 deletions doc/api/stream_iter.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,10 @@ are automatically UTF-8 encoded when passed to `from()`, `push()`, or
`pipeTo()`. This removes ambiguity around encodings and enables zero-copy
transfers between streams and native code.

Like [`Buffer.from()`][] for strings, small strings are encoded into a shared
pool: the resulting {Uint8Array} is a view of a larger {ArrayBuffer}, which
cannot be transferred.

### Batching

Each iteration yields a **batch** -- an {Array} of {Uint8Array} chunks
Expand All @@ -119,6 +123,10 @@ async function run() {
}
```

Some iterators of this module return iterator results (`{ done, value }`
objects) that do not inherit from `Object.prototype`. Code should only rely on
their `done` and `value` properties, as `for await...of` does.

### Transforms

Transforms come in two forms:
Expand All @@ -140,6 +148,15 @@ Both forms receive an `options` parameter with the following property:
can check `signal.aborted` or listen for the `'abort'` event to perform
early cleanup.

Each transform of a pipeline receives its own `options` object, the same one
for every call of a stateless transform, so a transform can modify its
`options` without affecting other transforms. The object does not inherit
from `Object.prototype`. The signal is created when `options.signal` is
first read, so a pipeline whose transforms never read it does not create one;
until then, `signal` is an accessor property. It is the same signal for every
transform of the pipeline. Transforms passed to [`pullSync()`][] receive no
`options`.

The flush signal (`null`) is sent after the source ends, giving transforms
a chance to emit trailing data (e.g., compression footers).

Expand Down Expand Up @@ -592,6 +609,10 @@ Objects implementing `Symbol.for('Stream.toAsyncStreamable')` or
precedence over the iteration protocols (`Symbol.asyncIterator`,
`Symbol.iterator`).

The readable of a [`push()`][] stream without transforms, the iterables
returned by [`fromReadable()`][], and the results of `from()` itself already
yield normalized batches, so `from()` returns them unchanged.

```mjs
import { Buffer } from 'node:buffer';
import { from, text } from 'node:stream/iter';
Expand Down Expand Up @@ -657,7 +678,9 @@ added:
* `writer` {Object} Destination with `write(chunk)` method.
* `options` {Object}
* `signal` {AbortSignal} Abort the pipeline. Aborting fails the destination
writer unless `preventFail` is `true`.
writer unless `preventFail` is `true`. The signal is passed to the
writer's `write()`, `writev()` and `end()` in an options object, the same
object for every call.
* `preventClose` {boolean} If `true`, do not call `writer.end()` when
the source ends. **Default:** `false`.
* `preventFail` {boolean} If `true`, do not call `writer.fail()` on
Expand Down Expand Up @@ -714,7 +737,7 @@ added:

* `source` {Iterable} The sync data source.
* `...transforms` {Function|Object} Zero or more sync transforms.
* `writer` {Object} Destination with `write(chunk)` method.
* `writer` {Object} Destination with a `writeSync(chunk)` method.
* `options` {Object}
* `failOnIncompleteClose` {boolean} If `true`, call `writer.fail()` when
`writer.endSync()` cannot close the writer synchronously. Ignored when
Expand All @@ -727,8 +750,11 @@ added:
Synchronous version of [`pipeTo()`][]. The `source`, all transforms, and the
`writer` must be synchronous. Cannot accept async iterables or promises.

The `writer` must have the `*Sync` methods (`writeSync`, `writevSync`,
`endSync`) and `fail()` for this to work.
The `writer` must have a `writeSync()` method. The other methods are
optional: `writevSync()` is used for batches of more than one chunk if it is
present, `endSync()` is called to close the writer (unless `preventClose` is
`true`), and `fail()` is called if the pipe fails (unless `preventFail` is
`true`). A writer without `endSync()` is not closed.

`pipeToSync()` never falls back to the asynchronous writer methods. If
`writer.endSync()` returns `-1` because the writer cannot close synchronously
Expand Down Expand Up @@ -2391,6 +2417,7 @@ console.log(textSync(stream)); // 'hello world'
[Iterable Streams API]: https://iter-streams.proposal.wintertc.org/
[`--experimental-stream-iter`]: cli.md#--experimental-stream-iter
[`Broadcast.from()`]: #broadcastfrominput-options
[`Buffer.from()`]: buffer.md#static-method-bufferfromstring-encoding
[`Share.from()`]: #static-method-sharefrominput-options
[`SyncShare.fromSync()`]: #static-method-syncsharefromsyncinput-options
[`array()`]: #arraysource-options
Expand All @@ -2406,6 +2433,7 @@ console.log(textSync(stream)); // 'hello world'
[`pipeTo()`]: #pipetosource-transforms-writer-options
[`pull()`]: #pullsource-transforms-options
[`pullSync()`]: #pullsyncsource-transforms
[`push()`]: #pushtransforms-options
[`share()`]: #sharesource-options
[`stream.Readable`]: stream.md#class-streamreadable
[`stream.Writable`]: stream.md#class-streamwritable
Expand Down
Loading