Release v1.15.4 from BuddhiLW/main

.dev/wasm_percall_findings.md

@@ -66,6 +66,62 @@ resolution inside `invoke` (zwasm/D-585) and the non-initial-thread stack
 query (zwasm/D-584), both upstream. `bench/wasm_percall.sh` is the standing
 measurement of this axis from here on.
 
+### After the zwasm/D-585 fix (measured 2026-10-22)
+
+`JitInstance` resolved every by-name invoke with `findExportFunc`, a walk of
+the section list plus an export-section decode, and the facade's `invokeJit`
+did it twice per call (signature, then invoke). zwasm `22a99bb1`
+(`fix/jit-percall` on the fork) serves the name from `CompiledWasm.exports`,
+the table compile already builds. The `big-*` cells of `bench/wasm_percall.clj`
+call a trivial export of a 47 KB module (`bench/wasm/percall/zigcool.wasm`),
+which makes a size-proportional cost visible. Median of 7 trials x 20k calls,
+ReleaseSafe `-Dwasm`, same host, load average 11-15, the before and after
+binaries run back to back:
+
+| cell              | before (d74e63b4) | after (22a99bb1) |
+|-------------------|-------------------|------------------|
+| JIT, 47 KB module | 5883              | 833              |
+| JIT, `add.wasm`   | 1031              | 882              |
+| interp, 47 KB     | 463               | 600              |
+| interp, `add.wasm`| 496               | 649              |
+
+The JIT no longer pays by module size: the 47 KB cell now equals the tiny
+one. The interpreter rows did not change in kind (its lookup was always a
+table loop); their drift is load noise. `perf` was unavailable on the host
+(`perf_event_paranoid`), so the attribution rests on the A/B: the fix changes
+only the export lookup. The JIT stays above the interpreter on a trivial
+callee; what remains is the entry thunk and zwasm/D-584's per-call stack
+query, which ADR-0195 only shrinks.
+
+### On the merged fork pin (measured 2026-10-22, zwasm `e9d74934`)
+
+The same fix as merged on the fork's `main` (fork PRs #2 and #3), full
+`bench/wasm_percall.clj`, median of 7 x 20k calls, ReleaseSafe `-Dwasm`, load
+average 16. With the interp thread-trap fix in the pin, `interp-worker` no
+longer traps, so the whole bench runs.
+
+| cell        | ns/call | | cell          | ns/call |
+|-------------|---------|-|---------------|---------|
+| big-jit     | 736     | | jit-main      | 614     |
+| big-interp  | 527     | | interp-main   | 479     |
+| big-auto    | 742     | | fn-call       | 179     |
+
+`:jit` on the 47 KB module is 1.4x `:interp` (the acceptance bound is 2x) and
+within noise of `:jit` on `add.wasm`.
+
+`perf record -g` on a loop of 400k `(wasm/call h "popcount" 255)` on the 47 KB
+module (`perf_event_paranoid` lowered to 1), stripped ReleaseSafe binaries:
+
+- v1.15.3 (zwasm `d74e63b4`), 7.2 to 9.3 us a call: 88% of samples in cljw,
+  about 65% in a single ~40-byte loop (`0x13e6910..0x13e6935`), entered from
+  two distinct call sites per `wasm/call` (7.7% and 4.9% of the hottest
+  address). That is the export-section walk, run twice per facade call
+  (signature, then invoke), as zwasm/D-585 describes.
+- this branch (zwasm `e9d74934`), 0.7 to 0.9 us a call, the same as
+  `add.wasm`: no hot loop. What remains is spread out: `pthread_getattr_np` and
+  `pthread_getaffinity_np` (zwasm/D-584's per-call stack query, about 6%) and
+  malloc/calloc/free (about 8%), none of which depends on the module.
+
 ## Two independent causes, both upstream, both already documented there
 
 zwasm ADR-0209 (`.dev/decisions/0209_percall_latency_bench.md` in the zwasm

.gitattributes

@@ -8,3 +8,7 @@
 
 # Keep read-only historical / generated content out of GitHub's language stats.
 docs/ja/archive/** linguist-documentation
+# Append-only ledgers: parallel branches each add rows at the end, so a union
+# merge is the correct resolution (the YAML and ledger gates still check it).
+.dev/optimizations.md merge=union
+.dev/debt.yaml merge=union

.github/workflows/bench-vs-bb.yml

@@ -0,0 +1,138 @@
+# bench-vs-bb: cljw vs babashka vs JVM Clojure on bench/vs_bb/workloads.
+#
+# Kanban 20261022012233-420c9fea, step 0 of decision 20261022012219-31249feb
+# (make cljw faster than bb): every perf step is judged by this table.
+#
+# This is a REPORT, not a gate. CI has exactly ONE gate configuration
+# (ci.yml -> scripts/ci_gate.sh -> test/cljw-test --only full, enforced by
+# scripts/check_gate_parity.sh); this workflow adds no tier to it. It never
+# fails a PR on a ratio: the job is continue-on-error and run.sh exits 0 on a
+# slow or mismatching workload. Its output is the step summary (the table plus
+# the YAML history row for this sha), so the ratio for any commit is on its
+# run page. A hosted runner is a noisy, shared machine: compare ratios
+# (cljw/bb on the same runner, same minute), not absolute milliseconds.
+#
+# Linux x86_64 only: the ratio is the signal, one host is enough, and the mac
+# leg would double the cost of a report.
+
+name: bench-vs-bb
+
+on:
+  pull_request:
+    paths-ignore: ['**/*.md', '.dev/**']
+  push:
+    branches: [main]
+    paths-ignore: ['**/*.md', '.dev/**']
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+concurrency:
+  group: bench-vs-bb-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  bench:
+    name: cljw vs bb vs JVM (report, non-blocking)
+    runs-on: ubuntu-22.04
+    timeout-minutes: 45
+    continue-on-error: true
+    steps:
+      - name: Checkout
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        with:
+          ref: ${{ github.sha }}
+
+      - name: Cache Zig 0.16.0
+        id: cache-zig
+        uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
+        with:
+          path: ~/.local/zig-0.16.0
+          key: zig-0.16.0-x86_64-linux
+
+      - name: Install Zig 0.16.0
+        if: steps.cache-zig.outputs.cache-hit != 'true'
+        shell: bash
+        run: |
+          set -euo pipefail
+          source .github/versions.lock
+          mkdir -p ~/.local
+          curl -fsSL "https://ziglang.org/download/${ZIG_VERSION}/zig-x86_64-linux-0.16.0.tar.xz" -o /tmp/zig.tar.xz
+          rm -rf "$HOME/.local/zig-x86_64-linux-0.16.0" "$HOME/.local/zig-0.16.0"
+          tar -xJf /tmp/zig.tar.xz -C ~/.local
+          mv "$HOME/.local/zig-x86_64-linux-0.16.0" "$HOME/.local/zig-0.16.0"
+
+      - name: Add Zig to PATH
+        shell: bash
+        run: |
+          echo "$HOME/.local/zig-0.16.0" >> "$GITHUB_PATH"
+          echo "ZIG_GLOBAL_CACHE_DIR=$HOME/.cache/zig" >> "$GITHUB_ENV"
+
+      # Read-only restore of the gate's Linux build cache (ci.yml saves it).
+      # Content-addressed, so a stale restore can only cost a rebuild.
+      - name: Restore Zig deps + build (read-only)
+        uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
+        with:
+          path: |
+            ~/.cache/zig
+            .zig-cache
+          key: x86_64-linux-zigbuild-${{ hashFiles('build.zig.zon', 'build.zig') }}-${{ github.run_id }}
+          restore-keys: |
+            x86_64-linux-zigbuild-${{ hashFiles('build.zig.zon', 'build.zig') }}-
+            x86_64-linux-zigbuild-
+
+      - name: Install Babashka (pinned, checksummed)
+        shell: bash
+        run: |
+          set -euo pipefail
+          source .github/versions.lock
+          archive="/tmp/babashka-${BABASHKA_VERSION}.tar.gz"
+          curl -fsSL "https://github.com/babashka/babashka/releases/download/v${BABASHKA_VERSION}/babashka-${BABASHKA_VERSION}-linux-amd64.tar.gz" -o "$archive"
+          actual=$(sha256sum "$archive" | awk '{print $1}')
+          [ "$actual" = "$BABASHKA_LINUX_AMD64_SHA256" ] || { echo "::error::Babashka checksum mismatch"; exit 1; }
+          mkdir -p "$HOME/.local/bin"
+          tar -xzf "$archive" -C "$HOME/.local/bin" bb
+          echo "$HOME/.local/bin" >> "$GITHUB_PATH"
+
+      # JVM column without the clojure CLI: the runner image ships a JDK, the
+      # jars come straight from Maven Central.
+      - name: Fetch JVM Clojure jars
+        shell: bash
+        run: |
+          set -euo pipefail
+          m=https://repo1.maven.org/maven2/org/clojure
+          d="$HOME/.cache/vs_bb_jars"; mkdir -p "$d"
+          for j in clojure/1.12.4/clojure-1.12.4 spec.alpha/0.5.238/spec.alpha-0.5.238 \
+                   core.specs.alpha/0.4.74/core.specs.alpha-0.4.74 data.json/2.5.1/data.json-2.5.1; do
+            curl -fsSL "$m/$j.jar" -o "$d/$(basename "$j").jar"
+          done
+          echo "CLJW_BENCH_JVM_CP=$(ls "$d"/*.jar | paste -sd:)" >> "$GITHUB_ENV"
+          java -version
+
+      - name: Build cljw ReleaseSafe -Dwasm
+        shell: bash
+        run: zig build -Dwasm -Doptimize=ReleaseSafe -p "$RUNNER_TEMP/cljw-bench"
+
+      - name: Run bench/vs_bb
+        shell: bash
+        run: |
+          set -uo pipefail
+          hist="$RUNNER_TEMP/vs_bb_row.yaml"
+          bash bench/vs_bb/run.sh --cljw "$RUNNER_TEMP/cljw-bench/bin/cljw" -n 3 --history "$hist" | tee "$RUNNER_TEMP/vs_bb_table.txt"
+          {
+            echo "## cljw vs bb vs JVM (${GITHUB_SHA::8})"
+            echo
+            echo "Non-blocking report. Ratios < 1.00 mean cljw is faster. Hosted runner: compare ratios, not ms."
+            echo
+            echo '```'
+            cat "$RUNNER_TEMP/vs_bb_table.txt"
+            echo '```'
+            echo
+            echo "<details><summary>YAML history row (append to bench/vs_bb/history.yaml)</summary>"
+            echo
+            echo '```yaml'
+            cat "$hist"
+            echo '```'
+            echo "</details>"
+          } >> "$GITHUB_STEP_SUMMARY"

.gitignore

@@ -59,3 +59,6 @@ __pycache__/
 # built on demand from jsonsum.go (docs/examples/polyglot/README.md), never
 # committed. The C, Zig and Rust guests are a few hundred bytes and are.
 docs/examples/polyglot/go/jsonsum.wasm
+
+# Per-worktree ling scratch and resume notes (never committed)
+.scratch/

CHANGELOG.md

@@ -7,6 +7,87 @@ first stable `1.0.0` tag; pre-1.0 `alpha` / `rc` tags may still change surfaces.
 
 ## [Unreleased]
 
+## [1.15.4] - 2026-10-23
+
+### Added
+
+- **`java.time.format`: `DateTimeFormatter`, `DateTimeFormatterBuilder`,
+  `ChronoField`, `ZoneId` and `ZoneOffset`.** Pattern letters
+  `G u y M L d D F E a h K k H m s S n A N V X x Z O`, optional sections,
+  SMART and STRICT resolvers and the 12 `ISO_*` constants. Letters cljw does
+  not implement raise `not_implemented` instead of formatting wrong text. This
+  was the last blocker for `malli.transform`.
+- **Primitive arrays cross `cljw.ffi` and wasm memory at memcpy speed
+  (ADR-0204).** `byte`/`short`/`int`/`long`/`float`/`double`/`char`/`boolean`
+  arrays hold raw native elements. A `:pointer` argument passes the array's
+  storage with no copy, `:bytes` is one memcpy, `wasm/mem-write!` accepts a
+  primitive array and the new `wasm/mem-read-array` returns one. `bytes?` is
+  true only for a byte array.
+- **`bench/vs_bb`:** 14 workloads that compare cljw with bb and JVM Clojure,
+  with a non-blocking CI report.
+- **`docs/examples/zig-native`:** one Zig source used both as a wasm guest and
+  as a `cljw.ffi` library.
+
+### Changed
+
+- **Core sequence functions are much faster (ADR-0205).** The seq of a vector
+  is now chunked by leaf, as clj's `PersistentVector$ChunkedSeq` is, and
+  `map`, `filter`, `into`, `mapv` and `reduce` go through one chunked-seq
+  boundary and the Zig `reduce`. On an 8000-element vector,
+  `(reduce + (map inc v))` went from 19271 us to 718 us and
+  `(count (filter even? v))` from 19646 us to 490 us. `(first (map f v))` now
+  realizes 32 elements, as clj does.
+- **Protocol calls hit a per-function dispatch cache (ADR-0206).** Each
+  protocol function remembers the method it resolved for the last receiver
+  type; every `extend`, `extend-type`, `extend-protocol` and type redefinition
+  invalidates it.
+- **A Clojure fn call no longer memsets its scratch buffers.** ReleaseSafe
+  filled about 2 KB of locals with 0xAA on every call. Skipping it cuts 18%
+  of instructions from `seq_dot`, 38% from `lazy_chain`, 35% from multimethod
+  dispatch and 10% from startup.
+- **String and vector hot paths:** `clojure.string/join`, `upper-case`,
+  `lower-case` and `split`, `subvec`, `str`, keyword interning and
+  `dorun`/`doall` take native fast paths. The `string_ops` bench went from
+  9.3x bb to 2.3x bb.
+- **Cold start does less work.** Deserializing the core image no longer maps
+  a page per quoted list. `cljw -e nil` takes 27.3 ms of CPU, against 38.8 ms
+  before and 29.1 ms for bb.
+- **The GC returns freed memory and reuses it.** Small cells live in 64 KiB
+  pages that are unmapped once empty, and String, array, transient-vector and
+  BigInt payload bytes now count toward the collect trigger, so long-lived
+  processes no longer hold their high-water mark.
+
+### Fixed
+
+- **`clojure.edn` reads the EDN dialect.** It used the permissive Clojure
+  reader and accepted EDN-invalid input; it now matches JVM Clojure's
+  `EdnReader` on 59 of 59 cases.
+- **wasm `i64` arguments and results cross exactly over the full range.**
+  Values past 2^47 came back as a lossy Double or were refused, and an
+  integral float of 2^63 crashed the host.
+- **A BigInt, Ratio or BigDecimal past about 1233 digits prints its digits.**
+  It printed the literal `(BigInt)`.
+- **A long-running server that reads stdin now gives memory back.** The
+  `*in*` and `System/in` readers kept every byte they had ever read; they now
+  drop what has been consumed. While the main thread waits on stdin, each idle
+  second runs a collect (when at least 1 MiB was allocated since the last one)
+  and returns freed memory to the OS, so garbage left by `future`s and worker
+  threads no longer waits for the next request. Socket and `StringBuilder`
+  buffers now count toward the collect trigger. A bb-mcp-shaped probe (20
+  `future`s each building 4 MB of JSON, then idle) settles at 129 MB instead
+  of holding 955 MB.
+
+- **A `wasm/call` on the JIT no longer costs time proportional to the
+  module.** zwasm resolved the export by re-parsing the module on every call;
+  it now reads the export table built at compile. A trivial export of a 47 KB
+  module went from about 5.9 us to 0.8 us a call, the same as a tiny module.
+  `bench/wasm_percall.clj` gains `big-*` cells that measure it.
+- **A wasm handle called from a `future` no longer traps on the interpreter.**
+  zwasm's interpreter cached the first calling thread's native-stack limit on
+  the instance, so a handle warmed on the runtime thread and then called from
+  another thread whose stack lay below that limit trapped
+  `CallStackExhausted`. The limit is now per calling thread (zwasm 79f3d63a).
+
 ## [1.15.3] - 2026-10-22
 
 ### Added

bench/ffi_prim_arrays.clj

@@ -0,0 +1,44 @@
+;; SPDX-License-Identifier: EPL-2.0
+;; Kanban CLJW-FFI-PRIM-ARRAY-BYTES (ADR-0204): what a 1000-element f64 dot
+;; costs through cljw.ffi, call only versus building its arguments.
+;;
+;;   cc -shared -fPIC -O2 -o /tmp/libdot.so test/e2e/fixtures/ffi/ffi_fixture.c
+;;   cljw bench/ffi_prim_arrays.clj /tmp/libdot.so
+;;
+;; Prints microseconds per call, best of 5 runs. `:pointer` passes a
+;; double-array's storage (only where arrays are typed); `:bytes` packs the
+;; doubles into a byte array first, the only route before ADR-0204.
+(require '[cljw.ffi :as ffi])
+
+(def lib (ffi/open (first *command-line-args*)))
+(def n 1000)
+(def xs (vec (range n)))
+
+(defn- best-us [iters f]
+  (apply min (for [_ (range 5)]
+               (let [t0 (System/nanoTime)]
+                 (dotimes [_ iters] (f))
+                 (/ (- (System/nanoTime) t0) 1000.0 iters)))))
+
+(defn- f64s->bytes [ds]
+  (byte-array
+   (mapcat (fn [d] (let [b (Double/doubleToRawLongBits d)]
+                     (for [i (range 8)] (unchecked-byte (bit-shift-right b (* 8 i))))))
+           ds)))
+
+(def dot-bytes (ffi/function lib "dot" [:bytes :bytes :long] :double))
+(let [b (f64s->bytes xs)]
+  (println "bytes-call-only-us" (best-us 2000 #(dot-bytes b b n)))
+  (println "bytes-pack-and-call-us" (best-us 5 #(let [b (f64s->bytes xs)] (dot-bytes b b n)))))
+
+(when (try (ffi/function lib "dot" [:pointer :pointer :long] :double)
+           ((ffi/function lib "dot" [:pointer :pointer :long] :double) (double-array 1) (double-array 1) 1)
+           true
+           (catch Exception _ false))
+  (let [dot (ffi/function lib "dot" [:pointer :pointer :long] :double)
+        a (double-array xs)]
+    (println "pointer-call-only-us" (best-us 2000 #(dot a a n)))
+    (println "pointer-build-and-call-us" (best-us 2000 #(let [a (double-array xs)] (dot a a n))))
+    (println "check" (dot a a n) (reduce + (map * xs xs)))))
+
+(println "clojure-dot-us" (best-us 20 #(reduce + (map * xs xs))))

bench/gc_heap_return.clj

@@ -0,0 +1,18 @@
+;; CLJW-GC-HEAP-RETURN acceptance probe. A bench, not a test: RSS is
+;; host-dependent. Steady-state bursts of one size must not grow the heap
+;; after the first, and a major collection must hand freed memory back to
+;; the OS. Run: cljw bench/gc_heap_return.clj
+(require (quote clojure.data.json) (quote clojure.string))
+(defn rss-mb [] (let [l (first (filter #(.startsWith ^String % "VmRSS") (clojure.string/split-lines (slurp "/proc/self/status"))))]
+                  (quot (Long/parseLong (re-find #"\d+" l)) 1024)))
+(println "start" (rss-mb) "MB")
+(def big (atom (vec (range 3000000))))
+(println "after 3M vector" (rss-mb) "MB")
+(reset! big nil)
+(System/gc) (System/gc)
+(println "after drop + System/gc" (rss-mb) "MB")
+(dotimes [_ 5] (count (vec (range 3000000))))
+(println "after 5 more 3M bursts" (rss-mb) "MB")
+(dotimes [i 200] (count (clojure.data.json/write-str (vec (range 20000)))))
+(System/gc)
+(println "after 200 x 20k JSON writes" (rss-mb) "MB")