diff --git a/.dev/accepted_divergences.yaml b/.dev/accepted_divergences.yaml index 1b41b63ec..c0b0c3c03 100644 --- a/.dev/accepted_divergences.yaml +++ b/.dev/accepted_divergences.yaml @@ -152,7 +152,7 @@ accepted: `(name (ns-name *ns*))` are clj-faithful ("user"); only the readable `#object[…]` form (pr / prn / print / println) diverges. derives_from: "ADR-0059 (no-JVM: no class FQN); AD-002 (opaque-ref identity-hash not reproducible); ADR-0083" - pin: "test/e2e/phase15_namespace.sh — `(str *ns*)` => user + `(pr-str *ns*)` => #object[Namespace \"user\"] (the cljw form locked)" + pin: "test/clj/suites/vars_namespaces_test.clj — `(str *ns*)` => user + `(pr-str *ns*)` => #object[Namespace \"user\"] (the cljw form locked)" - id: "AD-012" summary: "A PersistentQueue prints as a readable `#queue (…)` (cljw) instead of clj's opaque `#object[clojure.lang.PersistentQueue 0x… \"…@hash\"]`" example: '(pr-str (conj clojure.lang.PersistentQueue/EMPTY 1 2 3)) => cljw "#queue (1 2 3)" / clj "#object[clojure.lang.PersistentQueue 0x… \"clojure.lang.PersistentQueue@…\"]"' @@ -184,7 +184,7 @@ accepted: the internal algorithm divergence ADR-0090 reserved, with a pin asserting the observable serializability contract the retry-only engine upholds. derives_from: "ADR-0090 §3 (MVCC retry-only LockingTransaction) + §4 (no-barge)" - pin: "test/e2e/phase15_stm_dosync.sh — `dosync_concurrent_serializable` locks 4 threads × 100 `(dosync (alter c inc))` => exactly 400 (the serializable, no-lost-update result the retry-only engine produces under contention)" + pin: "test/clj/suites/concurrency_host_test.clj — `dosync_concurrent_serializable` locks 4 threads × 100 `(dosync (alter c inc))` => exactly 400 (the serializable, no-lost-update result the retry-only engine produces under contention)" - id: "AD-015" summary: "An ESCAPED `with-local-vars` local var, deref'd AFTER its dynamic extent, returns `nil` in cljw, where JVM returns an unreproducible `#object[clojure.lang.Var$Unbound 0x…]` opaque." example: '(var-get (with-local-vars [x 5] x)) => cljw nil / clj #object[Var$Unbound 0x…]' @@ -203,7 +203,7 @@ accepted: local var is meaningless after its extent in both runtimes; cljw returns a safe `nil` instead of an unreproducible opaque. derives_from: "F-006 (Vars gpa-owned + GC-membrane-filtered; no Unbound sentinel) + ADR-0097 (anon local Var lifetime) + the opaque-ref non-reproducibility class" - pin: "test/e2e/phase15_with_local_vars.sh — `escape-nil` asserts `(var-get (with-local-vars [x 5] x))` => nil (cljw's safe divergent value, not JVM's Var$Unbound opaque)" + pin: "test/clj/suites/vars_namespaces_test.clj — `escape-nil` asserts `(var-get (with-local-vars [x 5] x))` => nil (cljw's safe divergent value, not JVM's Var$Unbound opaque)" - id: "AD-016" summary: "`(biginteger x)` yields cljw's BigInt — prints `5N` (clj `5`) and `(class (biginteger 5))` is `BigInt` (clj `java.math.BigInteger`). cljw has no separate BigInteger type." example: '(biginteger 5) => cljw 5N / clj 5 ; (class (biginteger 5)) => cljw BigInt / clj java.math.BigInteger' @@ -268,23 +268,12 @@ accepted: derives_from: "F-005 (numeric tower: single internal f64, one clean float quot/rem/mod path) + AD-008 (cljw's cleaner numeric surface where clj throws)" pin: "test/clj/suites/numeric_tower_test.clj — the `non-finite-operands` deftest: quot_nan/rem_nan/mod_nan_d/mod_nan_v (NaN family) + quot_inf/rem_inf/mod_inf/quot_ninf (+/-Inf-dividend family) lock cljw's non-finite result (clj throws). Each assertion carries its original case name as a trailing comment." - id: "AD-019" - summary: "Java arrays are type-erased: element type hints (`^\"[B\"`, `^ints`, `^bytes`) are advisory only, and a typed-array constructor / `aset` does not coerce or type-check non-byte/short/char elements." - example: '(let [^"[B" b (byte-array 1)] (aset b 0 65) (aget b 0)) => 65 (a non-byte stored into a byte-typed array, no error); (class (object-array 0)) => cljw "array" / clj "[Ljava.lang.Object;"' + summary: "Array type arguments are advisory: `make-array` / `into-array` ignore their class argument and build an object array, type hints (`^\"[B\"`, `^ints`) are ignored, and `aset-long` / `aset-int` / `aset-double` store into an object array where clj throws. The typed constructors (`long-array`, `byte-array`, ...) DO build typed native storage (ADR-0204)." + example: '(let [c (make-array Long 2 2)] (aset-int c 0 0 3) (aget c 0 0)) => cljw 3 / clj IllegalArgumentException ; (class (object-array 0)) => cljw "array" / clj "[Ljava.lang.Object;"' why: |- - cljw arrays are one type-erased `[]Value` (ADR-0105): `byte-array`, - `int-array`, `object-array`, `make-array` all produce the same `.array` - shape. F-004 (uniform 8-byte Value) + F-005 (no primitive specialization) - delete the JVM's runtime array component-type, which on the JVM drives - boxing, `aset-X` coercion, and the `[Ljava.lang.Object;` class name. So a - `^"[B"` hint parses (reader.zig normalises it to `{:tag …}`) but is - ignored, and `aset` into a "typed" array never raises a type mismatch. - VALUE fidelity is still preserved where it is observable and value-changing - (F-011): per-constructor init defaults (0 / 0.0 / false / \space / nil) and - byte/short/char modular wrap are applied at construction / `aset-byte` — - those are NOT divergences. The simple class name (`array`, not the JVM - array-class encoding) is the AD-003 no-JVM-class surface applied to arrays. - derives_from: "F-004 (uniform Value) + F-005 (no primitive specialization) + ADR-0059 / AD-003 (no JVM class) + ADR-0105" - pin: "test/clj/suites/arrays_test.clj — type_hint_advisory (a ^\"[B\" array accepts a non-byte) + array_class_simple (class is `array`)" + cljw has no Class objects (ADR-0059 / AD-003), so `Long`, `Integer/TYPE` and a `^"[B"` hint carry no element kind for `make-array` / `into-array` to read. Those build object arrays, which is what `(make-array Long n)` is on the JVM too (a boxed `Long[]`). clj then refuses `aset-long` on a boxed array because `Array.setLong` wants a primitive `long[]`; cljw has no primitive-versus-boxed class to key that refusal on, so the store succeeds. The typed constructors build typed arrays (byte i8 ... double f64, ADR-0204): their values, `aset` strictness, `bytes?`, `Arrays/equals` and `Arrays/sort` match clj. The simple class name (`array`) is the AD-003 no-JVM-class surface applied to arrays. + derives_from: "ADR-0059 / AD-003 (no JVM Class objects, so a class argument names no element kind) + ADR-0204 (typed array storage for the typed constructors)" + pin: "test/e2e/phase14_arity_parity.sh — `AD-019 (aset-long a i j v)` / `AD-019 (aset-double a i j v)` / `AD-019 (aset-int a i j v)` lock the store into a make-array object array; test/clj/suites/arrays_test.clj — array_class_simple (class is `array`)" - id: "AD-020" summary: "A java.util.Random (any host_instance) prints opaquely as `#` — clj prints `#object[java.util.Random 0x… …]` with a non-reproducible identity hash." example: '(pr-str (java.util.Random. 0)) => cljw "#" / clj "#object[java.util.Random 0x.. java.util.Random@..]"' @@ -327,7 +316,7 @@ accepted: provably-JVM-internal prefixes defer. F-011: explicit error preserved, only its timing diverges. derives_from: "ADR-0113 (deferred host-class refs) + ADR-0059 (no JVM class) + provisional_marker.md (explicit runtime error OK, silent no-op forbidden)" - pin: "test/e2e/phase15_deferred_host_ref.sh — defines_and_loads + raises_when_called + typo_stays_loud" + pin: "test/clj/suites/forms_and_literals_test.clj (p15-deferred-host-ref: defines_and_loads + typo_stays_loud) + test/e2e/phase15_deferred_host_ref.sh (raises_when_called, the uncatchable call-time raise)" - id: "AD-023" summary: "Extending a protocol TO a host_inert java interface (`(extend-protocol P java.util.Map …)`) is a LOAD-ONLY no-op in cljw: the section loads but never dispatches, because no cljw value has a java.util.Map host type (cljw maps are native PersistentArrayMap/HashMap, not java.util.Map). A map receiver falls to the protocol's Object default (or raises if none)." example: '(do (defprotocol P (m [x])) (extend-protocol P java.util.Map (m [x] :map) Object (m [x] :obj)) (m {:a 1})) => cljw :obj (Map impl inert, falls to Object); clj => :map (a Clojure map IS a java.util.Map).' @@ -456,7 +445,7 @@ accepted: + the dangling D-232 frame-accessor reservation — D-438 / ADR-0140 are the real owner.) derives_from: "ADR-0059 (no JVM Throwable / StackTraceElement) + AD-024 (user-frames-only trace) + AD-003 (simple class name) + ADR-0140 (cljw-shaped frame seq)" - pin: "test/e2e/phase15_clojure_stacktrace.sh — caught_prints_user_frame + caught_no_marker lock the cljw-native per-frame format; print_cause_trace + unthrown_no_frames lock the frame-less marker + clj-faithful cause chain" + pin: "test/clj/suites/test_reporters_test.clj — caught_prints_user_frame + caught_no_marker lock the cljw-native per-frame format; print_cause_trace + unthrown_no_frames lock the frame-less marker + clj-faithful cause chain" - id: "AD-030" summary: "`(format \"%c\" )` is rejected by cljw for EVERY integer, where clj accepts an Integer (`(format \"%c\" (int 65))` → \"A\") but rejects a Long (`(format \"%c\" 65)` → IllegalFormatConversionException). cljw collapses int/Long into ONE integer type (F-005), so it cannot reproduce clj's Integer-accepts / Long-rejects split and chooses to reject all integers (D-267 option (b)). `%c` with an actual char (`\\A` / `(char 65)`) is clj-parity in both." example: '(format "%c" (int 65)) => cljw type_error / clj "A" ; (format "%c" 65) => cljw type_error / clj IllegalFormatConversionException ; (format "%c" (char 65)) => both "A"' @@ -552,24 +541,22 @@ accepted: summary: "The 2-arg array ctors `(byte-array size n)` / `(short-array size n)` / `(char-array size n)` FILL with a number init; clj throws (`IllegalArgumentException`, 'Don't know how to create ISeq from: java.lang.Long') because the JVM `Numbers` overload for those element types has no number-fill branch." example: "(vec (byte-array 3 65)) => cljw [65 65 65] / clj THROWS ; (int (aget (char-array 3 65) 0)) => cljw 65 / clj THROWS ; matching cases agree: (vec (int-array 5 7)) => [7 7 7 7 7], (vec (byte-array 3 [1 2])) => [1 2 0]" why: |- - cljw's typed array ctors compose over ONE uniform helper (`-array-from2`): - a `sequential?` init is copied, any other init fills every slot with - `(coerce init)` — applied identically to int/long/double/float/short/byte/ - char/boolean (F-011 single mechanism). clj's 2-arg behaviour is instead + Every typed constructor shares one native fill (`__array-fill`, ADR-0204): a + `sequential?` init is copied, any other init fills every slot, converted to + the element kind, identically for int/long/double/float/short/byte/char/ + boolean (F-011 single mechanism). clj's 2-arg behaviour is instead per-element-type: int/long/double/float/boolean accept a number-fill, but byte/short/char route an unrecognised init through `RT.seq` and THROW on a - number (a `Numbers.java` method-overload accident keyed on the distinct JVM - element types). cljw arrays are TYPE-ERASED (AD-019): there is no distinct - byte/short/char element type, so there is no per-type overload to key the - throw on. Reproducing clj's throw would require re-introducing the very - element-type distinction AD-019 denies — so the uniform fill is the FORCED - consequence of AD-019, not a convenience. (Devil's-advocate fork 2026-06-18 - confirmed: the per-type-throw mirror is the option that would contradict - AD-019; uniform + this AD is the finished-form-clean choice per F-002/F-011. - `object-array` keeps NO 2-arg form — cljw matches clj's arity error there.) - The matching cases (number-fill for int/long/double/float/boolean, seq-copy - for every type) are clj-byte-identical and corpus-pinned. - derives_from: "AD-019 (type-erased arrays — no distinct byte/short/char element type) + F-011 (uniform shared mechanism over per-JVM-type overload accidents). DA fork 2026-06-18 (private/notes/D446-array2arg-da-fork.md)." + number (a `Numbers.java` method-overload accident, not a designed contract). + Since ADR-0204 cljw arrays carry their element kind, so mirroring the throw is + possible; it is declined because it would add a per-kind branch whose only + purpose is reproducing that accident. (Devil's-advocate fork 2026-06-18 chose + uniform; re-examined 2026-10-22 when the type erasure it leaned on was + removed, and the overload-accident argument stands on its own.) `object-array` + keeps NO 2-arg form; cljw matches clj's arity error there. The matching cases + (number-fill for int/long/double/float/boolean, seq-copy for every kind) are + clj-byte-identical and corpus-pinned. + derives_from: "F-011 (uniform shared mechanism over per-JVM-type overload accidents): every typed constructor shares one native fill (ADR-0204), so a number init fills every kind alike. DA fork 2026-06-18 (private/notes/D446-array2arg-da-fork.md)." pin: "test/e2e/phase14_arity_parity.sh — `AD-036 (byte-array n num)` locks (vec (byte-array 3 65)) => [65 65 65] and `AD-036 (char-array n int)` locks (int (aget (char-array 3 65) 0)) => 65 (cljw fills where clj throws)" - id: "AD-037" summary: "`(str )` renders the realized ELEMENTS `(…)`; clj renders the LazySeq's identity-hash `toString` (`clojure.lang.LazySeq@`)." @@ -623,7 +610,7 @@ accepted: `testing` context line, `Testing `, and the summary — matches clj byte-for-line. derives_from: "ADR-0059 (no-JVM: no JVM stack / class-file source positions) + F-011 (behavioural equivalence — pass/fail/error outcomes + counts match; only the unreproducible location/trace text differs)." - pin: "test/e2e/phase15_clojure_test_output.sh (suffix present, deftest-var-sourced) + test/e2e/phase15_var_source_meta.sh (the Var meta channel + AOT round-trip)." + pin: "test/clj/suites/test_reporters_test.clj (suffix present, deftest-var-sourced) + test/clj/suites/concurrency_host_test.clj (the Var meta channel) + test/e2e/phase15_var_source_meta.sh (AOT round-trip)." - id: "AD-042" summary: "`(pr-str v)` / `(str v)` print the bare ISO toString for a `java.time.*` value (Instant, Duration, …), where clj's `pr` emits the opaque `#object[java.time.X 0x.. \"ISO\"]` identity form. cljw makes both str and pr the toString; the value's `(str …)`, method results, and `=` all match clj." example: "(pr-str (java.time.Instant/ofEpochSecond 0)) => cljw \"1970-01-01T00:00:00Z\" / clj \"#object[java.time.Instant 0x5d3411d \\\"1970-01-01T00:00:00Z\\\"]\". (str (java.time.Duration/ofSeconds 90)) => both \"PT1M30S\"." @@ -638,7 +625,7 @@ accepted: identity-hash wrapper — `(str v)`, the instance method results, and value `=` — matches clj exactly. derives_from: "ADR-0059 (no-JVM: no identity-hash `#object[…]` object form) + F-011 (behavioural equivalence — toString / method results / `=` match; only the unreproducible identity-wrapped pr form differs)." - pin: "test/e2e/phase15_java_time_instant.sh + test/e2e/phase15_java_time_duration.sh + test/e2e/phase15_java_time_local_date_time.sh + test/e2e/phase15_java_time_local_date.sh — lock `(str v)` = the bare ISO toString (Instant ISO_INSTANT variable fraction + `Z`; Duration `PT…`; LocalDateTime ISO-local conditional seconds + variable fraction; LocalDate `yyyy-MM-dd`; LocalTime ISO-local time) and the instance-method + value-`=` results; the absent `#object[…]` pr wrapper is the locked divergence." + pin: "test/clj/suites/java_time_values_test.clj — lock `(str v)` = the bare ISO toString (Instant ISO_INSTANT variable fraction + `Z`; Duration `PT…`; LocalDateTime ISO-local conditional seconds + variable fraction; LocalDate `yyyy-MM-dd`; LocalTime ISO-local time) and the instance-method + value-`=` results; the absent `#object[…]` pr wrapper is the locked divergence." - id: "AD-043" summary: "`(compare a b)` on java.time values (LocalDate/LocalDateTime …) and host enums returns the SIGN (-1/0/1) where clj returns a field-difference / ordinal-difference MAGNITUDE. The SIGN is identical, so sort order / branch behaviour match; only the non-zero magnitude differs. (Strings / chars / keywords / symbols return clj's exact Java compareTo magnitude since 2026-07-16 — `(compare \"a\" \"c\")` → -2 — so the divergence is TEMPORAL/ENUM-only.)" example: "(compare (java.time.LocalDate/of 2024 6 1) (java.time.LocalDate/of 2024 1 1)) => cljw 1 / clj 5. (compare java.time.DayOfWeek/MONDAY java.time.DayOfWeek/SUNDAY) => cljw -1 / clj -6 (ordinal diff). (compare (java.time.Instant/ofEpochSecond 5) (java.time.Instant/ofEpochSecond 9)) => -1 BOTH (Instant/Duration/LocalTime already match — they use a sign-returning Long/Integer.compare). (sort …) is identical either way." @@ -657,7 +644,7 @@ accepted: sign-test see identical results. The D-462 temporal `compare` arms follow the sign-based design; this AD records that residual class. derives_from: "java.lang.Comparable / java.util.Comparator contract (only the SIGN of compareTo/compare is specified; magnitude is implementation-defined) + ADR-0053 (cljw compare is Order-based) + ADR-0059 (no JVM-internal magnitude reproduction)." - pin: "test/e2e/phase15_java_time_sort.sh — locks the TEMPORAL sign results (`(compare LocalDate-jun LocalDate-jan)` => 1, clj 5) + `(sort [LocalDate …])` ordering + the string-family MAGNITUDE parity (`string-family-magnitude`: `(compare \"a\" \"c\")` => -2). test/e2e/phase15_java_time_dow_month.sh (block `enum-compare`) locks the host-enum sign result `(compare DayOfWeek/MONDAY DayOfWeek/SUNDAY)` => -1 (clj -6) + a sort by ordinal (ADR-0161)." + pin: "test/clj/suites/java_time_values_test.clj — locks the TEMPORAL sign results (`(compare LocalDate-jun LocalDate-jan)` => 1, clj 5) + `(sort [LocalDate …])` ordering + the string-family MAGNITUDE parity (`string-family-magnitude`: `(compare \"a\" \"c\")` => -2). test/clj/suites/java_time_values_test.clj (block `enum-compare`) locks the host-enum sign result `(compare DayOfWeek/MONDAY DayOfWeek/SUNDAY)` => -1 (clj -6) + a sort by ordinal (ADR-0161)." - id: "AD-044" summary: "`(class x)` / `(type x)` of a callable / seq-view / reference / transient returns a STABLE clj-faithful simple class name (`Fn` / `MultiFn` / `LazySeq` / `LongRange` / `Atom` / `Ref` / `Future` / `Promise` / …); clj returns a per-fn GENERATED class (`user$eval$fn__135`, `clojure.core$_PLUS_`) for closures/builtins and a per-call REIFY class for future/promise that cljw has no analog for." example: '(class (fn [x] x)) => cljw "Fn" / clj "user$eval$fn__135". (class +) => cljw "Fn" / clj "clojure.core$_PLUS_". (class (future 1)) => cljw "Future" / clj "core$future_call$reify__8578". (class (map inc [1])) => cljw "LazySeq" / clj "clojure.lang.LazySeq" (this one matches).' @@ -782,31 +769,6 @@ accepted: or throwing on `(/ 1.0 0.0)` too (breaking the common case) — both worse. derives_from: "F-005 (numeric tower = IEEE f64 semantics: float-zero divisor → ±Inf/NaN, integer-zero → throw) + ADR-0059 (no JVM Compiler — cljw has no operand-type inlining pass, so it cannot reproduce clj's inline-vs-runtime split) + F-011 (the common `(/ x 0.0)` literal path matches clj's inlined ±Inf exactly; only the rare runtime-dispatched 1-arg/apply/reduce path diverges)." pin: "test/e2e/phase14_float_div.sh — `ad050_recip`/`ad050_recip_n`/`ad050_apply`/`ad050_reduce` lock cljw's IEEE ±Inf form (NOT clj's throw) on the runtime paths; `pos_inf`/`int_zero` lock the clj-IDENTICAL inlined-±Inf + integer-throw surfaces." - - id: "AD-051" - summary: "`(bytes? x)` is true for ANY cljw array, where clj's `bytes?` is true only for a `byte[]`. cljw arrays are type-erased (AD-019) — `byte-array`/`int-array`/`object-array`/`make-array` share one runtime shape — so a byte-array is runtime-indistinguishable from any other array. The common positive guard `(bytes? (byte-array n))` matches clj (true); non-byte arrays diverge (clj false, cljw true)." - example: '(bytes? (byte-array 3)) => true in BOTH ; (bytes? (int-array 3)) => cljw true / clj false ; (bytes? (object-array 0)) => cljw true / clj false ; (bytes? "x") / (bytes? nil) / (bytes? [1 2]) => false in both' - why: |- - clj's `bytes?` is `(-> x class .getComponentType (= Byte/TYPE))` — it keys on - the JVM array's runtime component type. cljw deletes that component type: - every array (`byte-array`, `int-array`, `object-array`, `make-array`) is the - same type-erased `[]Value` (AD-019, derived from F-004 uniform 8-byte Value + - F-005 no primitive specialization). So cljw genuinely cannot tell a byte-array - from an int-array at runtime — the only signal available is `array?`. The - over-broad `(defn bytes? [x] (array? x))` is NOT a silent positive lie: in - cljw a byte-array IS usable as bytes (`(String. (byte-array [104 105]))` => - "hi", matching clj) AND, because all arrays share one representation, an - int/object array routed into byte-handling behaves identically to a byte-array - — there is no distinct byte-path to corrupt. The alternatives are worse: an - always-`false` `bytes?` would deny the predicate's DEFINING case - (`(bytes? (byte-array n))` => false), a negative lie on a capability cljw - demonstrably has; an element-inspection heuristic (`every? byte-range?`) dresses - an unknowable distinction as knowable (an int-array of 0-255 ints reads true) - and is the 個別最適化 F-013 closes. A Devil's-advocate fork (2026-06-25) - recommended always-`false` on a "positive answers can't be honored" premise; - the main loop overrode it because a probe falsified that premise (byte-arrays - ARE byte-usable in cljw). Record: `private/notes/2026-06-25-D472-bytes-da-fork.md`. - derives_from: "AD-019 (type-erased arrays — no per-array component type, so byte-array is runtime-indistinguishable from any other array) + F-004 (uniform 8-byte Value) + F-005 (no primitive specialization) + F-011 (the common positive guard `(bytes? (byte-array n))` matches clj true exactly; only non-byte arrays diverge) + F-013 (no bespoke per-array type-tag mechanism added to fake a distinction the representation does not carry)." - pin: "test/e2e/phase14_arity_parity.sh — `AD-051 (bytes? byte-array)`/`AD-051 (bytes? int-array)`/`AD-051 (bytes? object-array)` lock cljw's over-broad `true` (incl. the int/object-array cases where clj returns false). The clj-IDENTICAL cases (byte-array→true, string/nil/vector→false) are corpus-pinned in test/diff/clj_corpus/clojure_1_11_additions.txt." - id: "AD-052" summary: "format's `%t` date/time conversions render in UTC with fixed English names (month/day/am-pm) and a fixed `+0000`/`UTC` zone; the JVM renders in the process's default TIME ZONE and LOCALE (a JST/ja machine shows shifted hours and 金曜日/2月/午後)." example: '(format "%tH" 0) => cljw "00" (UTC) / clj-on-JST "09" ; (format "%ta" 1234567890123) => cljw "Fri" / clj-on-ja "金"' @@ -839,7 +801,7 @@ accepted: the namespace's existence; no mainline code enumerates (all-ns) expecting a closed set. derives_from: "ADR-0171 (rt merge — internals need a public qualified home for macro expansions) + ADR-0059 no-JVM (cljw owns the cljw.* namespace surface) + F-011 (every public var's home ns now MATCHES clj; only the extra internal ns is cljw-specific)." - pin: "test/e2e/phase15_namespace.sh — `ad053-internal-ns` locks (find-ns 'cljw.internal) present, `ad053-rt-gone` locks the old `rt` ns absent, `ad053-no-leak` locks __-internals out of clojure.core's publics." + pin: "test/clj/suites/vars_namespaces_test.clj — `ad053-internal-ns` locks (find-ns 'cljw.internal) present, `ad053-rt-gone` locks the old `rt` ns absent, `ad053-no-leak` locks __-internals out of clojure.core's publics." - id: "AD-054" summary: "Class-name completion (nREPL `completions` on an uppercase / dotted prefix) offers cljw's CLOSED class universe — the `rt.types` registry (host surfaces, deftypes, the java.lang/java.math default imports) — where mainline scans the whole JVM classpath and leaks internal classes (`java.lang.CharacterData00`, `sun.nio.cs.ext.Big5`, …)." example: "prefix \"Character\" => cljw [Character, java.lang.Character] / mainline additionally java.lang.CharacterData00…, com.sun.… (26 candidates). Every cljw candidate resolves; the mainline extras are not even loadable classes for user code." @@ -1018,10 +980,10 @@ accepted: summary: |- `(.totalMemory r)` / `(.freeMemory r)` / `(.maxMemory r)` answer from cljw's own mark-sweep accounting, not from a JVM heap that does not - exist. `totalMemory` is the live set or the adaptive collect threshold, - whichever is larger (cljw keeps no reserved-but-unused pool, so the - threshold is the honest analogue of committed heap); `freeMemory` is the - headroom between them; `maxMemory` is the per-eval ceiling + exist. `totalMemory` is the committed cell heap (mapped GC pages plus + live large cells), which grows with the heap and shrinks when a collect + returns empty pages to the OS (CLJW-GC-HEAP-RETURN); `freeMemory` is + that minus the live cells; `maxMemory` is the per-eval ceiling `CLJW_EVAL_MAX_HEAP_MB` sets (ADR-0125), or `Long/MAX_VALUE` when the heap is unmetered. The consequence worth stating: on an unmetered cljw a used/max heap- @@ -1032,7 +994,7 @@ accepted: that mean nothing. example: 'CLJW_EVAL_MAX_HEAP_MB unset: (.maxMemory (Runtime/getRuntime)) => 9223372036854775807 (no ceiling) / with CLJW_EVAL_MAX_HEAP_MB=64 => 67108864, and the pressure ratio becomes real.' derives_from: "ADR-0028 (mark-sweep + adaptive threshold) + ADR-0125 / D-352 (per-eval heap ceiling) + ADR-0059 (no JVM)." - pin: "test/e2e/concurrent_atomics.sh — runtime_memory_is_coherent + runtime_maxmemory_unmetered_is_exact_long." + pin: "test/e2e/concurrent_atomics.sh — runtime_memory_is_coherent + runtime_maxmemory_unmetered_is_exact_long; test/clj/suites/gc_heap_return_test.clj." - id: "AD-064" area: "concurrency" summary: |- @@ -1193,7 +1155,7 @@ accepted: - id: AD-076 area: strings summary: 'A string is indexed by Unicode CODE POINT, not by UTF-16 code unit. `count`, `subs`, `nth`, `get`, `seq`/`first`, `(map int s)` and the interop methods `.length`, `.charAt`, `.indexOf` and `.substring` all count a character outside the Basic Multilingual Plane (emoji, rare CJK, most historic scripts) as ONE position, where clj counts its surrogate pair as two. BMP text, including multibyte characters such as é or 世, indexes identically in both runtimes.' - example: '(count "🌍") => cljw 1 / clj 2. (count "ab🌍") => cljw 3 / clj 4. (.length "🌍") => cljw 1 / clj 2. (subs "a🌍b" 1 2) => cljw "🌍" / clj a lone high surrogate. (vec (map int "🌍")) => cljw [127757] / clj [55356 57101]. (str (first "🌍")) => cljw "🌍" / clj a lone high surrogate. (.indexOf "a🌍b" "b") => cljw 2 / clj 3. Measured on cljw v1.14.11; clj 1.12 column from GitHub issue #1.' + example: "(count \"\U0001F30D\") => cljw 1 / clj 2. (count \"ab\U0001F30D\") => cljw 3 / clj 4. (.length \"\U0001F30D\") => cljw 1 / clj 2. (subs \"a\U0001F30Db\" 1 2) => cljw \"\U0001F30D\" / clj a lone high surrogate. (vec (map int \"\U0001F30D\")) => cljw [127757] / clj [55356 57101]. (str (first \"\U0001F30D\")) => cljw \"\U0001F30D\" / clj a lone high surrogate. (.indexOf \"a\U0001F30Db\" \"b\") => cljw 2 / clj 3. Measured on cljw v1.14.11; clj 1.12 column from GitHub issue #1." why: |- UTF-16 is how the JVM happens to store a String, not part of Clojure's string semantics: clj's own `count` docstring says "number of items in @@ -1218,4 +1180,4 @@ accepted: protocol counting UTF-16 units). docs/clojure_vs_clojurewasm.md gives the porting guidance. derives_from: 'F-011 read at the VALUE level (a string is a sequence of characters; the JVM storage encoding is not surface), plus D-217 (a String is Indexed by codepoint) and D-179 / O-060 (string seq is a UTF-8 byte-offset view over code points). No JVM, ADR-0059: there is no UTF-16 backing store to expose.' - pin: test/clj/suites/string_code_point_test.clj (every expression in the example, language and interop surfaces) + test/golden/cases/string_unicode.clj (count of a string holding 🌍, and (count "🌍") => 1). + pin: "test/clj/suites/string_code_point_test.clj (every expression in the example, language and interop surfaces) + test/golden/cases/string_unicode.clj (count of a string holding \U0001F30D, and (count \"\U0001F30D\") => 1)." diff --git a/.dev/debt.yaml b/.dev/debt.yaml index 17decb763..ab816d4c2 100644 --- a/.dev/debt.yaml +++ b/.dev/debt.yaml @@ -1619,6 +1619,28 @@ active: barrier: |- tractable now; M-sized survey-first. Value rises with D-528 (real deps.edn corpus). last_reviewed: "2026-07-02" + - id: "D-588" + status: |- + open (2026-10-22, ADR-0205 DA fork point 4). clj exposes chunk-first (returns the current chunk as an IChunk, counted and indexable) and chunk-next; cljw has chunked-seq?, chunk-rest, chunk-buffer, chunk-append, chunk-cons but no chunk-first / chunk-next, so library code written against the IChunkedSeq API (chunked doseq/for expansions in user macros) fails to resolve. The boundary they would read now exists (runtime/collection/chunked_seq.zig). + category: clj-parity + barrier: |- + tractable now; needs a chunk value shape (an ArrayChunk view over the current chunk: count, nth, drop-first, reduce) for chunk-first to return. + - id: "D-589" + status: |- + open (2026-10-22, java.time.format DA review). The java.time.format engine (runtime/time/date_time_format.zig) landed beside three structural leftovers its devil-s-advocate fork named. (1) Two ISO printers: (str ld) / (str instant) go through instant.zig formatLocalDate / formatLocalTime / formatIsoInstant while (.format x ISO_*) goes through the engine; fold the str path onto the interned ISO programs and delete the instant.zig printers (F-011). (2) zone_offset.zig holds both the pure Zone arithmetic the engine borrows and the Value layer; split it into time/zone.zig + zone_value.zig like the formatter pair. (3) date_time_formatter_value.zig temporalOf / Target are an if-chain over the four temporal types; a temporal_view hook on TypeDescriptor (sibling of temporal_print) would let OffsetDateTime / ZonedDateTime join without editing the bridge. Also the SignStyle / TextStyle / ResolverStyle enums are not surfaced, so appendValue(f,min,max,SignStyle), appendText(f,TextStyle), withResolverStyle stay opaque_members. + category: "interop" + barrier: |- + tractable now; M-sized. (3) and the region-zone half of ZoneId also depend on the tz-database decision recorded on the ZonedDateTime compat_tiers row (user-owned). + last_reviewed: "2026-10-22" + - id: D-600 + status: open (2026-10-22, ADR-0206 DA review). The ProtocolFn dispatch cache (O-081) keys on rt.protocol_generation, a plain u32 bumped with +%= 1 by every protocol writer. Two extends racing on different threads can lose a bump, and a filler that resolved between them could then serve a stale method forever (F-015). Make the counter an atomic u64 (fetchAdd .release, load .acquire). The cache slot is also monomorphic and one per ProtocolFn, so polymorphic protocol calls thrash it and every miss writes a shared cache line. The finished form named by the DA is a small per-call-site polymorphic inline cache in VM bytecode, or a per-descriptor itable keyed by ProtocolFn id. Generation is also global (any extend flushes every protocol), unlike clj per-protocol -reset-methods. + category: performance + barrier: tractable now; S for the atomic counter, M for the polymorphic IC (touches vm.zig, coordinate with the VM call-path owner). + - id: D-610 + status: 'open (2026-10-22). After O-090, `cljw -e nil` runs 61.6M user instructions against bb 16.2M (taskset -c 2, perf stat -r 10); cljw kernel work fell 29.3M -> 12.3M and is now below bb (26.8M). Remaining attributed cost (perf, -Dprofile, 30 runs): the TLS segment is 1.17 MB, glibc memsets it for the main thread (dl_main, ~5%) and again for the eval thread (allocate_stack, ~4%); 868 KB of it is eval.backend.vm.vm_arena (threadlocal) and 256 KB std Thread.maybeAttachSignalStack. referAllImpl ~15% inclusive (every eager `(ns ...)` refers all of clojure.core); a getOrPut + reserve rewrite of it was drafted but not measured (build-lock contention), so it is not landed. readCollection ~9%, lazy loadRegionNamespace during the eager replay ~5%.' + category: perf + barrier: vm_arena out of TLS needs an edit in src/eval/backend/vm.zig (out of scope for CLJW-PERF-STARTUP); the refer cost needs a shared refer table (one clojure.core refer map per ns instead of a copy), an env design change. + last_reviewed: "2026-10-22" standing: - id: "D-549" status: |- @@ -6599,7 +6621,7 @@ discharged: barrier: "none — repro in hand: `(with-out-str (.write *out* \"hi\"))` => type_error (write on Keyword). General (any `(.write *out* …)` lib). Needs a cross-zone routing design (vtable hook / out_capture relocation / *out*-as-writer-value) — ADR-level. Take up as a focused cycle; pairs with the D-105 time/io build-out and the pprint/data.json writer surface." last_reviewed: "2026-06-14" - id: "D-438" - status: 'DONE 2026-06-14 (ADR-0140, landed same cycle it was minted). Caught-exception cljw frame-seq accessor: (stack-trace e) → a vector of {:ns :fn :file :line :column} maps, innermost-first (:fn bare, :ns separate), reading the ExInfo own deep-copied frames (ADR-0120); empty for a non-ex_info or never-thrown ex-info. Consumers: clojure.stacktrace/print-stack-trace prints / (:) per frame (AD-029 amended: marker→cljw frames, format still diverges from JVM); Throwable->map :trace/:at filled as the same maps (D-389 discharged; AD-033). DA fork chose Alt 2 (one map shape everywhere). user-frames-only via AD-024 (push-time elision). THIS ROW ALSO FIXES the dangling D-232 cross-reference: AD-029/D-389/sweep_plan named the validation-campaign row D-232 as the frame-accessor owner, but that owner row never existed (Reservation-as-bias) — D-438 + ADR-0140 are it. Pins: test/e2e/phase15_clojure_stacktrace.sh + phase14_throwable_map.sh.' + status: 'DONE 2026-06-14 (ADR-0140, landed same cycle it was minted). Caught-exception cljw frame-seq accessor: (stack-trace e) → a vector of {:ns :fn :file :line :column} maps, innermost-first (:fn bare, :ns separate), reading the ExInfo own deep-copied frames (ADR-0120); empty for a non-ex_info or never-thrown ex-info. Consumers: clojure.stacktrace/print-stack-trace prints / (:) per frame (AD-029 amended: marker→cljw frames, format still diverges from JVM); Throwable->map :trace/:at filled as the same maps (D-389 discharged; AD-033). DA fork chose Alt 2 (one map shape everywhere). user-frames-only via AD-024 (push-time elision). THIS ROW ALSO FIXES the dangling D-232 cross-reference: AD-029/D-389/sweep_plan named the validation-campaign row D-232 as the frame-accessor owner, but that owner row never existed (Reservation-as-bias) — D-438 + ADR-0140 are it. Pins: test/clj/suites/test_reporters_test.clj + phase14_throwable_map.sh.' category: "clj-parity" barrier: none — DONE this cycle (ADR-0140). last_reviewed: "2026-06-14" @@ -6763,7 +6785,7 @@ discharged: DISCHARGED — `.until` complete for LocalDate / LocalDateTime / Instant. last_reviewed: "2026-06-24" - id: "D-510" - status: "DISCHARGED (2026-06-24, ADR-0161, 2 cycles). Unified all four host enums (RoundingMode/ChronoUnit/DayOfWeek/Month) onto ONE comptime registry (runtime/host_enum.zig) + ONE flat rt.host_enum_consts[43] cache with a single interning entry point (DA Alt-3). Cycle 1: built the mechanism + migrated Group A (RoundingMode/ChronoUnit) behaviour-preserving; folded+deleted rounding_mode.zig/chrono_unit.zig. Cycle 2: folded Group B (DayOfWeek/Month) from .typed_instance getter-mint to .host_instance singletons — new surfaces DayOfWeek.zig/Month.zig (static fields + getValue/name/toString), getters return the interned singleton, TypeDescriptor.host_enum_idx drives unified pr-print + compare-by-ordinal; removed the typed_instance equal/compare/print arms + day_of_week_value.zig/month_value.zig + their per-Runtime descriptors. PARITY WINS: DayOfWeek/MONDAY..Month/DECEMBER static fields now resolve; (identical? (.getDayOfWeek monday) DayOfWeek/MONDAY) => true; all four enums JVM-Comparable (sign-based, AD-043 extended). compat_tiers: 4 entries → keyword host_enum (G3 green). MathContext stays a consumer (not a member). Pinned by test/e2e/phase15_java_time_dow_month.sh (enum-const/identity/compare blocks)." + status: "DISCHARGED (2026-06-24, ADR-0161, 2 cycles). Unified all four host enums (RoundingMode/ChronoUnit/DayOfWeek/Month) onto ONE comptime registry (runtime/host_enum.zig) + ONE flat rt.host_enum_consts[43] cache with a single interning entry point (DA Alt-3). Cycle 1: built the mechanism + migrated Group A (RoundingMode/ChronoUnit) behaviour-preserving; folded+deleted rounding_mode.zig/chrono_unit.zig. Cycle 2: folded Group B (DayOfWeek/Month) from .typed_instance getter-mint to .host_instance singletons — new surfaces DayOfWeek.zig/Month.zig (static fields + getValue/name/toString), getters return the interned singleton, TypeDescriptor.host_enum_idx drives unified pr-print + compare-by-ordinal; removed the typed_instance equal/compare/print arms + day_of_week_value.zig/month_value.zig + their per-Runtime descriptors. PARITY WINS: DayOfWeek/MONDAY..Month/DECEMBER static fields now resolve; (identical? (.getDayOfWeek monday) DayOfWeek/MONDAY) => true; all four enums JVM-Comparable (sign-based, AD-043 extended). compat_tiers: 4 entries → keyword host_enum (G3 green). MathContext stays a consumer (not a member). Pinned by test/clj/suites/java_time_values_test.clj (enum-const/identity/compare blocks)." category: "host-surface" barrier: |- ADR-0160 modelled java.math.RoundingMode as a host-enum object singleton @@ -6943,7 +6965,7 @@ discharged: barrier: "none — DISCHARGED." last_reviewed: "2026-06-20" - id: "D-475" - status: "DISCHARGED (2026-06-23, verified at resume). spec.alpha + spec.gen.alpha + core.specs.alpha are BUNDLED (src/lang/clj/clojure/spec/) and FULLY FUNCTIONAL: load + s/def/and/or/valid?/conform/cat/*/+/alt/?/&/coll-of/every/keys/map-of/tuple/merge/nilable/conformer/explain-data/explain-str/form all clj-verified in the current binary (incl. s/keys, the row's last-claimed blocker — the &-destructure path now works). s/form's rt/int? qualification is the accepted AD-049 (pin in phase15_spec.sh). e2e phase15_spec.sh + corpus spec.txt/core_specs_alpha.txt cover it. ORIGINAL: open (2026-06-20, fresh-namespace probe — HIGH VALUE, the highest-value remaining clj-parity gap). `clojure.spec.alpha` does NOT load — `(require '[clojure.spec.alpha])` → 'Could not locate'. spec is FUNDAMENTAL to modern Clojure (s/def / s/valid? / s/conform / s/cat / s/and / s/or / s/keys / s/coll-of / s/fdef / instrument), so a large class of real code + libs is blocked. FEASIBILITY (it is a SUBSTANTIAL multi-cycle unit, NOT a quick add): spec.alpha is mostly PURE-Clojure (over clojure.walk, which cljw has) — so it is BUNDLEABLE as src/lang/clj/clojure/spec/alpha.clj like clojure.set/walk/zip already are. Sources: `~/Documents/OSS/spec.alpha/src/main/clojure/clojure/spec/alpha.clj` (official) + a SCI-adapted reference at `~/Documents/OSS/.../babashka/src/babashka/impl/clojure/spec/alpha.clj` (bb also lacks full JVM — its adaptation shows the no-JVM-feasible subset). DEPENDENCY: spec.alpha `(:require clojure.spec.gen.alpha)` at load — gen LAZY-loads test.check via requiring-resolve, so it LOADS without test.check present (only s/gen/s/exercise fail at CALL time); bundle clojure.spec.gen.alpha too (it stays a lazy stub). RISKS to verify: (1) the regex-op spec machinery (s/cat/s/* /s/alt — `(instance? clojure.spec.alpha.Spec x)` protocol + deftype Specize/regex impls) exercises deftype/defprotocol/dynamic-var/metadata features cljw must fully support; (2) `(alias 'c 'clojure.core)` + the AOT cache_gen bootstrap of a ~1500-line .clj (cache_gen must compile it — watch for the host-class-in-bootstrap trap, memory `bootstrap-clj-host-class-unresolvable`, + regex-literal-in-bootstrap-.clj trap). PLAN: copy alpha.clj + gen/alpha.clj into the bundle's @embedFile FILES list, rebuild, fix the cljw feature gaps the load surfaces one by one, verify s/def/valid?/conform/cat/keys against the clj oracle, add a clj_corpus spec area. A focused FRESH-CONTEXT unit (the highest-value one available); also add a spec.alpha NEEDS-ROW to docs/works/ladder.md when landed. FEASIBILITY INVESTIGATED 2026-06-20 (de-risks the fresh cycle): bundle mechanism = add a `.{ .label, .source = @embedFile(\"clj/clojure/spec/alpha.clj\") }` entry to bootstrap.zig FILES (after clojure.walk + after the gen file). SIZES: official alpha.clj = 1996 lines (defprotocol Spec + Specize, 14 deftype/defprotocol constructs); gen/alpha.clj = 227 lines, NO deftypes, lazy-loads test.check via `dynaload`+`delay` (loads clean without test.check; s/gen fails only at call). FIRST CONCRETE BLOCKER: the OFFICIAL alpha.clj has a REGEX LITERAL `#\"(.*)\\$(.*?)(__[0-9]+)?\"` at line ~132 — a bundled/AOT .clj CANNOT carry a `#\"\"` literal (memory `regex_literal_unsupported_bootstrap_clj`); worse, it sits in a `(.. f getClass getName)` JVM-fn-class-name reflection path that does not apply to cljw (no-JVM fns have no class name). RECOMMENDATION: base the cljw port on the BABASHKA SCI-adapted version (`~/Documents/OSS/.../babashka/src/babashka/impl/clojure/spec/alpha.clj`) — bb also lacks full JVM, so it has ALREADY adapted the getClass/getName + other JVM paths to the no-JVM-feasible subset; diff it against the official to see exactly what cljw needs. Then iterate: bundle → rebuild → fix each cljw-feature gap cache_gen surfaces (expect the regex path, possibly deftype/protocol edge cases, dynamic-var/alias) → verify s/def/valid?/conform/cat/keys vs the clj oracle → clj_corpus spec area. Multi-cycle but the line-by-line blocker path is now mapped. LOAD-BLOCKER CHAIN (probed 2026-06-20 via `-cp /tmp/spec_cp` classpath load of the official src — this is the right dev harness: the classpath resolver loads .clj WITHOUT touching the @embedFile bundle, so the tracked build stays green while iterating blockers; bundle into FILES only once it loads + s/valid?/conform verify): #1 gen/alpha.clj `BigDecimal/valueOf` — FIXED commit 1a5e0ed0 (the static cljw.java.math.BigDecimal descriptor had an empty method_table; added valueOf + an .init populate fn). #2 gen/alpha.clj:188 `bytes?` unresolved — cljw lacks the predicate AND it is AD-019-blocked (JavaArray is fully type-erased, no element-type field, so byte[] is indistinguishable from int[]; = the known D-472). gen-builtins needs ALL 43 core type-predicates to resolve at analyze time (delay-body map KEYS); only bytes? is missing — so the port adapts that one gen-builtins entry (gen is generators-only, never forced without test.check) OR a deliberate bytes?=array? AD decision in the focused cycle (DA subagent on the AD-019 envelope). #3+ spec.alpha CORE (1996 lines) unprobed past gen; static scan: 12 `reify` of the Spec/Specize protocols (THE core spec mechanism — cljw reify-of-protocol must carry these), 2 `set!` on dynamic vars (*recursion-limit* etc.), defmulti/defmethod `describe`, `(alias 'c 'clojure.core)`, JVM-internal RT/checkSpecAsserts + Compiler/demunge (AD-008 deferred-host-ref handles those — ns loads, errors only if called). MILESTONE 2026-06-20: spec.alpha LOADS (`:loaded`) — 6 blockers cleared: 3 committed cljw clj-parity fixes [#1 BigDecimal/valueOf 1a5e0ed0; reify/deftype destructured method params 8c4f4827; (symbol var) f74f8f18] + 3 port adaptations in scratch /tmp/spec_cp [drop the bytes? gen-builtin (AD-019), `clojure.spec.alpha.Spec` FQCN→protocol var `Spec` (no-JVM, ADR-0059), checkSpecAsserts JVM-static→cljw atom]. REMAINING FUNCTIONAL BLOCKER (spec loads but s/valid? errors `Unable to resolve spec`): spec's `with-name` (alpha.clj:90) attaches `::name` metadata to every spec object, but a spec object is a `reify` instance and cljw reify instances are NOT IObj/metable — `(instance? clojure.lang.IObj )` is false + `(with-meta m)` raises `cannot attach metadata to reified_instance`, so `with-name` returns nil → the registry stores nil → lookup fails. clj's reify ALWAYS auto-implements IObj (metable); plain deftype does NOT (cljw already matches deftype). FIX (reify-only, ADR-0134-coupled — the instance?-IObj⟹with-meta-works invariant D-271): add a `meta: Value` slot to ReifiedInstance (16→24B + the @sizeOf asserts at type_descriptor.zig:476/797), init in allocReifiedInstance, mark it in traceReifiedInstance (mirror traceTypedInstance), a reifiedInstWithMeta minting helper, the withMetaFn/.reified_instance arm mints-fresh instead of raising, the metaFn/.reified_instance arm returns inst.meta, and isInstance/matchUserType answers IObj+IMeta true for reified_instance. Needs an inline ADR (extends ADR-0134 IObj activation to reify) + mandatory DA fork. Port adaptations still need promoting from /tmp into the tracked bundle once reify-IObj lands + s/valid?/conform verify green. MILESTONE-3 2026-06-20: reify-IObj-metadata LANDED (commit 458be654, ADR-0134 reify slice) → spec's registry resolves. BASIC SPECS NOW FUNCTIONAL + clj-verified: `(s/def ::age (s/and int? pos?))` + s/valid?/conform all correct. 5 GENERIC cljw clj-parity fixes extracted+committed this arc (BigDecimal/valueOf, reify/deftype destructured method params, symbol-of-var, reify-IObj-metadata) — these benefit ALL cljw code, not just spec. 4th port adaptation added in /tmp scratch: fn-sym→nil (alpha.clj:131 used `(.. f getClass getName)`+regex to name anon fns; cljw fns have no class name, so anon-fn specs describe as ::unknown via the specize* Object fallback — acceptable divergence). REMAINING DEEP TAIL (the regex-op + every machinery — spec-internal, diminishing generic-fix yield, a focused multi-cycle continuation): `s/coll-of`/`s/every` → alpha.clj:1291 `cpred` is nil at conform (every-impl closure-capture or opts-destructure issue — classify generic-cljw vs spec-internal); `s/or` → Exception; `s/keys` → alpha.clj:317 (unfn conj-on-symbol — cljw's postwalk-replace/destructure/conj all individually MATCH clj, so the trigger is a specific s/keys path, needs isolation); `s/cat`/`s/*` regex-op conform → `::invalid` (the regex-op state-machine — the hardest part). The 4 port adaptations (bytes?-drop, Spec-FQCN→var, checkSpecAsserts→atom, fn-sym→nil) live in /tmp/spec_cp and are re-derivable from this blocker chain; promote into the tracked bundle (src/lang/clj/clojure/spec/ + bootstrap.zig FILES, EPL variant-① header per clj_attribution.md) only once the regex-op tail is green. MILESTONE-4 2026-06-20: the regex-op tail is GREEN — 2 more generic cljw fixes landed [:keys [::ns-key] auto-resolve a17b97c8 (unblocked coll-of/s/* via every-impl's {:keys [::cpred]}); (conj map other-map) merge 8fc081d4 (unblocked the whole pcat* state machine → s/cat/s/*/s/+/s/alt conform)]. spec.alpha is now MOSTLY FUNCTIONAL + clj-verified: load + s/def + s/and (basic) + s/valid? + s/conform + s/cat + s/* + s/+ + s/alt + s/coll-of + s/keys all correct. The LAST blocker is the builtin-macro-shadow bug → D-476 (s/or mis-routes to core/or; strict s/and validates only the last pred). 7 generic clj-parity fixes total this arc (BigDecimal/valueOf, reify/deftype destructured params, symbol-of-var, reify-IObj-metadata, :keys auto-resolve, conj map-merge — all committed + benefit ALL cljw code). Resolve D-476, then promote the /tmp port into the bundle + verify the full surface. MILESTONE-6 2026-06-20: D-476 RESOLVED (7c40041f, macro-shadow gate + caseTest rt/or hygiene) → s/or + strict s/and work. 3 more generic fixes landed: clojure.walk over a cons/lazy-tailed list (cf0b8dd2), valueToForm .hash_map macro return (978b41e2). 10 generic cljw fixes total this arc; spec ~95% functional (s/def/and/or/valid?/conform/cat/*/+/alt/coll-of all clj-verified). THE ONE REMAINING BLOCKER = s/keys: cljw's `&`-destructure (sequentialDestructure, macro_transforms.zig:344) uses nth/nthnext, but clj's `& rest` expands to (seq g)/(first cur)/(next cur) — fails on the non-Indexed map at alpha.clj:837 `(loop [… [[k v] & ks] m] …)`. Fix = make the `&`-case seq-based (clj-faithful; NO-`&` keeps nth which clj-correctly errors on maps). Then drain any further conform-path blockers. Full spec spec/fix sequence is in `.dev/handover.md` First-commit-on-resume + `private/notes/spec-alpha-port-arc.md`." + status: "DISCHARGED (2026-06-23, verified at resume). spec.alpha + spec.gen.alpha + core.specs.alpha are BUNDLED (src/lang/clj/clojure/spec/) and FULLY FUNCTIONAL: load + s/def/and/or/valid?/conform/cat/*/+/alt/?/&/coll-of/every/keys/map-of/tuple/merge/nilable/conformer/explain-data/explain-str/form all clj-verified in the current binary (incl. s/keys, the row's last-claimed blocker — the &-destructure path now works). s/form's rt/int? qualification is the accepted AD-049 (pin in test/clj/suites/test_reporters_test.clj). e2e test/clj/suites/test_reporters_test.clj + corpus spec.txt/core_specs_alpha.txt cover it. ORIGINAL: open (2026-06-20, fresh-namespace probe — HIGH VALUE, the highest-value remaining clj-parity gap). `clojure.spec.alpha` does NOT load — `(require '[clojure.spec.alpha])` → 'Could not locate'. spec is FUNDAMENTAL to modern Clojure (s/def / s/valid? / s/conform / s/cat / s/and / s/or / s/keys / s/coll-of / s/fdef / instrument), so a large class of real code + libs is blocked. FEASIBILITY (it is a SUBSTANTIAL multi-cycle unit, NOT a quick add): spec.alpha is mostly PURE-Clojure (over clojure.walk, which cljw has) — so it is BUNDLEABLE as src/lang/clj/clojure/spec/alpha.clj like clojure.set/walk/zip already are. Sources: `~/Documents/OSS/spec.alpha/src/main/clojure/clojure/spec/alpha.clj` (official) + a SCI-adapted reference at `~/Documents/OSS/.../babashka/src/babashka/impl/clojure/spec/alpha.clj` (bb also lacks full JVM — its adaptation shows the no-JVM-feasible subset). DEPENDENCY: spec.alpha `(:require clojure.spec.gen.alpha)` at load — gen LAZY-loads test.check via requiring-resolve, so it LOADS without test.check present (only s/gen/s/exercise fail at CALL time); bundle clojure.spec.gen.alpha too (it stays a lazy stub). RISKS to verify: (1) the regex-op spec machinery (s/cat/s/* /s/alt — `(instance? clojure.spec.alpha.Spec x)` protocol + deftype Specize/regex impls) exercises deftype/defprotocol/dynamic-var/metadata features cljw must fully support; (2) `(alias 'c 'clojure.core)` + the AOT cache_gen bootstrap of a ~1500-line .clj (cache_gen must compile it — watch for the host-class-in-bootstrap trap, memory `bootstrap-clj-host-class-unresolvable`, + regex-literal-in-bootstrap-.clj trap). PLAN: copy alpha.clj + gen/alpha.clj into the bundle's @embedFile FILES list, rebuild, fix the cljw feature gaps the load surfaces one by one, verify s/def/valid?/conform/cat/keys against the clj oracle, add a clj_corpus spec area. A focused FRESH-CONTEXT unit (the highest-value one available); also add a spec.alpha NEEDS-ROW to docs/works/ladder.md when landed. FEASIBILITY INVESTIGATED 2026-06-20 (de-risks the fresh cycle): bundle mechanism = add a `.{ .label, .source = @embedFile(\"clj/clojure/spec/alpha.clj\") }` entry to bootstrap.zig FILES (after clojure.walk + after the gen file). SIZES: official alpha.clj = 1996 lines (defprotocol Spec + Specize, 14 deftype/defprotocol constructs); gen/alpha.clj = 227 lines, NO deftypes, lazy-loads test.check via `dynaload`+`delay` (loads clean without test.check; s/gen fails only at call). FIRST CONCRETE BLOCKER: the OFFICIAL alpha.clj has a REGEX LITERAL `#\"(.*)\\$(.*?)(__[0-9]+)?\"` at line ~132 — a bundled/AOT .clj CANNOT carry a `#\"\"` literal (memory `regex_literal_unsupported_bootstrap_clj`); worse, it sits in a `(.. f getClass getName)` JVM-fn-class-name reflection path that does not apply to cljw (no-JVM fns have no class name). RECOMMENDATION: base the cljw port on the BABASHKA SCI-adapted version (`~/Documents/OSS/.../babashka/src/babashka/impl/clojure/spec/alpha.clj`) — bb also lacks full JVM, so it has ALREADY adapted the getClass/getName + other JVM paths to the no-JVM-feasible subset; diff it against the official to see exactly what cljw needs. Then iterate: bundle → rebuild → fix each cljw-feature gap cache_gen surfaces (expect the regex path, possibly deftype/protocol edge cases, dynamic-var/alias) → verify s/def/valid?/conform/cat/keys vs the clj oracle → clj_corpus spec area. Multi-cycle but the line-by-line blocker path is now mapped. LOAD-BLOCKER CHAIN (probed 2026-06-20 via `-cp /tmp/spec_cp` classpath load of the official src — this is the right dev harness: the classpath resolver loads .clj WITHOUT touching the @embedFile bundle, so the tracked build stays green while iterating blockers; bundle into FILES only once it loads + s/valid?/conform verify): #1 gen/alpha.clj `BigDecimal/valueOf` — FIXED commit 1a5e0ed0 (the static cljw.java.math.BigDecimal descriptor had an empty method_table; added valueOf + an .init populate fn). #2 gen/alpha.clj:188 `bytes?` unresolved — cljw lacks the predicate AND it is AD-019-blocked (JavaArray is fully type-erased, no element-type field, so byte[] is indistinguishable from int[]; = the known D-472). gen-builtins needs ALL 43 core type-predicates to resolve at analyze time (delay-body map KEYS); only bytes? is missing — so the port adapts that one gen-builtins entry (gen is generators-only, never forced without test.check) OR a deliberate bytes?=array? AD decision in the focused cycle (DA subagent on the AD-019 envelope). #3+ spec.alpha CORE (1996 lines) unprobed past gen; static scan: 12 `reify` of the Spec/Specize protocols (THE core spec mechanism — cljw reify-of-protocol must carry these), 2 `set!` on dynamic vars (*recursion-limit* etc.), defmulti/defmethod `describe`, `(alias 'c 'clojure.core)`, JVM-internal RT/checkSpecAsserts + Compiler/demunge (AD-008 deferred-host-ref handles those — ns loads, errors only if called). MILESTONE 2026-06-20: spec.alpha LOADS (`:loaded`) — 6 blockers cleared: 3 committed cljw clj-parity fixes [#1 BigDecimal/valueOf 1a5e0ed0; reify/deftype destructured method params 8c4f4827; (symbol var) f74f8f18] + 3 port adaptations in scratch /tmp/spec_cp [drop the bytes? gen-builtin (AD-019), `clojure.spec.alpha.Spec` FQCN→protocol var `Spec` (no-JVM, ADR-0059), checkSpecAsserts JVM-static→cljw atom]. REMAINING FUNCTIONAL BLOCKER (spec loads but s/valid? errors `Unable to resolve spec`): spec's `with-name` (alpha.clj:90) attaches `::name` metadata to every spec object, but a spec object is a `reify` instance and cljw reify instances are NOT IObj/metable — `(instance? clojure.lang.IObj )` is false + `(with-meta m)` raises `cannot attach metadata to reified_instance`, so `with-name` returns nil → the registry stores nil → lookup fails. clj's reify ALWAYS auto-implements IObj (metable); plain deftype does NOT (cljw already matches deftype). FIX (reify-only, ADR-0134-coupled — the instance?-IObj⟹with-meta-works invariant D-271): add a `meta: Value` slot to ReifiedInstance (16→24B + the @sizeOf asserts at type_descriptor.zig:476/797), init in allocReifiedInstance, mark it in traceReifiedInstance (mirror traceTypedInstance), a reifiedInstWithMeta minting helper, the withMetaFn/.reified_instance arm mints-fresh instead of raising, the metaFn/.reified_instance arm returns inst.meta, and isInstance/matchUserType answers IObj+IMeta true for reified_instance. Needs an inline ADR (extends ADR-0134 IObj activation to reify) + mandatory DA fork. Port adaptations still need promoting from /tmp into the tracked bundle once reify-IObj lands + s/valid?/conform verify green. MILESTONE-3 2026-06-20: reify-IObj-metadata LANDED (commit 458be654, ADR-0134 reify slice) → spec's registry resolves. BASIC SPECS NOW FUNCTIONAL + clj-verified: `(s/def ::age (s/and int? pos?))` + s/valid?/conform all correct. 5 GENERIC cljw clj-parity fixes extracted+committed this arc (BigDecimal/valueOf, reify/deftype destructured method params, symbol-of-var, reify-IObj-metadata) — these benefit ALL cljw code, not just spec. 4th port adaptation added in /tmp scratch: fn-sym→nil (alpha.clj:131 used `(.. f getClass getName)`+regex to name anon fns; cljw fns have no class name, so anon-fn specs describe as ::unknown via the specize* Object fallback — acceptable divergence). REMAINING DEEP TAIL (the regex-op + every machinery — spec-internal, diminishing generic-fix yield, a focused multi-cycle continuation): `s/coll-of`/`s/every` → alpha.clj:1291 `cpred` is nil at conform (every-impl closure-capture or opts-destructure issue — classify generic-cljw vs spec-internal); `s/or` → Exception; `s/keys` → alpha.clj:317 (unfn conj-on-symbol — cljw's postwalk-replace/destructure/conj all individually MATCH clj, so the trigger is a specific s/keys path, needs isolation); `s/cat`/`s/*` regex-op conform → `::invalid` (the regex-op state-machine — the hardest part). The 4 port adaptations (bytes?-drop, Spec-FQCN→var, checkSpecAsserts→atom, fn-sym→nil) live in /tmp/spec_cp and are re-derivable from this blocker chain; promote into the tracked bundle (src/lang/clj/clojure/spec/ + bootstrap.zig FILES, EPL variant-① header per clj_attribution.md) only once the regex-op tail is green. MILESTONE-4 2026-06-20: the regex-op tail is GREEN — 2 more generic cljw fixes landed [:keys [::ns-key] auto-resolve a17b97c8 (unblocked coll-of/s/* via every-impl's {:keys [::cpred]}); (conj map other-map) merge 8fc081d4 (unblocked the whole pcat* state machine → s/cat/s/*/s/+/s/alt conform)]. spec.alpha is now MOSTLY FUNCTIONAL + clj-verified: load + s/def + s/and (basic) + s/valid? + s/conform + s/cat + s/* + s/+ + s/alt + s/coll-of + s/keys all correct. The LAST blocker is the builtin-macro-shadow bug → D-476 (s/or mis-routes to core/or; strict s/and validates only the last pred). 7 generic clj-parity fixes total this arc (BigDecimal/valueOf, reify/deftype destructured params, symbol-of-var, reify-IObj-metadata, :keys auto-resolve, conj map-merge — all committed + benefit ALL cljw code). Resolve D-476, then promote the /tmp port into the bundle + verify the full surface. MILESTONE-6 2026-06-20: D-476 RESOLVED (7c40041f, macro-shadow gate + caseTest rt/or hygiene) → s/or + strict s/and work. 3 more generic fixes landed: clojure.walk over a cons/lazy-tailed list (cf0b8dd2), valueToForm .hash_map macro return (978b41e2). 10 generic cljw fixes total this arc; spec ~95% functional (s/def/and/or/valid?/conform/cat/*/+/alt/coll-of all clj-verified). THE ONE REMAINING BLOCKER = s/keys: cljw's `&`-destructure (sequentialDestructure, macro_transforms.zig:344) uses nth/nthnext, but clj's `& rest` expands to (seq g)/(first cur)/(next cur) — fails on the non-Indexed map at alpha.clj:837 `(loop [… [[k v] & ks] m] …)`. Fix = make the `&`-case seq-based (clj-faithful; NO-`&` keeps nth which clj-correctly errors on maps). Then drain any further conform-path blockers. Full spec spec/fix sequence is in `.dev/handover.md` First-commit-on-resume + `private/notes/spec-alpha-port-arc.md`." category: "clj-parity" barrier: "none structurally (pure-Clojure, bundleable; gen is a lazy stub so it loads without test.check) — it is just SUBSTANTIAL + deserves a fresh focused cycle (copy 2 .clj into the bundle, AOT-rebuild, drain the cljw-feature gaps the load surfaces). The HIGHEST-value remaining clj-parity unit." last_reviewed: "2026-06-23" @@ -6982,9 +7004,9 @@ discharged: barrier: "none — reproduce, then either fix the map-state bug or make the class-lookup fall through cleanly to symbol_unresolved on a types miss." last_reviewed: "2026-06-13" - id: "D-463" - status: "DISCHARGED 2026-06-25 (debt audit — the lone residual is CLOSED: cljw now emits the per-var lifecycle events; a custom `report` reporter sees the full [:begin-test-ns :begin-test-var :end-test-var :end-test-ns] set bit-for-bit with clj, verified directly). ORIGINAL: PARTIAL (2026-06-18 — clojure.test report-format fidelity LANDED; only per-var lifecycle-event emission residual). The named scope is DONE: (1) `*test-out*` re-enabled (`(def ^:dynamic *test-out* *out*)` + `with-test-out` now rebinds *out* to it; `(resolve 'clojure.test/*test-out*)` non-nil, `(binding [*test-out* w] (run-tests))` redirects — the no-JVM deferral reason was obsolete); (2) FAIL/ERROR 'in' line uses `testing-vars-str` rendering clj's `(test-name)` list form; (3) `=` + generic-predicate `:actual` now the clj `(not (= 1 2))` / `(not (pos? -1))` bare-symbol not-form (added a clj-style `function?` guard to the :default assert-expr so a predicate call gets the (not …) treatment, a value/macro form the plain value); (4) context line now the clj `outer inner` join (was ` context: [..]`); (5) a `:begin-test-ns` 'Testing ' line. Mechanised: e2e phase15_clojure_test_output.sh (clj-grounded) + phase15_clojure_test_tap.sh golden updated (now `(pass-t)`/`(= 1 1)`/`(not (= 1 2))`, clj-faithful). Unmatchable parts → AD-041 (no source location ⇒ no ` (file:line)` suffix; no JVM stack ⇒ :error prints the ex value not a cause trace). RESIDUAL = per-var lifecycle events (begin/end-test-var, end-test-ns) are not emitted, so a custom reporter (tap) sees a subset of the events clj emits — low value (only affects custom-reporter diagnostic noise; the default reporter has empty methods for these)." + status: "DISCHARGED 2026-06-25 (debt audit — the lone residual is CLOSED: cljw now emits the per-var lifecycle events; a custom `report` reporter sees the full [:begin-test-ns :begin-test-var :end-test-var :end-test-ns] set bit-for-bit with clj, verified directly). ORIGINAL: PARTIAL (2026-06-18 — clojure.test report-format fidelity LANDED; only per-var lifecycle-event emission residual). The named scope is DONE: (1) `*test-out*` re-enabled (`(def ^:dynamic *test-out* *out*)` + `with-test-out` now rebinds *out* to it; `(resolve 'clojure.test/*test-out*)` non-nil, `(binding [*test-out* w] (run-tests))` redirects — the no-JVM deferral reason was obsolete); (2) FAIL/ERROR 'in' line uses `testing-vars-str` rendering clj's `(test-name)` list form; (3) `=` + generic-predicate `:actual` now the clj `(not (= 1 2))` / `(not (pos? -1))` bare-symbol not-form (added a clj-style `function?` guard to the :default assert-expr so a predicate call gets the (not …) treatment, a value/macro form the plain value); (4) context line now the clj `outer inner` join (was ` context: [..]`); (5) a `:begin-test-ns` 'Testing ' line. Mechanised: e2e test/clj/suites/test_reporters_test.clj (clj-grounded) + test/clj/suites/test_reporters_test.clj golden updated (now `(pass-t)`/`(= 1 1)`/`(not (= 1 2))`, clj-faithful). Unmatchable parts → AD-041 (no source location ⇒ no ` (file:line)` suffix; no JVM stack ⇒ :error prints the ex value not a cause trace). RESIDUAL = per-var lifecycle events (begin/end-test-var, end-test-ns) are not emitted, so a custom reporter (tap) sees a subset of the events clj emits — low value (only affects custom-reporter diagnostic noise; the default reporter has empty methods for these)." category: "clj-parity" - barrier: "none for the report-format scope (DONE). Remaining = per-var lifecycle event emission (begin/end-test-var, end-test-ns) — a small run-tests/test-var change to (do-report …) around each var, with matching report methods (default = no-op, like clj). Low priority: clj's default reporter ignores them; only custom reporters (tap) would gain the extra diagnostics. Take up if a real custom-reporter consumer needs the full event stream; mechanise by extending phase15_clojure_test_output.sh / _tap goldens." + barrier: "none for the report-format scope (DONE). Remaining = per-var lifecycle event emission (begin/end-test-var, end-test-ns) — a small run-tests/test-var change to (do-report …) around each var, with matching report methods (default = no-op, like clj). Low priority: clj's default reporter ignores them; only custom reporters (tap) would gain the extra diagnostics. Take up if a real custom-reporter consumer needs the full event stream; mechanise by extending test/clj/suites/test_reporters_test.clj / _tap goldens." last_reviewed: "2026-06-18" - id: "D-472" status: "DISCHARGED 2026-06-25 — `bytes?` landed as `(defn bytes? [x] (rt/array? x))` (core.clj), over-broad per AD-051: cljw arrays are type-erased (AD-019 — byte-array/int-array/object-array share one `[]Value` shape), so a byte-array is runtime-indistinguishable from any other array; cljw answers true for ANY array. The common positive guard `(bytes? (byte-array n))` matches clj; non-byte arrays diverge (clj false, cljw true). A Devil's-advocate fork (mandatory depth≥2) recommended always-`false`; the main loop OVERRODE it because a probe falsified the DA's premise — byte-arrays ARE byte-usable in cljw (`(String. (byte-array [104 105]))` => \"hi\"), and since all arrays share one representation an int-array routed into byte-handling behaves identically (no distinct byte-path to corrupt), so over-broad-positive is internally consistent, not a silent lie; always-`false` would fail the predicate's DEFINING case. clj-matching cases corpus-pinned (clojure_1_11_additions), divergent cases e2e-pinned (phase14_arity_parity AD-051). `uri?` half was already done 2026-06-20. Record: private/notes/2026-06-25-D472-bytes-da-fork.md." diff --git a/.dev/optimizations.md b/.dev/optimizations.md index 8020e25d9..fcbafc728 100644 --- a/.dev/optimizations.md +++ b/.dev/optimizations.md @@ -103,6 +103,9 @@ | O-039 | `runtime/numeric/promote.zig` (`operandManaged`/`OwnedManaged` + add/sub/mul BigInt else-branches) (ADR-0148; bigint_factorial) | The integer/BigInt arithmetic else-branch (reached when an operand is a heap BigInt) called `coerceToManaged` on BOTH operands, which **clones** a `.big_int` operand via `cloneWithDifferentAllocator` (alloc + limb memcpy) — yet `r.add`/`sub`/`mul` only READ them. `bigint_factorial` (`(reduce *' …)`, acc grows to ~9 limbs) clones the growing accumulator on every `*`. Same waste O-037 fixed for ratios, single-Managed flavour. | `operandManaged(rt, v, *OwnedManaged)` returns `*const Managed`: a BigInt aliases its stored Managed (zero alloc); an immediate Long materialises into a caller-stable `OwnedManaged` local (`deinit` only when active). The 3 else-branches use it; the `@addWithOverflow` both-int sub-paths keep `coerceToManaged` (both small, nothing to alias). | Removes the per-op accumulator clone. **`32_bigint_factorial` ReleaseSafe 26.4→22.4 ms (12 runs; 1.49×→~1.27× vs Babashka 17.7 ms).** ratio_sum unchanged (separate path). Broad win for all BigInt `+ - *`. | `zig build test -Dwasm` ×2 (diff oracle) + corpus + smoke; output unchanged (158 = digit count of 100!) | O-037, ADR-0148, D-450 | | O-038 | `runtime/numeric/promote.zig` (`ratioArith` scratch) + `runtime/numeric/ratio.zig` (`allocFromManagedPair` reduce-scratch) (ADR-0148; ratio_sum) | Each ratio op allocated its transient cross-multiply + gcd/divTrunc scratch as individual `Managed.init(rt.gc.infra)` calls — `ratioArith` 2-4 (rn/rd/lhs/rhs) + `allocFromManagedPair` 4 (gcd/r_num/r_den/rem) — so ~8 GPA malloc/free per `+`. `sample` profiling showed `libsystem_malloc` (`_xzm_*malloc`/`_xzm_free`/`memset`/`bzero`) ≈ 20% of leaf self-time. | Each function opens ONE `std.heap.ArenaAllocator` over `rt.gc.infra` and allocates all its scratch Managed from it; `defer arena.deinit()` bulk-frees on return. The two arenas don't alias (rn/rd live on `ratioArith`'s arena, read inside `allocFromManagedPair` which has its own). Result BigInts (numer/denom) still clone onto `gc.infra` (GC-managed, outlive the call). | Collapses ~8 individual malloc/free into ~2 arena chunk alloc/free per op. **`33_ratio_sum` 103.3→82.9 ms (12 runs); combined with O-037 108.1→82.9 (23%).** Re-profile: mem-leaf 227/1950 ≈ 11.6% (alloc no longer dominant; residual is interpreter dispatch + bignum compute → D-386). Broad win for all ratio arithmetic. | `zig build test -Dwasm` ×2 (diff oracle) + corpus 3157/3157 + smoke gate; output unchanged | O-037, ADR-0148, D-450 | | O-056 | `lang/clj/clojure/core.clj` `pop` (vector arm) + `lang/primitive/collection.zig` `__vector-pop` (new) | `(pop v)` on a vector was `(into [] (take (dec (count coll)) coll))` — an O(n) REBUILD of a whole fresh trie. Meanwhile `runtime/collection/vector.zig::pop` already implements the JVM `PersistentVector.pop()` algorithm correctly (tail fast path; else pull the last leaf out of the root and collapse a level), and was reachable ONLY from `java.util.ArrayDeque`. A stratified-design ABSTRACTION-BARRIER violation (Grokking Simplicity ch.8): the layer above re-implemented a barrier operation with high-level seq ops instead of calling through it | Expose the existing barrier fn as the `__vector-pop` primitive and have `pop` call it. No new lower-layer code — the "minimal interface" move: the barrier already had the op | O(n) -> O(log32 n), and it now SHARES structure with the parent instead of allocating a fresh trie. Measured 2000 pops, ReleaseSafe: n=1000 539.5 -> 1.14 ms; n=4000 2151.5 -> 1.30 ms; **n=16000 8278.3 -> 2.47 ms (~3352x)**. Reference JVM clj 1.12.4 on the same box/probe: 1.93 ms — cljw was ~4288x SLOWER than clj here and is now within ~1.3x | `scripts/check_corpus_regression.sh` **4347/4347** golden clj outputs reproduced + e2e `phase14_peek_pop`, `phase14_deftype_collection_base`, `phase14_deftype_object`, `phase14_catch_internal` + spot checks (meta preserved, list arm -> rest, queue arm -> `__queue-pop`, empty still raises IllegalStateException) | D-044, ADR-0148, `.dev/bench/` | +| O-061 | `runtime/collection/chunked_seq.zig` (NEW, the IChunkedSeq boundary) + `lang/primitive/sequence.zig` (`chunked-seq?` / `__chunk-count` / `__chunk-nth` / `chunk-rest`) + `lang/primitive/chunk_transform.zig` (O-032 drains) | The seq of a vector (an `.array_seq` view, O-058) answered `chunked-seq?` false, so `map`/`filter`/`keep`/`map-indexed` over a vector built one lazy node + one interpreted thunk per element; clj's `PersistentVector$ChunkedSeq` realizes a 32-element leaf per step (a laziness divergence too: `(first (map f v))` ran f once, clj 32 times) | One boundary answers isChunked / chunkCount / chunkNth / chunkRest for `.chunked_cons` and the vector-backed view, with clj's leaf geometry (cursor i exposes i..(i\|31)+1, bounded by count). subvec and Java-array seqs stay unchunked (clj APersistentVector$Seq / ArraySeq are not IChunkedSeq). Naive fallback: per-element lazy walk | 8000-element vector, ReleaseSafe -Dwasm, per call (bb): `(reduce + (map inc v))` 19271 -> 1204 us (420); `(count (filter even? v))` 19646 -> 927 (221); `(into [] (map inc v))` 18082 -> 1306 (302) | `chunked_seq.zig` units (leaf alignment, mid-leaf cursor, tail) + `test/clj/suites/chunked_seq_test.clj` `a-vector-seq-is-chunked-by-leaf` (realization counts 32 / 27, chunk-rest counts 38 / 38, measured on bb) + full suites | ADR-0205, O-032, O-058 | +| O-062 | `lang/primitive/map_step.zig` (NEW, `cljw.internal/__map-step`) + `core.clj` `-map-n-lazy` / `map` 2- and 3-coll arities | Multi-coll `map` interpreted a let / and / seq / first / rest / cons body per element | The step body (seq every source, f on the firsts, then the rests, cons onto `(self f rests..)`) runs in Zig; the lazy node and the recursion stay in core.clj, so the unchunked one-f-per-node shape, its order and its realization counts are clj's. Naive fallback: the `.clj` body | see ADR-0205 Consequences | suites + `chunked_seq_test.clj` `multi-coll-map-is-unchunked` | ADR-0205 | +| O-064 | `eval/backend/tree_walk.zig` (`uninitScratch`; `callMethodImpl` locals, `evalCall` args + arg locs, `evalRecur`, `evalConstructorCall`) + `eval/backend/vm.zig` (`eval` handler stack) | ReleaseSafe fills every `= undefined` local with 0xAA, not only Debug. The per-call scratch windows (`[MAX_LOCALS]Value`, about 2 KB each) were memset on every Clojure fn call: 19% of instructions in `(reduce + (map * xs xs))`, 18-59% of samples in transduce/map after ADR-0205 | `uninitScratch(T)` returns `undefined` with runtime safety off outside Debug. Every user reads only a prefix it has just written, so the poison caught nothing; Debug keeps it, and the callers' own checks stay on | User instructions, ReleaseSafe -Dwasm, `taskset -c 2`: seq_dot_1k 16.02G -> 13.15G (-18%), lazy_chain -38%, multimethod_dispatch -35%, ratio_sum -27%, protocol_dispatch -26%, transduce -24%, gc_large_heap -50%, startup 74.2M -> 66.9M (-10%); no cell of 01-45 regressed | `test/run_all.sh --smoke` (unit VM + TreeWalk, corpus, clj suites, e2e) | card CLJW-PERF-UNDEFINED-SCRATCH, knowledge 20261022011049-02b6329c | | O-060 | `runtime/collection/string_seq.zig` (NEW, `.string_seq` A13 — a day-1 reservation with no producer) + `lang/primitive/sequence.zig` (`seqFn`/`countFn`/`firstFn`/`restFn`/`nextFn` + rev-walk `.string` arm → view) + `.string_seq` arms across `equal.zig` (isSequential via the interface_membership SSOT / isCountable / seqLen / valueHash / isSeqKeyValue) / `print.zig` (`printStringSeq`, ports-threaded) / `higher_order.zig` (fuse-base + chunkClassify) / `clojure_lang_method.zig` (native-collection arms) / `lazy_seq.zig` / `metadata.zig` / `meta.zig` / `runtime.zig` (GC trace registration) | `(seq s)` built an EAGER n-cell codepoint cons chain by asking `charset.codepointAt(s, i)` per element, and that restarts a `Utf8Iterator` from byte 0 every call — n calls at O(i) each = **O(n²)** overall. Measured cljw eager path: 2,000 chars 27 ms → 146,670 chars **89,910 ms**; downstream, bb-mcp paid ~79 s/boot asking `(seq value)` as a truthiness test on a 146,673-char nREPL response | A byte-offset VIEW satisfying the seq protocol: `StringSeq{backing, offset}`, two words. The cursor is a BYTE offset, not a codepoint index (an index would reintroduce the byte-0 walk): `first` decodes in place at `offset`, `rest` advances by the width it just read, `count` is a single forward byte scan (UTF-8 gives no length arithmetic — unlike the vector view's subtraction — but the scan allocates nothing and is no worse than `(count s)`). Consumers (`printStringSeq`, both `equal.zig` cursors) take the byte walk too rather than `nth(i)` (`nth` on UTF-8 is O(i) however the view is built, so the index-walk shape copied from array_seq would rebuild the quadratic inside the fix). `decodeAt` is total (malformed byte → width-1 raw), so a walk always advances — a view can never wedge or loop. Slot from the F-004 A13 day-1 reservation (already declared across interface_membership + class_name displayClassName), NO `heap_tag.unallocated` slot spent — the same position `.array_seq` (O-058) was in | **O(n²) → O(n).** NEW cljw ReleaseSafe end-to-end (incl. process startup + O(n) string build): `(count (seq (str-of-n "x")))` n=100k **0.13 s**, n=500k **0.60 s** (linear); the eager path's 89,910 ms at 146k chars is now sub-second (**>10,000× at that size**). clj JVM isolates the same seq+count to ~0 ms (StringSeq view + JIT); cljw's view is likewise O(1)/step, its residual wall time being the O(n) string construction, not the seq walk. Broad win: every hand-written `(seq s)` / `doseq`/`for` over a string and every `(seq value)` truthiness probe on a large string | `zig build -Dwasm -Doptimize=ReleaseSafe` (string-seq binary compiles) + corpus `string_seq.txt` (33 exprs vs clj, `33/33 reproduced`) + the 47-assertion observable-surface pin set, landed as e2e `string_seq_view.sh` and since migrated verbatim to `test/clj/suites/string_seq_view_test.clj` (vs clj: prints-as-seq, seq?/sequential?/coll?/not-string/not-vector, class=StringSeq, UTF-8 count/first/rest/nth/last, second/fnext/nthrest, `=`/hash with list AND vector + map-key + set-dedup + length-mismatch loses, vec/into/apply-str/reverse/take/drop/cons/concat, map/filter/reduce, empty→nil, scale count 20k + utf8 10k) | D-179, O-058, F-004, `feat/string-seq-view` | | O-059 | `runtime/collection/sub_vector.zig` (NEW, `.sub_vector` D11) + `lang/clj/clojure/core.clj` `subvec` (fast-path) + `lang/primitive/collection.zig` (`__subvec` prim + `conjOne`/`getFn`/`nthFn`/`assocFn`/`containsQFn`/`__vector-pop`/`isEntrySeqTag`) + `.sub_vector` arms across `interface_membership.zig` (16 arrays) / `sequence.zig` / `core.zig` / `equal.zig` / `print.zig` / `compare.zig` / `lookup.zig` + `tree_walk.zig` / `meta.zig` / `metadata.zig` / `class_name.zig` / `lazy_seq.zig` / `walk.zig` / `json.zig` / `higher_order.zig` / `intrinsic.zig` / `serialize.zig` / `clojure_lang_method.zig` / `multimethod.zig` / `transient_array_map.zig` / `array_seq.zig` (backing arm). `vector.subvec`→`copyRange` (materializing; LBQ compaction) | `clojure.core/subvec` on a vector was `(into [] (take (- end start) (drop start v)))` — an eager O(n) rebuild of a whole fresh trie per call, the LAST catastrophic collection gap (measured 7480x slower than clj). JVM never copies: `subvec` returns an `APersistentVector$SubVector` sharing the parent's root+tail (D-044/D-583). The Zig `vector.subvec` (eager) was reachable only from LBQ compaction, which genuinely wants a copy — so the view had to be a NEW representation, not a rewrite of it | A new REPRESENTATION satisfying the existing IPersistentVector protocol: `SubVector{start,end,parent,meta}`, 32 bytes fixed whatever the span. `count = end-start` (O(1)), `nth(i) = parent.nth(start+i)`. JVM `SubVector` semantics: FLATTEN on nest (parent is always a plain `.vector`), never-empty (empty range → EMPTY plain vector), write ops stay SubVectors via `parent.assocN` (`conj`/`assoc`/`pop`, structural sharing — never materialize). Slot came from the F-004 `box`→`sub_vector` amendment (user-selected, ADR-0179), 64-slot layout unchanged. Interface membership matches JVM ground truth exactly: `.sub_vector` joins every array `.vector` is in EXCEPT `EDITABLE_TAGS` (no transient) and `IREDUCEINIT_TAGS` (SubVector is not IReduceInit) — verified live against clj 1.11.1. seq/rest/next route through the O-058 `.array_seq` view (its backing switch gained a `.sub_vector` arm), so a subvec's seq is O(1) too. `=`/hash reuse the equal.zig `.vec` cursor by init'ing it with the parent + absolute [start,end) indices — zero new cursor code | O(n) alloc + trie rebuild -> O(1). Measured 2000 subvec ops on a 16000-element vector, ReleaseSafe: **9215 -> 0.77 ms (~11967x)**. Reference JVM clj 1.11.1, same box/probe: 1.06 ms — cljw was ~8693x SLOWER than clj and is now at PARITY (slightly faster). Broad win: `subvec` is the idiomatic vector slice, and it now shares structure instead of rebuilding | `zig build test -Dwasm` ×2 (VM + tree_walk diff oracle, all Values equal) + new `sub_vector.zig` units (layout, flatten-on-nest, never-empty, conj/assoc/pop parent-untouched, with-meta) + the 52-assertion observable-surface pin set (vector?/hash-eq/map-key/set-dedup/compare/into-{}/walk/nested/structural-sharing/meta/class=SubVector/empty+full-range/OOB-throws), landed as e2e `subvec_view.sh` and since migrated verbatim to `test/clj/suites/subvec_view_test.clj` + corpus `subvec.txt` (31 exprs) + `phase14_subvec.sh` (bounds) still green | D-583, D-044, O-058, ADR-0179, F-004, `private/notes/subvec-seqview-design.md` | | O-058 | `runtime/collection/array_seq.zig` (NEW, `.array_seq` A14) + `lang/primitive/sequence.zig` (`seqFn`/`restFn`/`nextFn`/`countFn`/`firstFn` + the three seq-walk helpers) + `.array_seq` arms in `equal.zig` / `print.zig` / `higher_order.zig` / `lazy_seq.zig` / `collection.zig` (`nthFn`) / `clojure_lang_method.zig` | `(seq v)` / `(rest v)` / `(next v)` on a vector built an EAGER `PersistentList` copy (`vectorToList` / `vectorTailAsList`): n `consHeap` allocations AND n trie descents per call. Clojure never copies — `PersistentVector$ChunkedSeq` is a VIEW holding `(vector, index)`. `.dev/bench/README.md` ranked this #2 and called SubVector (#1) its prerequisite, on the reading that a chunked seq's `next` must be "the vector minus its first 32 elements". It is not: `next` of a seq is a SEQ, and clj's own answer is a ChunkedSeq over the SAME vector at a higher index. So #2 never depended on #1, and the `heap_tag` census question #1 raises does not gate it | A new REPRESENTATION satisfying the existing seq protocol (OCP — no consumer changes, no `if` at call sites): `ArraySeq{backing, index, meta}`, 32 bytes, fixed size whatever it spans. `first` = one indexed read, `rest` = the same view at `index+1`, `count` = `backingCount - index`, `nth` = one indexed read (clj WALKS here — ChunkedSeq is Counted, not Indexed). Invariant: `make` answers nil when the view would be empty, mirroring `.range`, which is what makes `first` total. Costs NO slot from `heap_tag.unallocated`: A14 `.array_seq` was a day-1 reservation already declared across every `interface_membership.zig` protocol table and already named in `class_name.zig` but had no producer — `equal.zig`'s header literally anticipated "when a producer mints those tags". It also REPAIRS a latent census inaccuracy: A14 was classified boxed-in-production while nothing produced it | O(n) alloc + O(n log32 n) -> O(1) for all three. Measured 2000 ops on a 16000-element vector, ReleaseSafe: **`seq` 2123 -> 0.36 ms (~5897x), `rest` 2253 -> 0.39 ms (~5777x), `next` 2164 -> 0.38 ms (~5695x)**. Reference JVM clj 1.12.4, same box and probe: 0.60 / 0.87 / 0.99 ms — cljw was 2184-3549x SLOWER than clj on these and is now 1.7-2.6x FASTER. Broad win: this is what every hand-written `loop`/`recur` seq walk and every library seq walk pays (the idiomatic `reduce`/`doseq`/`into` paths already bypassed `seq` via O-002/O-004/O-032, which is why no single benchmark regressed on it) | `zig build test -Dwasm -Doptimize=ReleaseSafe` x2 (VM + tree_walk diff oracle) + new array_seq.zig units (layout, never-empty invariant, count-at-offset, rest/drop termination, saturating drop) + new the 76-assertion observable-surface pin set, landed as e2e `vector_seq_view.sh` and since split — 75 expression pins migrated to `test/clj/suites/vector_seq_view_test.clj`, the GC-torture rooting case kept in the e2e because CLJW_GC_TORTURE is read at process start. Pins: print-as-seq, `=`/`hash`/map-key interop with list AND vector, length-mismatch must lose, count exact at every offset, nth + default + OOB + negative, empty/one-element boundaries, next-walk visits each exactly once, 5000-element multi-level trie, nested collections) + `CLJW_GC_TORTURE=1` on the view (proves the trace fn reaches `backing`) + `phase16_gc_torture` | D-044, O-056, O-057, ADR-0179, F-004, `.dev/bench/README.md` | @@ -113,6 +116,17 @@ | O-051 | `runtime/collection/map.zig` (`arrayMapKeywordSlot` helper + `get`/`contains` keyword fast path) (ADR-0165/ADR-0148; destructure, gc_large_heap) | An `.array_map` `get`/`contains` scanned every entry through `keyEq`→`equal.eqConsult`→`isSimpleEqKey`×2→`keyEqValue` (a `!bool` error-union chain) even when the lookup key is a keyword — yet keywords are INTERNED, so `=` over two keywords IS NaN-box payload bit-identity. destructure (`{:keys [a b c]} m` ×100k = 3 keyword gets/iter) + every keyword config-map lookup pay the chain per entry. (O-049 cut the TLV read inside `eqConsult`; this removes the call chain + error union entirely for the dominant keyword case.) | `arrayMapKeywordSlot(am, kw)`: when `k.tag()==.keyword`, scan `am.entries` comparing `@intFromEnum(entry_key)==@intFromEnum(kw)` inline (no call, no `try`), returning the entry index or null. `get`/`contains` take it on the keyword branch; non-keyword keys keep the general `keyEq` path. Correct because keyword interning makes bit-equality ⟺ `=`, and a keyword can only bit-match a keyword entry (distinct NaN-box tags). The `>8`-entry `.hash_map` path is unchanged (keyword-keyed maps are overwhelmingly small). | **clean old-vs-new ReleaseSafe binary A/B (hyperfine -N, 30 runs, same session): `37_destructure` 51.5→48.1 ms (−6.6%); `27_gc_large_heap` 33.2→31.7 ms (−4.5%); 300k-get microbench 23.7→21.1 ms (−11.0%); map-destructure micro 34.8→32.6 (−6.3%).** Broad win for every keyword `(get map k)` / `(contains? map k)` / `:keys` destructure. | `zig build test -Dwasm` ×2 (diff oracle TreeWalk≡VM, identical Values) + new map.zig unit test (keyword hit/miss + mixed keyword/int/string keys, re-interned keyword identity) + lint | O-048, O-049, O-026, ADR-0165, ADR-0148, D-450, D-520 | | O-050 | `runtime/numeric/promote.zig` (`ratioArith` add/sub branch) + `runtime/numeric/ratio.zig` (`allocFromReducedManagedPair`/`finishReducedPair`) (ADR-0148; ratio_sum) | The BIG-tier ratio add `(an*bd ± bn*ad)/(ad*bd)` reduced via a full gcd in `allocFromManagedPair`. O-046's i64 small tier covers tiny ratios, but the harmonic-sum ACCUMULATOR (`(reduce + …)`) grows past i64 → big-tier add ~45×/call, each paying the big gcd. | Knuth TAOCP 4.5.1 / GMP `mpq_add` gcd-first add: `g = gcd(ad, bd)` FIRST (the added term's denom ≤ 50 → a tiny gcd), reduce by g; coprime denominators (g=1) SKIP the final gcd entirely, else a small `gcd(t, g≤50)`. Result is coprime → `allocFromReducedManagedPair` (factored post-gcd tail of `allocFromManagedPair`, no redundant final gcd; zero-numerator→int-0 guarded). mul/div keep the full reduce. | **`33_ratio_sum` ReleaseSafe 39.6→31.0 ms (hyperfine -N, 15 runs, 1.28×±0.20 over the pre-Knuth baseline, same machine/session A/B).** Complements O-046 (small tier); broad win for all big-tier ratio `+`/`-`. | `zig build test -Dwasm` ×2 (diff oracle) + corpus 3656 + 10 clj-verified edge cases (coprime / g>1 reduction / zero / integer-collapse / negative / large-coprime / harmonic) | O-037, O-046, ADR-0148, D-450 | | O-054 (renumbered 2026-08-05 — the 2026-07-07 D-529 commit duplicated O-051, which the 2026-06-24 keyword-map-get row already held; external O-051 references all mean that row) | ~30 sites across `runtime/` + `lang/primitive/` (grep `PERF:` without an `O-NNN` ref for the live list) | Each site's naive form is stated IN its own marker comment (e.g. "empty + N×conj" vs `fromSlice`; seqFn walk vs index-walk; per-element vs chunk-append) | Self-documenting micro fast-paths: the marker text itself records the naive↔optimized pair, so a dedicated ledger row per site would duplicate the comment without adding information. This umbrella row makes that class OFFICIAL (D-529 audit, 2026-07-07): a `PERF:` marker whose comment states its own naive form and cites no `O-NNN` belongs to this row (O-054) implicitly. New MEASURED optimizations still get their own row. | n/a (micro; individually unmeasured by design) | the dual-backend diff oracle + corpus cover each site's behaviour | D-529 | +| O-080 | `eval/backend/tree_walk.zig::callMethodBody` (borrows a `tw_scratch` window) | `var locals: [MAX_LOCALS]Value = undefined;` declared inside the call body, which ReleaseSafe 0xAA-fills (a 2 KB memset) on EVERY fn call, silently undoing O-015's exact-count init | `callMethodBody` borrows an exact-size window from the threadlocal `tw_scratch` (PR #80), which neither 0xAA-fills nor costs native stack per re-entry; it replaced both the original `@setRuntimeSafety(false)` wrapper and the interim O-064 `uninitScratch` buffer (one mechanism for per-call scratch). Only `locals[0..fs]` is ever initialised, published as a GC root or read (O-015 contract unchanged) | The fill was 52% of protocol_dispatch's cycles under perf. Removing it is a per-call win for every TreeWalk-entered call (protocol methods, bootstrap .clj fns), not only protocol calls | `protocol_dispatch_cache_test` + `type_redefine_test` suites, `--compare` dual-backend, smoke `zig_build_test_vm`/`zig_build_test_tree_walk` | ADR-0206, O-015 | +| O-081 | `runtime/dispatch.zig::dispatchCached` + `dispatch/method_table.zig::DispatchCache` (embedded in `protocol.zig::ProtocolFn`) | `callProtocolFn` built a fresh `CallSite` per call, so every protocol call re-ran the method-table scan with string compares, then the ADR-0187 / IRecord / Object fallback chain | One seqlock slot per `ProtocolFn` memoises `(receiver descriptor, protocol_generation) -> resolved method Value` for the WHOLE `resolveProtocolMethod` chain. Every writer of a method table or fallback table bumps `protocol_generation` (extend-type/extend-protocol/extend, ADR-0187 targets, `addProtocolImpl` after the membership write, type redefine), so a mismatch is the whole invalidation story. Descriptors are retired, never freed (D-587), so a cached address has no ABA | A hit costs one descriptor resolve + 4 atomic loads instead of a table scan. bench/vs_bb protocol_dispatch: see the PR for before/after against bb | `test/clj/suites/protocol_dispatch_cache_test.clj` (extend after a failed first call, re-extend, Object default displaced, alternating receivers, record redefine per vintage, defprotocol re-eval, native receivers, reify) = JVM 21/21, `--compare` OK on both backends; `DispatchCache` unit test | ADR-0206 | +| O-070 | `lang/primitive/string.zig` (`-join` leaf, any seqable) + `lang/clj/clojure/string.clj` `join` (bench/vs_bb string_ops) | 2-arity `join` was `(apply str (interpose sep coll))`: a lazy interpose seq (closure call + lazy cell per element) plus an `apply` arg spill; profiled ~87% of string_ops | `-join` is clj's StringBuilder loop in Zig: vector arm indexes directly, every other seqable walks `seq`/`first`/`next` under a GC-root frame; sep and elements render through `print.writeStrValue` (the `str` renderer), so nil→"" and char/keyword/number render as `str` would. Both arities call it | string_ops cljw 3760→943 ms in-proc (with O-071/O-072; 9.33×→2.27× vs bb) | spot vs clj: vector/list/string/map/empty-set colls, nil sep, nil/char/keyword elements, non-seqable raises IAE | O-012 | +| O-071 | `runtime/charset.zig` (`upperCaseAlloc`/`lowerCaseAlloc` ASCII fast path) | upper/lower-case walked a `Utf8Iterator` + SpecialCasing/simple-map table lookup per codepoint even for pure-ASCII input | OR-reduce the bytes; all-ASCII → byte-for-byte `std.ascii` fold (every ASCII codepoint's full mapping is its ASCII fold, 1:1). Non-ASCII keeps the table walk (ß→SS unchanged) | part of string_ops 3760→943 ms | `(upper-case "aßc")`→"ASSC", ASCII round trips | D-057 | +| O-072 | `runtime/regex/match.zig` (`Scanner`) + `runtime/regex/replace.zig` (`splitToVector`) | `split` allocated+freed a fresh VM thread-list pair per match via `findFrom`, then built the result with N persistent `conj`s | One `Scanner` (one VM state) for the whole split; pieces published by `vector.fromSlice` inside a fabrication region | part of string_ops 3760→943 ms | `(split "a,b,,c,," #",")`, `(split "abc" #"")` == clj; regex_parity e2e | O-035, ADR-0150 | +| O-073 | `lang/primitive/collection.zig` (`__subvec-fast`) + `core.clj` `subvec` | `subvec` ran ~10 interpreted checks + an `->idx` closure alloc per call before the O(1) `__subvec` view | `__subvec-fast` answers the in-bounds immediate-int vector/subvec case natively; nil (never a real subvec result) falls through to the unchanged coercing/throwing path | vector_assoc 1452→1092 ms (with O-074; 3.24×→1.94× vs bb) | subvec nested/1-arity/float index/OOB IOOBE/nil NPE == clj | O-059, D-583 | +| O-074 | `runtime/collection/vector.zig` (`copyHamt`/`newTail`) | node copies did `.{ .header = … }` (nil-init all 32 slots) then `@memcpy`: every slot written twice on every conj/assoc/pop/pushTail | Write header + slots once; tails copy the live prefix and nil-fill only the dead suffix | part of vector_assoc 1452→1092 ms | zig unit tests + diff oracle | O-040 | +| O-075 | `lang/primitive/core.zig` (`strFn` stack-buffer path) | multi-arg `str` always built a heap `Allocating` writer (gpa alloc+free) and dispatched `writeStrValue` per arg | when every arg is a string / immediate int / nil and the result fits 256 bytes, concatenate into a stack buffer and allocate the string once; anything else takes the unchanged general path | `(str "item-" i "-x")` x500k 316→144 ms (bb 125) | spot vs clj: nil/int/negative/mixed/float+kw+char fallback/overflow past 256 bytes | O-042 | +| O-076 | `runtime/keyword.zig` (`stackKey` probe in `internUnlocked`/`findUnlocked`) | every `keyword` intern allocated and freed a `ns/name` lookup key even on a hit | probe the table with a stack-built key (or the name itself when there is no ns); allocate only on a miss or a key over 128 bytes | `(keyword "abc")` x500k 235→104 ms; vector_assoc ~950→~800 ms in-proc (A/B, loaded box) | `=` identity of re-interned long/ns keywords, `find-keyword` hit/miss | O-051 | +| O-077 | `lang/primitive/chunk_transform.zig` (`__dorun`) + `core.clj` `dorun` 1-arity | `dorun` (and so `doall`) stepped `(next s)` through the interpreter per element | native walk: chunked_cons skips its whole realized chunk via `chunk-rest`, other seqs step with `seqable.next` (realization runs once per element, in order) | `(doall (map identity (range 2000)))` x250 225→135 ms | side-effect counts, lazy/concat mixes, nil/vector/string/map, non-seqable raises IAE | O-032 | +| O-090 | `eval/bytecode/serialize.zig` (`readValue` `.list` arm), cold-start axis | Each quoted-list constant decoded from the AOT bootstrap blob took its forward-read scratch buffer from `std.heap.page_allocator` | The buffer comes from the reader's allocator (`ctx.allocator`), as `readCollection` already does; the count header is bounds-checked against the remaining bytes first | `page_allocator` is one `mmap` + `munmap` + first-touch page fault per list, 927 per cold start. `perf stat -r 20` on `cljw -e nil`, ReleaseSafe: task-clock 38.8 → 27.3 ms, kernel instructions 22.2M → 7.9M, page faults 2741 → 1792 (bb: 29.1 ms, 4868 faults); pinned `taskset -c 2`: kernel instructions 29.3M → 12.3M (bb 26.8M), user unchanged 61.6M (bb 16.2M, the residual is D-610) | `test/run_all.sh --smoke` + `perf stat`/`strace -c` A/B | card CLJW-PERF-STARTUP | ## Identified high-ROI candidates (measured, not yet implemented) diff --git a/.dev/wasm_percall_findings.md b/.dev/wasm_percall_findings.md index 6bba0bb65..88869b585 100644 --- a/.dev/wasm_percall_findings.md +++ b/.dev/wasm_percall_findings.md @@ -66,6 +66,62 @@ resolution inside `invoke` (zwasm/D-585) and the non-initial-thread stack query (zwasm/D-584), both upstream. `bench/wasm_percall.sh` is the standing measurement of this axis from here on. +### After the zwasm/D-585 fix (measured 2026-10-22) + +`JitInstance` resolved every by-name invoke with `findExportFunc`, a walk of +the section list plus an export-section decode, and the facade's `invokeJit` +did it twice per call (signature, then invoke). zwasm `22a99bb1` +(`fix/jit-percall` on the fork) serves the name from `CompiledWasm.exports`, +the table compile already builds. The `big-*` cells of `bench/wasm_percall.clj` +call a trivial export of a 47 KB module (`bench/wasm/percall/zigcool.wasm`), +which makes a size-proportional cost visible. Median of 7 trials x 20k calls, +ReleaseSafe `-Dwasm`, same host, load average 11-15, the before and after +binaries run back to back: + +| cell | before (d74e63b4) | after (22a99bb1) | +|-------------------|-------------------|------------------| +| JIT, 47 KB module | 5883 | 833 | +| JIT, `add.wasm` | 1031 | 882 | +| interp, 47 KB | 463 | 600 | +| interp, `add.wasm`| 496 | 649 | + +The JIT no longer pays by module size: the 47 KB cell now equals the tiny +one. The interpreter rows did not change in kind (its lookup was always a +table loop); their drift is load noise. `perf` was unavailable on the host +(`perf_event_paranoid`), so the attribution rests on the A/B: the fix changes +only the export lookup. The JIT stays above the interpreter on a trivial +callee; what remains is the entry thunk and zwasm/D-584's per-call stack +query, which ADR-0195 only shrinks. + +### On the merged fork pin (measured 2026-10-22, zwasm `e9d74934`) + +The same fix as merged on the fork's `main` (fork PRs #2 and #3), full +`bench/wasm_percall.clj`, median of 7 x 20k calls, ReleaseSafe `-Dwasm`, load +average 16. With the interp thread-trap fix in the pin, `interp-worker` no +longer traps, so the whole bench runs. + +| cell | ns/call | | cell | ns/call | +|-------------|---------|-|---------------|---------| +| big-jit | 736 | | jit-main | 614 | +| big-interp | 527 | | interp-main | 479 | +| big-auto | 742 | | fn-call | 179 | + +`:jit` on the 47 KB module is 1.4x `:interp` (the acceptance bound is 2x) and +within noise of `:jit` on `add.wasm`. + +`perf record -g` on a loop of 400k `(wasm/call h "popcount" 255)` on the 47 KB +module (`perf_event_paranoid` lowered to 1), stripped ReleaseSafe binaries: + +- v1.15.3 (zwasm `d74e63b4`), 7.2 to 9.3 us a call: 88% of samples in cljw, + about 65% in a single ~40-byte loop (`0x13e6910..0x13e6935`), entered from + two distinct call sites per `wasm/call` (7.7% and 4.9% of the hottest + address). That is the export-section walk, run twice per facade call + (signature, then invoke), as zwasm/D-585 describes. +- this branch (zwasm `e9d74934`), 0.7 to 0.9 us a call, the same as + `add.wasm`: no hot loop. What remains is spread out: `pthread_getattr_np` and + `pthread_getaffinity_np` (zwasm/D-584's per-call stack query, about 6%) and + malloc/calloc/free (about 8%), none of which depends on the module. + ## Two independent causes, both upstream, both already documented there zwasm ADR-0209 (`.dev/decisions/0209_percall_latency_bench.md` in the zwasm diff --git a/.gitattributes b/.gitattributes index 1b30fff8b..6b5e6ef7c 100644 --- a/.gitattributes +++ b/.gitattributes @@ -8,3 +8,7 @@ # Keep read-only historical / generated content out of GitHub's language stats. docs/ja/archive/** linguist-documentation +# Append-only ledgers: parallel branches each add rows at the end, so a union +# merge is the correct resolution (the YAML and ledger gates still check it). +.dev/optimizations.md merge=union +.dev/debt.yaml merge=union diff --git a/.github/workflows/bench-vs-bb.yml b/.github/workflows/bench-vs-bb.yml new file mode 100644 index 000000000..b01908e27 --- /dev/null +++ b/.github/workflows/bench-vs-bb.yml @@ -0,0 +1,138 @@ +# bench-vs-bb: cljw vs babashka vs JVM Clojure on bench/vs_bb/workloads. +# +# Kanban 20261022012233-420c9fea, step 0 of decision 20261022012219-31249feb +# (make cljw faster than bb): every perf step is judged by this table. +# +# This is a REPORT, not a gate. CI has exactly ONE gate configuration +# (ci.yml -> scripts/ci_gate.sh -> test/cljw-test --only full, enforced by +# scripts/check_gate_parity.sh); this workflow adds no tier to it. It never +# fails a PR on a ratio: the job is continue-on-error and run.sh exits 0 on a +# slow or mismatching workload. Its output is the step summary (the table plus +# the YAML history row for this sha), so the ratio for any commit is on its +# run page. A hosted runner is a noisy, shared machine: compare ratios +# (cljw/bb on the same runner, same minute), not absolute milliseconds. +# +# Linux x86_64 only: the ratio is the signal, one host is enough, and the mac +# leg would double the cost of a report. + +name: bench-vs-bb + +on: + pull_request: + paths-ignore: ['**/*.md', '.dev/**'] + push: + branches: [main] + paths-ignore: ['**/*.md', '.dev/**'] + workflow_dispatch: + +permissions: + contents: read + +concurrency: + group: bench-vs-bb-${{ github.ref }} + cancel-in-progress: true + +jobs: + bench: + name: cljw vs bb vs JVM (report, non-blocking) + runs-on: ubuntu-22.04 + timeout-minutes: 45 + continue-on-error: true + steps: + - name: Checkout + uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 + with: + ref: ${{ github.sha }} + + - name: Cache Zig 0.16.0 + id: cache-zig + uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 + with: + path: ~/.local/zig-0.16.0 + key: zig-0.16.0-x86_64-linux + + - name: Install Zig 0.16.0 + if: steps.cache-zig.outputs.cache-hit != 'true' + shell: bash + run: | + set -euo pipefail + source .github/versions.lock + mkdir -p ~/.local + curl -fsSL "https://ziglang.org/download/${ZIG_VERSION}/zig-x86_64-linux-0.16.0.tar.xz" -o /tmp/zig.tar.xz + rm -rf "$HOME/.local/zig-x86_64-linux-0.16.0" "$HOME/.local/zig-0.16.0" + tar -xJf /tmp/zig.tar.xz -C ~/.local + mv "$HOME/.local/zig-x86_64-linux-0.16.0" "$HOME/.local/zig-0.16.0" + + - name: Add Zig to PATH + shell: bash + run: | + echo "$HOME/.local/zig-0.16.0" >> "$GITHUB_PATH" + echo "ZIG_GLOBAL_CACHE_DIR=$HOME/.cache/zig" >> "$GITHUB_ENV" + + # Read-only restore of the gate's Linux build cache (ci.yml saves it). + # Content-addressed, so a stale restore can only cost a rebuild. + - name: Restore Zig deps + build (read-only) + uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 + with: + path: | + ~/.cache/zig + .zig-cache + key: x86_64-linux-zigbuild-${{ hashFiles('build.zig.zon', 'build.zig') }}-${{ github.run_id }} + restore-keys: | + x86_64-linux-zigbuild-${{ hashFiles('build.zig.zon', 'build.zig') }}- + x86_64-linux-zigbuild- + + - name: Install Babashka (pinned, checksummed) + shell: bash + run: | + set -euo pipefail + source .github/versions.lock + archive="/tmp/babashka-${BABASHKA_VERSION}.tar.gz" + curl -fsSL "https://github.com/babashka/babashka/releases/download/v${BABASHKA_VERSION}/babashka-${BABASHKA_VERSION}-linux-amd64.tar.gz" -o "$archive" + actual=$(sha256sum "$archive" | awk '{print $1}') + [ "$actual" = "$BABASHKA_LINUX_AMD64_SHA256" ] || { echo "::error::Babashka checksum mismatch"; exit 1; } + mkdir -p "$HOME/.local/bin" + tar -xzf "$archive" -C "$HOME/.local/bin" bb + echo "$HOME/.local/bin" >> "$GITHUB_PATH" + + # JVM column without the clojure CLI: the runner image ships a JDK, the + # jars come straight from Maven Central. + - name: Fetch JVM Clojure jars + shell: bash + run: | + set -euo pipefail + m=https://repo1.maven.org/maven2/org/clojure + d="$HOME/.cache/vs_bb_jars"; mkdir -p "$d" + for j in clojure/1.12.4/clojure-1.12.4 spec.alpha/0.5.238/spec.alpha-0.5.238 \ + core.specs.alpha/0.4.74/core.specs.alpha-0.4.74 data.json/2.5.1/data.json-2.5.1; do + curl -fsSL "$m/$j.jar" -o "$d/$(basename "$j").jar" + done + echo "CLJW_BENCH_JVM_CP=$(ls "$d"/*.jar | paste -sd:)" >> "$GITHUB_ENV" + java -version + + - name: Build cljw ReleaseSafe -Dwasm + shell: bash + run: zig build -Dwasm -Doptimize=ReleaseSafe -p "$RUNNER_TEMP/cljw-bench" + + - name: Run bench/vs_bb + shell: bash + run: | + set -uo pipefail + hist="$RUNNER_TEMP/vs_bb_row.yaml" + bash bench/vs_bb/run.sh --cljw "$RUNNER_TEMP/cljw-bench/bin/cljw" -n 3 --history "$hist" | tee "$RUNNER_TEMP/vs_bb_table.txt" + { + echo "## cljw vs bb vs JVM (${GITHUB_SHA::8})" + echo + echo "Non-blocking report. Ratios < 1.00 mean cljw is faster. Hosted runner: compare ratios, not ms." + echo + echo '```' + cat "$RUNNER_TEMP/vs_bb_table.txt" + echo '```' + echo + echo "
YAML history row (append to bench/vs_bb/history.yaml)" + echo + echo '```yaml' + cat "$hist" + echo '```' + echo "
" + } >> "$GITHUB_STEP_SUMMARY" diff --git a/.gitignore b/.gitignore index 9dc848bfc..ae37174e6 100644 --- a/.gitignore +++ b/.gitignore @@ -59,3 +59,6 @@ __pycache__/ # built on demand from jsonsum.go (docs/examples/polyglot/README.md), never # committed. The C, Zig and Rust guests are a few hundred bytes and are. docs/examples/polyglot/go/jsonsum.wasm + +# Per-worktree ling scratch and resume notes (never committed) +.scratch/ diff --git a/CHANGELOG.md b/CHANGELOG.md index e3c08c07a..0301a48e1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,87 @@ first stable `1.0.0` tag; pre-1.0 `alpha` / `rc` tags may still change surfaces. ## [Unreleased] +## [1.15.4] - 2026-10-23 + +### Added + +- **`java.time.format`: `DateTimeFormatter`, `DateTimeFormatterBuilder`, + `ChronoField`, `ZoneId` and `ZoneOffset`.** Pattern letters + `G u y M L d D F E a h K k H m s S n A N V X x Z O`, optional sections, + SMART and STRICT resolvers and the 12 `ISO_*` constants. Letters cljw does + not implement raise `not_implemented` instead of formatting wrong text. This + was the last blocker for `malli.transform`. +- **Primitive arrays cross `cljw.ffi` and wasm memory at memcpy speed + (ADR-0204).** `byte`/`short`/`int`/`long`/`float`/`double`/`char`/`boolean` + arrays hold raw native elements. A `:pointer` argument passes the array's + storage with no copy, `:bytes` is one memcpy, `wasm/mem-write!` accepts a + primitive array and the new `wasm/mem-read-array` returns one. `bytes?` is + true only for a byte array. +- **`bench/vs_bb`:** 14 workloads that compare cljw with bb and JVM Clojure, + with a non-blocking CI report. +- **`docs/examples/zig-native`:** one Zig source used both as a wasm guest and + as a `cljw.ffi` library. + +### Changed + +- **Core sequence functions are much faster (ADR-0205).** The seq of a vector + is now chunked by leaf, as clj's `PersistentVector$ChunkedSeq` is, and + `map`, `filter`, `into`, `mapv` and `reduce` go through one chunked-seq + boundary and the Zig `reduce`. On an 8000-element vector, + `(reduce + (map inc v))` went from 19271 us to 718 us and + `(count (filter even? v))` from 19646 us to 490 us. `(first (map f v))` now + realizes 32 elements, as clj does. +- **Protocol calls hit a per-function dispatch cache (ADR-0206).** Each + protocol function remembers the method it resolved for the last receiver + type; every `extend`, `extend-type`, `extend-protocol` and type redefinition + invalidates it. +- **A Clojure fn call no longer memsets its scratch buffers.** ReleaseSafe + filled about 2 KB of locals with 0xAA on every call. Skipping it cuts 18% + of instructions from `seq_dot`, 38% from `lazy_chain`, 35% from multimethod + dispatch and 10% from startup. +- **String and vector hot paths:** `clojure.string/join`, `upper-case`, + `lower-case` and `split`, `subvec`, `str`, keyword interning and + `dorun`/`doall` take native fast paths. The `string_ops` bench went from + 9.3x bb to 2.3x bb. +- **Cold start does less work.** Deserializing the core image no longer maps + a page per quoted list. `cljw -e nil` takes 27.3 ms of CPU, against 38.8 ms + before and 29.1 ms for bb. +- **The GC returns freed memory and reuses it.** Small cells live in 64 KiB + pages that are unmapped once empty, and String, array, transient-vector and + BigInt payload bytes now count toward the collect trigger, so long-lived + processes no longer hold their high-water mark. + +### Fixed + +- **`clojure.edn` reads the EDN dialect.** It used the permissive Clojure + reader and accepted EDN-invalid input; it now matches JVM Clojure's + `EdnReader` on 59 of 59 cases. +- **wasm `i64` arguments and results cross exactly over the full range.** + Values past 2^47 came back as a lossy Double or were refused, and an + integral float of 2^63 crashed the host. +- **A BigInt, Ratio or BigDecimal past about 1233 digits prints its digits.** + It printed the literal `(BigInt)`. +- **A long-running server that reads stdin now gives memory back.** The + `*in*` and `System/in` readers kept every byte they had ever read; they now + drop what has been consumed. While the main thread waits on stdin, each idle + second runs a collect (when at least 1 MiB was allocated since the last one) + and returns freed memory to the OS, so garbage left by `future`s and worker + threads no longer waits for the next request. Socket and `StringBuilder` + buffers now count toward the collect trigger. A bb-mcp-shaped probe (20 + `future`s each building 4 MB of JSON, then idle) settles at 129 MB instead + of holding 955 MB. + +- **A `wasm/call` on the JIT no longer costs time proportional to the + module.** zwasm resolved the export by re-parsing the module on every call; + it now reads the export table built at compile. A trivial export of a 47 KB + module went from about 5.9 us to 0.8 us a call, the same as a tiny module. + `bench/wasm_percall.clj` gains `big-*` cells that measure it. +- **A wasm handle called from a `future` no longer traps on the interpreter.** + zwasm's interpreter cached the first calling thread's native-stack limit on + the instance, so a handle warmed on the runtime thread and then called from + another thread whose stack lay below that limit trapped + `CallStackExhausted`. The limit is now per calling thread (zwasm 79f3d63a). + ## [1.15.3] - 2026-10-22 ### Added diff --git a/bench/ffi_prim_arrays.clj b/bench/ffi_prim_arrays.clj new file mode 100644 index 000000000..92140f35b --- /dev/null +++ b/bench/ffi_prim_arrays.clj @@ -0,0 +1,44 @@ +;; SPDX-License-Identifier: EPL-2.0 +;; Kanban CLJW-FFI-PRIM-ARRAY-BYTES (ADR-0204): what a 1000-element f64 dot +;; costs through cljw.ffi, call only versus building its arguments. +;; +;; cc -shared -fPIC -O2 -o /tmp/libdot.so test/e2e/fixtures/ffi/ffi_fixture.c +;; cljw bench/ffi_prim_arrays.clj /tmp/libdot.so +;; +;; Prints microseconds per call, best of 5 runs. `:pointer` passes a +;; double-array's storage (only where arrays are typed); `:bytes` packs the +;; doubles into a byte array first, the only route before ADR-0204. +(require '[cljw.ffi :as ffi]) + +(def lib (ffi/open (first *command-line-args*))) +(def n 1000) +(def xs (vec (range n))) + +(defn- best-us [iters f] + (apply min (for [_ (range 5)] + (let [t0 (System/nanoTime)] + (dotimes [_ iters] (f)) + (/ (- (System/nanoTime) t0) 1000.0 iters))))) + +(defn- f64s->bytes [ds] + (byte-array + (mapcat (fn [d] (let [b (Double/doubleToRawLongBits d)] + (for [i (range 8)] (unchecked-byte (bit-shift-right b (* 8 i)))))) + ds))) + +(def dot-bytes (ffi/function lib "dot" [:bytes :bytes :long] :double)) +(let [b (f64s->bytes xs)] + (println "bytes-call-only-us" (best-us 2000 #(dot-bytes b b n))) + (println "bytes-pack-and-call-us" (best-us 5 #(let [b (f64s->bytes xs)] (dot-bytes b b n))))) + +(when (try (ffi/function lib "dot" [:pointer :pointer :long] :double) + ((ffi/function lib "dot" [:pointer :pointer :long] :double) (double-array 1) (double-array 1) 1) + true + (catch Exception _ false)) + (let [dot (ffi/function lib "dot" [:pointer :pointer :long] :double) + a (double-array xs)] + (println "pointer-call-only-us" (best-us 2000 #(dot a a n))) + (println "pointer-build-and-call-us" (best-us 2000 #(let [a (double-array xs)] (dot a a n)))) + (println "check" (dot a a n) (reduce + (map * xs xs))))) + +(println "clojure-dot-us" (best-us 20 #(reduce + (map * xs xs)))) diff --git a/bench/gc_heap_return.clj b/bench/gc_heap_return.clj new file mode 100644 index 000000000..8de3eb0aa --- /dev/null +++ b/bench/gc_heap_return.clj @@ -0,0 +1,18 @@ +;; CLJW-GC-HEAP-RETURN acceptance probe. A bench, not a test: RSS is +;; host-dependent. Steady-state bursts of one size must not grow the heap +;; after the first, and a major collection must hand freed memory back to +;; the OS. Run: cljw bench/gc_heap_return.clj +(require (quote clojure.data.json) (quote clojure.string)) +(defn rss-mb [] (let [l (first (filter #(.startsWith ^String % "VmRSS") (clojure.string/split-lines (slurp "/proc/self/status"))))] + (quot (Long/parseLong (re-find #"\d+" l)) 1024))) +(println "start" (rss-mb) "MB") +(def big (atom (vec (range 3000000)))) +(println "after 3M vector" (rss-mb) "MB") +(reset! big nil) +(System/gc) (System/gc) +(println "after drop + System/gc" (rss-mb) "MB") +(dotimes [_ 5] (count (vec (range 3000000)))) +(println "after 5 more 3M bursts" (rss-mb) "MB") +(dotimes [i 200] (count (clojure.data.json/write-str (vec (range 20000))))) +(System/gc) +(println "after 200 x 20k JSON writes" (rss-mb) "MB") diff --git a/bench/vs_bb/README.md b/bench/vs_bb/README.md new file mode 100644 index 000000000..6d0acd919 --- /dev/null +++ b/bench/vs_bb/README.md @@ -0,0 +1,69 @@ +# bench/vs_bb: cljw vs babashka vs JVM Clojure + +Step 0 of decision `20261022012219-31249feb` (make cljw faster than bb, four +measured steps). Kanban `20261022012233-420c9fea`. Each perf step is judged by +this table. + +## Why a new set + +`bench/benchmarks/*/bench.clj` totals 28-190 ms per run against a startup of +23-27 ms, so startup dominates and the real gap stays hidden (cljw/bb read +1.4-2.5x there, while per-call library code measured 22-37x). Every workload +here runs for at least 300 ms on bb, and the runner reports startup on its own +line. + +## Workloads (`workloads/*.clj`) + +| file | bound by | +|------|----------| +| fn_call, fib_recursive, dot_loop_nth, protocol_dispatch | interpreter (user code) | +| reduce_map, lazy_chain, transduce, arrays | core library (seq, transducer, array constructors) | +| into_collections, vector_assoc | persistent collections | +| string_ops | strings and regex | +| numeric_doubles | double arithmetic (NaN-boxing vs boxed Double) | +| gc_churn | allocation and GC with a retained heap | +| json_roundtrip | clojure.data.json write and read | + +Each file is plain portable Clojure that runs unchanged on cljw, bb and JVM +Clojure. It prints `RESULT `, which the runner checks for equality across +runtimes, and `INPROC_MS `, the time measured inside the script without +startup or namespace loading. + +## Runner + + bash bench/vs_bb/run.sh # builds ReleaseSafe -Dwasm into /tmp, 3 runs + bash bench/vs_bb/run.sh --cljw PATH -n 5 --only reduce_map,arrays + bash bench/vs_bb/run.sh --perf --probe # + instructions:u, + per-call probe and RSS probe + +Columns: `*_in` is the in-process ms, `*_wl` the process wall ms, `c/bb_in` the +headline ratio cljw/bb on in-process time (below 1.00, cljw is faster), +`c/jvm_in` cljw over warm-ish JVM. The wall column uses hyperfine when it is +installed, otherwise the min of N. Each run appends one YAML row to +`history.yaml`. + +On a shared or loaded host, read the ratios. Absolute ms are not comparable +across rows. With `--perf`, the `instructions:u` count is the load-independent +check. + +`probe/` holds the original per-call probes (`probe2.clj`: subvec, dot via +reduce/map, dot via loop/nth, byte-array) and the RSS probe (`heap.clj`, with +`heap_bb.clj` for bb, which has no `slurp` of `/proc` files). + +## CI + +`.github/workflows/bench-vs-bb.yml` runs this on ubuntu for every PR and every +push to main. The job is `continue-on-error`, and `run.sh` exits 0 on a slow or +mismatching workload, so the job is a report and never fails a PR. The table +and the YAML row for that sha are in the run's step summary. It is not part of +the gate, so CI still has one gate configuration (memory +`20260911001202-2ee19e2f`). + +## Why bb failed two bench/benchmarks programs + +- `16_protocol_dispatch`: `(extend-type PersistentArrayMap ...)` names the class + unqualified. JVM Clojure and cljw import it implicitly, but SCI does not, so + bb reports `Unable to resolve symbol: PersistentArrayMap`. Here the workload + dispatches on a `defrecord` instead. +- `39_json_parse`: `(def doc ...)` clashes with `clojure.repl/doc`, which bb + refers into `user` (`doc already refers to #'clojure.repl/doc`). JVM `clojure` + run as a script and cljw do not refer it. Here the var is named `docs`. diff --git a/bench/vs_bb/first-run-2026-10-22.txt b/bench/vs_bb/first-run-2026-10-22.txt new file mode 100644 index 000000000..20ed88f90 --- /dev/null +++ b/bench/vs_bb/first-run-2026-10-22.txt @@ -0,0 +1,45 @@ + +cljw vs bb vs JVM commit e1708566 runs=3 (min) load=18.85 wall=min-of-N + cljw ClojureWasm v1.15.3 (ReleaseSafe, wasm) + bb babashka v1.13.224 + jvm clojure 1.12.4 on openjdk version "25.0.4.1" 2026-08-18 + startup (-e nil) ms: cljw 41 bb 28 jvm 1095 cljw/bb 1.46 + idle peak RSS MB: cljw 12 bb 60 jvm 108 + +workload cljw_in bb_in jvm_in | cljw_wl bb_wl jvm_wl | c/bb_in c/bb_wl c/jvm_in | cljwMB bbMB check +arrays 7436 346 484 | 7489 387 1654 | 21.49 19.35 15.36 | 28 131 ok +dot_loop_nth 747 374 78 | 787 411 1202 | 2.00 1.91 9.58 | 13 110 ok +fib_recursive 324 271 32 | 364 312 1105 | 1.20 1.17 10.12 | 13 104 ok +fn_call 743 485 68 | 781 528 1238 | 1.53 1.48 10.93 | 13 110 ok +gc_churn 1589 603 235 | 1910 656 1394 | 2.64 2.91 6.76 | 379 180 ok +into_collections 5985 484 372 | 6133 529 1664 | 12.37 11.59 16.09 | 129 133 ok +json_roundtrip 415 371 479 | 502 409 2358 | 1.12 1.23 0.87 | 50 112 ok +lazy_chain 10258 548 462 | 10511 601 1634 | 18.72 17.49 22.20 | 115 197 ok +numeric_doubles 1082 565 32 | 1116 600 1038 | 1.92 1.86 33.81 | 13 110 ok +protocol_dispatch 1568 344 50 | 1597 389 1052 | 4.56 4.11 31.36 | 13 110 ok +reduce_map 9939 656 740 | 10368 707 2184 | 15.15 14.66 13.43 | 146 163 ok +string_ops 3760 403 496 | 3903 439 1784 | 9.33 8.89 7.58 | 121 110 ok +transduce 5751 358 195 | 5790 389 1400 | 16.06 14.88 29.49 | 27 110 ok +vector_assoc 1452 448 450 | 1563 500 1667 | 3.24 3.13 3.23 | 168 163 ok + +geomean in-process ratio: cljw/bb 4.80 cljw/jvm 10.94 (< 1.00 means cljw is faster) +columns: *_in = ms inside the script (INPROC_MS), *_wl = process wall ms, ratios on min values + +per-call probe (probe/probe2.clj, us per call, min of 7 after warmup): + cljw {:version "1.12.0", :subvec-us 1.3556124999999999, :dot-1k-reduce-map-us 2169.15017, :dot-1k-loop-nth-us 279.632315, :byte-array-vec-8000-us 3095.86732} + bb {:version "1.12.6-SCI", :subvec-us 0.110318, :dot-1k-reduce-map-us 97.765145, :dot-1k-loop-nth-us 246.26002, :byte-array-vec-8000-us 88.45394} + jvm {:version "1.12.4", :subvec-us 0.094437, :dot-1k-reduce-map-us 79.32263, :dot-1k-loop-nth-us 7.8361350000000005, :byte-array-vec-8000-us 52.08972} +RSS probe (probe/heap.clj; bb uses heap_bb.clj): + cljw: + start 13 MB + after 3M vector 44 MB + after drop + System/gc 44 MB + after 5 more 3M bursts 153 MB + after 200 x 20k JSON writes 149 MB + bb: + start 63 MB + after 3M vector 157 MB + after drop + System/gc 120 MB + after 5 more 3M bursts 174 MB + after 200 x 20k JSON writes 177 MB +EXIT 0 diff --git a/bench/vs_bb/history.yaml b/bench/vs_bb/history.yaml new file mode 100644 index 000000000..ffbcc05f9 --- /dev/null +++ b/bench/vs_bb/history.yaml @@ -0,0 +1,74 @@ +# bench/vs_bb history: one row per run of bench/vs_bb/run.sh (append-only). +- date: "2026-10-22T17:12:15Z" + commit: "e1708566" + host: "Linux x86_64" + ci: false + load: "18.85" + runs: 3 + wall_source: min-of-n + versions: + cljw: "ClojureWasm v1.15.3 (ReleaseSafe, wasm)" + bb: "babashka v1.13.224" + jvm: "clojure 1.12.4 on openjdk version 25.0.4.1 2026-08-18" + startup_ms: + cljw: 41 + bb: 28 + jvm: 1095 + geomean_inproc: {cljw_over_bb: 4.80, cljw_over_jvm: 10.94} + workloads: + arrays: + cljw: {inproc_ms: 7436, wall_ms: 7489, rss_kb: 29408} + bb: {inproc_ms: 346, wall_ms: 387, rss_kb: 134180} + jvm: {inproc_ms: 484, wall_ms: 1654, rss_kb: 684272} + dot_loop_nth: + cljw: {inproc_ms: 747, wall_ms: 787, rss_kb: 13920} + bb: {inproc_ms: 374, wall_ms: 411, rss_kb: 112724} + jvm: {inproc_ms: 78, wall_ms: 1202, rss_kb: 135384} + fib_recursive: + cljw: {inproc_ms: 324, wall_ms: 364, rss_kb: 13488} + bb: {inproc_ms: 271, wall_ms: 312, rss_kb: 107028} + jvm: {inproc_ms: 32, wall_ms: 1105, rss_kb: 136592} + fn_call: + cljw: {inproc_ms: 743, wall_ms: 781, rss_kb: 13544} + bb: {inproc_ms: 485, wall_ms: 528, rss_kb: 113044} + jvm: {inproc_ms: 68, wall_ms: 1238, rss_kb: 195412} + gc_churn: + cljw: {inproc_ms: 1589, wall_ms: 1910, rss_kb: 388684} + bb: {inproc_ms: 603, wall_ms: 656, rss_kb: 184532} + jvm: {inproc_ms: 235, wall_ms: 1394, rss_kb: 339132} + into_collections: + cljw: {inproc_ms: 5985, wall_ms: 6133, rss_kb: 132752} + bb: {inproc_ms: 484, wall_ms: 529, rss_kb: 136404} + jvm: {inproc_ms: 372, wall_ms: 1664, rss_kb: 313268} + json_roundtrip: + cljw: {inproc_ms: 415, wall_ms: 502, rss_kb: 52216} + bb: {inproc_ms: 371, wall_ms: 409, rss_kb: 114860} + jvm: {inproc_ms: 479, wall_ms: 2358, rss_kb: 564780} + lazy_chain: + cljw: {inproc_ms: 10258, wall_ms: 10511, rss_kb: 118440} + bb: {inproc_ms: 548, wall_ms: 601, rss_kb: 202388} + jvm: {inproc_ms: 462, wall_ms: 1634, rss_kb: 528180} + numeric_doubles: + cljw: {inproc_ms: 1082, wall_ms: 1116, rss_kb: 13872} + bb: {inproc_ms: 565, wall_ms: 600, rss_kb: 113400} + jvm: {inproc_ms: 32, wall_ms: 1038, rss_kb: 137836} + protocol_dispatch: + cljw: {inproc_ms: 1568, wall_ms: 1597, rss_kb: 13920} + bb: {inproc_ms: 344, wall_ms: 389, rss_kb: 113396} + jvm: {inproc_ms: 50, wall_ms: 1052, rss_kb: 168344} + reduce_map: + cljw: {inproc_ms: 9939, wall_ms: 10368, rss_kb: 150236} + bb: {inproc_ms: 656, wall_ms: 707, rss_kb: 167112} + jvm: {inproc_ms: 740, wall_ms: 2184, rss_kb: 689760} + string_ops: + cljw: {inproc_ms: 3760, wall_ms: 3903, rss_kb: 124628} + bb: {inproc_ms: 403, wall_ms: 439, rss_kb: 112892} + jvm: {inproc_ms: 496, wall_ms: 1784, rss_kb: 445768} + transduce: + cljw: {inproc_ms: 5751, wall_ms: 5790, rss_kb: 27776} + bb: {inproc_ms: 358, wall_ms: 389, rss_kb: 113232} + jvm: {inproc_ms: 195, wall_ms: 1400, rss_kb: 359324} + vector_assoc: + cljw: {inproc_ms: 1452, wall_ms: 1563, rss_kb: 172408} + bb: {inproc_ms: 448, wall_ms: 500, rss_kb: 167600} + jvm: {inproc_ms: 450, wall_ms: 1667, rss_kb: 558992} diff --git a/bench/vs_bb/probe/heap.clj b/bench/vs_bb/probe/heap.clj new file mode 100644 index 000000000..527dd0d46 --- /dev/null +++ b/bench/vs_bb/probe/heap.clj @@ -0,0 +1,15 @@ +(require (quote clojure.data.json) (quote clojure.string)) +(defn rss-mb [] (let [l (first (filter #(.startsWith ^String % "VmRSS") (clojure.string/split-lines (slurp "/proc/self/status"))))] + (Long/parseLong (re-find #"\d+" l)))) +(defn kb->mb [k] (quot k 1024)) +(println "start" (kb->mb (rss-mb)) "MB") +(def big (atom (vec (range 3000000)))) +(println "after 3M vector" (kb->mb (rss-mb)) "MB") +(reset! big nil) +(System/gc) (System/gc) +(println "after drop + System/gc" (kb->mb (rss-mb)) "MB") +(dotimes [_ 5] (count (vec (range 3000000)))) +(println "after 5 more 3M bursts" (kb->mb (rss-mb)) "MB") +(dotimes [i 200] (count (clojure.data.json/write-str (vec (range 20000))))) +(System/gc) +(println "after 200 x 20k JSON writes" (kb->mb (rss-mb)) "MB") diff --git a/bench/vs_bb/probe/heap_bb.clj b/bench/vs_bb/probe/heap_bb.clj new file mode 100644 index 000000000..d20b78cae --- /dev/null +++ b/bench/vs_bb/probe/heap_bb.clj @@ -0,0 +1,15 @@ +(require (quote clojure.data.json) (quote clojure.string)) +(defn rss-mb [] (let [l (first (filter #(.startsWith ^String % "VmRSS") (java.nio.file.Files/readAllLines (java.nio.file.Paths/get "/proc/self/status" (into-array String [])))))] + (Long/parseLong (re-find #"\d+" l)))) +(defn kb->mb [k] (quot k 1024)) +(println "start" (kb->mb (rss-mb)) "MB") +(def big (atom (vec (range 3000000)))) +(println "after 3M vector" (kb->mb (rss-mb)) "MB") +(reset! big nil) +(System/gc) (System/gc) +(println "after drop + System/gc" (kb->mb (rss-mb)) "MB") +(dotimes [_ 5] (count (vec (range 3000000)))) +(println "after 5 more 3M bursts" (kb->mb (rss-mb)) "MB") +(dotimes [i 200] (count (clojure.data.json/write-str (vec (range 20000))))) +(System/gc) +(println "after 200 x 20k JSON writes" (kb->mb (rss-mb)) "MB") diff --git a/bench/vs_bb/probe/probe.clj b/bench/vs_bb/probe/probe.clj new file mode 100644 index 000000000..957460895 --- /dev/null +++ b/bench/vs_bb/probe/probe.clj @@ -0,0 +1,12 @@ +(defn bench [f n] + (f) + (let [runs (for [_ (range 7)] + (let [t0 (System/nanoTime)] (dotimes [_ n] (f)) (- (System/nanoTime) t0)))] + (/ (double (apply min runs)) 1e6))) +(def v (vec (range 16000))) +(def xs (mapv double (range 1 1001))) +(prn {:empty-dotimes-20000-ms (let [t0 (System/nanoTime)] (dotimes [_ 20000]) (/ (- (System/nanoTime) t0) 1e6)) + :subvec-20000-ms (bench #(subvec v 100 15000) 1) + :subvec-x20-ms (bench #(dotimes [_ 20] (subvec v 100 15000)) 1) + :dot-1k-ms (bench #(reduce + (map * xs xs)) 1) + :dot-1k-loop-ms (bench #(loop [i 0 acc 0.0] (if (< i 1000) (recur (inc i) (+ acc (* (nth xs i) (nth xs i)))) acc)) 1)}) diff --git a/bench/vs_bb/probe/probe2.clj b/bench/vs_bb/probe/probe2.clj new file mode 100644 index 000000000..0e260ba53 --- /dev/null +++ b/bench/vs_bb/probe/probe2.clj @@ -0,0 +1,12 @@ +(defn per-call-us [f n] + (dotimes [_ 2000] (f)) + (let [runs (for [_ (range 7)] + (let [t0 (System/nanoTime)] (dotimes [_ n] (f)) (- (System/nanoTime) t0)))] + (/ (double (apply min runs)) n 1e3))) +(def v (vec (range 16000))) +(def xs (mapv double (range 1 1001))) +(prn {:version (clojure-version) + :subvec-us (per-call-us #(subvec v 100 15000) 2000) + :dot-1k-reduce-map-us (per-call-us #(reduce + (map * xs xs)) 200) + :dot-1k-loop-nth-us (per-call-us #(loop [i 0 acc 0.0] (if (< i 1000) (recur (inc i) (+ acc (* (nth xs i) (nth xs i)))) acc)) 200) + :byte-array-vec-8000-us (let [bv (vec (repeat 8000 1))] (per-call-us #(byte-array bv) 50))}) diff --git a/bench/vs_bb/run.sh b/bench/vs_bb/run.sh new file mode 100755 index 000000000..94c98f029 --- /dev/null +++ b/bench/vs_bb/run.sh @@ -0,0 +1,282 @@ +#!/usr/bin/env bash +# bench/vs_bb/run.sh: cljw vs babashka vs JVM Clojure on the same sources. +# +# Kanban 20261022012233-420c9fea (CLJW-BENCH-VS-BB), step 0 of decision +# 20261022012219-31249feb: every perf step toward "cljw faster than bb" is +# judged by this table. +# +# Every workload in workloads/*.clj runs >= 300 ms on bb (so startup does not +# hide the gap, which it did in bench/benchmarks/*: 28-190 ms totals against +# 23-27 ms startup), prints `RESULT ` (checked equal across runtimes) +# and `INPROC_MS ` (time measured INSIDE the script, startup and load +# excluded). The runner reports, per runtime: +# startup min wall of `-e nil` +# wall min wall of the whole process +# inproc min of INPROC_MS +# rss peak RSS (GNU time only) +# instr user-space instructions (perf stat, only with --perf and a usable perf) +# and the ratios cljw/bb and cljw/JVM on inproc (the headline) and on wall. +# Min-of-N by default; with hyperfine installed the wall column comes from +# `hyperfine --runs N` (its min) instead. +# +# Release-only measurement (rule perf_measure_release, memory +# 20260909234455-390c5332): the default cljw is a ReleaseSafe -Dwasm build +# into a SEPARATE prefix, never the Debug zig-out/bin/cljw. +# +# Usage: +# bash bench/vs_bb/run.sh # build ReleaseSafe, 3 runs, all runtimes +# bash bench/vs_bb/run.sh --cljw /path/cljw # time a given binary (no build) +# bash bench/vs_bb/run.sh -n 5 --only fib_recursive,reduce_map +# bash bench/vs_bb/run.sh --no-jvm --perf --probe +# bash bench/vs_bb/run.sh --no-history # print only, no YAML row +# CLJW_BENCH_JVM_CP=a.jar:b.jar bash bench/vs_bb/run.sh # JVM column without the clojure CLI +# +# Output: the table on stdout, and one YAML row appended to +# bench/vs_bb/history.yaml (override with --history FILE). Exit status is 0 +# unless the runner itself breaks; a slow or mismatching workload is REPORTED, +# never fatal (the CI job is non-blocking by design). +set -uo pipefail +HERE="$(cd "$(dirname "$0")" && pwd)" +ROOT="$(cd "$HERE/../.." && pwd)" + +N=3 +CLJW="" +ONLY="" +WANT_JVM=1 +WANT_PERF=0 +WANT_PROBE=0 +HISTORY="$HERE/history.yaml" +MODE="${CLJW_BENCH_MODE:-ReleaseSafe}" +CLJ_VERSION="1.12.4" +JSON_VERSION="2.5.1" + +while [[ $# -gt 0 ]]; do + case "$1" in + -n|--runs) N="$2"; shift 2 ;; + --cljw) CLJW="$2"; shift 2 ;; + --only) ONLY="$2"; shift 2 ;; + --no-jvm) WANT_JVM=0; shift ;; + --perf) WANT_PERF=1; shift ;; + --probe) WANT_PROBE=1; shift ;; + --history) HISTORY="$2"; shift 2 ;; + --no-history) HISTORY=""; shift ;; + -h|--help) sed -n '2,40p' "$0"; exit 0 ;; + *) echo "run.sh: unknown argument '$1'" >&2; exit 2 ;; + esac +done + +# ---------------------------------------------------------------- binaries +if [[ -z "$CLJW" ]]; then + PREFIX="${TMPDIR:-/tmp}/cljw-bench-vs-bb" + echo "[vs_bb] building cljw $MODE -Dwasm into $PREFIX" >&2 + # shellcheck source=/dev/null + source "$ROOT/scripts/lib_build_lock.sh" + ( cd "$ROOT" && with_build_lock zig build -Dwasm -Doptimize="$MODE" -p "$PREFIX" ) >&2 \ + || { echo "run.sh: cljw build failed" >&2; exit 1; } + CLJW="$PREFIX/bin/cljw" +fi +[[ -x "$CLJW" ]] || { echo "run.sh: no cljw binary at $CLJW" >&2; exit 1; } + +RUNTIMES=(cljw) +command -v bb >/dev/null 2>&1 && RUNTIMES+=(bb) || echo "[vs_bb] bb not found: cljw/bb column empty" >&2 + +JVM_CP="" +if [[ "$WANT_JVM" == 1 && -n "${CLJW_BENCH_JVM_CP:-}" ]] && command -v java >/dev/null 2>&1; then + # CI path: a classpath of Maven Central jars, no clojure CLI needed. + JVM_CP="$CLJW_BENCH_JVM_CP"; RUNTIMES+=(jvm) +elif [[ "$WANT_JVM" == 1 ]] && command -v clojure >/dev/null 2>&1 && command -v java >/dev/null 2>&1; then + JVM_CP="$(clojure -Spath -Sdeps "{:deps {org.clojure/clojure {:mvn/version \"$CLJ_VERSION\"} org.clojure/data.json {:mvn/version \"$JSON_VERSION\"}}}" 2>/dev/null || true)" + [[ -n "$JVM_CP" ]] && RUNTIMES+=(jvm) || echo "[vs_bb] clojure -Spath failed: no JVM column" >&2 +fi + +# GNU time gives peak RSS; BSD/macOS time has a different flag set, skip there. +GTIME="" +if [[ -x /usr/bin/time ]] && /usr/bin/time -f '%M' -o /dev/null true 2>/dev/null; then GTIME=/usr/bin/time; fi +HYPERFINE="" +command -v hyperfine >/dev/null 2>&1 && HYPERFINE=hyperfine +PERF="" +if [[ "$WANT_PERF" == 1 ]] && command -v perf >/dev/null 2>&1 \ + && perf stat -x, -e instructions:u -o /dev/null true 2>/dev/null; then PERF=perf; fi +[[ "$WANT_PERF" == 1 && -z "$PERF" ]] && echo "[vs_bb] perf stat unusable here (perf_event_paranoid?): no instr column" >&2 + +cmd_for() { # runtime file-or-empty -> argv on stdout (one string) + local rt="$1" f="$2" + case "$rt" in + cljw) [[ -n "$f" ]] && echo "$CLJW $f" || echo "$CLJW -e nil" ;; + bb) [[ -n "$f" ]] && echo "bb $f" || echo "bb -e nil" ;; + jvm) [[ -n "$f" ]] && echo "java -cp $JVM_CP clojure.main $f" || echo "java -cp $JVM_CP clojure.main -e nil" ;; + esac +} + +TMP="$(mktemp -d "${TMPDIR:-/tmp}/vs_bb.XXXXXX")" +trap 'rm -rf "$TMP"' EXIT + +now_ns() { date +%s%N; } + +# measure RT FILE -> sets M_WALL M_INPROC M_RESULT M_RSS M_INSTR M_ERR (min over N) +measure() { + local rt="$1" f="$2" cmd i t0 t1 w ip rss + cmd="$(cmd_for "$rt" "$f")" + M_WALL=""; M_INPROC=""; M_RESULT=""; M_RSS=""; M_INSTR=""; M_ERR="" + # one warmup run (page cache, JIT-free but disk-warm), not counted + $cmd >/dev/null 2>&1 + for ((i = 0; i < N; i++)); do + t0=$(now_ns) + if [[ -n "$GTIME" ]]; then + $GTIME -f '%M' -o "$TMP/rss" $cmd >"$TMP/out" 2>"$TMP/err" + else + $cmd >"$TMP/out" 2>"$TMP/err" + fi + local rc=$? + t1=$(now_ns) + if [[ $rc -ne 0 ]]; then + M_ERR="ERR$rc"; head -c 400 "$TMP/err" >"$TMP/lasterr.$rt"; return + fi + w=$(( (t1 - t0) / 1000000 )) + [[ -z "$M_WALL" || $w -lt $M_WALL ]] && M_WALL=$w + ip="$(awk '/^INPROC_MS /{print $2}' "$TMP/out")" + [[ -n "$ip" ]] && { [[ -z "$M_INPROC" || $ip -lt $M_INPROC ]] && M_INPROC=$ip; } + M_RESULT="$(awk '/^RESULT /{print $2}' "$TMP/out")" + if [[ -n "$GTIME" ]]; then + rss="$(tail -1 "$TMP/rss")" + [[ -z "$M_RSS" || $rss -gt $M_RSS ]] && M_RSS=$rss + fi + done + if [[ -n "$HYPERFINE" ]]; then + if $HYPERFINE -N --runs "$N" --warmup 0 --export-json "$TMP/hf.json" "$cmd" >/dev/null 2>&1; then + local hmin + hmin="$(python3 -c 'import json,sys; print(int(json.load(open(sys.argv[1]))["results"][0]["min"]*1000))' "$TMP/hf.json" 2>/dev/null || true)" + [[ -n "$hmin" ]] && M_WALL="$hmin" + fi + fi + if [[ -n "$PERF" ]]; then + perf stat -x, -e instructions:u -o "$TMP/perf" $cmd >/dev/null 2>&1 \ + && M_INSTR="$(awk -F, '/instructions/{print $1; exit}' "$TMP/perf")" + fi +} + +ratio() { # a b -> a/b with 2 decimals, or "-" + [[ -n "$1" && -n "$2" && "$2" != 0 ]] && awk -v a="$1" -v b="$2" 'BEGIN{printf "%.2f", a/b}' || echo "-" +} + +version_of() { + case "$1" in + cljw) "$CLJW" --version 2>&1 | sed -n 1p ;; + bb) bb --version 2>&1 | sed -n 1p ;; + jvm) echo "clojure $CLJ_VERSION on $(java -version 2>&1 | sed -n 1p)" ;; + esac +} + +# ---------------------------------------------------------------- workloads +WORKLOADS=() +for f in "$HERE"/workloads/*.clj; do + name="$(basename "$f" .clj)" + if [[ -n "$ONLY" ]] && [[ ",$ONLY," != *",$name,"* ]]; then continue; fi + WORKLOADS+=("$name") +done + +declare -A WALL INPROC RES RSS INSTR ERR STARTUP STARTUP_RSS +for rt in "${RUNTIMES[@]}"; do + measure "$rt" "" + STARTUP[$rt]="${M_WALL:-$M_ERR}"; STARTUP_RSS[$rt]="$M_RSS" +done + +for name in "${WORKLOADS[@]}"; do + f="$HERE/workloads/$name.clj" + for rt in "${RUNTIMES[@]}"; do + echo "[vs_bb] $name on $rt" >&2 + measure "$rt" "$f" + WALL[$name.$rt]="$M_WALL"; INPROC[$name.$rt]="$M_INPROC"; RES[$name.$rt]="$M_RESULT" + RSS[$name.$rt]="$M_RSS"; INSTR[$name.$rt]="$M_INSTR"; ERR[$name.$rt]="$M_ERR" + if [[ -n "$M_ERR" ]]; then echo "[vs_bb] $rt failed: $(tr '\n' ' ' <"$TMP/lasterr.$rt" | head -c 200)" >&2; fi + done +done + +# ---------------------------------------------------------------- table +has() { local x; for x in "${RUNTIMES[@]}"; do [[ "$x" == "$1" ]] && return 0; done; return 1; } +cell() { local v="${ERR[$1.$2]:-}"; [[ -n "$v" ]] && echo "$v" || echo "${3:--}"; } + +COMMIT="$(git -C "$ROOT" rev-parse --short HEAD 2>/dev/null || echo unknown)" +LOAD="$(awk '{print $1}' /proc/loadavg 2>/dev/null || uptime | awk -F'load averages?: ' '{print $2}' | awk '{print $1}' | tr -d ,)" +echo +echo "cljw vs bb vs JVM commit $COMMIT runs=$N (min) load=$LOAD wall=$([[ -n "$HYPERFINE" ]] && echo hyperfine || echo min-of-N)" +for rt in "${RUNTIMES[@]}"; do printf ' %-5s %s\n' "$rt" "$(version_of "$rt")"; done +printf ' startup (-e nil) ms:'; for rt in "${RUNTIMES[@]}"; do printf ' %s %s' "$rt" "${STARTUP[$rt]}"; done +printf ' cljw/bb %s\n' "$(has bb && ratio "${STARTUP[cljw]}" "${STARTUP[bb]}" || echo -)" +[[ -n "$GTIME" ]] && { printf ' idle peak RSS MB:'; for rt in "${RUNTIMES[@]}"; do printf ' %s %s' "$rt" "$(( ${STARTUP_RSS[$rt]:-0} / 1024 ))"; done; echo; } +echo +printf '%-18s %8s %8s %8s | %8s %8s %8s | %7s %7s %7s | %6s %6s %s\n' \ + workload cljw_in bb_in jvm_in cljw_wl bb_wl jvm_wl c/bb_in c/bb_wl c/jvm_in cljwMB bbMB check +geo_bb=0; geo_bb_n=0; geo_jvm=0; geo_jvm_n=0 +for name in "${WORKLOADS[@]}"; do + rb="-"; rj="-"; rw="-" + has bb && rb="$(ratio "${INPROC[$name.cljw]}" "${INPROC[$name.bb]}")" && rw="$(ratio "${WALL[$name.cljw]}" "${WALL[$name.bb]}")" + has jvm && rj="$(ratio "${INPROC[$name.cljw]}" "${INPROC[$name.jvm]}")" + [[ "$rb" != "-" ]] && { geo_bb=$(awk -v g="$geo_bb" -v r="$rb" 'BEGIN{print g+log(r)}'); geo_bb_n=$((geo_bb_n+1)); } + [[ "$rj" != "-" ]] && { geo_jvm=$(awk -v g="$geo_jvm" -v r="$rj" 'BEGIN{print g+log(r)}'); geo_jvm_n=$((geo_jvm_n+1)); } + check=ok + for rt in "${RUNTIMES[@]}"; do + [[ -n "${ERR[$name.$rt]}" ]] && { check="$rt:${ERR[$name.$rt]}"; continue; } + [[ "${RES[$name.$rt]}" != "${RES[$name.cljw]}" ]] && check="MISMATCH($rt)" + done + printf '%-18s %8s %8s %8s | %8s %8s %8s | %7s %7s %7s | %6s %6s %s\n' "$name" \ + "$(cell "$name" cljw "${INPROC[$name.cljw]}")" "$(cell "$name" bb "${INPROC[$name.bb]:-}")" "$(cell "$name" jvm "${INPROC[$name.jvm]:-}")" \ + "${WALL[$name.cljw]:--}" "${WALL[$name.bb]:--}" "${WALL[$name.jvm]:--}" \ + "$rb" "$rw" "$rj" \ + "$(( ${RSS[$name.cljw]:-0} / 1024 ))" "$(( ${RSS[$name.bb]:-0} / 1024 ))" "$check" + [[ -n "$PERF" ]] && printf '%-18s instructions:u cljw %s bb %s jvm %s\n' "" \ + "${INSTR[$name.cljw]:--}" "${INSTR[$name.bb]:--}" "${INSTR[$name.jvm]:--}" +done +GEO_BB="-"; GEO_JVM="-" +[[ $geo_bb_n -gt 0 ]] && GEO_BB=$(awk -v g="$geo_bb" -v n="$geo_bb_n" 'BEGIN{printf "%.2f", exp(g/n)}') +[[ $geo_jvm_n -gt 0 ]] && GEO_JVM=$(awk -v g="$geo_jvm" -v n="$geo_jvm_n" 'BEGIN{printf "%.2f", exp(g/n)}') +echo +echo "geomean in-process ratio: cljw/bb $GEO_BB cljw/jvm $GEO_JVM (< 1.00 means cljw is faster)" +echo "columns: *_in = ms inside the script (INPROC_MS), *_wl = process wall ms, ratios on min values" + +# ---------------------------------------------------------------- probe (per-call micro cells) +if [[ "$WANT_PROBE" == 1 ]]; then + echo + echo "per-call probe (probe/probe2.clj, us per call, min of 7 after warmup):" + for rt in "${RUNTIMES[@]}"; do + printf ' %-5s ' "$rt"; $(cmd_for "$rt" "$HERE/probe/probe2.clj") 2>&1 | tail -1 + done + echo "RSS probe (probe/heap.clj; bb uses heap_bb.clj):" + for rt in cljw bb; do + has "$rt" || continue + h="$HERE/probe/heap.clj"; [[ "$rt" == bb ]] && h="$HERE/probe/heap_bb.clj" + echo " $rt:"; $(cmd_for "$rt" "$h") 2>&1 | sed 's/^/ /' + done +fi + +# ---------------------------------------------------------------- YAML history row +if [[ -n "$HISTORY" ]]; then + { + [[ -s "$HISTORY" ]] || echo "# bench/vs_bb history: one row per run of bench/vs_bb/run.sh (append-only)." + echo "- date: \"$(date -u +%Y-%m-%dT%H:%M:%SZ)\"" + echo " commit: \"$COMMIT\"" + echo " host: \"$(uname -sm)\"" + echo " ci: ${CI:-false}" + echo " load: \"$LOAD\"" + echo " runs: $N" + echo " wall_source: $([[ -n "$HYPERFINE" ]] && echo hyperfine || echo min-of-n)" + echo " versions:" + for rt in "${RUNTIMES[@]}"; do echo " $rt: \"$(version_of "$rt" | tr -d '"')\""; done + echo " startup_ms:" + for rt in "${RUNTIMES[@]}"; do echo " $rt: ${STARTUP[$rt]}"; done + echo " geomean_inproc: {cljw_over_bb: $GEO_BB, cljw_over_jvm: $GEO_JVM}" + echo " workloads:" + for name in "${WORKLOADS[@]}"; do + echo " $name:" + for rt in "${RUNTIMES[@]}"; do + if [[ -n "${ERR[$name.$rt]}" ]]; then echo " $rt: {error: ${ERR[$name.$rt]}}"; continue; fi + printf ' %s: {inproc_ms: %s, wall_ms: %s' "$rt" "${INPROC[$name.$rt]:-null}" "${WALL[$name.$rt]:-null}" + [[ -n "${RSS[$name.$rt]}" ]] && printf ', rss_kb: %s' "${RSS[$name.$rt]}" + [[ -n "${INSTR[$name.$rt]}" ]] && printf ', instructions: %s' "${INSTR[$name.$rt]}" + echo "}" + done + done + } >>"$HISTORY" + echo "[vs_bb] appended a row to $HISTORY" >&2 +fi +exit 0 diff --git a/bench/vs_bb/workloads/arrays.clj b/bench/vs_bb/workloads/arrays.clj new file mode 100644 index 000000000..a4805dde9 --- /dev/null +++ b/bench/vs_bb/workloads/arrays.clj @@ -0,0 +1,10 @@ +;; core-library-bound: byte-array / long-array / double-array from an 8000-element vector. +(def bv (vec (repeat 8000 1))) +(defn work [reps] + (loop [k 0 s 0] + (if (< k reps) + (recur (inc k) (+ s (alength (byte-array bv)) (alength (long-array bv)) (alength (double-array bv)))) + s))) +(let [t0 (System/nanoTime) r (work 900)] + (println "RESULT" r) + (println "INPROC_MS" (quot (- (System/nanoTime) t0) 1000000))) diff --git a/bench/vs_bb/workloads/dot_loop_nth.clj b/bench/vs_bb/workloads/dot_loop_nth.clj new file mode 100644 index 000000000..c0b14903a --- /dev/null +++ b/bench/vs_bb/workloads/dot_loop_nth.clj @@ -0,0 +1,9 @@ +;; interpreter-bound: loop/recur dot product of 1000 doubles via nth. +(def xs (mapv double (range 1 1001))) +(defn dot [] + (loop [i 0 acc 0.0] + (if (< i 1000) (recur (inc i) (+ acc (* (nth xs i) (nth xs i)))) acc))) +(defn work [reps] (loop [k 0 s 0.0] (if (< k reps) (recur (inc k) (+ s (dot))) s))) +(let [t0 (System/nanoTime) r (work 2500)] + (println "RESULT" (long r)) + (println "INPROC_MS" (quot (- (System/nanoTime) t0) 1000000))) diff --git a/bench/vs_bb/workloads/fib_recursive.clj b/bench/vs_bb/workloads/fib_recursive.clj new file mode 100644 index 000000000..50370628e --- /dev/null +++ b/bench/vs_bb/workloads/fib_recursive.clj @@ -0,0 +1,5 @@ +;; interpreter-bound: non-tail recursion, integer arithmetic. +(defn fib [n] (if (< n 2) n (+ (fib (- n 1)) (fib (- n 2))))) +(let [t0 (System/nanoTime) r (fib 30)] + (println "RESULT" r) + (println "INPROC_MS" (quot (- (System/nanoTime) t0) 1000000))) diff --git a/bench/vs_bb/workloads/fn_call.clj b/bench/vs_bb/workloads/fn_call.clj new file mode 100644 index 000000000..62d0a3d66 --- /dev/null +++ b/bench/vs_bb/workloads/fn_call.clj @@ -0,0 +1,8 @@ +;; interpreter-bound: empty fn call plus loop/recur dispatch overhead. +(defn noop [x] x) +(defn work [n] + (loop [i 0 acc 0] + (if (< i n) (recur (inc i) (+ acc (noop i))) acc))) +(let [t0 (System/nanoTime) r (work 4000000)] + (println "RESULT" r) + (println "INPROC_MS" (quot (- (System/nanoTime) t0) 1000000))) diff --git a/bench/vs_bb/workloads/gc_churn.clj b/bench/vs_bb/workloads/gc_churn.clj new file mode 100644 index 000000000..8f8087e50 --- /dev/null +++ b/bench/vs_bb/workloads/gc_churn.clj @@ -0,0 +1,11 @@ +;; GC heavy: many short-lived maps and vectors over a retained large heap. +(defn work [n] + (let [retained (vec (map (fn [i] {:id i :v [i (inc i)]}) (range 200000)))] + (loop [i 0 s 0] + (if (< i n) + (let [m {:a i :b [i i i] :c {:d (str i)}}] + (recur (inc i) (+ s (count (:b m)) (count (:c m))))) + (+ s (count retained) (:id (nth retained 1234))))))) +(let [t0 (System/nanoTime) r (work 600000)] + (println "RESULT" r) + (println "INPROC_MS" (quot (- (System/nanoTime) t0) 1000000))) diff --git a/bench/vs_bb/workloads/into_collections.clj b/bench/vs_bb/workloads/into_collections.clj new file mode 100644 index 000000000..89887534c --- /dev/null +++ b/bench/vs_bb/workloads/into_collections.clj @@ -0,0 +1,16 @@ +;; collections: into {} / into #{} / frequencies / group-by / sort-by / update. +(defn work [reps] + (loop [k 0 s 0] + (if (< k reps) + (let [v (vec (range 20000)) + m (into {} (map (fn [i] [i (* 2 i)])) v) + st (into #{} (map #(mod % 1000)) v) + fr (frequencies (map #(mod % 97) v)) + g (group-by #(mod % 13) v) + srt (sort-by - v) + m2 (reduce (fn [acc i] (update acc (mod i 50) (fnil + 0) i)) {} v)] + (recur (inc k) (+ s (count m) (count st) (count fr) (count g) (first srt) (get m2 7)))) + s))) +(let [t0 (System/nanoTime) r (work 15)] + (println "RESULT" r) + (println "INPROC_MS" (quot (- (System/nanoTime) t0) 1000000))) diff --git a/bench/vs_bb/workloads/json_roundtrip.clj b/bench/vs_bb/workloads/json_roundtrip.clj new file mode 100644 index 000000000..daa071dd5 --- /dev/null +++ b/bench/vs_bb/workloads/json_roundtrip.clj @@ -0,0 +1,14 @@ +;; library: clojure.data.json write-str of a 20k vector plus read-str of a 500-record document. +(require '[clojure.data.json :as json]) +(def payload (vec (range 20000))) +(def docs + (json/write-str + {:users (vec (map (fn [i] {:id i :name (str "u" i) :tags ["a" "b"] :score (* 1.5 i)}) (range 500)))})) +(defn work [reps] + (loop [k 0 s 0] + (if (< k reps) + (recur (inc k) (+ s (count (json/write-str payload)) (count (get (json/read-str docs) "users")))) + s))) +(let [t0 (System/nanoTime) r (work 120)] + (println "RESULT" r) + (println "INPROC_MS" (quot (- (System/nanoTime) t0) 1000000))) diff --git a/bench/vs_bb/workloads/lazy_chain.clj b/bench/vs_bb/workloads/lazy_chain.clj new file mode 100644 index 000000000..8c10c0d68 --- /dev/null +++ b/bench/vs_bb/workloads/lazy_chain.clj @@ -0,0 +1,9 @@ +;; core-library-bound: a lazy map/filter/take chain over an infinite range. +(defn work [reps] + (loop [k 0 s 0] + (if (< k reps) + (recur (inc k) (+ s (reduce + (take 50000 (filter odd? (map #(* 3 %) (range))))))) + s))) +(let [t0 (System/nanoTime) r (work 15)] + (println "RESULT" r) + (println "INPROC_MS" (quot (- (System/nanoTime) t0) 1000000))) diff --git a/bench/vs_bb/workloads/numeric_doubles.clj b/bench/vs_bb/workloads/numeric_doubles.clj new file mode 100644 index 000000000..65c3a1658 --- /dev/null +++ b/bench/vs_bb/workloads/numeric_doubles.clj @@ -0,0 +1,20 @@ +;; numeric doubles: mandelbrot escape-time loop plus a Math/sqrt sum. +(defn mandel [w h maxit] + (loop [y 0 total 0] + (if (< y h) + (recur (inc y) + (loop [x 0 total total] + (if (< x w) + (let [cr (- (* 3.0 (/ x (double w))) 2.0) + ci (- (* 2.0 (/ y (double h))) 1.0) + it (loop [zr 0.0 zi 0.0 i 0] + (if (and (< i maxit) (< (+ (* zr zr) (* zi zi)) 4.0)) + (recur (+ (- (* zr zr) (* zi zi)) cr) (+ (* 2.0 zr zi) ci) (inc i)) + i))] + (recur (inc x) (+ total it))) + total))) + total))) +(defn sqrts [n] (loop [i 0 s 0.0] (if (< i n) (recur (inc i) (+ s (Math/sqrt (double i)))) s))) +(let [t0 (System/nanoTime) r (+ (mandel 240 160 60) (long (sqrts 1500000)))] + (println "RESULT" r) + (println "INPROC_MS" (quot (- (System/nanoTime) t0) 1000000))) diff --git a/bench/vs_bb/workloads/protocol_dispatch.clj b/bench/vs_bb/workloads/protocol_dispatch.clj new file mode 100644 index 000000000..adae50006 --- /dev/null +++ b/bench/vs_bb/workloads/protocol_dispatch.clj @@ -0,0 +1,9 @@ +;; interpreter-bound: protocol dispatch on a record (portable: names no host class). +(defprotocol ICompute (compute [this x])) +(defrecord Scaler [factor] ICompute (compute [_ x] (* factor x))) +(defn work [n] + (let [s (->Scaler 3)] + (loop [i 0 sum 0] (if (< i n) (recur (inc i) (+ sum (compute s i))) sum)))) +(let [t0 (System/nanoTime) r (work 2000000)] + (println "RESULT" r) + (println "INPROC_MS" (quot (- (System/nanoTime) t0) 1000000))) diff --git a/bench/vs_bb/workloads/reduce_map.clj b/bench/vs_bb/workloads/reduce_map.clj new file mode 100644 index 000000000..4f678151a --- /dev/null +++ b/bench/vs_bb/workloads/reduce_map.clj @@ -0,0 +1,7 @@ +;; core-library-bound: (reduce + (map * xs xs)) over 100k doubles (lazy seq machinery). +(def xs (mapv double (range 100000))) +(defn work [reps] + (loop [k 0 s 0.0] (if (< k reps) (recur (inc k) (+ s (reduce + (map * xs xs)))) s))) +(let [t0 (System/nanoTime) r (work 40)] + (println "RESULT" (long (/ r 1e9))) + (println "INPROC_MS" (quot (- (System/nanoTime) t0) 1000000))) diff --git a/bench/vs_bb/workloads/string_ops.clj b/bench/vs_bb/workloads/string_ops.clj new file mode 100644 index 000000000..fe54779aa --- /dev/null +++ b/bench/vs_bb/workloads/string_ops.clj @@ -0,0 +1,15 @@ +;; strings: str, join, split, replace, upper-case, regex scan. +(require '[clojure.string :as str]) +(defn work [reps] + (loop [k 0 s 0] + (if (< k reps) + (let [line (str/join "," (map #(str "item-" % "-x") (range 2000))) + parts (str/split line #",") + up (str/upper-case line) + rep (str/replace line "item" "it") + hits (count (re-seq #"\d+" line))] + (recur (inc k) (+ s (count parts) (count up) (count rep) hits))) + s))) +(let [t0 (System/nanoTime) r (work 250)] + (println "RESULT" r) + (println "INPROC_MS" (quot (- (System/nanoTime) t0) 1000000))) diff --git a/bench/vs_bb/workloads/transduce.clj b/bench/vs_bb/workloads/transduce.clj new file mode 100644 index 000000000..51df15183 --- /dev/null +++ b/bench/vs_bb/workloads/transduce.clj @@ -0,0 +1,10 @@ +;; core-library-bound: transducers (transduce, into with an xform). +(def xf (comp (map inc) (filter even?) (map #(* % %)))) +(defn work [reps] + (loop [k 0 s 0] + (if (< k reps) + (recur (inc k) (+ s (transduce xf + (range 100000)) (count (into [] xf (range 50000))))) + s))) +(let [t0 (System/nanoTime) r (work 40)] + (println "RESULT" r) + (println "INPROC_MS" (quot (- (System/nanoTime) t0) 1000000))) diff --git a/bench/vs_bb/workloads/vector_assoc.clj b/bench/vs_bb/workloads/vector_assoc.clj new file mode 100644 index 000000000..561f429ee --- /dev/null +++ b/bench/vs_bb/workloads/vector_assoc.clj @@ -0,0 +1,11 @@ +;; collections: persistent vector conj/assoc/subvec and keyword-map assoc/dissoc churn. +(defn work [n] + (let [v (loop [i 0 v []] (if (< i n) (recur (inc i) (conj v i)) v)) + v2 (loop [i 0 v v] (if (< i n) (recur (inc i) (assoc v i (- i))) v)) + m (loop [i 0 m {}] (if (< i n) (recur (inc i) (assoc m (keyword (str "k" (mod i 5000))) i)) m)) + m2 (reduce dissoc m (take 1000 (keys m))) + sv (loop [i 0 s 0] (if (< i 2000) (recur (inc i) (+ s (count (subvec v2 i (+ i 1000))))) s))] + (+ (count v) (reduce + v2) (count m2) sv))) +(let [t0 (System/nanoTime) r (work 300000)] + (println "RESULT" r) + (println "INPROC_MS" (quot (- (System/nanoTime) t0) 1000000))) diff --git a/bench/wasm/percall/zigcool.wasm b/bench/wasm/percall/zigcool.wasm new file mode 100755 index 000000000..6af41044a Binary files /dev/null and b/bench/wasm/percall/zigcool.wasm differ diff --git a/bench/wasm/percall/zigcool.zig b/bench/wasm/percall/zigcool.zig new file mode 100644 index 000000000..29ecc355f --- /dev/null +++ b/bench/wasm/percall/zigcool.zig @@ -0,0 +1,78 @@ +//! The 47 KB module behind bench/wasm_percall.clj's `big-*` cells. Its size +//! comes from the compile-time tables; the bench calls only `popcount`, so a +//! per-call cost that grows with the module is all the cells can show. +//! Rebuild: zig build-exe -target wasm32-freestanding -fno-entry -rdynamic -O ReleaseSmall zigcool.zig + +const std = @import("std"); + +// The first 10000 primes, computed by the compiler: the binary ships the table. +const PRIMES = blk: { + @setEvalBranchQuota(200_000_000); + var table: [10_000]u32 = undefined; + var n: usize = 0; + var c: u32 = 2; + while (n < table.len) : (c += 1) { + var is_p = true; + for (table[0..n]) |p| { + if (p * p > c) break; + if (c % p == 0) { + is_p = false; + break; + } + } + if (is_p) { + table[n] = c; + n += 1; + } + } + break :blk table; +}; + +/// The nth prime (0-based) from the compile-time table, 0 when out of range. +export fn nth_prime(i: u32) u32 { + return if (i < PRIMES.len) PRIMES[i] else 0; +} + +// A perfect-hash keyword map, also built at compile time. +const OPS = std.StaticStringMap(i32).initComptime(.{ + .{ "add", 1 }, .{ "sub", 2 }, .{ "mul", 3 }, .{ "div", 4 }, .{ "pow", 5 }, +}); + +/// Opcode for an operator name, -1 when unknown. +export fn opcode(ptr: [*]const u8, len: usize) i32 { + return OPS.get(ptr[0..len]) orelse -1; +} + +/// Dot product of two f64 arrays, 4 lanes at a time with @Vector. +export fn dot(a: [*]const f64, b: [*]const f64, len: usize) f64 { + const V = @Vector(4, f64); + var acc: V = @splat(0); + var i: usize = 0; + while (i + 4 <= len) : (i += 4) { + const va: V = a[i..][0..4].*; + const vb: V = b[i..][0..4].*; + acc += va * vb; + } + var sum = @reduce(.Add, acc); + while (i < len) : (i += 1) sum += a[i] * b[i]; + return sum; +} + +/// Set bits in x. +export fn popcount(x: u64) u32 { + return @popCount(x); +} + +/// x with its 64 bits in reverse order. +export fn bitrev(x: u64) u64 { + return @bitReverse(x); +} + +/// BLAKE3 hex digest of the input into `out` (64 bytes + NUL). +export fn blake3(ptr: [*]const u8, len: usize, out: [*]u8) void { + var d: [32]u8 = undefined; + std.crypto.hash.Blake3.hash(ptr[0..len], &d, .{}); + const hex = std.fmt.bytesToHex(d, .lower); + @memcpy(out[0..64], &hex); + out[64] = 0; +} diff --git a/bench/wasm_percall.clj b/bench/wasm_percall.clj index 850e61001..1bb1e0772 100644 --- a/bench/wasm_percall.clj +++ b/bench/wasm_percall.clj @@ -13,7 +13,10 @@ (`-worker`, via `future`), plus `fn-call`: a plain Clojure function call in the same process. That last row is the platform constant, kept in its own datum key so a regression in cljw's call path is distinguishable from one in - the engine. Copies the discipline of zwasm's own bench/latency runner. + the engine. `big-auto | big-jit | big-interp` call a trivial export of a + 47 KB module: a per-call cost that grows with module size shows up as the + gap between `big-jit` and `jit-main` ([CLJW-WASM-JIT-PERCALL-SCALES], + zwasm/D-585). Copies the discipline of zwasm's own bench/latency runner. Run (needs a -Dwasm ReleaseSafe binary): ./zig-out/bin/cljw -cp bench -m wasm-percall [--n=20000] [--trials=7] [--yaml=FILE] @@ -23,6 +26,12 @@ (:require [clojure.string :as str])) (def add-wasm "bench/wasm/ffi/add.wasm") +(def three-f64-wasm "test/e2e/fixtures/wasm/three_f64.wasm") +(def echo-component "test/e2e/fixtures/wasm/echo.component.wasm") + +;; 47 KB, built from bench/wasm/percall/zigcool.zig with +;; zig build-exe -target wasm32-freestanding -fno-entry -rdynamic -O ReleaseSmall zigcool.zig +(def big-wasm "bench/wasm/percall/zigcool.wasm") (defn parse-args [args] (reduce (fn [opts arg] @@ -39,6 +48,9 @@ (defn burn-wasm [m n] (loop [i 0] (when (< i n) (wasm/call m "add" i 1) (recur (inc i))))) +(defn burn-big [m n] + (loop [i 0] (when (< i n) (wasm/call m "popcount" i) (recur (inc i))))) + (defn burn-fn [n] (loop [i 0] (when (< i n) (plain-add i 1) (recur (inc i))))) @@ -50,18 +62,41 @@ (long (/ (double (- (System/nanoTime) t0)) n)))) (defn cells - "Ordered [name thunk] pairs. Each thunk runs one trial of n calls." + "Ordered [name thunk] pairs. Each thunk runs one trial of n calls. + + Every cell owns its instance. An `:interp` instance traps once it is called + from a thread other than the one that first ran it (staging 2026-10-21, + zwasm d74e63b4; card CLJW-WASM-INTERP-THREAD-HOP). `future` may land each + trial on a different pool thread, so `interp-worker` instantiates on the + worker inside its trial; that one load, amortised over n calls, is part of + its number. + + `core-f64x3` / `component-f64` pin [CLJW-WASM-CALL-22X]: core wasm/call + against component-call on an f64 shape. They measured 22x apart on v1.10.18 + (14.2 us vs 0.65 us), before ADR-0195 moved the program off the initial + thread and before the per-handle exportSig cache. The cells stay so the two + surfaces cannot drift apart unseen." [n] - (let [auto (wasm/load add-wasm {:fuel 0}) - jit (wasm/load add-wasm {:engine :jit :fuel 0}) - interp (wasm/load add-wasm {:engine :interp :fuel 0}) - on-worker (fn [f] #(deref (future (f))))] - [["auto-main" #(burn-wasm auto n)] - ["auto-worker" (on-worker #(burn-wasm auto n))] - ["jit-main" #(burn-wasm jit n)] - ["jit-worker" (on-worker #(burn-wasm jit n))] - ["interp-main" #(burn-wasm interp n)] - ["interp-worker" (on-worker #(burn-wasm interp n))] + (let [ld (fn [opts] (wasm/load add-wasm (assoc opts :fuel 0))) + big (fn [opts] (wasm/load big-wasm (assoc opts :fuel 0))) + three (wasm/load three-f64-wasm) + comp (wasm/load-component echo-component) + on-worker (fn [f] #(deref (future (f)))) + auto-m (ld {}) auto-w (ld {}) + jit-m (ld {:engine :jit}) jit-w (ld {:engine :jit}) + int-m (ld {:engine :interp}) + big-auto (big {}) big-jit (big {:engine :jit}) big-interp (big {:engine :interp})] + [["auto-main" #(burn-wasm auto-m n)] + ["auto-worker" (on-worker #(burn-wasm auto-w n))] + ["jit-main" #(burn-wasm jit-m n)] + ["jit-worker" (on-worker #(burn-wasm jit-w n))] + ["interp-main" #(burn-wasm int-m n)] + ["interp-worker" (on-worker #(burn-wasm (ld {:engine :interp}) n))] + ["core-f64x3" #(loop [i 0] (when (< i n) (wasm/call three "sum3" 1.5 2.5 3.5) (recur (inc i))))] + ["component-f64" #(loop [i 0] (when (< i n) (wasm/component-call comp "echo-f64" 1.5) (recur (inc i))))] + ["big-auto" #(burn-big big-auto n)] + ["big-jit" #(burn-big big-jit n)] + ["big-interp" #(burn-big big-interp n)] ["fn-call" #(burn-fn n)]])) (defn stats diff --git a/build.zig.zon b/build.zig.zon index 20cbb2185..7da3d294a 100644 --- a/build.zig.zon +++ b/build.zig.zon @@ -1,6 +1,6 @@ .{ .name = .cljw, - .version = "1.15.3", + .version = "1.15.4", .fingerprint = 0x1869d207073beffa, .minimum_zig_version = "0.16.0", .dependencies = .{ @@ -14,6 +14,18 @@ // zwasm v2 embedding API (F-001 / D-037). `lazy` so the DEFAULT build + // gate never resolve it — only `-Dwasm` (b.lazyDependency) does. // + // 2026-10-22: the fork's `main` is cljw's zwasm integration line: + // upstream zwasm/zwasm `main` plus patches found and proven here, each + // listed with its evidence in the fork's FORK.md and upstreamed once a + // cljw release has carried it. Pin a commit of fork `main`, never a + // fix branch. Now at e9d74934, on d74e63b4: + // fork PR #2 the interp's native-stack limit is the calling + // thread's, not the Runtime's (CLJW-WASM-INTERP-THREAD-TRAP) + // fork PR #3 a by-name JIT invoke resolves the export from + // `CompiledWasm.exports` instead of re-parsing the module + // per call (zwasm/D-585, CLJW-WASM-JIT-PERCALL-SCALES) + // fork PR #4 FORK.md, the evidence ledger + // // 2026-10-21: PINNED to `main` of THIS FORK, BuddhiLW/zwasm, at // d74e63b4 (fork main fast-forwarded to upstream zwasm/zwasm main, // version string still 2.7.0, no newer tag). Upstream now carries all @@ -46,8 +58,8 @@ // embedding-API delta is additive (five engine hooks, // `Instance.engine()`, two appended trap kinds); one behaviour change, // `.auto` no longer retries the interpreter on a JIT validity verdict. - // The per-call JIT cost (zwasm/D-584 / D-585, - // `.dev/wasm_percall_findings.md`) is still open. + // The per-call JIT cost: zwasm/D-585 is fixed by the 2026-10-22 pin, + // zwasm/D-584 is avoided by ADR-0195 (`.dev/wasm_percall_findings.md`). // // Bumping: `zig fetch` rejects the annotated TAG object (NotACommit); // peel it with `git rev-parse 'vX.Y.Z^{}'` to get the commit sha. A @@ -68,8 +80,8 @@ // moment anything else claims that name. The `.hash` is // content-addressed, so the host of the URL never changes it. .zwasm = .{ - .url = "git+https://github.com/BuddhiLW/zwasm.git?ref=main#d74e63b4aac1c7d392fc7e562467232945788dfe", - .hash = "zwasm-2.7.0-FT1Fv-zingBTL0it1Jtud_E9LvRZ6jkeZWH66zgwYpsw", + .url = "git+https://github.com/BuddhiLW/zwasm.git?ref=main#e9d74934ec3212e502b101feab7673bf11c910ba", + .hash = "zwasm-2.7.0-FT1Fv3v5ngAavfQgV-JntjzckyYqMLUUCfYssfSrFN9S", .lazy = true, }, }, diff --git a/data/compat_tiers.yaml b/data/compat_tiers.yaml index d7c5d1a0b..3ff542883 100644 --- a/data/compat_tiers.yaml +++ b/data/compat_tiers.yaml @@ -400,7 +400,7 @@ tiers: impl: runtime/time/instant.zig # D-425: ctor (java.util.Date. [ms]) + .getTime landed; setTime/toString/ # toInstant are a follow-up (NOT listed — anti-overclaim, D-177). - methods: [after, before, getTime, toInstant] + methods: [after, before, from, getTime, toInstant] clojure_peer_vars: [clojure.core/inst-ms] - fqn: java.util.Random cljw_ns: cljw.java.util.Random @@ -474,11 +474,11 @@ tiers: # Value-wrap layer (F-009 wrap slot, G3-exempt): wraps instant.zig's # epoch data into a .typed_instance Value. wrap: runtime/time/instant_value.zig - methods: [getEpochSecond, getNano, isAfter, isBefore, minus, minusMillis, minusNanos, minusSeconds, now, ofEpochMilli, ofEpochSecond, parse, plus, plusMillis, plusNanos, plusSeconds, toEpochMilli, until] + methods: [from, getEpochSecond, getNano, isAfter, isBefore, minus, minusMillis, minusNanos, minusSeconds, now, ofEpochMilli, ofEpochSecond, parse, plus, plusMillis, plusNanos, plusSeconds, toEpochMilli, until] static_fields: ["EPOCH"] opaque_members: [MIN, MAX] clojure_peer_vars: [] - status: "WIRED (D-462) - statics now/ofEpochSecond([secs] | [secs nanos])/ofEpochMilli/parse mint a `.typed_instance` Instant VALUE (instant_value.zig, per-Runtime descriptor); instance methods getEpochSecond/getNano/toEpochMilli + value `=` + `(str i)` = ISO_INSTANT (variable fraction + Z) all reachable. pr-str divergence = AD-042 (no JVM `#object[…]` identity form). Pinned by test/e2e/phase15_java_time_instant.sh. Arithmetic methods (plusSeconds/minus*/isBefore/…) are NOT implemented (would be a follow-up unit, not listed above per anti-D-177). Cross-type ops plus/minus (Duration arg, with sub-second carry) ARE implemented + reachable (pinned by test/e2e/phase15_java_time_duration_between.sh). Sibling java.time reservations (ZonedDateTime) remain UNWIRED." + status: "WIRED (D-462) - statics now/ofEpochSecond([secs] | [secs nanos])/ofEpochMilli/parse mint a `.typed_instance` Instant VALUE (instant_value.zig, per-Runtime descriptor); instance methods getEpochSecond/getNano/toEpochMilli + value `=` + `(str i)` = ISO_INSTANT (variable fraction + Z) all reachable. pr-str divergence = AD-042 (no JVM `#object[…]` identity form). Pinned by test/clj/suites/java_time_values_test.clj. Arithmetic methods (plusSeconds/minus*/isBefore/…) are NOT implemented (would be a follow-up unit, not listed above per anti-D-177). Cross-type ops plus/minus (Duration arg, with sub-second carry) ARE implemented + reachable (pinned by test/clj/suites/java_time_values_test.clj). Sibling java.time reservations (ZonedDateTime) remain UNWIRED." - fqn: java.time.LocalDate cljw_ns: cljw.java.time.LocalDate keyword: local_date @@ -489,10 +489,10 @@ tiers: # Value-wrap layer (F-009 wrap slot, G3-exempt): mints the # .typed_instance LocalDate Value (1 field [epoch_day]). wrap: runtime/time/local_date_value.zig - methods: [atStartOfDay, atTime, getDayOfMonth, getDayOfWeek, getDayOfYear, getMonth, getMonthValue, getYear, isAfter, isBefore, isEqual, isLeapYear, lengthOfMonth, minusDays, minusMonths, minusWeeks, minusYears, now, of, ofEpochDay, parse, plusDays, plusMonths, plusWeeks, plusYears, until] + methods: [atStartOfDay, atTime, format, from, getDayOfMonth, getDayOfWeek, getDayOfYear, getMonth, getMonthValue, getYear, isAfter, isBefore, isEqual, isLeapYear, lengthOfMonth, minusDays, minusMonths, minusWeeks, minusYears, now, of, ofEpochDay, parse, plusDays, plusMonths, plusWeeks, plusYears, until] static_fields: ["EPOCH", "MAX", "MIN"] clojure_peer_vars: [] - status: "WIRED (D-462) - statics of(y m d)/now/parse mint a `.typed_instance` LocalDate VALUE (local_date_value.zig, per-Runtime descriptor) carrying [epoch_day] (signed days since 1970-01-01 via the Hinnant civil algorithm reused from instant.zig); instance methods getYear/getMonthValue/getDayOfMonth/plusDays + value `=` + `(str d)` = ISO `yyyy-MM-dd` (4-digit zero-padded year, e.g. LocalDate.of(1 1 1) → \"0001-01-01\") all reachable. pr-str divergence = AD-042 (no JVM `#object[…]` identity form). Pinned by test/e2e/phase15_java_time_local_date.sh. `now` is UTC-based (cljw has no zone DB). getDayOfWeek/getMonth return the DayOfWeek/Month enum VALUE types (`(str …)` = enum NAME, getValue = ISO ordinal); getDayOfYear returns an int 1..366 — pinned by test/e2e/phase15_java_time_dow_month.sh." + status: "WIRED (D-462) - statics of(y m d)/now/parse mint a `.typed_instance` LocalDate VALUE (local_date_value.zig, per-Runtime descriptor) carrying [epoch_day] (signed days since 1970-01-01 via the Hinnant civil algorithm reused from instant.zig); instance methods getYear/getMonthValue/getDayOfMonth/plusDays + value `=` + `(str d)` = ISO `yyyy-MM-dd` (4-digit zero-padded year, e.g. LocalDate.of(1 1 1) → \"0001-01-01\") all reachable. pr-str divergence = AD-042 (no JVM `#object[…]` identity form). Pinned by test/clj/suites/java_time_values_test.clj. `now` is UTC-based (cljw has no zone DB). getDayOfWeek/getMonth return the DayOfWeek/Month enum VALUE types (`(str …)` = enum NAME, getValue = ISO ordinal); getDayOfYear returns an int 1..366 — pinned by test/clj/suites/java_time_values_test.clj." - fqn: java.time.LocalTime cljw_ns: cljw.java.time.LocalTime keyword: local_time @@ -503,10 +503,10 @@ tiers: # Value-wrap layer (F-009 wrap slot, G3-exempt): mints the # .typed_instance LocalTime Value (1 field [nano_of_day]). wrap: runtime/time/local_time_value.zig - methods: [getHour, getMinute, getNano, getSecond, isAfter, isBefore, minusHours, minusMinutes, minusNanos, minusSeconds, now, of, ofNanoOfDay, ofSecondOfDay, parse, plusHours, plusMinutes, plusNanos, plusSeconds] + methods: [format, from, getHour, getMinute, getNano, getSecond, isAfter, isBefore, minusHours, minusMinutes, minusNanos, minusSeconds, now, of, ofNanoOfDay, ofSecondOfDay, parse, plusHours, plusMinutes, plusNanos, plusSeconds] static_fields: ["MAX", "MIDNIGHT", "MIN", "NOON"] clojure_peer_vars: [] - status: "WIRED (D-462) - statics of(h mi [s [n]] arity 2-4)/now/parse mint a `.typed_instance` LocalTime VALUE (local_time_value.zig, per-Runtime descriptor) carrying [nano_of_day] (i64 in [0, 86_400_000_000_000)); instance methods getHour/getMinute/getSecond/getNano + value `=` + `(str t)` = ISO local time (conditional :ss omitted when sec=nano=0, variable fraction, e.g. \"12:30\" / \"12:30:45.500\") all reachable. The date/time format+parse halves are shared with LocalDate/LocalDateTime via instant.zig (F-009 neutral home). pr-str divergence = AD-042 (no JVM `#object[…]` identity form). Pinned by test/e2e/phase15_java_time_local_date.sh. `now` is UTC-based (cljw has no zone DB)." + status: "WIRED (D-462) - statics of(h mi [s [n]] arity 2-4)/now/parse mint a `.typed_instance` LocalTime VALUE (local_time_value.zig, per-Runtime descriptor) carrying [nano_of_day] (i64 in [0, 86_400_000_000_000)); instance methods getHour/getMinute/getSecond/getNano + value `=` + `(str t)` = ISO local time (conditional :ss omitted when sec=nano=0, variable fraction, e.g. \"12:30\" / \"12:30:45.500\") all reachable. The date/time format+parse halves are shared with LocalDate/LocalDateTime via instant.zig (F-009 neutral home). pr-str divergence = AD-042 (no JVM `#object[…]` identity form). Pinned by test/clj/suites/java_time_values_test.clj. `now` is UTC-based (cljw has no zone DB)." - fqn: java.time.LocalDateTime cljw_ns: cljw.java.time.LocalDateTime keyword: local_date_time @@ -517,10 +517,10 @@ tiers: # Value-wrap layer (F-009 wrap slot, G3-exempt): mints the # .typed_instance LocalDateTime Value + the ISO-local formatter. wrap: runtime/time/local_date_time_value.zig - methods: [getDayOfMonth, getDayOfWeek, getDayOfYear, getHour, getMinute, getMonth, getMonthValue, getNano, getSecond, getYear, isAfter, isBefore, isEqual, minus, minusDays, minusHours, minusMinutes, minusMonths, minusNanos, minusSeconds, minusWeeks, minusYears, now, of, parse, plus, plusDays, plusHours, plusMinutes, plusMonths, plusNanos, plusSeconds, plusWeeks, plusYears, toLocalDate, toLocalTime, until] + methods: [format, from, getDayOfMonth, getDayOfWeek, getDayOfYear, getHour, getMinute, getMonth, getMonthValue, getNano, getSecond, getYear, isAfter, isBefore, isEqual, minus, minusDays, minusHours, minusMinutes, minusMonths, minusNanos, minusSeconds, minusWeeks, minusYears, now, of, parse, plus, plusDays, plusHours, plusMinutes, plusMonths, plusNanos, plusSeconds, plusWeeks, plusYears, toLocalDate, toLocalTime, until] static_fields: ["MAX", "MIN"] clojure_peer_vars: [] - status: "WIRED (D-462) - statics of(y m d h mi [s [n]] arity 5-7)/now/parse mint a `.typed_instance` LocalDateTime VALUE (local_date_time_value.zig, per-Runtime descriptor) carrying [epoch_day, nano_of_day] (epoch_day via the Hinnant civil algorithm reused from instant.zig); instance methods getYear/getMonthValue/getDayOfMonth/getHour/getMinute/getSecond/getNano + toLocalDate/toLocalTime (return the sibling LocalDate/LocalTime VALUE types) + value `=` + `(str d)` = ISO local date-time (conditional :ss omitted when sec=nano=0, variable fraction; the date/time halves are shared with LocalDate/LocalTime via instant.zig) all reachable. pr-str divergence = AD-042 (no JVM `#object[…]` identity form). Pinned by test/e2e/phase15_java_time_local_date_time.sh + toLocalDate/toLocalTime by test/e2e/phase15_java_time_local_date.sh. Arithmetic plusDays/minusDays/plusWeeks/minusWeeks (epoch_day) + plusHours/minusHours/plusMinutes/minusMinutes/plusSeconds/minusSeconds/plusNanos/minusNanos (nano_of_day with midnight carry/borrow into epoch_day) are implemented + reachable (pinned by test/e2e/phase15_java_time_ldt_arith.sh). Cross-type ops plus/minus (Duration arg, with midnight carry/borrow into epoch_day) ARE implemented + reachable (pinned by test/e2e/phase15_java_time_duration_between.sh). Calendar arithmetic plusMonths/minusMonths/plusYears/minusYears (civil month/year shift on the DATE part with JVM day-clamping, time-of-day kept; shares instant.zig's addMonthsToEpochDay/addYearsToEpochDay with LocalDate) ARE implemented + reachable (pinned by test/e2e/phase15_java_time_ldt_calendar.sh). `now` is UTC-based (cljw has no zone DB). getDayOfWeek/getMonth return the DayOfWeek/Month enum VALUE types (`(str …)` = enum NAME, getValue = ISO ordinal); getDayOfYear returns an int 1..366 — pinned by test/e2e/phase15_java_time_dow_month.sh." + status: "WIRED (D-462) - statics of(y m d h mi [s [n]] arity 5-7)/now/parse mint a `.typed_instance` LocalDateTime VALUE (local_date_time_value.zig, per-Runtime descriptor) carrying [epoch_day, nano_of_day] (epoch_day via the Hinnant civil algorithm reused from instant.zig); instance methods getYear/getMonthValue/getDayOfMonth/getHour/getMinute/getSecond/getNano + toLocalDate/toLocalTime (return the sibling LocalDate/LocalTime VALUE types) + value `=` + `(str d)` = ISO local date-time (conditional :ss omitted when sec=nano=0, variable fraction; the date/time halves are shared with LocalDate/LocalTime via instant.zig) all reachable. pr-str divergence = AD-042 (no JVM `#object[…]` identity form). Pinned by test/clj/suites/java_time_values_test.clj + toLocalDate/toLocalTime by test/clj/suites/java_time_values_test.clj. Arithmetic plusDays/minusDays/plusWeeks/minusWeeks (epoch_day) + plusHours/minusHours/plusMinutes/minusMinutes/plusSeconds/minusSeconds/plusNanos/minusNanos (nano_of_day with midnight carry/borrow into epoch_day) are implemented + reachable (pinned by test/clj/suites/java_time_values_test.clj). Cross-type ops plus/minus (Duration arg, with midnight carry/borrow into epoch_day) ARE implemented + reachable (pinned by test/clj/suites/java_time_values_test.clj). Calendar arithmetic plusMonths/minusMonths/plusYears/minusYears (civil month/year shift on the DATE part with JVM day-clamping, time-of-day kept; shares instant.zig's addMonthsToEpochDay/addYearsToEpochDay with LocalDate) ARE implemented + reachable (pinned by test/clj/suites/java_time_values_test.clj). `now` is UTC-based (cljw has no zone DB). getDayOfWeek/getMonth return the DayOfWeek/Month enum VALUE types (`(str …)` = enum NAME, getValue = ISO ordinal); getDayOfYear returns an int 1..366 — pinned by test/clj/suites/java_time_values_test.clj." - fqn: java.time.DayOfWeek cljw_ns: cljw.java.time.DayOfWeek keyword: host_enum @@ -532,7 +532,7 @@ tiers: methods: [getValue, name, of, valueOf, values] static_fields: ["FRIDAY", "MONDAY", "SATURDAY", "SUNDAY", "THURSDAY", "TUESDAY", "WEDNESDAY"] clojure_peer_vars: [] - status: "WIRED (ADR-0161, folds D-462) — host-enum singleton (.host_instance, ordinal in state[0]). Static field DayOfWeek/MONDAY..SUNDAY resolves; LocalDate/LocalDateTime.getDayOfWeek returns the SAME interned singleton (identical? parity). (str)/.name = enum NAME, getValue = 1-7 (ISO Mon=1), value `=` by identity, compare by ordinal (sign, AD-043). Pinned by test/e2e/phase15_java_time_dow_month.sh." + status: "WIRED (ADR-0161, folds D-462) — host-enum singleton (.host_instance, ordinal in state[0]). Static field DayOfWeek/MONDAY..SUNDAY resolves; LocalDate/LocalDateTime.getDayOfWeek returns the SAME interned singleton (identical? parity). (str)/.name = enum NAME, getValue = 1-7 (ISO Mon=1), value `=` by identity, compare by ordinal (sign, AD-043). Pinned by test/clj/suites/java_time_values_test.clj." - fqn: java.time.Month cljw_ns: cljw.java.time.Month keyword: host_enum @@ -544,7 +544,7 @@ tiers: methods: [getValue, name, of, valueOf, values] static_fields: ["APRIL", "AUGUST", "DECEMBER", "FEBRUARY", "JANUARY", "JULY", "JUNE", "MARCH", "MAY", "NOVEMBER", "OCTOBER", "SEPTEMBER"] clojure_peer_vars: [] - status: "WIRED (ADR-0161, folds D-462) — host-enum singleton (.host_instance, ordinal in state[0]). Static field Month/JANUARY..DECEMBER resolves; LocalDate/LocalDateTime.getMonth returns the SAME interned singleton (identical? parity). (str)/.name = enum NAME, getValue = 1-12, value `=` by identity, compare by ordinal (sign, AD-043). Pinned by test/e2e/phase15_java_time_dow_month.sh." + status: "WIRED (ADR-0161, folds D-462) — host-enum singleton (.host_instance, ordinal in state[0]). Static field Month/JANUARY..DECEMBER resolves; LocalDate/LocalDateTime.getMonth returns the SAME interned singleton (identical? parity). (str)/.name = enum NAME, getValue = 1-12, value `=` by identity, compare by ordinal (sign, AD-043). Pinned by test/clj/suites/java_time_values_test.clj." - fqn: java.time.Duration cljw_ns: cljw.java.time.Duration keyword: duration @@ -558,7 +558,7 @@ tiers: methods: [abs, between, dividedBy, getNano, getSeconds, isNegative, isZero, minus, minusDays, minusHours, minusMillis, minusMinutes, minusNanos, minusSeconds, multipliedBy, negated, of, ofDays, ofHours, ofMillis, ofMinutes, ofNanos, ofSeconds, parse, plus, plusDays, plusHours, plusMillis, plusMinutes, plusNanos, plusSeconds, toMillis, toMinutes] static_fields: ["ZERO"] clojure_peer_vars: [] - status: "WIRED (D-462) - statics ofSeconds([secs] | [secs nanoAdj])/ofMillis/ofNanos/ofMinutes/ofHours/ofDays mint a NORMALIZED (nanos∈[0,1e9), seconds carries sign) `.typed_instance` Duration VALUE (duration_value.zig, per-Runtime descriptor); instance methods getSeconds/getNano/toMillis/toMinutes + value `=` + `(str d)` = ISO-8601 `PT…` (the exact JDK Duration.toString) all reachable. pr-str divergence = AD-042 (no JVM `#object[…]` identity form). Pinned by test/e2e/phase15_java_time_duration.sh. The static factory `between(a b)` mints the elapsed Duration `b - a`, dispatching on the temporal type of BOTH args (Instant/LocalDateTime/LocalTime; mismatched/non-temporal pair → type_arg_invalid) — pinned by test/e2e/phase15_java_time_duration_between.sh. Remaining arithmetic methods (toNanos/…) are NOT implemented (would be a follow-up unit, not listed above per anti-D-177)." + status: "WIRED (D-462) - statics ofSeconds([secs] | [secs nanoAdj])/ofMillis/ofNanos/ofMinutes/ofHours/ofDays mint a NORMALIZED (nanos∈[0,1e9), seconds carries sign) `.typed_instance` Duration VALUE (duration_value.zig, per-Runtime descriptor); instance methods getSeconds/getNano/toMillis/toMinutes + value `=` + `(str d)` = ISO-8601 `PT…` (the exact JDK Duration.toString) all reachable. pr-str divergence = AD-042 (no JVM `#object[…]` identity form). Pinned by test/clj/suites/java_time_values_test.clj. The static factory `between(a b)` mints the elapsed Duration `b - a`, dispatching on the temporal type of BOTH args (Instant/LocalDateTime/LocalTime; mismatched/non-temporal pair → type_arg_invalid) — pinned by test/clj/suites/java_time_values_test.clj. Remaining arithmetic methods (toNanos/…) are NOT implemented (would be a follow-up unit, not listed above per anti-D-177)." - fqn: java.time.ZonedDateTime cljw_ns: cljw.java.time.ZonedDateTime keyword: zoned_date_time @@ -569,7 +569,30 @@ tiers: methods: [] clojure_peer_vars: [] status: "reservation - DEFERRED (D-462): blocked on a timezone database. ZonedDateTime's headline use is named zones (ZoneId/of \"Asia/Tokyo\" -> offset+DST resolution), which needs a bundled IANA tzdata cljw does not ship (binary-size + update-cadence scope decision, user/ADR-owned, not autonomous). A fixed-offset-only subset (ZoneOffset) would just be OffsetDateTime and miss the point. Instant/LocalDateTime/LocalDate/LocalTime/Duration ARE wired (D-462); ZonedDateTime waits for a tz-DB decision." - - {fqn: "java.time.ZoneId", native_ns: "cljw.host.java.time.ZoneId", phase: 6} + - fqn: java.time.ZoneId + cljw_ns: cljw.java.time.ZoneId + keyword: zone_offset + tier: A + phase: 15 + files: + surface: runtime/java/time/ZoneId.zig + impl: runtime/time/zone_offset.zig + methods: [of, systemDefault] + clojure_peer_vars: [] + status: "WIRED (fixed offsets only) - ZoneId/of resolves an offset id (Z, +hh[:mm[:ss]]) to a ZoneOffset and UTC / GMT / UT (optionally +offset) to an offset-backed ZoneRegion; a region id such as Europe/Paris raises zone_id_unknown (DateTimeException) because cljw ships no tz database (the ZonedDateTime row records that decision). systemDefault is the UTC region, matching the UTC-based now factories. Instances list java.time.ZoneId as host supertype. Pinned by test/clj/suites/java_time_format_test.clj." + opaque_members: [getAvailableZoneIds, getRules, ofOffset] + - fqn: java.time.ZoneOffset + cljw_ns: cljw.java.time.ZoneOffset + keyword: zone_offset + tier: A + phase: 15 + files: + surface: runtime/java/time/ZoneOffset.zig + impl: runtime/time/zone_offset.zig + methods: [getId, getTotalSeconds, normalized, of, ofHours, ofHoursMinutes, ofHoursMinutesSeconds, ofTotalSeconds] + static_fields: ["MAX", "MIN", "UTC"] + clojure_peer_vars: [] + status: "WIRED - a .typed_instance [total-seconds, prefix] value; = and hash by value; (str z) = the id (Z, +02:00). Factories validate with the JVM DateTimeException texts (hours -18..18, sign agreement, the +-18:00 bound). Pinned by test/clj/suites/java_time_format_test.clj." # java.net - {fqn: "java.net.URL", native_ns: "cljw.host.java.net.URL", phase: 14} # java.net.URI / URLEncoder — minimal host surfaces unblocking hiccup @@ -666,6 +689,45 @@ tiers: methods: [name, valueOf, values] static_fields: ["CENTURIES", "DAYS", "DECADES", "ERAS", "FOREVER", "HALF_DAYS", "HOURS", "MICROS", "MILLENNIA", "MILLIS", "MINUTES", "MONTHS", "NANOS", "SECONDS", "WEEKS", "YEARS"] clojure_peer_vars: [] + - fqn: java.time.temporal.ChronoField + cljw_ns: cljw.java.time.temporal.ChronoField + keyword: host_enum + tier: A + phase: 15 + files: + surface: runtime/java/time/temporal/ChronoField.zig + impl: runtime/host_enum.zig + methods: [name, valueOf, values] + static_fields: ["ALIGNED_DAY_OF_WEEK_IN_MONTH", "ALIGNED_DAY_OF_WEEK_IN_YEAR", "ALIGNED_WEEK_OF_MONTH", "ALIGNED_WEEK_OF_YEAR", "AMPM_OF_DAY", "CLOCK_HOUR_OF_AMPM", "CLOCK_HOUR_OF_DAY", "DAY_OF_MONTH", "DAY_OF_WEEK", "DAY_OF_YEAR", "EPOCH_DAY", "ERA", "HOUR_OF_AMPM", "HOUR_OF_DAY", "INSTANT_SECONDS", "MICRO_OF_DAY", "MICRO_OF_SECOND", "MILLI_OF_DAY", "MILLI_OF_SECOND", "MINUTE_OF_DAY", "MINUTE_OF_HOUR", "MONTH_OF_YEAR", "NANO_OF_DAY", "NANO_OF_SECOND", "OFFSET_SECONDS", "PROLEPTIC_MONTH", "SECOND_OF_DAY", "SECOND_OF_MINUTE", "YEAR", "YEAR_OF_ERA"] + clojure_peer_vars: [] + status: "WIRED - the 30 enum constants as host-enum singletons (ADR-0161 registry); (str f) = display name (HourOfDay), (.name f) = enum name. The field argument of DateTimeFormatterBuilder and Parsed.getLong / isSupported; ranges and semantics live in runtime/time/date_time_format.zig::Field. Pinned by test/clj/suites/java_time_format_test.clj." + - fqn: java.time.format.DateTimeFormatter + cljw_ns: cljw.java.time.format.DateTimeFormatter + keyword: date_time_format + tier: A + phase: 15 + files: + surface: runtime/java/time/format/DateTimeFormatter.zig + impl: runtime/time/date_time_format.zig + wrap: runtime/time/date_time_formatter_value.zig + methods: [format, getZone, ofPattern, parse, withLocale, withZone] + static_fields: ["BASIC_ISO_DATE", "ISO_DATE", "ISO_DATE_TIME", "ISO_INSTANT", "ISO_LOCAL_DATE", "ISO_LOCAL_DATE_TIME", "ISO_LOCAL_TIME", "ISO_OFFSET_DATE", "ISO_OFFSET_DATE_TIME", "ISO_OFFSET_TIME", "ISO_ORDINAL_DATE", "ISO_TIME"] + clojure_peer_vars: [] + status: "WIRED - the JDK 25 printer-parser model one to one: pattern letters G u y M L d D F E a h K k H m s S n A N V X x Z O, quoted literals, [optional] sections, adjacent value parsing, the SMART (ofPattern / builder) and STRICT (ISO constants) resolvers, toString in the JVM form, English text. format reads LocalDate / LocalTime / LocalDateTime / Instant / Parsed (withZone re-projects an Instant); parse returns a Parsed TemporalAccessor. Refused loudly (not_implemented): the week-based, quarter, zone-name, day-period, Julian and padding letters (Y w W e c Q q z v B g p); region zones need the absent tz database. Drives malli.transform string<->date. Pinned by test/clj/suites/java_time_format_test.clj." + opaque_members: [ISO_WEEK_DATE, ISO_ZONED_DATE_TIME, RFC_1123_DATE_TIME, getLocale, getResolverStyle, ofLocalizedDate, ofLocalizedDateTime, ofLocalizedTime, withResolverStyle] + - fqn: java.time.format.DateTimeFormatterBuilder + cljw_ns: cljw.java.time.format.DateTimeFormatterBuilder + keyword: date_time_format + tier: A + phase: 15 + files: + surface: runtime/java/time/format/DateTimeFormatterBuilder.zig + impl: runtime/time/date_time_format.zig + wrap: runtime/time/date_time_formatter_value.zig + methods: [append, appendFraction, appendInstant, appendLiteral, appendOffset, appendOffsetId, appendOptional, appendPattern, appendText, appendValue, appendZoneId, optionalEnd, optionalStart, parseCaseInsensitive, parseCaseSensitive, parseDefaulting, parseLenient, parseStrict, toFormatter] + clojure_peer_vars: [] + status: "WIRED - a mutable host builder; each call mutates it and returns it, toFormatter freezes a SMART formatter. appendValue takes (field) or (field width); the SignStyle / TextStyle overloads are absent because those enums are not surfaced. Pinned by test/clj/suites/java_time_format_test.clj." + opaque_members: [appendChronologyId, appendLocalized, appendLocalizedOffset, appendValueReduced, appendZoneOrOffsetId, appendZoneRegionId, padNext] # java.security - fqn: java.security.MessageDigest cljw_ns: cljw.java.security.MessageDigest @@ -743,7 +805,7 @@ tiers: # directory (verified)" claim is stale and superseded. Tier letter stays # B (a status-truth fix, not a tier change — tier_classification.md). clojure.math: {status: "implemented (clojure/math.clj; loaded by bootstrap.zig; corpus clojure_math.txt + math_doc_claims.txt)"} - clojure.spec.alpha: {status: "implemented (clojure/spec/alpha.clj, 2019-line port; loaded by bootstrap.zig; e2e phase15_spec.sh + corpus spec.txt; D-475). Generative subset (s/gen, s/exercise) throws at call time — clojure.test.check not bundled"} + clojure.spec.alpha: {status: "implemented (clojure/spec/alpha.clj, 2019-line port; loaded by bootstrap.zig; e2e test/clj/suites/test_reporters_test.clj + corpus spec.txt; D-475). Generative subset (s/gen, s/exercise) throws at call time — clojure.test.check not bundled"} clojure.spec.gen.alpha: {status: "implemented (clojure/spec/gen/alpha.clj; loaded by bootstrap.zig). Combinators/prims dynaload clojure.test.check.* at call time; unbundled so they throw when forced"} C: description: "Thin reflective implementation via TypeDescriptor walk." diff --git a/data/placement.yaml b/data/placement.yaml index 04d946bb4..64ce45bfc 100644 --- a/data/placement.yaml +++ b/data/placement.yaml @@ -24,9 +24,9 @@ schema_version: "2.0" summary: namespaces_required: 36 - total: 1456 - zig: 424 - clj: 806 + total: 1464 + zig: 431 + clj: 807 value: 220 unbound: 6 vars: @@ -69,6 +69,9 @@ vars: "cljw.internal/__agent-await": { impl: zig, flags: "-" } "cljw.internal/__agent-fail-mode?": { impl: zig, flags: "-" } "cljw.internal/__agent-set-fail-mode": { impl: zig, flags: "-" } + "cljw.internal/__array-fill": { impl: zig, flags: "-" } + "cljw.internal/__array-from": { impl: zig, flags: "-" } + "cljw.internal/__array-kind": { impl: zig, flags: "-" } "cljw.internal/__array-make": { impl: zig, flags: "-" } "cljw.internal/__assertion-error": { impl: zig, flags: "-" } "cljw.internal/__chunk-count": { impl: zig, flags: "-" } @@ -83,6 +86,7 @@ vars: "cljw.internal/__defrecord!": { impl: zig, flags: "-" } "cljw.internal/__deftype!": { impl: zig, flags: "-" } "cljw.internal/__delay-create": { impl: zig, flags: "-" } + "cljw.internal/__dorun": { impl: zig, flags: "-" } "cljw.internal/__dump-host-classes": { impl: zig, flags: "-" } "cljw.internal/__dump-placement": { impl: zig, flags: "-" } "cljw.internal/__entry-at": { impl: zig, flags: "-" } @@ -101,6 +105,7 @@ vars: "cljw.internal/__make-protocol!": { impl: zig, flags: "-" } "cljw.internal/__make-protocol-fn!": { impl: zig, flags: "-" } "cljw.internal/__map->record": { impl: zig, flags: "-" } + "cljw.internal/__map-step": { impl: zig, flags: "-" } "cljw.internal/__methods": { impl: zig, flags: "-" } "cljw.internal/__native-type": { impl: zig, flags: "-" } "cljw.internal/__open-input-stream": { impl: zig, flags: "-" } @@ -128,6 +133,7 @@ vars: "cljw.internal/__string-reader": { impl: zig, flags: "-" } "cljw.internal/__string-writer": { impl: zig, flags: "-" } "cljw.internal/__subvec": { impl: zig, flags: "-" } + "cljw.internal/__subvec-fast": { impl: zig, flags: "-" } "cljw.internal/__unsupported-host-ref": { impl: zig, flags: "-" } "cljw.internal/__update-in": { impl: zig, flags: "-" } "cljw.internal/__vector-pop": { impl: zig, flags: "-" } @@ -239,6 +245,7 @@ vars: "clojure.core/->>": { impl: value, flags: "macro" } "clojure.core/->Eduction": { impl: clj, flags: "-" } "clojure.core/-apply-to": { impl: value, flags: "-" } + "clojure.core/-array-elems": { impl: clj, flags: "private" } "clojure.core/-array-from": { impl: clj, flags: "private" } "clojure.core/-array-from2": { impl: clj, flags: "private" } "clojure.core/-as-transient": { impl: value, flags: "-" } @@ -285,6 +292,7 @@ vars: "clojure.core/-lookup": { impl: value, flags: "-" } "clojure.core/-map-indexed-lazy": { impl: clj, flags: "private" } "clojure.core/-map-lazy": { impl: clj, flags: "-" } + "clojure.core/-map-n-lazy": { impl: clj, flags: "private" } "clojure.core/-map-n-step": { impl: clj, flags: "-" } "clojure.core/-merge-sorted": { impl: clj, flags: "-" } "clojure.core/-meta": { impl: value, flags: "-" } @@ -310,9 +318,8 @@ vars: "clojure.core/-sorted-seq-from": { impl: value, flags: "-" } "clojure.core/-spread": { impl: clj, flags: "-" } "clojure.core/-sub-sequence": { impl: value, flags: "-" } + "clojure.core/-subvec-slow": { impl: clj, flags: "private" } "clojure.core/-take-eager": { impl: zig, flags: "private,zig-leaf" } - "clojure.core/-to-byte": { impl: clj, flags: "private" } - "clojure.core/-to-short": { impl: clj, flags: "private" } "clojure.core/-tset-contains?": { impl: value, flags: "-" } "clojure.core/-tx-seq-pump": { impl: clj, flags: "-" } "clojure.core/-tx-seq-run": { impl: clj, flags: "-" } @@ -1482,6 +1489,7 @@ vars: "wasm/load": { impl: zig, flags: "-" } "wasm/load-component": { impl: zig, flags: "-" } "wasm/mem-read": { impl: zig, flags: "-" } + "wasm/mem-read-array": { impl: zig, flags: "-" } "wasm/mem-size": { impl: zig, flags: "-" } "wasm/mem-write!": { impl: zig, flags: "-" } "wasm/resource-drop": { impl: zig, flags: "-" } diff --git a/docs/README.md b/docs/README.md index ea8b950ea..1d13379f1 100644 --- a/docs/README.md +++ b/docs/README.md @@ -20,6 +20,9 @@ GitHub wiki. - [`examples/ffi/`](./examples/ffi/README.md): `cljw.ffi`, calling a native shared library (C, Go c-shared, Rust cdylib, Zig) directly, with the hive C ABI as the worked example. +- [`examples/zig-native/`](./examples/zig-native/README.md): one Zig source + (comptime tables, `@Vector` SIMD, bit builtins, `std.crypto`) called as a + wasm guest and as a native library, with the per-call costs of each route. - [`works/ladder.md`](./works/ladder.md) — which real-world Clojure libraries load and run today. diff --git a/docs/clojure_vs_clojurewasm.md b/docs/clojure_vs_clojurewasm.md index d37bd8fc3..bc7d3a545 100644 --- a/docs/clojure_vs_clojurewasm.md +++ b/docs/clojure_vs_clojurewasm.md @@ -84,7 +84,7 @@ the JVM's `hashCode`/`hasheq` split. | `hash` / `.hashCode` values | JVM/Murmur3 values | cljw-native values (intra-cljw consistent) | AD-009 | | `APersistentMap/mapHash` vs `/mapHasheq` | distinct (additive `hashCode` vs murmur) | both = the single `(hash m)` content hash | AD-028 | | `ns-interns`/`ns-publics` of `clojure.core` | includes `reduce`, `+`, … | omits the `rt`-referred primitives (`ns-map` includes them) | AD-011 | -| `(class (object-array 0))` + typed arrays | `[Ljava.lang.Object;`, `aset` type-checks | `array`; type-erased `[]Value`, `^"[B"` hints advisory | AD-019 | +| `(class (object-array 0))`, `make-array` / `into-array` type | `[Ljava.lang.Object;`; the class picks the kind | `array`; the class arg is ignored (object array), hints advisory | AD-019 | | Unresolved `clojure.lang.*` / `.asm.*` ref | resolves to the JVM class | namespace LOADS; the ref errors only if evaluated (call-time) | AD-022 | | `(extend-protocol P java.util.Map …)` | covers all clj maps (a map IS a `java.util.Map`) | LOAD-ONLY no-op; a map receiver falls to `Object` | AD-023 | | `java.util.Map` methods under a `clojure.lang.*` deftype section | dispatch (the interfaces extend the java ones) | accepted-and-dropped; a later `(.iterator x)` is method-not-found | AD-027 | @@ -205,7 +205,8 @@ bridge to the JVM are not part of the runtime (Tier C/D): cljw provides its own host-class surface (a curated set, see [`data/compat_tiers.yaml`](../data/compat_tiers.yaml)) rather than open JVM interop. -Note that Java *arrays* are implemented (type-erased — see AD-019), as is +Note that Java *arrays* are implemented (typed native storage for the typed +constructors, ADR-0204; see AD-019 for the class argument), as is `with-local-vars` (AD-015); they are no longer in this list. ## How to read this page diff --git a/docs/examples/ffi/README.md b/docs/examples/ffi/README.md index 1cadddabc..7087c443e 100644 --- a/docs/examples/ffi/README.md +++ b/docs/examples/ffi/README.md @@ -22,10 +22,37 @@ ordinary Clojure fn. No libffi, no binding generator, no rebuild of cljw. | `(string ptr)` | copy a NUL-terminated C string; 0 or nil gives nil | | `(bytes ptr n)` | copy `n` bytes into a byte array | -Types: `:void` (return only), `:int` (C int), `:long` (int64), `:double`, -`:pointer` (address as integer, nil is NULL), `:string` (a copy both ways, -NULL is nil, a returned string is never freed), `:bytes` (argument only, a -copy of a byte array). +| type | C side | Clojure side | +|---|---|---| +| `:void` | `void` (return only) | nil | +| `:int` | `int` | integer, truncated to 32 bits; a return is sign-extended | +| `:long` | `int64_t` / `long` | integer | +| `:double` | `double` | float, or an integer widened | +| `:pointer` | `T *` | an address integer, nil (NULL), or a primitive array: its storage, no copy, C reads and writes it in place for the call | +| `:string` | `const char *` | a copy both ways; NULL is nil; a returned string is never freed | +| `:bytes` | `const uint8_t *` | argument only: a copy of a byte array (one memcpy), so C writes do not reach it | + +A primitive array is contiguous native storage of its element width, so it is +a C array as it stands: + +| constructor | C element | +|---|---| +| `byte-array` | `int8_t` | +| `short-array` | `int16_t` | +| `int-array` | `int32_t` | +| `long-array` | `int64_t` | +| `float-array` | `float` | +| `double-array` | `double` | +| `char-array` | `uint32_t` (a code point) | +| `boolean-array` | `uint8_t` (0 or 1) | + +```clojure +(def dot (ffi/function lib "dot" [:pointer :pointer :long] :double)) +(dot (double-array xs) (double-array ys) (count xs)) +``` + +`object-array`, `to-array` and `into-array` hold Clojure values and are not +accepted as `:pointer`. Limits: at most 6 integer-class and 8 `:double` arguments, no varargs (`printf`), no structs by value, no `float`. Available on x86_64 and aarch64, diff --git a/docs/examples/polyglot/README.md b/docs/examples/polyglot/README.md index 698bac5bc..0bd9aa781 100644 --- a/docs/examples/polyglot/README.md +++ b/docs/examples/polyglot/README.md @@ -13,6 +13,10 @@ against the committed guests, then against guests rebuilt from the recipe in each source header, then `hosts.cljc` on every Clojure host on PATH. What this page claims is what that gate checks. +For Zig beyond scalars (compile-time tables, SIMD, `std.crypto`), and the +same source as a native library through `cljw.ffi`, see +[`../zig-native/`](../zig-native/README.md). + ## Three shapes of guest | guest | source | built with | size | crosses the boundary as | cljw entry point | diff --git a/docs/examples/wasm/README.md b/docs/examples/wasm/README.md index 6d8caa80d..a0f6f7422 100644 --- a/docs/examples/wasm/README.md +++ b/docs/examples/wasm/README.md @@ -30,6 +30,18 @@ own path, so results, fuel limits and traps behave the same. (.sum_f64 k buf 4) ;=> 10.0 ``` +`wasm/mem-write!` takes a vector, or a primitive array (`double-array`, +`long-array`, ...). An array whose element matches the dtype (`:f64` and a +double-array, `:i64` and a long-array, `:i32` and an int-array) is copied in +with one memcpy; a vector is encoded one element at a time. +`wasm/mem-read-array` is the bulk read: it answers a primitive array of the +dtype's width (`:u16` widens to an int-array, `:u32` to a long-array). + +```clojure +(wasm/mem-write! k :f64 buf (double-array [1 2 3 4])) +(vec (wasm/mem-read-array k :f64 buf 4)) ;=> [1.0 2.0 3.0 4.0] +``` + ## Run it The **released binaries** (GitHub releases, `brew install diff --git a/docs/examples/zig-native/README.md b/docs/examples/zig-native/README.md new file mode 100644 index 000000000..3d2b01cec --- /dev/null +++ b/docs/examples/zig-native/README.md @@ -0,0 +1,72 @@ +# Zig from Clojure: one source, two FFI routes + +[`kernel.zig`](./kernel.zig) holds things Zig does well and Clojure does not, +and [`zig_native.clj`](./zig_native.clj) calls each of them twice: as a +sandboxed wasm guest and as a native library through `cljw.ffi`. + +```sh +cd docs/examples/zig-native +cljw zig_native.clj # PASS zig-wasm, SKIP zig-native +zig build-lib -dynamic -O ReleaseFast kernel.zig +cljw zig_native.clj # PASS zig-wasm, PASS zig-native +``` + +`test/e2e/phase16_zig_native.sh` runs it in CI against the committed +`kernel.wasm`, then rebuilds both artifacts from the recipe in the source +header and runs it again. + +| export | Zig feature | from Clojure | +|---|---|---| +| `nth_prime` | a 10,000-prime table computed by the compiler (`comptime`) | `(nth-prime 9999) ;=> 104729`, a table lookup | +| `opcode` | `std.StaticStringMap`, a keyword table built at compile time | `"pow" ;=> 5`, `"mod" ;=> -1` | +| `dot` | `@Vector(4, f64)` SIMD, four lanes per step | equals `(reduce + (map * xs xs))` | +| `popcount`, `bitrev` | `@popCount`, `@bitReverse` | `(popcount 255) ;=> 8` | +| `blake3` | `std.crypto.hash.Blake3` | `"abc"` hashes to the reference digest | + +## Two routes + +| | wasm (`wasm/load`) | native (`cljw.ffi`) | +|---|---|---| +| build | `zig build-exe -target wasm32-freestanding` | `zig build-lib -dynamic` | +| artifact | `kernel.wasm`, 47 KB, any OS | `libkernel.so` / `.dylib`, per platform | +| safety | sandboxed: fuel, memory cap, a fault is an exception | none: a bad pointer crashes the process | +| data | the guest owns its buffers (`f64_addr`, `byte_addr`), the host fills them with `wasm/mem-write!` (a `double-array` is one memcpy) | `:string` and `:bytes` arguments are copied in; `:pointer` takes a primitive array's storage (no copy) or an address the host owns | +| signatures | `wasm/exports` reads them back | written by hand: `[:double :double] :double` | +| platforms | everywhere cljw runs | x86_64 and aarch64, Linux and macOS | + +Only `export fn` with C-ABI types crosses either route. Slices, error unions, +optionals, generics and structs stay inside Zig: flatten them to pointer plus +length and status codes in a thin export. + +## What it costs + +v1.15.3, ReleaseSafe, nanoseconds per call, best of 5 runs of 20,000 calls, +on a loaded host (ratios are stable, absolute numbers are not): + +| call | ns | +|---|---| +| empty Clojure fn | 167 | +| `cljw.ffi` native call | 342 | +| wasm, `:interp` | 550 | +| wasm, `:jit`, a tiny module | 1,321 | +| wasm, `:jit`, this 47 KB module | 5,600 | + +| work | Clojure | Zig through `cljw.ffi` | +|---|---|---| +| dot product, 1,000 doubles | 1.85 ms | 12.2 µs | +| dot product, 100,000 doubles | 357 ms | 1.2 to 5.5 ms, mostly copying the `:bytes` arguments | +| the 1,000th prime | 294 ms by trial division | 1 µs from the compile-time table | + +Cross the boundary with big pieces of work, not per element: building a +`:bytes` argument one double at a time in Clojure (`f64s->bytes` here) costs +more than the whole computation. + +## Known limits + +- wasm `i64` values cross exactly only within ±2^47: larger results come back + as a lossy `Double` and larger arguments are rejected + (CLJW-WASM-I64-FULL-RANGE). The native route carries all 64 bits. +- `:jit` call overhead grows with module size (CLJW-WASM-JIT-PERCALL-SCALES). +- Primitive arrays are contiguous native storage (ADR-0204): pass a + `double-array` as `:pointer` instead of packing `:bytes`. Native buffers + that outlive one call are CLJW-FFI-NATIVE-BUFFER. diff --git a/docs/examples/zig-native/kernel.wasm b/docs/examples/zig-native/kernel.wasm new file mode 100755 index 000000000..f70be8f80 Binary files /dev/null and b/docs/examples/zig-native/kernel.wasm differ diff --git a/docs/examples/zig-native/kernel.zig b/docs/examples/zig-native/kernel.zig new file mode 100644 index 000000000..0e7b36052 --- /dev/null +++ b/docs/examples/zig-native/kernel.zig @@ -0,0 +1,102 @@ +//! What Zig is good at, called from Clojure. One source, two builds: +//! +//! native (cljw.ffi): zig build-lib -dynamic -O ReleaseFast kernel.zig +//! wasm (wasm/load): zig build-exe kernel.zig -target wasm32-freestanding \ +//! -O ReleaseSmall -fno-entry -rdynamic -femit-bin=kernel.wasm +//! +//! Every export takes and returns C-ABI scalars or pointers. The wasm build +//! cannot see host memory, so it hands out its own buffers (`f64_addr`, +//! `byte_addr`) for the host to fill with wasm/mem-write!. +const std = @import("std"); + +// The first 10000 primes, computed by the compiler. The binary ships the table. +const PRIMES = blk: { + @setEvalBranchQuota(200_000_000); + var table: [10_000]u32 = undefined; + var n: usize = 0; + var c: u32 = 2; + while (n < table.len) : (c += 1) { + var is_p = true; + for (table[0..n]) |p| { + if (p * p > c) break; + if (c % p == 0) { + is_p = false; + break; + } + } + if (is_p) { + table[n] = c; + n += 1; + } + } + break :blk table; +}; + +/// The ith prime (0-based) from the compile-time table; 0 when out of range. +export fn nth_prime(i: u32) u32 { + return if (i < PRIMES.len) PRIMES[i] else 0; +} + +// A keyword table, also built at compile time. +const OPS = std.StaticStringMap(i32).initComptime(.{ + .{ "add", 1 }, .{ "sub", 2 }, .{ "mul", 3 }, .{ "div", 4 }, .{ "pow", 5 }, +}); + +/// Opcode for an operator name of `len` bytes at `ptr`; -1 when unknown. +export fn opcode(ptr: [*]const u8, len: usize) i32 { + return OPS.get(ptr[0..len]) orelse -1; +} + +/// Dot product of two f64 arrays of `len` elements, four lanes at a time. +export fn dot(a: [*]const f64, b: [*]const f64, len: usize) f64 { + const V = @Vector(4, f64); + var acc: V = @splat(0); + var i: usize = 0; + while (i + 4 <= len) : (i += 4) { + const va: V = a[i..][0..4].*; + const vb: V = b[i..][0..4].*; + acc += va * vb; + } + var sum = @reduce(.Add, acc); + while (i < len) : (i += 1) sum += a[i] * b[i]; + return sum; +} + +/// Number of set bits in x. +export fn popcount(x: u64) u32 { + return @popCount(x); +} + +/// x with its 64 bits in reverse order. +export fn bitrev(x: u64) u64 { + return @bitReverse(x); +} + +/// BLAKE3 of `len` bytes at `ptr`, written to `out` as 64 hex chars and a NUL. +export fn blake3(ptr: [*]const u8, len: usize, out: [*]u8) void { + var d: [32]u8 = undefined; + std.crypto.hash.Blake3.hash(ptr[0..len], &d, .{}); + const hex = std.fmt.bytesToHex(d, .lower); + @memcpy(out[0..64], &hex); + out[64] = 0; +} + +// Guest-owned buffers for the wasm build; unused by the native build. +var f64_buf: [4096]f64 = undefined; +var byte_buf: [4096]u8 = undefined; + +/// Address and capacity (in elements) of the f64 buffer. +export fn f64_addr() usize { + return @intFromPtr(&f64_buf); +} +export fn f64_cap() usize { + return f64_buf.len; +} + +/// Address and capacity (in bytes) of the byte buffer. +export fn byte_addr() usize { + return @intFromPtr(&byte_buf); +} +export fn byte_cap() usize { + return byte_buf.len; +} diff --git a/docs/examples/zig-native/zig_native.clj b/docs/examples/zig-native/zig_native.clj new file mode 100644 index 000000000..d0b44ee1c --- /dev/null +++ b/docs/examples/zig-native/zig_native.clj @@ -0,0 +1,68 @@ +;; What Zig is good at, from Clojure: compile-time tables, @Vector SIMD, bit +;; builtins and std.crypto, through both FFI routes. +;; Run from this directory: cljw zig_native.clj +;; +;; The wasm leg uses the committed kernel.wasm. The native leg needs the shared +;; library built from the same source (see kernel.zig) and is skipped without it. +(ns zig-native.demo + (:require [cljw.ffi :as ffi] + [clojure.java.io :as io])) + +(def blake3-abc "6437b3ac38465133ffb63b75273a8db548c558465d79db03fd359c6cd5bd9d85") +(def xs (mapv double (range 1 101))) +(def xs-dot (reduce + (map * xs xs))) ; 338350.0 + +;; --- wasm: sandboxed, the guest owns its buffers --------------------------- +(def w (wasm/load "kernel.wasm" {:fuel 10000000})) +(assert (= [2 3 5 7919 104729] (mapv #(.nth_prime w %) [0 1 2 999 9999]))) +(assert (= 8 (.popcount w 255))) +;; wasm i64 crosses exactly only within ±2^47 today (CLJW-WASM-I64-FULL-RANGE). +(assert (= (bit-shift-left 1 43) (.bitrev w (bit-shift-left 1 20)))) +(let [fa (.f64_addr w)] + (wasm/mem-write! w :f64 fa xs) + (assert (= xs-dot (.dot w fa fa (count xs))))) +(let [ba (.byte_addr w) + put! (fn [s] (wasm/mem-write! w :u8 ba (mapv int s)) (count s))] + (assert (= 5 (.opcode w ba (put! "pow")))) + (assert (= -1 (.opcode w ba (put! "mod")))) + (let [out (+ ba 128)] + (.blake3 w ba (put! "abc") out) + (assert (= blake3-abc (apply str (map char (wasm/mem-read w :u8 out 64))))))) +(println "PASS zig-wasm") + +;; --- native: cljw.ffi, full speed, no sandbox ------------------------------ +(def lib-file (first (filter #(.exists (io/file %)) ["libkernel.so" "libkernel.dylib"]))) + +(defn f64s->bytes + "Little-endian f64 bytes for a :bytes argument." + [ds] + (byte-array (mapcat (fn [d] (let [b (Double/doubleToRawLongBits (double d))] + (map #(unchecked-byte (bit-shift-right b (* 8 %))) (range 8)))) + ds))) + +(if lib-file + (let [lib (ffi/open (str "./" lib-file)) + libc (ffi/open (if (= lib-file "libkernel.dylib") "libSystem.dylib" "libc.so.6")) + malloc (ffi/function libc "malloc" [:long] :pointer) + free (ffi/function libc "free" [:pointer] :void) + nth-prime (ffi/function lib "nth_prime" [:int] :int) + opcode (ffi/function lib "opcode" [:string :long] :int) + dot (ffi/function lib "dot" [:bytes :bytes :long] :double) + popcount (ffi/function lib "popcount" [:long] :int) + bitrev (ffi/function lib "bitrev" [:long] :long) + blake3* (ffi/function lib "blake3" [:string :long :pointer] :void) + blake3 (fn [s] (let [out (malloc 65)] + (try (blake3* s (count s) out) (ffi/string out) + (finally (free out)))))] + (assert (= [2 3 5 7919 104729] (mapv nth-prime [0 1 2 999 9999]))) + (assert (= {"add" 1 "pow" 5 "mod" -1} + (into {} (map (juxt identity #(opcode % (count %)))) ["add" "pow" "mod"]))) + (let [b (f64s->bytes xs)] (assert (= xs-dot (dot b b (count xs))))) + (assert (= 8 (popcount 255))) + (assert (= Long/MIN_VALUE (bitrev 1))) ; the full 64 bits cross the C ABI + (assert (= blake3-abc (blake3 "abc"))) + (ffi/close lib) + (println "PASS zig-native")) + (println "SKIP zig-native (libkernel not built; see kernel.zig)")) + +(println "DONE") diff --git a/docs/works/collection_performance.md b/docs/works/collection_performance.md index be1e35d73..416ff9f28 100644 --- a/docs/works/collection_performance.md +++ b/docs/works/collection_performance.md @@ -1,19 +1,21 @@ -# Collection performance — cljw against JVM Clojure, measured +# Collection performance: cljw against JVM Clojure, measured The question this page answers came from a user: *for performance, should cljw reimplement the same immutable algorithms core Clojure does?* It is answered by measurement rather than by reading either implementation. The internal ledger of individual optimizations lives in `.dev/optimizations.md`; -this page is the user-facing view — what you can expect, and where cljw is still -slower. +this page is the user-facing view: what you can expect, where cljw is still +slower, and what to do about it. **Short version.** The persistent data structures are in the same league as the JVM's, trading wins op by op. The *interpreter* is far behind the JVM's JIT and -will stay behind until cljw compiles. Every catastrophic gap ever measured here +will stay behind until cljw compiles, which makes tight numeric loops 25 to 75 +times slower than warm JVM Clojure. Every catastrophic gap ever measured here turned out to be a specific algorithmic mistake in cljw, not a consequence of -Zig-vs-Java or of the GC; three of the four found so far are fixed, and one -remains. +Zig against Java or of the GC; all four found so far are fixed. For hot numeric +kernels, call compiled code: the same work in Zig through `cljw.ffi` runs about +150 times faster than in cljw. ## Method @@ -31,12 +33,12 @@ cljw .dev/bench/scaling.clj # cljw side, human-readable ``` Sub-microsecond results are reported best-of-7 with an empty-loop baseline -subtracted — without that, per-call noise and loop overhead swamp the signal and +subtracted. Without that, per-call noise and loop overhead swamp the signal and produce confident-looking nonsense in both directions. ## Two different questions, which are easy to conflate -### 1. Interpretation speed — the JVM is far ahead +### 1. Interpretation speed: the JVM is far ahead | per 20000 iterations | clj 1.12.4 | cljw | |---|---|---| @@ -46,12 +48,12 @@ An empty loop costs cljw ~45 ns per iteration against the JVM's ~2.2 ns. That is not Zig losing to Java. It is an **interpreter** (tree-walk / VM) losing to a **JIT** that has compiled the loop to native code with the induction variable in a register. No amount of collection work changes it; only a compiler would. This -is tracked as the runtime's own "gap area III" (fusion → a narrow ARM64 JIT). +is tracked as the runtime's own "gap area III" (fusion, then a narrow ARM64 JIT). Every number in the next section carries this overhead. Read them as *relative* structural cost, not as a language comparison. -### 2. The persistent structures themselves — same league, trading wins +### 2. The persistent structures themselves: same league, trading wins | per 20000 ops, n=16000 | clj 1.12.4 | cljw | faster | |---|---|---|---| @@ -61,42 +63,43 @@ structural cost, not as a language comparison. | map `dissoc` | **2.62 ms** | 8.15 ms | clj, 3.1× | | vector `conj` | **1.75 ms** | 7.62 ms | clj, 4.4× | -cljw is ahead on both `assoc`es — its bump-allocating GC makes the path-copy -cheaper — and behind on `conj` and `dissoc`. There are no 1000× effects in +cljw is ahead on both `assoc`es (its bump-allocating GC makes the path-copy +cheaper) and behind on `conj` and `dissoc`. There are no 1000× effects in either direction once the algorithms match. So: this is not "Zig beats the JVM". It is "the data structures are sound, and -the catastrophic numbers are bugs." +the catastrophic numbers were bugs." -## The catastrophic gaps were all algorithmic +## The catastrophic gaps were all algorithmic, and all are fixed -Four operations were once 1000×–7000× slower than JVM Clojure. Every one was a -specific mistake, of exactly two kinds. +Four operations were once 1000× to 12000× slower than JVM Clojure. Every one was +a specific mistake, of exactly two kinds. -**An abstraction-barrier violation** — the Clojure layer re-implementing an -operation the Zig layer already provided correctly: +**An abstraction-barrier violation**: the Clojure layer re-implementing an +operation the Zig layer already provided correctly. | op | was | now | vs clj | |---|---|---|---| -| `pop` (vector) | 8278 ms | **2.47 ms** | 1.93 ms — within 1.3× | -| `count` (list) | 481 ms | **0.19 ms** | 0.18 ms — parity | +| `pop` (vector) | 8278 ms | **2.47 ms** | 1.93 ms, within 1.3× | +| `count` (list) | 481 ms | **0.19 ms** | 0.18 ms, parity | `pop` was `(into [] (take (dec (count coll)) coll))` in `core.clj`, rebuilding a whole fresh trie, while `vector.zig` already had a faithful `PersistentVector.pop()` that nothing called. `count` on a list walked the chain -even though every cons cell already stored its length — the stored number was +even though every cons cell already stored its length: the stored number was distrusted because a cell can head a mixed chain where it means something else. The fix was to make the field say *when* it is exact, not to compute it again. -**Materialising where Clojure keeps a view** — the more systemic kind: +**Materialising where Clojure keeps a view**: the more systemic kind. | op | was | now | vs clj | |---|---|---|---| -| `seq` (vector) | 2123 ms | **0.36 ms** | 0.60 ms — cljw 1.7× faster | -| `rest` (vector) | 2253 ms | **0.39 ms** | 0.87 ms — cljw 2.2× faster | -| `next` (vector) | 2164 ms | **0.38 ms** | 0.99 ms — cljw 2.6× faster | +| `seq` (vector) | 2123 ms | **0.36 ms** | 0.60 ms, cljw 1.7× faster | +| `rest` (vector) | 2253 ms | **0.39 ms** | 0.87 ms, cljw 2.2× faster | +| `next` (vector) | 2164 ms | **0.38 ms** | 0.99 ms, cljw 2.6× faster | +| `subvec` (2000 ops) | 9215 ms | **0.77 ms** | 1.06 ms, cljw 1.4× faster | -`(seq v)` built an eager `PersistentList` copy — one allocation and one trie +`(seq v)` built an eager `PersistentList` copy: one allocation and one trie descent per element, per call. Clojure never copies: its seq over a vector is a **view** holding `(vector, index)`. cljw now does the same, so all three are O(1), and `nth` on such a seq is O(1) too (Clojure walks there). @@ -104,33 +107,80 @@ O(1), and `nth` on such a seq is O(1) too (Clojure walks there). This one mattered out of proportion to any single benchmark, because `seq` / `rest` / `next` are what every hand-written `loop`/`recur` walk and every library seq walk uses. It did not show up as a benchmark regression precisely -because the *idiomatic* paths — `reduce`, `doseq`, `into` — already bypassed +because the *idiomatic* paths (`reduce`, `doseq`, `into`) already bypassed `seq` with index-walk and chunk-drain fast paths. -## What is still slow +`subvec` was the last of the four. It rebuilt a whole trie per call where +Clojure returns a `SubVector` sharing the parent's root and tail. The fix +(O-059, 2026-08-22; its JVM reference was clj 1.11.1) is a new representation +with the same JVM semantics: a +32-byte `SubVector{start, end, parent}` whatever the span, flattened on nesting, +with `conj`/`assoc`/`pop` staying views. Its seq goes through the same view as a +vector's, so slicing and then walking is O(1) to start. -| op | clj 1.12.4 | cljw | | +## What is still slow: per-element work in tight loops + +With the structures sound, what remains is the interpreter's cost per element. +Measured 2026-10-22, cljw v1.15.3 against warm JVM Clojure 1.12.6 (2000 warmup +calls, best of 7), on a loaded host, so read the ratios rather than the +absolute numbers: + +| per call | clj 1.12.6 | cljw | cljw slower by | |---|---|---|---| -| **`subvec`** | **1.06 ms** | **9215 ms** | **~7500× slower** | +| `subvec` (constant time on both) | 0.08 µs | 1.0 µs | 13× | +| dot product of 1000 doubles, `(reduce + (map * xs xs))` | 85 µs | 2177 µs | 26× | +| the same as `loop`/`recur` with `nth` | 9.3 µs | 271 µs | 29× | +| `(byte-array v)` from an 8000-element vector | 40 µs | 3001 µs | 74× | + +Each element costs several boxed operations at roughly 170 ns each, against a +JIT that compiles the loop to machine code. Two practical consequences: + +- **Pick the loop shape.** In cljw a `loop`/`recur` over indices ran 8× faster + than `(reduce + (map * xs xs))` over the same vector, because it skips building a + lazy seq of boxed products. The JVM shows the same ordering (9×). +- **Do not build native data one element at a time.** Packing doubles into a + `byte-array` in Clojure costs more than any computation you then hand it to. -`subvec` eagerly copies where Clojure returns a `SubVector` sharing the parent's -root and tail. It is the last measured catastrophic gap, and the fix is a real -design decision rather than a missing call: cljw's heap-tag space is a fixed -64-slot census, so a new vector *representation* has to either spend one of the -reserved names or thread an offset through every existing vector operation. -Until it lands, prefer `(into [] (take n (drop k v)))`-style slicing only where -you would have paid for a copy anyway, and avoid `subvec` in a loop. +## Tight numeric loops: hand them to compiled code + +cljw's answer to the interpreter gap is not to out-run the JIT in Clojure. It +is to make calling compiled code cheap, from the same binary and REPL. The +same dot product and a prime search, measured the same day on v1.15.3: + +| work | cljw Clojure | Zig through `cljw.ffi` | +|---|---|---| +| dot product, 1000 doubles | 1.85 ms | **12.2 µs** (about 150×) | +| dot product, 100000 doubles | 357 ms | **1.2 to 5.5 ms**, mostly copying the arguments | +| the 1000th prime | 294 ms by trial division | **1 µs** from a table Zig computed at compile time | + +| one call | cost | +|---|---| +| empty Clojure fn | 167 ns | +| native function through `cljw.ffi` | 342 ns | +| wasm export, `:interp` engine | 550 ns | +| wasm export, `:jit` engine | 1.3 to 5.6 µs (grows with module size, being fixed) | + +The rule: cross the boundary with a whole batch (hash the batch, take the dot +of the vector), never once per element. Today the cost that remains is getting +data across: `:bytes` arguments are copied on every call. Primitive arrays +passed without the per-element packing, and native buffers that stay on the +native side between calls, are in progress. +[`../examples/zig-native/`](../examples/zig-native/README.md) has the runnable +version of every number in this section. ## What this means if you are choosing cljw - **Data-structure-heavy code is fine.** Persistent vectors, maps and sets perform comparably to the JVM's, sometimes better. -- **Tight numeric loops are not cljw's strength yet.** The interpreter overhead - dominates, and that is a compiler problem, not a library one. -- **Startup is where cljw wins outright** — milliseconds against the JVM's - ~1–2 s, which is the whole point for CLI tools, scripts and serverless. See +- **Tight numeric loops in Clojure are 25 to 75× slower than warm JVM + Clojure.** That is a compiler problem, not a library one. Until cljw + compiles, move the hot kernel to Zig, Rust or C through `cljw.ffi` (native, + full speed) or `wasm/load` (sandboxed), and keep the program logic in + Clojure. +- **Startup is where cljw wins outright**: milliseconds against the JVM's + ~1 to 2 s, which is the whole point for CLI tools, scripts and serverless. See [`binary_size.md`](./binary_size.md) for the size side of the same story. -- **If you hit something pathologically slow, it is probably a bug.** Every gap - on this page was, and each was found by running a probe rather than by reading - code. `.dev/bench/` has the harness; an issue with a reproducing form is - useful. +- **If you hit something pathologically slow, it is probably a bug.** Every + catastrophic gap on this page was, and each was found by running a probe + rather than by reading code. `.dev/bench/` has the harness; an issue with a + reproducing form is useful. diff --git a/scripts/check_changelog_reachability.sh b/scripts/check_changelog_reachability.sh index d55aab570..da1d8833b 100755 --- a/scripts/check_changelog_reachability.sh +++ b/scripts/check_changelog_reachability.sh @@ -6,9 +6,9 @@ # 1. A shipped release has no heading of its own. v1.14.9, v1.14.10 and # v1.14.11 appended their entries under `## [1.14.8]` and nothing noticed # (fixed by bba2e79f). Every `vX.Y.Z` tag reachable from HEAD, from v1.0.0 -# on, must have a `## [X.Y.Z] - YYYY-MM-DD` heading, and the newest tag's -# heading must be the first one after `## [Unreleased]`, or the second when -# the first is the untagged release build.zig.zon names. +# on, must have a `## [X.Y.Z] - YYYY-MM-DD` heading, and only untagged +# releases between the newest tag and the version build.zig.zon names may +# sit between `## [Unreleased]` and the newest tag's heading. # # 2. `[Unreleased]` describes something that has already shipped. An entry # that is also present in the newest tag's own `[Unreleased]` section went @@ -55,10 +55,10 @@ newest_tag="$(printf '%s\n' "$release_tags" | sed -n 1p)" # Version of each `## [X.Y.Z] - YYYY-MM-DD` heading, in file order. dated_headings="$(sed -nE 's/^## \[([0-9]+\.[0-9]+\.[0-9]+)\] - [0-9]{4}-[0-9]{2}-[0-9]{2}$/\1/p' "$CHANGELOG")" -# The first `## ` heading after `## [Unreleased]`, verbatim. -first_after_unreleased="$(awk ' +# Every `## ` heading after `## [Unreleased]`, verbatim, in file order. +headings_after_unreleased="$(awk ' /^## \[Unreleased\]/ { seen = 1; next } - seen && /^## / { print; exit } + seen && /^## / { print } ' "$CHANGELOG")" if [ -z "$newest_tag" ]; then @@ -82,31 +82,35 @@ else newest_v="${newest_tag#v}" - # A hand-stamped MINOR/MAJOR heading precedes its tag: allowed only while - # build.zig.zon names that version, it is newer, and no such tag exists. + # A heading above the newest tag's names a release this clone has no tag + # for: the pending one build.zig.zon names (stamped before the release + # workflow tags it, which on a mirror races the CI run on the same merge), + # or one a mirror never tagged (clojurewasm/ClojureWasm has no v1.15.2, its + # release job failed first). Each must be dated, newer than the newest tag, + # no newer than build.zig.zon, and untagged. pending_v="$(sed -nE 's/^[[:space:]]*\.version = "([0-9]+\.[0-9]+\.[0-9]+)",[[:space:]]*$/\1/p' build.zig.zon 2>/dev/null | sed -n 1p)" - if [ -n "$pending_v" ] \ - && ! git rev-parse -q --verify "refs/tags/v$pending_v" >/dev/null \ - && [ "$(printf '%s\n%s\n' "$newest_v" "$pending_v" | sort -V | sed -n 2p)" = "$pending_v" ] \ - && [ "$pending_v" != "$newest_v" ]; then - case "$first_after_unreleased" in - "## [$pending_v] - "*) - first_after_unreleased="$(awk ' - /^## \[Unreleased\]/ { seen = 1; next } - seen && /^## / { n++; if (n == 2) { print; exit } } - ' "$CHANGELOG")" - echo "check_changelog_reachability: ok, [$pending_v] is the pending release build.zig.zon names (no v$pending_v tag yet)" - ;; + found_newest=0 + while IFS= read -r heading; do + [ -n "$heading" ] || continue + case "$heading" in + "## [$newest_v] - "*) found_newest=1; break ;; esac + hv="$(sed -nE 's/^## \[([0-9]+\.[0-9]+\.[0-9]+)\] - [0-9]{4}-[0-9]{2}-[0-9]{2}$/\1/p' <<<"$heading")" + if [ -n "$hv" ] && [ -n "$pending_v" ] \ + && ! git rev-parse -q --verify "refs/tags/v$hv" >/dev/null \ + && [ "$(printf '%s\n%s\n' "$newest_v" "$hv" | sort -V | sed -n 2p)" = "$hv" ] \ + && [ "$(printf '%s\n%s\n' "$hv" "$pending_v" | sort -V | sed -n 2p)" = "$pending_v" ]; then + echo "check_changelog_reachability: ok, [$hv] is an untagged release between $newest_tag and build.zig.zon's $pending_v" + continue + fi + violation "'$heading' sits above $newest_tag's dated heading but is not an untagged release between $newest_tag and build.zig.zon's ${pending_v:-}." + echo " The newest release goes directly under [Unreleased]." >&2 + found_newest=1 + break + done <<< "$headings_after_unreleased" + if [ "$found_newest" -eq 0 ]; then + violation "no '## [$newest_v] - YYYY-MM-DD' heading after [Unreleased]." fi - - case "$first_after_unreleased" in - "## [$newest_v] - "*) ;; - *) - violation "the first heading after [Unreleased] is '${first_after_unreleased:-}', not $newest_tag's dated heading." - echo " The newest release goes directly under [Unreleased]." >&2 - ;; - esac fi # ---- 2. nothing under [Unreleased] has already shipped ---------------------- diff --git a/scripts/check_e2e_count.sh b/scripts/check_e2e_count.sh index bfcd955c2..35bd7433f 100755 --- a/scripts/check_e2e_count.sh +++ b/scripts/check_e2e_count.sh @@ -23,7 +23,7 @@ # bash 3.2 compatible (the macOS host runs the system bash). set -uo pipefail -BASELINE=322 +BASELINE=263 SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" E2E_DIR="${E2E_DIR:-$SCRIPT_DIR/../test/e2e}" diff --git a/src/eval/analyzer/analyzer.zig b/src/eval/analyzer/analyzer.zig index 5723828b2..a7850bc44 100644 --- a/src/eval/analyzer/analyzer.zig +++ b/src/eval/analyzer/analyzer.zig @@ -64,6 +64,7 @@ const map_collection = @import("../../runtime/collection/map.zig"); const set_collection = @import("../../runtime/collection/set.zig"); const literal = @import("../../runtime/collection/literal.zig"); const big_int = @import("../../runtime/numeric/big_int.zig"); +const decimal_text = @import("../../runtime/numeric/decimal_text.zig"); const promote = @import("../../runtime/numeric/promote.zig"); const big_decimal = @import("../../runtime/numeric/big_decimal.zig"); const ratio_mod = @import("../../runtime/numeric/ratio.zig"); @@ -435,9 +436,13 @@ fn staticFieldValue(rt: *Runtime, sf: *const @import("../../runtime/type_descrip .time_local_date_epoch => @import("../../runtime/time/local_date_value.zig").make(rt, 0) catch |e| return coerceConstErr(e), .time_local_date_time_min => @import("../../runtime/time/local_date_time_value.zig").make(rt, MIN_EPOCH_DAY, 0) catch |e| return coerceConstErr(e), .time_local_date_time_max => @import("../../runtime/time/local_date_time_value.zig").make(rt, MAX_EPOCH_DAY, 86_399_999_999_999) catch |e| return coerceConstErr(e), + .time_zone_offset_utc => @import("../../runtime/time/zone_offset.zig").make(rt, .{ .seconds = 0 }) catch |e| return coerceConstErr(e), + .time_zone_offset_min => @import("../../runtime/time/zone_offset.zig").make(rt, .{ .seconds = -18 * 3600 }) catch |e| return coerceConstErr(e), + .time_zone_offset_max => @import("../../runtime/time/zone_offset.zig").make(rt, .{ .seconds = 18 * 3600 }) catch |e| return coerceConstErr(e), }, .host_enum => |he| try @import("../../runtime/host_enum.zig").singleton(rt, @enumFromInt(he.enum_idx), he.ordinal), .math_context => |which| try @import("../../runtime/math_context.zig").singleton(rt, which), + .date_time_formatter => |idx| @import("../../runtime/time/date_time_formatter_value.zig").isoConstant(rt, idx) catch |e| return coerceConstErr(e), // BigDecimal ZERO/ONE/TWO/TEN (ADR-0174 D7): a scale-0 BigDecimal // lifted via the constructor at analyze time. .big_decimal => |i| @import("../../runtime/numeric/big_decimal.zig").allocFromI64Scale(rt, i, 0) catch |e| return coerceConstErr(e), @@ -1430,7 +1435,18 @@ const try_form = @import("try_form.zig"); /// the first error in the text is the one raised. Every other lift (quote, /// macro arguments, metadata) is `.lenient`: code was already checked as it /// was read. -const LiftChecks = enum { lenient, strict }; +/// `.edn` is `.strict` for `clojure.edn` reads (ADR-0200 Layer B): a +/// `#ns.Record{...}` tag is NOT a record constructor there, clj's EdnReader +/// resolves every tag through `:readers` / `:default` only. +const LiftChecks = enum { + lenient, + strict, + edn, + + fn checksDuplicates(self: LiftChecks) bool { + return self != .lenient; + } +}; /// Form atom → Value lift. Handles nil / bool / int / float / char / /// string / keyword / symbol and the collection literals, interning @@ -1447,6 +1463,13 @@ pub fn formToValueStrict(rt: *Runtime, env: *Env, form: Form) AnalyzeError!Value return lift(rt, env, form, .strict); } +/// `formToValueStrict` for a `clojure.edn` read: additionally, a +/// `#ns.Name{...}` tag is resolved only through `*data-readers*` / +/// `*default-data-reader-fn*`, never as a record literal (clj EdnReader). +pub fn formToValueEdn(rt: *Runtime, env: *Env, form: Form) AnalyzeError!Value { + return lift(rt, env, form, .edn); +} + /// D-186: honour a reader `^meta` map on a literal. The reader (readMeta) /// parks normalised meta on `Form.meta`; lift it and attach it as clj's /// MetaReader does (`meta_mod.readerAttachOrNull`: reset on a reference, @@ -1559,7 +1582,7 @@ fn liftTagged(rt: *Runtime, env: *Env, t: TaggedForm, loc: SourceLocation, check // by simple name and then re-check `defining_ns`, which silently // rejected a legitimate `#aaa.Point{…}` as soon as some other namespace // defined a record also called `Point` and displaced the entry. - if (t.tag.ns == null) { + if (t.tag.ns == null and checks != .edn) { if (std.mem.findScalar(u8, t.tag.name, '.') != null) { if (rt.types.get(t.tag.name)) |td| { const map_arg = inner.tag() == .array_map or inner.tag() == .hash_map; @@ -1656,7 +1679,7 @@ fn mapFormToValue(rt: *Runtime, env: *Env, entries: []const Form, loc: SourceLoc else => |e| return e, }; roots[0] = out; - if (checks == .strict and duplicate == null) { + if (checks.checksDuplicates() and duplicate == null) { if (map_collection.count(out) == before) { duplicate = entries[i]; } else if (repeatedReaderConstantNan(entries[0..i], entries[i], 2)) { @@ -1691,7 +1714,7 @@ fn setFormToValue(rt: *Runtime, env: *Env, items: []const Form, checks: LiftChec else => |e| return e, }; roots[0] = out; - if (checks == .strict and duplicate == null) { + if (checks.checksDuplicates() and duplicate == null) { if (set_collection.count(out) == before) { duplicate = item; } else if (repeatedReaderConstantNan(items[0..idx], item, 1)) { @@ -1738,7 +1761,7 @@ fn listFormToValue(rt: *Runtime, env: *Env, items: []const Form, checks: LiftChe // The chain is consed from the end. A strict (data) lift must still // LIFT left to right, so the first error in the text is the one raised, // as clj reads it: it lifts into a vector first. - const lifted: ?Value = if (checks == .strict) try vectorFormToValue(rt, env, items, checks) else null; + const lifted: ?Value = if (checks.checksDuplicates()) try vectorFormToValue(rt, env, items, checks) else null; if (lifted) |v| roots[2] = v; while (i > 0) { i -= 1; @@ -1800,7 +1823,7 @@ pub fn valueToForm( .ratio => blk: { const lit = switch (ratio_mod.parts(v)) { .small => |s| try std.fmt.allocPrint(arena, "{d}/{d}", .{ s.n, s.d }), - .big => |b| try std.fmt.allocPrint(arena, "{f}/{f}", .{ b.n.m, b.d.m }), + .big => |b| try decimal_text.fractionAlloc(arena, b.n.m, b.d.m), }; break :blk .{ .data = .{ .ratio_literal = lit }, .location = call_loc }; }, @@ -1810,9 +1833,9 @@ pub fn valueToForm( // back as a BigInt literal and break downstream i64-only ops // (unsigned-bit-shift-right et al.). .big_int => if (big_int.originOf(v) == .long) blk: { - const n = promote.exactI64(v) catch break :blk Form{ .data = .{ .big_int_literal = try std.fmt.allocPrint(arena, "{f}", .{big_int.asManaged(v)}) }, .location = call_loc }; + const n = promote.exactI64(v) catch break :blk Form{ .data = .{ .big_int_literal = try decimal_text.decimalAlloc(arena, big_int.asManaged(v)) }, .location = call_loc }; break :blk Form{ .data = .{ .integer = n }, .location = call_loc }; - } else .{ .data = .{ .big_int_literal = try std.fmt.allocPrint(arena, "{f}", .{big_int.asManaged(v)}) }, .location = call_loc }, + } else .{ .data = .{ .big_int_literal = try decimal_text.decimalAlloc(arena, big_int.asManaged(v)) }, .location = call_loc }, .big_decimal => blk: { var aw: std.Io.Writer.Allocating = .init(arena); print_mod.writeBigDecimalDigits(&aw.writer, v) catch return error.OutOfMemory; diff --git a/src/eval/backend/tree_walk.zig b/src/eval/backend/tree_walk.zig index 3430533c3..02ed683a0 100644 --- a/src/eval/backend/tree_walk.zig +++ b/src/eval/backend/tree_walk.zig @@ -74,6 +74,63 @@ const gc_heap_mod = @import("../../runtime/gc/gc_heap.zig"); /// frame-size known at analyse time. pub const MAX_LOCALS: u16 = 256; +/// A stack scratch array the caller writes before it reads. Plain `undefined` +/// costs a 0xAA memset in ReleaseSafe, not only in Debug: for a +/// `[MAX_LOCALS]Value` window that is 2 KB written on EVERY fn call, measured +/// at 19% of all instructions in `(reduce + (map * xs xs))`. Each user reads +/// only a prefix it has just written (a frame's `locals[0..fs]`, a published +/// args prefix), so the poison catches nothing there. Debug keeps it: safety +/// is switched off outside Debug and only inside this fn, so the caller's own +/// checks are untouched. +// PERF: skip the safe-mode 0xAA fill of the per-call frame/arg windows [refs: O-064] +pub fn uninitScratch(comptime T: type) T { + @setRuntimeSafety(@import("builtin").mode == .Debug); + return undefined; +} + +/// Per-thread scratch for the per-call windows that used to be native-stack +/// arrays: a fn activation's locals window (`callMethodImpl`) and a call's +/// evaluated args + their source locs (`evalCall` & co.). Each borrower takes +/// a window at the watermark and restores it on return (normal or error), the +/// same discipline as vm.zig's `VmArena`. Off the native stack because these +/// frames are paid once per native `eval` re-entry, and a HOF recursing +/// through Zig (chunked `map`, `reduce`) re-enters per level: the budget +/// ADR-0195 meters is native bytes, so a 2 KB window per frame was a 2 KB +/// slice of every level of user recursion. Threadlocal static (tbss): nothing +/// to allocate or free. +const TwScratch = struct { + vals: [TW_SCRATCH_VALS]Value = undefined, + val_top: u32 = 0, + locs: [TW_SCRATCH_LOCS]SourceLocation = undefined, + loc_top: u32 = 0, +}; +const TW_SCRATCH_VALS: usize = 32 * 1024; +const TW_SCRATCH_LOCS: usize = 8 * 1024; +threadlocal var tw_scratch: TwScratch = .{}; + +/// Static TLS this file adds; vm.zig asserts it together with `VmArena` +/// against the ADR-0195 reserve. +pub const TLS_BYTES: usize = @sizeOf(TwScratch); + +/// Borrow `n` Values at the scratch watermark; the caller restores +/// `tw_scratch.val_top` to the returned base on exit. Exhaustion is deep +/// recursion: a catchable StackOverflowError, like the native guard. +fn borrowVals(n: usize, loc: SourceLocation) ![]Value { + const base = tw_scratch.val_top; + if (@as(usize, base) + n > TW_SCRATCH_VALS) + return error_catalog.raise(.stack_overflow, loc, .{ .max = TW_SCRATCH_VALS }); + tw_scratch.val_top = base + @as(u32, @intCast(n)); + return tw_scratch.vals[base..][0..n]; +} + +fn borrowLocs(n: usize, loc: SourceLocation) ![]SourceLocation { + const base = tw_scratch.loc_top; + if (@as(usize, base) + n > TW_SCRATCH_LOCS) + return error_catalog.raise(.stack_overflow, loc, .{ .max = TW_SCRATCH_LOCS }); + tw_scratch.loc_top = base + @as(u32, @intCast(n)); + return tw_scratch.locs[base..][0..n]; +} + /// TreeWalk error surface. Aliases `error_mod.ClojureWasmError` so /// calls to `error_catalog.raise(.code, loc, args)` type-check; the /// backend still only **emits** `error.TypeError` (non-callable @@ -380,6 +437,8 @@ const deserialized_fn_body: Node = .{ .constant = .{ .value = .nil_val } }; pub const SerializedMethod = struct { arity: u16, has_rest: bool, + /// The method's exact frame high-water (ADR-0130); 0 = full MAX_LOCALS. + frame_slots: u16 = 0, bytecode: ?*const BytecodeChunk, }; @@ -404,6 +463,7 @@ pub fn allocFunctionFromSerialized( .params = &.{}, .body = &deserialized_fn_body, .bytecode = m.bytecode, + .frame_slots = m.frame_slots, }; } var variadic_method: ?FunctionMethod = null; @@ -413,6 +473,7 @@ pub fn allocFunctionFromSerialized( .params = &.{}, .body = &deserialized_fn_body, .bytecode = vm.bytecode, + .frame_slots = vm.frame_slots, }; cell.f.* = .{ @@ -711,12 +772,14 @@ fn evalConstructorCall(rt: *Runtime, env: *Env, locals: []Value, n: node_mod.Int // (special_forms.constructInstance) so TreeWalk + VM construct // identically — incl. the java-surface `` path `(java.io.File. // "path")` rides (ADR-0036 dual-backend parity; D-196 blocker 3). - var buf: [MAX_LOCALS]Value = undefined; if (n.args.len > MAX_LOCALS) return error_catalog.raise(.call_args_exceed_max_locals, n.loc, .{ .got = n.args.len, .max = MAX_LOCALS }); + const scratch_base = tw_scratch.val_top; + defer tw_scratch.val_top = scratch_base; + const buf = try borrowVals(n.args.len, n.loc); // GC-ROOT: A6b — progressive arg publication (see ArgsFrame). var af: ArgsFrame = undefined; - af.push(&buf); + af.push(buf.ptr); defer af.pop(); for (n.args, 0..) |arg_node, i| { buf[i] = try eval(rt, env, locals, &arg_node); @@ -1036,9 +1099,11 @@ fn evalRecur(rt: *Runtime, env: *Env, locals: []Value, n: node_mod.RecurNode) an // // GC-ROOT: A6b — already-evaluated recur values are reachable only // through this buffer while a LATER arg's eval allocates (D-556). - var local_buf: [MAX_LOCALS]Value = undefined; + const scratch_base = tw_scratch.val_top; + defer tw_scratch.val_top = scratch_base; + const local_buf = try borrowVals(n.args.len, n.loc); var af: ArgsFrame = undefined; - af.push(&local_buf); + af.push(local_buf.ptr); defer af.pop(); for (n.args, 0..) |*a, i| { local_buf[i] = try eval(rt, env, locals, a); @@ -1176,15 +1241,21 @@ const ArgsFrame = struct { fn evalCall(rt: *Runtime, env: *Env, locals: []Value, n: node_mod.CallNode) !Value { const callee = try eval(rt, env, locals, n.callee); // Slot 0 carries the callee; args fill 1..; the frame publishes each. - var args_buf: [MAX_LOCALS + 1]Value = undefined; if (n.args.len > MAX_LOCALS) return error_catalog.raise(.call_args_exceed_max_locals, n.loc, .{ .got = n.args.len, .max = MAX_LOCALS }); + const val_base = tw_scratch.val_top; + const loc_base = tw_scratch.loc_top; + defer { + tw_scratch.val_top = val_base; + tw_scratch.loc_top = loc_base; + } + const args_buf = try borrowVals(n.args.len + 1, n.loc); + const arg_locs = try borrowLocs(n.args.len, n.loc); args_buf[0] = callee; var af: ArgsFrame = undefined; - af.push(&args_buf); + af.push(args_buf.ptr); defer af.pop(); af.publish(1); - var arg_locs: [MAX_LOCALS]SourceLocation = undefined; for (n.args, 0..) |*a, i| { args_buf[1 + i] = try eval(rt, env, locals, a); af.publish(2 + i); @@ -1303,19 +1374,24 @@ pub fn treeWalkCall( // clojure.lang.IFn `-invoke` is callable as `(inst args…)`. `-invoke` // receives (this, …args). Falls through to value_not_callable when the // instance does not implement IFn. Shared treeWalkCall = both backends. - .typed_instance, .reified_instance => blk: { - var cs: dispatch.CallSite = .{}; - const inv_args = try rt.gpa.alloc(Value, args.len + 1); - defer rt.gpa.free(inv_args); - inv_args[0] = callee; - @memcpy(inv_args[1..], args); - if (try dispatch.dispatchOrNull(rt, env, &cs, callee, "IFn", "-invoke", inv_args, loc)) |v| break :blk v; - break :blk error_catalog.raise(.value_not_callable, loc, .{ .actual = @tagName(callee.tag()) }); - }, + .typed_instance, .reified_instance => callIFnInstance(rt, env, callee, args, loc), else => |t| error_catalog.raise(.value_not_callable, loc, .{ .actual = @tagName(t) }), }; } +/// `(inst args…)` on an IFn deftype/reify. Out of line (`noinline`) so its +/// CallSite + arg copy are not part of `treeWalkCall`'s frame, which every +/// call of every backend pays (ADR-0195 native budget). +noinline fn callIFnInstance(rt: *Runtime, env: *Env, callee: Value, args: []const Value, loc: SourceLocation) anyerror!Value { + var cs: dispatch.CallSite = .{}; + const inv_args = try rt.gpa.alloc(Value, args.len + 1); + defer rt.gpa.free(inv_args); + inv_args[0] = callee; + @memcpy(inv_args[1..], args); + if (try dispatch.dispatchOrNull(rt, env, &cs, callee, "IFn", "-invoke", inv_args, loc)) |v| return v; + return error_catalog.raise(.value_not_callable, loc, .{ .actual = @tagName(callee.tag()) }); +} + /// Dispatch `(m receiver args...)` where `m` is a `.protocol_fn` /// Value. The receiver is `args[0]` (cw v1 mirrors JVM Clojure's /// "first arg is the dispatch target" convention); the impl @@ -1327,22 +1403,21 @@ pub fn treeWalkCall( /// `op_method_call`; this protocol-fn path keeps a per-call CallSite, /// so every invocation pays one method_table linear scan — small N, /// Clojure protocols typically ≤ 8 methods.) -pub fn callProtocolFn(rt: *Runtime, env: *Env, callee: Value, args: []const Value, loc: SourceLocation) anyerror!Value { +pub noinline fn callProtocolFn(rt: *Runtime, env: *Env, callee: Value, args: []const Value, loc: SourceLocation) anyerror!Value { if (args.len < 1) return error_catalog.raise(.arity_below_min, loc, .{ .fn_name = "protocol-fn", .got = args.len, .min = 1 }); const pfn = protocol_mod.asProtocolFn(callee); const receiver = args[0]; // D-314 / ADR-0144: `:extend-via-metadata true` protocols dispatch via the // receiver's metadata (per-VALUE) ahead of the per-type impl. It MUST run - // before `dispatch.dispatch` (which owns the per-TYPE CallSite cache), so a + // before `dispatch.dispatchCached` (which owns the per-TYPE ProtocolFn cache), so a // meta hit never poisons the type cache. Gated on the flag → other protocols // pay nothing. Both backends share this entry (VM wires callFn=treeWalkCall). if (try meta_mod.metaDispatch(rt, env, pfn.descriptor, receiver, pfn.methodName(), args, loc)) |v| return v; - var cs: method_table.CallSite = .{}; - return dispatch.dispatch( + return dispatch.dispatchCached( rt, env, - &cs, + pfn.cache(), receiver, pfn.descriptor.fqcn(), pfn.methodName(), @@ -1387,28 +1462,34 @@ pub fn callFunctionBindingRest(rt: *Runtime, env: *Env, fn_val: Value, args: []c /// (the §9.2.S perf-parity work flattening `op_call`) reuses this exact /// binder rather than re-deriving the closure/arg/rest layout, so the two /// activation paths cannot drift. +/// The exact slot window a call of method `m` binds and runs in: `fs`. +// PERF: init + GC-root only the slots this method actually uses, not all 256 +// — cuts the ~2 KB per-call nil-init on the hottest path [refs: O-015] +// ADR-0130 frame-rooting: m.frame_slots is the analyzer's exact per-method +// high-water (≥ arg_region by construction; the @max defends a runtime/compile +// slot_base skew). Sentinel 0 (a no-arg/no-local/no-capture body, or a +// non-analyzer FnMethod) falls back to the full MAX_LOCALS. A too-low fs is a +// GC UAF / slot_out_of_range (the O-005 failure modes) — locked by the +// CLJW_GC_TORTURE `nested_deep` e2e. +pub fn frameSlots(f: *const Function, m: *const FunctionMethod, loc: SourceLocation) !usize { + const arg_region: usize = @as(usize, f.slot_base) + m.arity + @intFromBool(m.has_rest); + const fs: usize = if (m.frame_slots == 0) MAX_LOCALS else @max(@as(usize, m.frame_slots), arg_region); + if (fs > MAX_LOCALS) + return error_catalog.raise(.fn_frame_exceeds_max_locals, loc, .{ .base = f.slot_base, .arity = m.arity, .max = MAX_LOCALS }); + return fs; +} + pub fn bindCallFrame( rt: *Runtime, f: *const Function, m: *const FunctionMethod, args: []const Value, - locals: *[MAX_LOCALS]Value, + locals: []Value, rest_mode: RestMode, loc: SourceLocation, ) !usize { - const arg_region: usize = @as(usize, f.slot_base) + m.arity + @intFromBool(m.has_rest); - // PERF: init + GC-root only the slots this method actually uses, not all 256 - // — cuts the ~2 KB per-call nil-init on the hottest path [refs: O-015] - // ADR-0130 frame-rooting: m.frame_slots is the analyzer's exact per-method - // high-water (≥ arg_region by construction; the @max defends a runtime/compile - // slot_base skew). Sentinel 0 (a no-arg/no-local/no-capture body, or a - // non-analyzer FnMethod) falls back to the full MAX_LOCALS. A too-low fs is a - // GC UAF / slot_out_of_range (the O-005 failure modes) — locked by the - // CLJW_GC_TORTURE `nested_deep` e2e. - const fs: usize = if (m.frame_slots == 0) MAX_LOCALS else @max(@as(usize, m.frame_slots), arg_region); - if (fs > MAX_LOCALS) - return error_catalog.raise(.fn_frame_exceeds_max_locals, loc, .{ .base = f.slot_base, .arity = m.arity, .max = MAX_LOCALS }); - + const fs = try frameSlots(f, m, loc); + std.debug.assert(locals.len >= fs); @memset(locals[0..fs], .nil_val); if (f.closure_bindings) |snap| { @memcpy(locals[0..snap.len], snap); @@ -1439,6 +1520,14 @@ pub fn bindCallFrame( } fn callMethodImpl(rt: *Runtime, env: *Env, fn_val: Value, args: []const Value, loc: SourceLocation, rest_mode: RestMode) !Value { + // PERF: no per-call locals buffer on the native stack: callMethodBody + // borrows an exact-size window from `tw_scratch`, which neither + // 0xaa-fills in ReleaseSafe (O-080's original goal) nor costs native stack + // per re-entry (PR #80 recursion depth) [refs: O-064, O-080] + return callMethodBody(rt, env, fn_val, args, loc, rest_mode); +} + +fn callMethodBody(rt: *Runtime, env: *Env, fn_val: Value, args: []const Value, loc: SourceLocation, rest_mode: RestMode) !Value { const f = fn_val.decodePtr(*Function); // Row 7.8 cycle 1 (ADR-0041): linear scan over `methods`, fixed- // arity wins on exact match; fall through to `variadic` when @@ -1462,7 +1551,11 @@ fn callMethodImpl(rt: *Runtime, env: *Env, fn_val: Value, args: []const Value, l // a raw `*Function` (ADR-0184 A, DA gap 1) — the callee slot must // already be live then; the locals window starts empty and is widened // to the bound slots after the binder returns. [ref: .dev/gc_rooting.md §A] - var locals: [MAX_LOCALS]Value = undefined; + // The window comes from `tw_scratch`, not the native stack, sized to the + // method's exact frame (`frameSlots`, the binder's own computation). + const scratch_base = tw_scratch.val_top; + defer tw_scratch.val_top = scratch_base; + const locals = try borrowVals(try frameSlots(f, m, loc), loc); var gc_stack: [1]Value = .{.nil_val}; var gc_sp: u16 = 0; var gc_frame: root_set.EvalFrame = .{ @@ -1475,7 +1568,7 @@ fn callMethodImpl(rt: *Runtime, env: *Env, fn_val: Value, args: []const Value, l root_set.eval_frame_head = &gc_frame; defer root_set.eval_frame_head = gc_frame.parent; - const fs = try bindCallFrame(rt, f, m, args, &locals, rest_mode, loc); + const fs = try bindCallFrame(rt, f, m, args, locals, rest_mode, loc); gc_frame.locals = locals[0..fs]; // VM backend hook: when this method carries compiled bytecode and // the vtable has the `evalChunk` slot wired (vm.installVTable), run @@ -1535,7 +1628,9 @@ pub fn selectMethod(f: *const Function, n: usize) ?*const FunctionMethod { return null; } -fn raiseArityNotMatched(f: *const Function, got: usize, loc: SourceLocation) anyerror { +// `noinline`: its 256-byte message buffer must not land in callMethodImpl's +// frame, which every native eval re-entry pays (ADR-0195 budget). +noinline fn raiseArityNotMatched(f: *const Function, got: usize, loc: SourceLocation) anyerror { // For single-arity fixed fns, preserve the existing // arity_not_expected / arity_below_min diagnostics so existing // user-visible error text stays stable. diff --git a/src/eval/backend/vm.zig b/src/eval/backend/vm.zig index 0199de64f..1bb5d6675 100644 --- a/src/eval/backend/vm.zig +++ b/src/eval/backend/vm.zig @@ -89,6 +89,9 @@ const ARENA_SLOTS: usize = 1 << 14; // 16384; keeps `op_top` within u16 /// `VmFrame`; exceeding this → catchable `StackOverflow` (matches clj). The /// operand arena (~8 slots/fib-frame) caps comparable depth, so the two align. const FRAMES_MAX: usize = 2048; +/// Handler-arena capacity: the live handlers of every nested `eval` on the +/// thread together. A window that cannot open a slot raises stack_overflow. +const HANDLER_ARENA_SLOTS: usize = 8192; /// The thread's reusable VM arenas: the operand stack + its parallel loc stack + /// the flattened-call-frame locals arena + the in-VM frame stack, as inline /// arrays so the storage is threadlocal STATIC (BSS, demand-paged) — nothing to @@ -105,6 +108,13 @@ const VmArena = struct { local_top: u16 = 0, frames: [FRAMES_MAX]VmFrame = undefined, frame_top: u16 = 0, + /// try/catch + cleanup handlers. Each `eval` borrows a window of up to + /// `HANDLER_STACK_MAX` starting at `handler_top`, publishing + /// `handler_top = its base + its live count` before every step so a nested + /// eval borrows above the live ones. Off the native stack: the window was a + /// 768-byte `eval` local paid by every re-entry (ADR-0195 budget). + handlers: [HANDLER_ARENA_SLOTS]Handler = undefined, + handler_top: u16 = 0, }; threadlocal var vm_arena: VmArena = .{}; @@ -121,9 +131,10 @@ threadlocal var stack_base: usize = 0; const STACK_BUDGET_BYTES = runtime_thread.STACK_BUDGET_BYTES; comptime { - // The threadlocal arena is static TLS, carved out of the runtime stack - // under glibc pthreads; it must fit the reserve the margin accounts for. - runtime_thread.assertTlsFits(@sizeOf(VmArena)); + // The threadlocal arenas (this VmArena + tree_walk's per-call scratch) are + // static TLS, carved out of the runtime stack under glibc pthreads; together + // they must fit the reserve the margin accounts for. + runtime_thread.assertTlsFits(@sizeOf(VmArena) + tree_walk.TLS_BYTES); } /// An in-VM call frame. A flattened `op_call` pushes one + continues @@ -242,6 +253,7 @@ pub fn eval( const op_base = ar.op_top; const local_base_entry = ar.local_top; const frame_base = ar.frame_top; + const handler_base = ar.handler_top; const head_entry = root_set.eval_frame_head; // Publish this eval's env for the alloc-driven GC torture mode: a collect // forced from inside `gc.alloc` has no `env`, so it reads this threadlocal. @@ -258,13 +270,14 @@ pub fn eval( ar.frame_top = frame_base; ar.op_top = op_base; ar.local_top = local_base_entry; + ar.handler_top = handler_base; root_set.eval_frame_head = head_entry; root_set.active_env = active_env_entry; } if (ar.frame_top >= FRAMES_MAX) return error_catalog.raise(.stack_overflow, .{}, .{ .max = FRAMES_MAX }); - var handlers: [HANDLER_STACK_MAX]Handler = undefined; + const handlers: []Handler = ar.handlers[handler_base..@min(@as(usize, handler_base) + HANDLER_STACK_MAX, HANDLER_ARENA_SLOTS)]; var handler_count: u16 = 0; // The base frame: caller-owned locals + the base chunk. Its gc_frame roots @@ -348,8 +361,10 @@ pub fn eval( // with the alloc-boundary site via the shared `bytes_since_last_gc` reset; // one predicted-not-taken compare per op when the threshold is not yet crossed. rt.gc.maybeAutoCollect(); + // Publish this eval's live handlers so a nested eval borrows above them. + ar.handler_top = handler_base + handler_count; var flatten_req: ?FlattenReq = null; - const step_result = stepOnce(rt, env, cur.locals, cur.chunk, &ar.stack, &ar.loc, &ar.op_top, &ip, &handlers, &handler_count, &flatten_req); + const step_result = stepOnce(true, rt, env, cur.locals, cur.chunk, &ar.stack, &ar.loc, &ar.op_top, &ip, handlers, &handler_count, &flatten_req); // A flattenPush FRAMES_MAX overflow must reach the shared error arm // below (`else |err|` — unwind + synthesise the catalog error into a // catchable thrown value + jump to the handler) so THIS eval's own @@ -473,6 +488,11 @@ pub fn eval( // fib_recursive 40→33 ms, tak 15→13. The // naive form is a plain `fn`; behaviour is identical (diff oracle). [refs: O-017] inline fn stepOnce( + /// `true` for the copy inlined into `eval`'s loop: the cold opcodes + /// (`isColdOp`) leave it for the out-of-line `stepCold` copy, so their + /// scratch (fmt buffers, the make-fn stackFallback, literal builders) is + /// not part of the frame every `eval` re-entry pays (ADR-0195 budget). + comptime hot: bool, rt: *Runtime, env: *Env, locals: []Value, @@ -481,13 +501,20 @@ inline fn stepOnce( loc_stack: []SourceLocation, sp_ptr: *u16, ip_ptr: *usize, - handlers: *[HANDLER_STACK_MAX]Handler, + handlers: []Handler, handler_count_ptr: *u16, /// When `op_call` resolves a monomorphic bytecode callee it sets /// this (and returns `null`/cont, WITHOUT popping the callee+args) so the eval /// loop flattens it into an in-VM frame instead of a host `vt.callFn` re-entry. flatten_out: *?FlattenReq, ) anyerror!?Value { + if (hot) { + const at = ip_ptr.*; + if (at < chunk.instructions.len and isColdOp(chunk.instructions[at].op())) { + @branchHint(.unlikely); + return stepCold(rt, env, locals, chunk, stack, loc_stack, sp_ptr, ip_ptr, handlers, handler_count_ptr, flatten_out); + } + } var sp = sp_ptr.*; var ip = ip_ptr.*; var handler_count = handler_count_ptr.*; @@ -548,7 +575,7 @@ inline fn stepOnce( const count: u16 = instr.operand >> 8; tree_walk.patchLetfnClosures(locals, base, count); }, - .op_def, .op_def_unbound => { + .op_def, .op_def_unbound => if (hot) unreachable else { const var_idx = instr.operand & opcode_mod.DEF_VAR_IDX_MASK; if (var_idx >= chunk.constants.len) return raiseInternal("vm: def Var index out of range"); @@ -570,7 +597,7 @@ inline fn stepOnce( stack[sp] = Value.encodeHeapPtr(.var_ref, var_ptr); sp += 1; }, - .op_var_meta => { + .op_var_meta => if (hot) unreachable else { // Stack is [.., var, meta] — pop the meta map, set it as // the Var's meta (the merged user ^meta/:doc + :line/:column/:file // the analyzer minted); the var stays as def's result value. @@ -593,7 +620,7 @@ inline fn stepOnce( stack[sp] = var_ptr.deref(); sp += 1; }, - .op_ns_import => { + .op_ns_import => if (hot) unreachable else { // Register one `(:import …)` simple->fqcn into the // current ns. Pushes nil (the ns form's running value). if (instr.operand >= chunk.import_sites.len) @@ -607,7 +634,7 @@ inline fn stepOnce( stack[sp] = Value.nil_val; sp += 1; }, - .op_set_var => { + .op_set_var => if (hot) unreachable else { if (instr.operand >= chunk.constants.len) return raiseInternal("vm: op_set_var constant index out of range"); if (sp == 0) return raiseInternal("vm: op_set_var on empty stack"); @@ -621,7 +648,7 @@ inline fn stepOnce( return error_catalog.raise(.var_set_not_bound, instr_loc, .{ .@"var" = full }); } }, - .op_set_field => { + .op_set_field => if (hot) unreachable else { // `(set! field v)` on a deftype mutable field. Stack: // [receiver, value]. operand = field-name String constant. if (instr.operand >= chunk.constants.len) @@ -1002,7 +1029,7 @@ inline fn stepOnce( stack[sp] = stack[sp - 1]; sp += 1; }, - .op_throw => { + .op_throw => if (hot) unreachable else { if (sp == 0) return raiseInternal("vm: op_throw on empty stack"); sp -= 1; dispatch.last_thrown_exception = stack[sp]; @@ -1016,7 +1043,7 @@ inline fn stepOnce( dispatch.last_thrown_context = error_mod.snapshotContext(); return error.ThrownValue; }, - .op_make_fn => { + .op_make_fn => if (hot) unreachable else { // The compiler stashes either a final closure-less // Function (slot_base == 0) or a template Function // (slot_base > 0, closure_bindings still null) in the @@ -1090,7 +1117,7 @@ inline fn stepOnce( if (sp < instr.operand) return raiseInternal("vm: op_recur underflow"); }, - .op_invoke_builtin => { + .op_invoke_builtin => if (hot) unreachable else { // Reserved for analyzer-resolved direct builtin calls; // the compiler does not emit this (every call routes // through `op_call` + vtable). Per @@ -1102,8 +1129,7 @@ inline fn stepOnce( const offset: i16 = @bitCast(instr.operand); const catch_ip = applyJump(ip, offset) orelse return raiseInternal("vm: op_push_handler target out of range"); - if (handler_count >= HANDLER_STACK_MAX) - return raiseInternal("vm: handler stack overflow"); + if (handler_count >= handlers.len) return handlerOverflow(handlers.len, instr_loc); handlers[handler_count] = .{ .catch_ip = catch_ip, .saved_sp = sp, .kind = .catch_clause }; handler_count += 1; }, @@ -1111,8 +1137,7 @@ inline fn stepOnce( const offset: i16 = @bitCast(instr.operand); const cleanup_ip = applyJump(ip, offset) orelse return raiseInternal("vm: op_push_cleanup target out of range"); - if (handler_count >= HANDLER_STACK_MAX) - return raiseInternal("vm: handler stack overflow"); + if (handler_count >= handlers.len) return handlerOverflow(handlers.len, instr_loc); handlers[handler_count] = .{ .catch_ip = cleanup_ip, .saved_sp = sp, .kind = .cleanup }; handler_count += 1; }, @@ -1132,7 +1157,7 @@ inline fn stepOnce( return raiseInternal("vm: op_pop_handler on empty handler stack"); handler_count -= 1; }, - .op_push_binding_frame => { + .op_push_binding_frame => if (hot) unreachable else { // Pops 2N entries [encVar0, val0, …] and installs a // BindingFrame on the per-thread binding chain (the // threadlocal is load-bearing: Clojure dynamic bindings @@ -1173,7 +1198,7 @@ inline fn stepOnce( // refresh in case this frame rebinds *ns*. env.refreshCurrentNs(); }, - .op_pop_binding_frame => { + .op_pop_binding_frame => if (hot) unreachable else { const f = env_mod.current_frame orelse return raiseInternal("vm: op_pop_binding_frame on empty frame stack"); env_mod.popFrame(); @@ -1182,7 +1207,7 @@ inline fn stepOnce( // Restore current_ns to the outer *ns* after the frame pops. env.refreshCurrentNs(); }, - .op_match_class => { + .op_match_class => if (hot) unreachable else { if (sp == 0) return raiseInternal("vm: op_match_class on empty stack"); if (instr.operand >= chunk.constants.len) @@ -1197,7 +1222,7 @@ inline fn stepOnce( stack[sp] = if (matches) Value.true_val else Value.false_val; sp += 1; }, - .op_match_type_keyword => { + .op_match_type_keyword => if (hot) unreachable else { if (sp == 0) return raiseInternal("vm: op_match_type_keyword on empty stack"); if (instr.operand >= chunk.constants.len) @@ -1210,7 +1235,7 @@ inline fn stepOnce( stack[sp] = if (matches) Value.true_val else Value.false_val; sp += 1; }, - .op_in_ns => { + .op_in_ns => if (hot) unreachable else { // Mirror of tree_walk's evalInNs. `(in-ns 'foo)` is a // naked ns switch — NO auto-refer of clojure.core. // `.clj` heads use `(ns foo (:refer-clojure))`, which @@ -1229,7 +1254,7 @@ inline fn stepOnce( stack[sp] = Env.nsValue(target_ns); sp += 1; }, - .op_ns_with_refer_clojure => { + .op_ns_with_refer_clojure => if (hot) unreachable else { // Mirror of tree_walk's evalNs when `refer_clojure = true`: // op_in_ns logic + referAll(clojure.core). clojure.core is // the single auto-refer source (Zig builtins intern there @@ -1248,7 +1273,7 @@ inline fn stepOnce( stack[sp] = Value.nil_val; sp += 1; }, - .op_ns_with_filter => { + .op_ns_with_filter => if (hot) unreachable else { // Mirror of tree_walk's evalNs refer-clojure branch // with the `:exclude`/`:only` filter. Enter the ns, apply the // ns meta + docstring, then refer clojure.core @@ -1275,7 +1300,7 @@ inline fn stepOnce( stack[sp] = Value.nil_val; sp += 1; }, - .op_require => { + .op_require => if (hot) unreachable else { // Mirror of tree_walk's evalRequire. if (instr.operand >= chunk.constants.len) return raiseInternal("vm: op_require constant index out of range"); @@ -1292,7 +1317,7 @@ inline fn stepOnce( stack[sp] = Value.nil_val; sp += 1; }, - .op_require_with_libspec => { + .op_require_with_libspec => if (hot) unreachable else { // Mirror of tree_walk's evalRequire full body. Pops the // LibspecEntry from the chunk side-table, runs the // op_require prelude, then applies alias + refers. @@ -1342,7 +1367,7 @@ inline fn stepOnce( // past `compiler.literal_step` elements the literal is built in steps, // each `op_*_extend` appending one step to the collection below it, so // the operand stack never holds more than one step (D-346). - inline .op_vector_literal, .op_map_literal, .op_set_literal => |op| { + inline .op_vector_literal, .op_map_literal, .op_set_literal => |op| if (hot) unreachable else { const n: u16 = instr.operand; if (sp < n) return raiseInternal("vm: collection literal underflows operand stack"); if (op == .op_map_literal and n % 2 != 0) return raiseInternal("vm: odd map literal"); @@ -1351,7 +1376,7 @@ inline fn stepOnce( stack[sp] = coll; sp += 1; }, - inline .op_vector_extend, .op_map_extend, .op_set_extend => |op| { + inline .op_vector_extend, .op_map_extend, .op_set_extend => |op| if (hot) unreachable else { // The collection under construction stays in its (rooted) stack // slot while each append allocates. const n: u16 = instr.operand; @@ -1360,7 +1385,7 @@ inline fn stepOnce( try literal.extend(rt, comptime literalKind(op), &stack[sp - n - 1], stack[sp - n .. sp]); sp -= n; }, - .op_ctor_call => { + .op_ctor_call => if (hot) unreachable else { // operand = index into the ctor_sites side-table (the // class name carries full width, no 8-bit name_idx packing). if (instr.operand >= chunk.ctor_sites.len) @@ -1378,7 +1403,7 @@ inline fn stepOnce( stack[sp] = new_val; sp += 1; }, - .op_method_call => { + .op_method_call => if (hot) unreachable else { // operand = call_site_idx. The unified instance-member // resolver (ADR-0050) runs field-first (deftype/record // field_layout), then method_table; `field_only` (the @@ -1463,7 +1488,7 @@ inline fn stepOnce( } } }, - .op_static_method_call => { + .op_static_method_call => if (hot) unreachable else { // operand = call_site_idx. Static dispatch — the // descriptor is the analyze-time pointer in // the call-site (no receiver to derive it from). Raw @@ -1504,6 +1529,64 @@ inline fn stepOnce( return null; } +/// The out-of-line copy of `stepOnce` for the cold opcodes. `noinline` is the +/// point: its frame exists only while a cold op runs, never across the +/// `vt.callFn` re-entries the hot copy makes. +noinline fn stepCold( + rt: *Runtime, + env: *Env, + locals: []Value, + chunk: *const BytecodeChunk, + stack: []Value, + loc_stack: []SourceLocation, + sp_ptr: *u16, + ip_ptr: *usize, + handlers: []Handler, + handler_count_ptr: *u16, + flatten_out: *?FlattenReq, +) anyerror!?Value { + return stepOnce(false, rt, env, locals, chunk, stack, loc_stack, sp_ptr, ip_ptr, handlers, handler_count_ptr, flatten_out); +} + +/// Opcodes the hot `stepOnce` hands to `stepCold`: definitions, namespace +/// forms, fn construction, dynamic binding, throw/catch matching, interop and +/// collection literals. Each either runs once per form or allocates anyway, so +/// the extra call is noise; what they would cost inline is native stack on +/// every `eval` frame. +fn isColdOp(op: Opcode) bool { + return switch (op) { + .op_def, + .op_def_unbound, + .op_var_meta, + .op_ns_import, + .op_set_var, + .op_set_field, + .op_throw, + .op_make_fn, + .op_invoke_builtin, + .op_push_binding_frame, + .op_pop_binding_frame, + .op_match_class, + .op_match_type_keyword, + .op_in_ns, + .op_ns_with_refer_clojure, + .op_ns_with_filter, + .op_require, + .op_require_with_libspec, + .op_ctor_call, + .op_method_call, + .op_static_method_call, + .op_vector_literal, + .op_map_literal, + .op_set_literal, + .op_vector_extend, + .op_map_extend, + .op_set_extend, + => true, + else => false, + }; +} + fn matchExceptionClass(class_name: []const u8, thrown: Value) bool { // Delegate to the shared host-class hierarchy table in // `runtime/error/host_class.zig`. Mirror of @@ -1522,6 +1605,14 @@ fn matchExceptionTypeKeyword(rt: *Runtime, kw_val: Value, thrown: Value) bool { return @intFromEnum(got) == @intFromEnum(kw_val); } +/// No handler slot left. A full `HANDLER_STACK_MAX` window is one eval's +/// try nesting (internal limit); a short one means the thread's handler +/// arena is spent by nested evals, i.e. deep recursion: StackOverflowError. +fn handlerOverflow(window: usize, loc: SourceLocation) anyerror { + if (window >= HANDLER_STACK_MAX) return raiseInternal("vm: handler stack overflow"); + return error_catalog.raise(.stack_overflow, loc, .{ .max = HANDLER_ARENA_SLOTS }); +} + fn raiseInternal(comptime detail: []const u8) anyerror { return error_catalog.raiseInternal(.{}, detail); } diff --git a/src/eval/bytecode/serialize.zig b/src/eval/bytecode/serialize.zig index 7e607f8da..eac881f79 100644 --- a/src/eval/bytecode/serialize.zig +++ b/src/eval/bytecode/serialize.zig @@ -79,6 +79,10 @@ const tree_walk = @import("../backend/tree_walk.zig"); const Function = tree_walk.Function; pub const MAGIC: [4]u8 = .{ 'C', 'L', 'J', 'W' }; +// v14: a fn method carries its `frame_slots` (ADR-0130). Without it a +// deserialized method fell back to the full MAX_LOCALS window per call, so +// every bootstrap-cache fn on a recursion path (`map`, `first`, ...) paid 256 +// slots per level. A v13 method has no such field, so v13 is rejected. // v13: a ctor site carries `definer_ns`, the namespace its form was analyzed // in, so a bare user-type name is qualified lexically instead of by the // caller's namespace (ADR-0198, ADR-0201). A v12 ctor site has no such field, @@ -94,7 +98,7 @@ pub const MAGIC: [4]u8 = .{ 'C', 'L', 'J', 'W' }; // v10: def operands capture Var constants instead of caller-relative names. // v9 added op_var_meta; v8 unified host-class FQCNs; v7 compacted WireInstr // and interned the name pool; v6 merged rt into clojure.core. -pub const VERSION: u16 = 13; +pub const VERSION: u16 = 14; pub const SerializeError = error{ OutOfMemory, @@ -534,6 +538,7 @@ fn writeEntries(ctx: *WriteCtx, w: *std.Io.Writer, coll: Value, comptime kind: e fn writeFnMethod(ctx: *WriteCtx, w: *std.Io.Writer, m: tree_walk.FunctionMethod) SerializeError!void { try writeU16(w, m.arity); try writeU8(w, @intFromBool(m.has_rest)); + try writeU16(w, m.frame_slots); // v14 if (m.bytecode) |chunk| { try writeU8(w, 1); // v7: nested fn-method chunks are HEADERLESS bodies (no repeated @@ -607,13 +612,14 @@ fn readValue(ctx: *ReadCtx, r: *ByteReader) DeserializeError!Value { // nil (D-164) — fold-from-nil would otherwise lose the distinct // `()` for a quoted empty list baked into the AOT blob. if (n == 0) return list_mod.emptyList(rt) catch return DeserializeError.OutOfMemory; - // Read forward into a stack-allocated buffer, then cons-fold - // back to preserve the head-first source order. Caps at a - // bounded stack alloc; very-large quoted lists are rare in - // the constant pool — long sequences come through `list` - // primitive at runtime. - const buf = std.heap.page_allocator.alloc(Value, n) catch return DeserializeError.OutOfMemory; - defer std.heap.page_allocator.free(buf); + // Read forward into a scratch buffer, then cons-fold back to + // preserve the head-first source order. + // PERF: scratch from the reader's allocator, as `readCollection`'s + // is; `page_allocator` cost one mmap + munmap + page fault per + // quoted list, 927 of them on every cold start [refs: O-090]. + if (n > r.bytes.len - r.pos) return DeserializeError.BytecodeTruncated; + const buf = ctx.allocator.alloc(Value, n) catch return DeserializeError.OutOfMemory; + defer ctx.allocator.free(buf); // Fabrication region: `buf` elements + the growing `lst` are Zig // locals across the nested readValue/cons allocs — a collect in // the window would sweep the in-progress structure (the D-563b @@ -732,6 +738,7 @@ fn readFnMethod(ctx: *ReadCtx, r: *ByteReader) DeserializeError!tree_walk.Serial const allocator = ctx.allocator; const arity = try r.readU16(); const has_rest = (try r.readU8()) != 0; + const frame_slots = try r.readU16(); // v14 const has_bytecode = try r.readU8(); var bytecode: ?*const BytecodeChunk = null; if (has_bytecode != 0) { @@ -749,7 +756,7 @@ fn readFnMethod(ctx: *ReadCtx, r: *ByteReader) DeserializeError!tree_walk.Serial sub.* = chunk; bytecode = sub; } - return .{ .arity = arity, .has_rest = has_rest, .bytecode = bytecode }; + return .{ .arity = arity, .has_rest = has_rest, .frame_slots = frame_slots, .bytecode = bytecode }; } // --- Chunk serialize / deserialize --- diff --git a/src/eval/reader.zig b/src/eval/reader.zig index 350ef81bf..f6a56caf8 100644 --- a/src/eval/reader.zig +++ b/src/eval/reader.zig @@ -32,6 +32,10 @@ const SourceLocation = error_mod.SourceLocation; /// just admits future kinds without churning every signature. pub const ReadError = error_mod.ClojureWasmError; +/// Re-exported for callers that pick a reader dialect (`edn.zig`). +pub const Dialect = tok_mod.Dialect; +pub const ReadForm = Form; + /// Process-wide counter behind the `#()` param gensyms (`p1__N#`), as clj's /// RT.nextID: each literal takes the next id, from any thread. var fn_lit_ids: std.atomic.Value(u32) = .init(1); @@ -75,6 +79,17 @@ pub const Reader = struct { /// reader first, as clj's does. form_checks: bool = true, + /// Called with each `#_`-discarded form the moment it is read, so a DATA + /// read can materialize it (run its tag readers, check its duplicates) + /// before it is dropped, as clj's reader does (ADR-0200 A4). Null for a + /// code read, whose forms are never lifted here. + discard_hook: ?DiscardHook = null, + + pub const DiscardHook = struct { + ctx: *anyopaque, + f: *const fn (ctx: *anyopaque, form: Form) ReadError!void, + }; + pub fn init(allocator: std.mem.Allocator, source: []const u8) Reader { return .{ .tokenizer = Tokenizer.init(source), @@ -83,6 +98,15 @@ pub const Reader = struct { }; } + /// A reader for `dialect` (ADR-0200 Layer B). `.edn` is clj's EdnReader + /// surface; the tokenizer enforces it, since every EDN restriction is a + /// lexical one (a macro character or a dispatch character). + pub fn initDialect(allocator: std.mem.Allocator, source: []const u8, dialect: tok_mod.Dialect) Reader { + var r = init(allocator, source); + r.tokenizer.dialect = dialect; + return r; + } + /// Read one Form. Returns `null` on clean EOF. pub fn read(self: *Reader) ReadError!?Form { if (try self.readValue()) |f| return f; @@ -915,8 +939,9 @@ pub const Reader = struct { const discard_loc = self.locOf(tok); // Consume the discard target (a real form; readValue skips any nested // no-value macros). A boundary before a target is incomplete (`[#_]`). - _ = (try self.readValue()) orelse + const target = (try self.readValue()) orelse return error_catalog.raise(.discard_reader_macro_incomplete, discard_loc, .{}); + if (self.discard_hook) |h| try h.f(h.ctx, target); // `#_` itself contributes nothing: signal skip + return a placeholder // the caller (readValue) discards. self.skip = true; diff --git a/src/eval/tokenizer.zig b/src/eval/tokenizer.zig index 2d1a6e5d8..f08808123 100644 --- a/src/eval/tokenizer.zig +++ b/src/eval/tokenizer.zig @@ -112,16 +112,31 @@ pub const Token = struct { }; /// Stateful tokenizer. +/// Which surface syntax the tokenizer accepts (ADR-0200 Layer B). `.clojure` +/// is clj's LispReader; `.edn` is clj's EdnReader, a strict subset: no +/// syntax-quote / unquote / deref macros (`` ` `` `~` `@` are invalid leading +/// AND invalid constituent characters), `'` is a symbol constituent that may +/// also start a symbol (`'a` is the symbol `'a`), no `::auto` keywords, and the +/// only `#` dispatches are `#{` `#_` `##` `#^` `#:ns{` and `#tag`. +pub const Dialect = enum { clojure, edn }; + pub const Tokenizer = struct { source: []const u8, pos: u32 = 0, line: u32 = 1, column: u16 = 0, + dialect: Dialect = .clojure, pub fn init(source: []const u8) Tokenizer { return .{ .source = source }; } + /// A char clj's EdnReader rejects inside a token ("Invalid constituent + /// character"). In `.clojure` these terminate the token instead. + fn isEdnBadConstituent(self: *const Tokenizer, c: u8) bool { + return self.dialect == .edn and (c == '@' or c == '~' or c == '`'); + } + /// Return the next token; advances state. Idempotent at EOF. pub fn next(self: *Tokenizer) Token { self.skipWhitespace(); @@ -133,6 +148,14 @@ pub const Tokenizer = struct { const start_col = self.column; const ch = self.source[self.pos]; + if (self.dialect == .edn) switch (ch) { + // clj EdnReader: no quote macro, `'` is a symbol constituent that + // can start one; `` ` `` `~` `@` are "Invalid leading character". + '\'' => return self.readSymbol(start, start_line, start_col), + '`', '~', '@' => return self.singleChar(.invalid, start, start_line, start_col), + else => {}, + }; + switch (ch) { '(' => return self.singleChar(.lparen, start, start_line, start_col), ')' => return self.singleChar(.rparen, start, start_line, start_col), @@ -246,6 +269,10 @@ pub const Tokenizer = struct { if (self.pos - start <= 1) { return self.makeToken(.invalid, start, start_line, start_col); // bare ':' } + // clj EdnReader has no auto-resolved keywords: `::kw` is "Invalid token". + if (self.dialect == .edn and self.source[start + 1] == ':') + return self.makeToken(.invalid, start, start_line, start_col); + if (self.endsAtEdnBadConstituent()) return self.makeToken(.invalid, start, start_line, start_col); return self.makeToken(.keyword, start, start_line, start_col); } @@ -345,11 +372,22 @@ pub const Tokenizer = struct { fn readSymbol(self: *Tokenizer, start: u32, start_line: u32, start_col: u16) Token { while (self.pos < self.source.len and isSymbolChar(self.source[self.pos])) self.advance(); + if (self.endsAtEdnBadConstituent()) return self.makeToken(.invalid, start, start_line, start_col); // `nil` / `true` / `false` come back as plain symbol tokens; // the reader (next phase) re-classifies them. return self.makeToken(.symbol, start, start_line, start_col); } + /// `.edn` only: the token just scanned stopped at `@` `~` or `` ` ``, which + /// clj's EdnReader rejects as an "Invalid constituent character". Consume + /// the rest of the run so the error names the whole token. + fn endsAtEdnBadConstituent(self: *Tokenizer) bool { + if (self.pos >= self.source.len or !self.isEdnBadConstituent(self.source[self.pos])) return false; + while (self.pos < self.source.len and + (isSymbolChar(self.source[self.pos]) or self.isEdnBadConstituent(self.source[self.pos]))) self.advance(); + return true; + } + fn readDispatch(self: *Tokenizer, start: u32, start_line: u32, start_col: u16) Token { self.advance(); // '#' @@ -357,6 +395,22 @@ pub const Tokenizer = struct { return self.makeToken(.invalid, start, start_line, start_col); } + // clj EdnReader's dispatch table is `#{` `#_` `##` `#^` `#:` and `#tag` + // only; every other dispatch char is "No dispatch macro for: c", and + // `#::` (auto-resolved namespaced map) is an invalid token. + if (self.dialect == .edn) switch (self.source[self.pos]) { + '!', '"', '(', '\'', '?', '=', '<' => { + self.advance(); + return self.makeToken(.invalid, start, start_line, start_col); + }, + ':' => if (self.pos + 1 < self.source.len and self.source[self.pos + 1] == ':') { + self.advance(); + self.advance(); + return self.makeToken(.invalid, start, start_line, start_col); + }, + else => {}, + }; + switch (self.source[self.pos]) { '_' => { self.advance(); @@ -755,3 +809,32 @@ test "a number token ends at a macro char, even ones a symbol keeps" { try testing.expectEqual(TokenKind.integer, u.next().kind); try testing.expectEqual(TokenKind.lparen, u.next().kind); } + +fn ednKinds(src: []const u8) TokenKind { + var t = Tokenizer.init(src); + t.dialect = .edn; + return t.next().kind; +} + +test "edn dialect rejects the macro characters and dispatches clj's EdnReader lacks" { + for ([_][]const u8{ "`a", "~a", "@a", "#'a", "#(x)", "#\"r\"", "#=(x)", "#?(:clj 1)", "#!x", "::kw", "#::{}", "a@b", "a~b", "a`b", ":k@b" }) |src| + try testing.expectEqual(TokenKind.invalid, ednKinds(src)); +} + +test "edn dialect reads quote as a symbol constituent and keeps the EDN dispatches" { + var t = Tokenizer.init("'a #{ #_ ##Inf #:a{ #tag"); + t.dialect = .edn; + const q = t.next(); + try testing.expectEqual(TokenKind.symbol, q.kind); + try testing.expectEqualStrings("'a", q.text(t.source)); + try testing.expectEqual(TokenKind.set_open, t.next().kind); + try testing.expectEqual(TokenKind.discard, t.next().kind); + try testing.expectEqual(TokenKind.symbolic, t.next().kind); + _ = t.next(); // Inf + try testing.expectEqual(TokenKind.ns_map, t.next().kind); + _ = t.next(); // { + try testing.expectEqual(TokenKind.tagged, t.next().kind); + // The clojure dialect is unchanged. + var c = Tokenizer.init("`a"); + try testing.expectEqual(TokenKind.syntax_quote, c.next().kind); +} diff --git a/src/lang/clj/cljw/ffi.clj b/src/lang/clj/cljw/ffi.clj index f8d8fcb3f..11f32d348 100644 --- a/src/lang/clj/cljw/ffi.clj +++ b/src/lang/clj/cljw/ffi.clj @@ -8,7 +8,8 @@ ;; (:require #?(:cljw [cljw.ffi :as ffi] :rust [clojure.rust.ffi :as ffi])) ;; ;; Types: :void (return only) :int :long :double :pointer :string :bytes -;; (argument only). At most 6 integer-class and 8 :double arguments; no +;; (argument only). A :pointer argument also takes a primitive array (its +;; storage, no copy); :bytes copies a byte array. At most 6 integer-class and 8 :double arguments; no ;; varargs, no structs by value, no float. Failures are ex-info with ;; {:ffi/error :open|:symbol|:signature|:arity|:arg-type|:closed ...}. ;; Require-on-demand, and present only on x86_64 / aarch64 Linux and macOS. diff --git a/src/lang/clj/clojure/core.clj b/src/lang/clj/clojure/core.clj index b5cea0def..ca1813ba8 100644 --- a/src/lang/clj/clojure/core.clj +++ b/src/lang/clj/clojure/core.clj @@ -168,6 +168,12 @@ (chunk-cons (cljw.internal/__chunk-map-step f s) (-map-lazy f (chunk-rest s))) (cons (f (first s)) (-map-lazy f (rest s)))) nil))))) +;; PERF: O-062 internal lazy body for 2/3-coll `map`: one lazy node per element +;; (clj's multi-coll map is unchunked), whose body is the Zig `__map-step` +;; (seq all, f on the firsts, recur on the rests). Passes itself as `self`. +(def ^:private -map-n-lazy + (fn* ([f c1 c2] (lazy-seq (cljw.internal/__map-step f -map-n-lazy c1 c2))) + ([f c1 c2 c3] (lazy-seq (cljw.internal/__map-step f -map-n-lazy c1 c2 c3))))) ;; Forward-declared so map's n-ary arm resolves it at def time (D-446). (declare -map-n-step) (def map @@ -186,19 +192,10 @@ ;; PERF: O-023 stamp the fusion descriptor on the outer node so `reduce` ;; fuses; the lazy body (`-map-lazy`) is unchanged → seq ops identical. (-lazy-set-fuse (-map-lazy f coll) (-fuse-descriptor (map f) coll))) - ([f c1 c2] - (lazy-seq - (let [s1 (seq c1) s2 (seq c2)] - (if (and s1 s2) - (cons (f (first s1) (first s2)) (map f (rest s1) (rest s2))) - nil)))) - ([f c1 c2 c3] - (lazy-seq - (let [s1 (seq c1) s2 (seq c2) s3 (seq c3)] - (if (and s1 s2 s3) - (cons (f (first s1) (first s2) (first s3)) - (map f (rest s1) (rest s2) (rest s3))) - nil)))) + ;; PERF: O-062 the 2/3-coll step body runs in Zig (`__map-step`): same + ;; unchunked one-f-per-node shape as clj, the lazy node stays here. + ([f c1 c2] (-map-n-lazy f c1 c2)) + ([f c1 c2 c3] (-map-n-lazy f c1 c2 c3)) ;; n-ary: zip 4+ colls, stopping at the shortest (clj parity). ([f c1 c2 c3 & colls] (map (fn* [args] (apply f args)) (-map-n-step (conj colls c3 c2 c1)))))) @@ -516,8 +513,9 @@ ;; bootstrap-order-safe. The lazy-seq realization utilities (clojure.core). (def dorun (fn* ([coll] - (loop* [s (seq coll)] - (if s (recur (next s)) nil))) + ;; PERF: the seq/next walk runs natively, a chunk at a time over a + ;; chunked seq. [refs: O-077] + (cljw.internal/__dorun coll)) ([n coll] (loop* [i n s (seq coll)] (if (if (pos? i) s nil) (recur (dec i) (next s)) nil))))) @@ -731,9 +729,8 @@ ;; defaults to (count v)) as a vector. For a vector this is JVM's O(1) ;; shared-structure VIEW (the `.sub_vector` tag; D-583); a non-vector ;; seqable falls back to the eager take/drop build. -(def subvec - (fn* ([v start] (subvec v start (count v))) - ([v start end] +(def ^:private -subvec-slow + (fn* ([v start end] (when (nil? v) (throw (NullPointerException. "subvec: nil vector"))) (when-not (vector? v) @@ -763,6 +760,15 @@ ;; O(1) shared-structure view, preserving numeric coercion and bounds. (cljw.internal/__subvec v start end))))) +(def subvec + (fn* ([v start] (subvec v start (count v))) + ([v start end] + ;; PERF: in-bounds integer indices over a vector are answered by one + ;; native step; nil means "not that shape", so everything below is the + ;; coercing / throwing path. [refs: O-073] + (or (cljw.internal/__subvec-fast v start end) + (-subvec-slow v start end))))) + ;; `(bounded-count n coll)` — for a `counted?` coll return its FULL count (clj: ;; `(bounded-count 3 (range 100))` → 100, range is O(1) counted); otherwise walk ;; at most n elements so it terminates on infinite / expensive seqs (D-134). @@ -1020,7 +1026,8 @@ ;; `(mapv f coll)` — eager `map` returning a vector. Single-coll form. (def mapv - (fn* ([f coll] (reduce (fn* [acc x] (conj acc (f x))) [] coll)) + ;; PERF: single-coll form builds through a transient, clj's own shape (O(n) persistent! over a flat buffer vs N persistent conjs) [refs: O-003, D-180] + (fn* ([f coll] (persistent! (reduce (fn* [v o] (conj! v (f o))) (transient []) coll))) ([f c1 c2] (vec (map f c1 c2))) ([f c1 c2 c3] (vec (map f c1 c2 c3))) ([f c1 c2 c3 & colls] (vec (apply map f c1 c2 c3 colls))))) @@ -2744,109 +2751,96 @@ (throw (IllegalArgumentException. (str "Not a qualified symbol: " sym)))))) -;; --- Java arrays (ADR-0105 / D-287) --- -;; Type-erased uniform []Value over the cljw.internal/__array-make + aget/aset/alength/ -;; aclone host primitives. Per-constructor init defaults + byte/short/char wrap -;; give clj-faithful VALUES (F-011); the element type itself is erased (AD-019), -;; so int/long/double/float arrays do not coerce non-byte elements. Arrays use -;; identity equality. +;; --- Java arrays (ADR-0105 / D-287, ADR-0204) --- +;; An array has an element kind: `object-array` / `to-array` / `into-array` / +;; `make-array` hold any value, the typed constructors hold native elements +;; (byte i8 ... double f64). The cljw.internal/__array-make / __array-from / +;; __array-fill constructors are native, so an n-element array is one call. +;; Constructors convert like clj's RT.longArray and friends (wrap, truncate); +;; `aset` is strict (clj's "argument type mismatch"). Arrays use identity +;; equality. + +(defn- -array-elems + "A vector or array holding `x`'s elements, the input of `__array-from`." + [x] (if (or (vector? x) (array? x)) x (vec x))) (defn- -array-from - "size-or-seq → array. A number makes n slots of `default`; otherwise the - seqable is materialised and each element passed through `coerce`." - [size-or-seq default coerce] + "size-or-seq → array of `kind`. A number makes n default slots; otherwise + each element of the seqable is converted to the kind." + [kind size-or-seq] (if (number? size-or-seq) - (cljw.internal/__array-make size-or-seq default) - (let [s (vec size-or-seq) - n (count s) - a (cljw.internal/__array-make n default)] - (dotimes [i n] (aset a i (coerce (nth s i)))) - a))) + (cljw.internal/__array-make kind (long size-or-seq)) + (cljw.internal/__array-from kind (-array-elems size-or-seq) nil))) (defn- -array-from2 - "size + init-val-or-seq → array (the 2-arg ctor form). A `sequential?` init is - copied (`size` slots, the rest left at `default`); any other init fills every - slot with `(coerce init)`. Uniform across element types — cljw arrays are - type-erased (AD-019), so unlike clj there is no per-type Number/Character - overload accident: `(byte-array 3 65)` / `(char-array 3 65)` fill here but - throw on the JVM (AD-036)." - [size init default coerce] - (let [a (cljw.internal/__array-make size default)] - (if (sequential? init) - (let [s (vec init) - n (min size (count s))] - (dotimes [i n] (aset a i (coerce (nth s i))))) - (dotimes [i size] (aset a i (coerce init)))) - a)) - -(defn- -to-byte [v] - (let [b (bit-and v 255)] (if (>= b 128) (- b 256) b))) -(defn- -to-short [v] - (let [s (bit-and v 65535)] (if (>= s 32768) (- s 65536) s))) + "size + init-val-or-seq → array of `kind` (the 2-arg ctor form). A + `sequential?` init is copied (`size` slots, the rest left at the default); + any other init fills every slot. Uniform across kinds, so unlike clj there + is no per-type Number/Character overload accident: `(byte-array 3 65)` / + `(char-array 3 65)` fill here but throw on the JVM (AD-036)." + [kind size init] + (if (sequential? init) + (cljw.internal/__array-from kind (-array-elems init) (long size)) + (cljw.internal/__array-fill (cljw.internal/__array-make kind (long size)) init))) ;; object-array has NO clj 2-arg form (clj throws an arity error) — stays 1-arg. -(defn object-array [x] (-array-from x nil identity)) +(defn object-array [x] (-array-from :object x)) ;; The typed ctors take `([size-or-seq] [size init-val-or-seq])` (clj parity). -(defn int-array ([x] (-array-from x 0 identity)) ([n init] (-array-from2 n init 0 identity))) -(defn long-array ([x] (-array-from x 0 identity)) ([n init] (-array-from2 n init 0 identity))) -(defn double-array ([x] (-array-from x 0.0 identity)) ([n init] (-array-from2 n init 0.0 identity))) -(defn float-array ([x] (-array-from x 0.0 identity)) ([n init] (-array-from2 n init 0.0 identity))) -(defn short-array ([x] (-array-from x 0 -to-short)) ([n init] (-array-from2 n init 0 -to-short))) -(defn byte-array ([x] (-array-from x 0 -to-byte)) ([n init] (-array-from2 n init 0 -to-byte))) -(defn char-array ([x] (-array-from x \space char)) ([n init] (-array-from2 n init \space char))) -(defn boolean-array ([x] (-array-from x false boolean)) ([n init] (-array-from2 n init false boolean))) +(defn int-array ([x] (-array-from :int x)) ([n init] (-array-from2 :int n init))) +(defn long-array ([x] (-array-from :long x)) ([n init] (-array-from2 :long n init))) +(defn double-array ([x] (-array-from :double x)) ([n init] (-array-from2 :double n init))) +(defn float-array ([x] (-array-from :float x)) ([n init] (-array-from2 :float n init))) +(defn short-array ([x] (-array-from :short x)) ([n init] (-array-from2 :short n init))) +(defn byte-array ([x] (-array-from :byte x)) ([n init] (-array-from2 :byte n init))) +(defn char-array ([x] (-array-from :char x)) ([n init] (-array-from2 :char n init))) +(defn boolean-array ([x] (-array-from :boolean x)) ([n init] (-array-from2 :boolean n init))) (defn make-array - "Allocate an array. The `type` arg is accepted but ignored (cljw arrays are - type-erased, AD-019). Multi-dim builds nested arrays." - ([type len] (cljw.internal/__array-make len nil)) + "Allocate an object array. The `type` arg is accepted but ignored: cljw has + no Class objects, and `(make-array Long n)` is a `Long[]` on the JVM too. + Multi-dim builds nested arrays." + ([type len] (cljw.internal/__array-make :object len)) ([type d & more] - (let [a (cljw.internal/__array-make d nil)] + (let [a (cljw.internal/__array-make :object d)] (dotimes [i d] (aset a i (apply make-array type more))) a))) (defn to-array - "coll → an array of its elements (clj returns Object[])." - [coll] (-array-from (vec coll) nil identity)) + "coll → an object array of its elements (clj returns Object[])." + [coll] (-array-from :object (-array-elems coll))) (defn bytes? - "Returns true if x is a cljw array. JVM Clojure's `bytes?` is true only for a - `byte[]`, but cljw arrays are type-erased (AD-019): `byte-array`/`int-array`/ - `object-array` share one runtime shape, so a byte-array is indistinguishable - from any other array. cljw therefore answers true for any array (AD-051) — the - common positive guard `(bytes? (byte-array n))` matches clj; non-byte arrays - diverge (clj false, cljw true). A byte-array IS usable as bytes in cljw - (`(String. (byte-array ...))` works), so the over-broad positive is consistent - with cljw's uniform-array model rather than a silent lie." - [x] (array? x)) + "Returns true if x is a byte array." + [x] (and (array? x) (= :byte (cljw.internal/__array-kind x)))) (defn into-array - "coll → array; the optional leading `type` is accepted and ignored." + "coll → an object array; the optional leading `type` is accepted and + ignored (a boxed `Long[]` on the JVM is an object array here too)." ([coll] (to-array coll)) ([type coll] (to-array coll))) -;; aset-* typed setters. clj coerces the value to the element type; cljw applies -;; only the value-changing modular wraps (byte/short/char), the rest are aset. +;; aset-* typed setters: the value is coerced as clj's `(int v)` / `(byte v)` +;; ... do (a range check, not a wrap), then stored. (defn aset-int - ([a i v] (aset a i v)) + ([a i v] (aset a i (int v))) ([a i j & more] (apply aset-int (aget a i) j more))) (defn aset-long - ([a i v] (aset a i v)) + ([a i v] (aset a i (long v))) ([a i j & more] (apply aset-long (aget a i) j more))) (defn aset-float - ([a i v] (aset a i v)) + ([a i v] (aset a i (float v))) ([a i j & more] (apply aset-float (aget a i) j more))) (defn aset-double - ([a i v] (aset a i v)) + ([a i v] (aset a i (double v))) ([a i j & more] (apply aset-double (aget a i) j more))) (defn aset-boolean ([a i v] (aset a i (boolean v))) ([a i j & more] (apply aset-boolean (aget a i) j more))) (defn aset-byte - ([a i v] (aset a i (-to-byte v))) + ([a i v] (aset a i (byte v))) ([a i j & more] (apply aset-byte (aget a i) j more))) (defn aset-short - ([a i v] (aset a i (-to-short v))) + ([a i v] (aset a i (short v))) ([a i j & more] (apply aset-short (aget a i) j more))) (defn aset-char ([a i v] (aset a i (char v))) diff --git a/src/lang/clj/clojure/spec/gen/alpha.clj b/src/lang/clj/clojure/spec/gen/alpha.clj index 1a3163c60..8a37b9b10 100644 --- a/src/lang/clj/clojure/spec/gen/alpha.clj +++ b/src/lang/clj/clojure/spec/gen/alpha.clj @@ -7,9 +7,9 @@ ;; terms of this license. You must not remove this notice, or any other, from ;; this software. ;; -;; clojure.spec.gen.alpha — by Rich Hickey. Reproduced in ClojureWasm with one -;; no-JVM adaptation (the `bytes?` gen-builtin dropped — Java arrays are -;; type-erased, AD-019); redistributed under EPL-2.0 per EPL-1.0 §7. +;; clojure.spec.gen.alpha — by Rich Hickey. Reproduced in ClojureWasm (the +;; `bytes?` gen-builtin, once dropped for type-erased arrays, is back since +;; ADR-0204); redistributed under EPL-2.0 per EPL-1.0 §7. ;; ClojureWasm changes (c) the ClojureWasm authors. (ns clojure.spec.gen.alpha (:refer-clojure :exclude [boolean bytes cat hash-map list map not-empty set vector @@ -159,6 +159,7 @@ gens, each of which should generate something sequential." simple-symbol? (symbol) qualified-symbol? (such-that qualified? (symbol-ns)) uuid? (uuid) + bytes? (bytes) uri? (fmap #(java.net.URI/create (str "http://" % ".com")) (uuid)) decimal? (fmap #(BigDecimal/valueOf %) (double* {:infinite? false :NaN? false})) diff --git a/src/lang/clj/clojure/string.clj b/src/lang/clj/clojure/string.clj index 3ce5d183f..58043bf0d 100644 --- a/src/lang/clj/clojure/string.clj +++ b/src/lang/clj/clojure/string.clj @@ -68,21 +68,14 @@ ;; `-capitalize` leaf, which already does exactly this over codepoints. (def capitalize (fn* [s] (-capitalize s))) -;; join: 1-arity (coll) = (apply str coll); 2-arity (sep coll) interposes the -;; separator and builds the result in one `apply str`. Variadic via `[& args]` -;; + count discrimination since cw v1 lacks multi-arity `fn*` (D-070). +;; join: 1-arity (coll) = (apply str coll); 2-arity (sep coll) puts `(str sep)` +;; between the `str` of each element of `(seq coll)`. Both arities are the +;; `-join` leaf: one native walk + one buffer, no interpose seq. ;; (join sep) transducer arity is deferred (D-070 + ADR-0033 D6a). -;; -;; PERF: the old 2-arity `(reduce (fn [acc x] (str acc sep x)) nil coll)` copied -;; the growing accumulator string on EVERY element → O(n²) in total length. The -;; JVM idiom `(apply str (interpose sep coll))` walks once and builds the result -;; in a single `str` pass (O(n) — `apply str` is the native variadic builder). -;; [refs: O-012] +;; [refs: O-012, O-070] (def join - (fn* [& args] - (if (= 1 (count args)) - (apply str (first args)) - (apply str (interpose (first args) (first (rest args))))))) + (fn* ([coll] (-join coll)) + ([separator coll] (-join separator coll)))) ;; ---------------------------------------------------------------- ;; Row 7.12 cycle 3 (D-078): `replace` / `replace-first` Pattern A diff --git a/src/lang/diff_test.zig b/src/lang/diff_test.zig index 9906b16db..4fcb7eb7b 100644 --- a/src/lang/diff_test.zig +++ b/src/lang/diff_test.zig @@ -875,7 +875,7 @@ test "diff: Java array aset/aget + seq (ADR-0105)" { // same BuiltinFn — this case locks that aset's in-place mutation + the // array-as-Seqable reduce path agree on TreeWalk and VM. try f.check( - \\(let* [a (cljw.internal/__array-make 3 0)] + \\(let* [a (cljw.internal/__array-make :long 3)] \\ (aset a 0 10) (aset a 1 20) (aset a 2 12) \\ (reduce + a)) , 42); diff --git a/src/lang/primitive.zig b/src/lang/primitive.zig index 3d808e2be..af563db99 100644 --- a/src/lang/primitive.zig +++ b/src/lang/primitive.zig @@ -55,6 +55,7 @@ const reduced_prim = @import("primitive/reduced.zig"); const namespace_prim = @import("primitive/namespace.zig"); const macroexpand_prim = @import("primitive/macroexpand.zig"); const array_prim = @import("primitive/array.zig"); +const map_step = @import("primitive/map_step.zig"); pub const RegisterError = error{ InternalNamespaceMissing, @@ -114,6 +115,7 @@ pub fn registerAll(env: *Env) !void { try namespace_prim.register(env, clojure_core_ns); // ADR-0083 ns-reflection try macroexpand_prim.register(env, clojure_core_ns); // D-229 macroexpand-1/macroexpand try array_prim.register(env, clojure_core_ns); // ADR-0105 / D-287 Java arrays + try map_step.register(env, clojure_core_ns); // ADR-0205 multi-coll map step // Phase 14 row 14.1 (D-079 discharge): walk every // `runtime/java//.zig`'s `___HOST_EXTENSION` diff --git a/src/lang/primitive/array.zig b/src/lang/primitive/array.zig index fb8870a12..615b708a0 100644 --- a/src/lang/primitive/array.zig +++ b/src/lang/primitive/array.zig @@ -1,14 +1,15 @@ // SPDX-License-Identifier: EPL-2.0 -//! Java-array primitives — the Layer-1 host surface over -//! `runtime/collection/java_array.zig` (ADR-0105 / D-287). +//! Java-array primitives: the Layer-1 host surface over +//! `runtime/collection/java_array.zig` (ADR-0105 / D-287, ADR-0204). //! -//! Only the irreducible host operations live here: `__array-make` (sized + -//! init-filled), `aget` (2-arg), `aset` (3-arg), `alength`, `aclone`, -//! `array?`. The full clojure.core surface (object-array / int-array / -//! byte-array / make-array / to-array / into-array / aset-* / amap / areduce / -//! multi-dim aget+aset) is composed from these in `core.clj`, where seq -//! walking + per-type init defaults + byte/short/char wrap are idiomatic and -//! the seq fns already handle GC rooting (F-009 layering). +//! The irreducible host operations: the three constructors `__array-make` +//! (sized, default-filled), `__array-from` (a bulk copy of a vector or array, +//! converting each element) and `__array-fill` (every slot one value), then +//! `aget`, `aset`, `alength`, `aclone`, `array?` and `__array-kind`. The +//! clojure.core surface (object-array / int-array / byte-array / make-array / +//! to-array / into-array / aset-* / amap / areduce) is composed from these in +//! `core.clj`. The constructors are native so building an n-element array is +//! one call, not n `aset`s. const Value = @import("../../runtime/value/value.zig").Value; const Runtime = @import("../../runtime/runtime.zig").Runtime; @@ -18,6 +19,9 @@ const error_mod = @import("../../runtime/error/info.zig"); const error_catalog = @import("../../runtime/error/catalog.zig"); const SourceLocation = error_mod.SourceLocation; const java_array = @import("../../runtime/collection/java_array.zig"); +const vector_mod = @import("../../runtime/collection/vector.zig"); +const sub_vector = @import("../../runtime/collection/sub_vector.zig"); +const keyword_mod = @import("../../runtime/keyword.zig"); fn requireArray(v: Value, fn_name: []const u8, loc: SourceLocation) !void { if (v.tag() != .array) @@ -38,38 +42,105 @@ fn asIndex(v: Value, fn_name: []const u8, loc: SourceLocation) !i64 { return v.asInteger(); } -/// `(rt/__array-make size init-val)` — allocate a `size`-element array with -/// every slot set to `init-val`. The clj per-constructor default (0 / 0.0 / -/// false / \space / nil) is chosen by the core.clj caller (F-011). +fn asSize(v: Value, fn_name: []const u8, loc: SourceLocation) !u32 { + const n = try asIndex(v, fn_name, loc); + if (n < 0 or n > std_max_u32) + return error_catalog.raise(.type_arg_invalid, loc, .{ .fn_name = fn_name, .expected = "a non-negative size", .actual = "out of range" }); + return @intCast(n); +} + +const std_max_u32: i64 = 0xFFFF_FFFF; + +/// The element kind a keyword (`:long`, `:object`, …) names. +fn asKind(v: Value, fn_name: []const u8, loc: SourceLocation) !java_array.Kind { + if (v.tag() == .keyword) { + if (java_array.Kind.fromName(keyword_mod.asKeyword(v).name)) |k| return k; + } + return error_catalog.raise(.type_arg_invalid, loc, .{ .fn_name = fn_name, .expected = "an array element kind keyword", .actual = @tagName(v.tag()) }); +} + +/// `(__array-make kind size)`: a `size`-element array of `kind`, every slot +/// at the kind's default. fn arrayMakeFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { _ = env; try error_catalog.checkArity("__array-make", args, 2, loc); - const n = try asIndex(args[0], "__array-make", loc); - if (n < 0) - return error_catalog.raise(.type_arg_invalid, loc, .{ .fn_name = "__array-make", .expected = "a non-negative size", .actual = "negative" }); - return java_array.make(rt, @intCast(n), args[1]); + return java_array.make(rt, try asKind(args[0], "__array-make", loc), try asSize(args[1], "__array-make", loc)); +} + +/// `(__array-from kind src size)`: an array of `kind` holding `src`'s +/// elements, each converted as clj's `RT.longArray` and friends do (a float +/// truncates into an integer kind, an integer wraps to the width). `src` is a +/// vector or an array; `size` nil means `(count src)`, a number pads with the +/// default or keeps the first `size` elements. The fill runs after the one +/// allocation, so no element is live across a GC. +fn arrayFromFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("__array-from", args, 3, loc); + const kind = try asKind(args[0], "__array-from", loc); + const src = args[1]; + const n_src: u32 = switch (src.tag()) { + .vector => vector_mod.count(src), + .sub_vector => sub_vector.count(src), + .array => java_array.alength(src), + else => return error_catalog.raise(.type_arg_invalid, loc, .{ .fn_name = "__array-from", .expected = "a vector or an array", .actual = @tagName(src.tag()) }), + }; + const n = if (args[2].isNil()) n_src else try asSize(args[2], "__array-from", loc); + const out = try java_array.make(rt, kind, n); + const dst = java_array.asArray(out); + const m = @min(n, n_src); + if (src.tag() == .array and java_array.asArray(src).kind == kind) { + java_array.copy(java_array.asArray(src), 0, dst, 0, m); + return out; + } + for (0..m) |i| { + const e = switch (src.tag()) { + .vector => vector_mod.nth(src, @intCast(i)), + .sub_vector => sub_vector.nth(src, @intCast(i)), + else => try java_array.get(rt, java_array.asArray(src), i), + }; + try java_array.store(dst, i, e, .convert, "array constructor", loc); + } + return out; +} + +/// `(__array-fill array v)`: every slot set to `v` (converted), returns the +/// array. The `(int-array n init)` form. +fn arrayFillFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = rt; + _ = env; + try error_catalog.checkArity("__array-fill", args, 2, loc); + try requireArray(args[0], "__array-fill", loc); + try java_array.fill(java_array.asArray(args[0]), args[1], .convert, "array constructor", loc); + return args[0]; } -/// `(aget array idx & idxs)` — element read, multi-dim by recursion exactly +/// `(__array-kind array)`: the element kind keyword (`:byte`, `:object`, …). +fn arrayKindFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("__array-kind", args, 1, loc); + try requireArray(args[0], "__array-kind", loc); + return keyword_mod.intern(rt, null, @tagName(java_array.asArray(args[0]).kind)); +} + +/// `(aget array idx & idxs)`: element read, multi-dim by recursion exactly /// as clj's variadic `aget` composes: `(aget a i j)` reads the nested array /// at `i`, then `j` within it. Implemented directly in the builtin rather /// than as a core.clj variadic (the D-446 barrier sketched that shape) so the /// hot 2-arg path stays a single builtin call. fn agetFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { - _ = rt; _ = env; try error_catalog.checkArityMin("aget", args, 2, loc); var v = args[0]; for (args[1..]) |idx| { try requireArray(v, "aget", loc); - v = try java_array.aget(v, try asIndex(idx, "aget", loc), "aget", loc); + v = try java_array.aget(rt, v, try asIndex(idx, "aget", loc), "aget", loc); } return v; } -/// `(aset array idx val)` — 3-arg in-place write, returns `val`. +/// `(aset array idx val)`: 3-arg in-place write, returns `val`. The value +/// must fit the element kind (clj's "argument type mismatch" otherwise). fn asetFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { - _ = rt; _ = env; try error_catalog.checkArityMin("aset", args, 3, loc); // `(aset a i j … v)`: every arg but the last two walks into a nested @@ -78,7 +149,7 @@ fn asetFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) any var v = args[0]; for (args[1 .. args.len - 2]) |idx| { try requireArray(v, "aset", loc); - v = try java_array.aget(v, try asIndex(idx, "aset", loc), "aset", loc); + v = try java_array.aget(rt, v, try asIndex(idx, "aset", loc), "aset", loc); } try requireArray(v, "aset", loc); return java_array.aset(v, try asIndex(args[args.len - 2], "aset", loc), args[args.len - 1], "aset", loc); @@ -110,6 +181,9 @@ const Entry = struct { name: []const u8, f: *const fn (*Runtime, *Env, []const V const ENTRIES = [_]Entry{ .{ .name = "__array-make", .f = &arrayMakeFn }, + .{ .name = "__array-from", .f = &arrayFromFn }, + .{ .name = "__array-fill", .f = &arrayFillFn }, + .{ .name = "__array-kind", .f = &arrayKindFn }, .{ .name = "aget", .f = &agetFn }, .{ .name = "aset", .f = &asetFn }, .{ .name = "alength", .f = &alengthFn }, diff --git a/src/lang/primitive/chunk_transform.zig b/src/lang/primitive/chunk_transform.zig index c89f43a0f..982423fe2 100644 --- a/src/lang/primitive/chunk_transform.zig +++ b/src/lang/primitive/chunk_transform.zig @@ -27,8 +27,10 @@ const error_mod = @import("../../runtime/error/info.zig"); const error_catalog = @import("../../runtime/error/catalog.zig"); const SourceLocation = error_mod.SourceLocation; const chunked_cons = @import("../../runtime/collection/chunked_cons.zig"); +const chunked_seq = @import("../../runtime/collection/chunked_seq.zig"); const root_set = @import("../../runtime/gc/root_set.zig"); const invokeCallable = @import("higher_order.zig").invokeCallable; +const seqable = @import("../../runtime/seqable.zig"); /// PERF: in-Zig chunk-map drain — `f` over all of `s`'s current chunk → a filled /// ChunkBuffer (the `.clj` arm does `(chunk-cons (-map-lazy f (chunk-rest @@ -37,7 +39,7 @@ pub fn chunkMapStepFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceL try error_catalog.checkArity("__chunk-map-step", args, 2, loc); const f = args[0]; const s = args[1]; - const cnt = chunked_cons.currentChunkCount(s); + const cnt = chunked_seq.chunkCount(s); // GC-ROOT: O-032 — reentrant-primitive frame [f, s, buf] (mirrors reduceFn, // ADR-0094) [ref: .dev/gc_rooting.md]. Push BEFORE the buf alloc (slot 2 nil) @@ -55,7 +57,7 @@ pub fn chunkMapStepFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceL var i: u32 = 0; while (i < cnt) : (i += 1) { - const elt = chunked_cons.currentChunkNth(s, i); + const elt = chunked_seq.chunkNth(s, i); const mapped = try invokeCallable(rt, env, f, &.{elt}, loc); // chunkAppend is non-allocating + runs AFTER the call returns, so `mapped` // is never live-across-collect unrooted. @@ -64,6 +66,34 @@ pub fn chunkMapStepFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceL return buf; } +/// `(__dorun coll)`: the 1-arity `dorun` walk, `(loop [s (seq coll)] (when s +/// (recur (next s))))`, in Zig. A chunked_cons skips its whole current chunk +/// in one step (`chunk-rest`, the elements are already realized), which is +/// what clj's `next` on a ChunkedCons amounts to for a forcing walk. Every +/// other seq is stepped with `seqable.next`, so lazy realization (and any side +/// effect it carries) runs exactly once per element, in order. Returns nil. +/// PERF: the `.clj` loop paid an interpreted `next` call per element (bench +/// vs_bb string_ops `doall (map …)`). [refs: O-077] +pub fn dorunFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + try error_catalog.checkArity("__dorun", args, 1, loc); + // GC-ROOT: frame [cursor]; `next` may realize a lazy cell (re-enters eval). + var gc_roots: [1]Value = .{.nil_val}; + var gc_sp: u16 = 1; + var gc_frame: root_set.EvalFrame = .{ .stack = &gc_roots, .sp = &gc_sp, .locals = &.{}, .parent = root_set.eval_frame_head }; + root_set.eval_frame_head = &gc_frame; + defer root_set.eval_frame_head = gc_frame.parent; + + gc_roots[0] = try seqable.seq(rt, env, args[0], loc); + while (gc_roots[0].tag() != .nil) { + if (gc_roots[0].tag() == .chunked_cons and chunked_cons.currentChunkCount(gc_roots[0]) > 0) { + gc_roots[0] = try seqable.seq(rt, env, chunked_cons.chunkRest(gc_roots[0]), loc); + } else { + gc_roots[0] = try seqable.next(rt, env, gc_roots[0], loc); + } + } + return .nil_val; +} + /// PERF: in-Zig chunk-filter drain — `pred` over all of `s`'s current chunk → /// a ChunkBuffer of survivors (partial; empty → the `.clj` `chunk-cons` returns /// the tail directly). [refs: O-032, O-004] @@ -71,7 +101,7 @@ pub fn chunkFilterStepFn(rt: *Runtime, env: *Env, args: []const Value, loc: Sour try error_catalog.checkArity("__chunk-filter-step", args, 2, loc); const pred = args[0]; const s = args[1]; - const cnt = chunked_cons.currentChunkCount(s); + const cnt = chunked_seq.chunkCount(s); // GC-ROOT: O-032 — frame [pred, s, buf]; same discipline as chunkMapStepFn. var gc_roots: [3]Value = .{ pred, s, .nil_val }; @@ -85,7 +115,7 @@ pub fn chunkFilterStepFn(rt: *Runtime, env: *Env, args: []const Value, loc: Sour var i: u32 = 0; while (i < cnt) : (i += 1) { - const elt = chunked_cons.currentChunkNth(s, i); + const elt = chunked_seq.chunkNth(s, i); if ((try invokeCallable(rt, env, pred, &.{elt}, loc)).isTruthy()) { _ = chunked_cons.chunkAppend(buf, elt); } diff --git a/src/lang/primitive/collection.zig b/src/lang/primitive/collection.zig index 7e0bfeece..14222aa57 100644 --- a/src/lang/primitive/collection.zig +++ b/src/lang/primitive/collection.zig @@ -505,9 +505,9 @@ pub fn getFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) .array => blk: { if (k.tag() != .integer) break :blk default; const idx = k.asInteger(); - const items = java_array.asArray(coll).items(); - if (idx < 0 or idx >= items.len) break :blk default; - break :blk items[@intCast(idx)]; + const arr = java_array.asArray(coll); + if (idx < 0 or idx >= arr.len) break :blk default; + break :blk try java_array.get(rt, arr, @intCast(idx)); }, // A String is index-gettable (clj `(get "abc" 1)` → \b): return the // codepoint char, else default (OOR / non-integer key). D-217. @@ -652,7 +652,7 @@ pub fn nthFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) if (has_default) break :blk default; break :blk error_catalog.raise(.index_out_of_range, loc, .{ .fn_name = "nth" }); } - break :blk array_seq.nth(coll, @intCast(idx)); + break :blk try array_seq.nth(rt, coll, @intCast(idx)); }, // `nth` into UTF-8 is a forward walk by nature, so this arm is O(idx) // rather than the vector view's O(1). The bounds check rides on that @@ -688,7 +688,7 @@ pub fn nthFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) if (has_default) break :blk default; break :blk error_catalog.raise(.index_out_of_range, loc, .{ .fn_name = "nth" }); } - break :blk try java_array.aget(coll, idx, "nth", loc); + break :blk try java_array.aget(rt, coll, idx, "nth", loc); }, // A live transient vector is Indexed (clj parity, D-199) — same // index discipline as a persistent vector. @@ -1291,6 +1291,28 @@ pub fn subvecFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocatio return sub_vector.make(rt, v, start, end); } +/// `(__subvec-fast v start end)` — the common `subvec` call answered in one +/// native step: `v` a vector (or subvec view), `start`/`end` immediate +/// integers with `0 <= start <= end <= (count v)` → the O(1) view. Any other +/// shape (nil, a non-vector, a float / ratio / non-number index, an +/// out-of-bounds range) answers nil, and `clojure.core/subvec` falls through +/// to its full coercion + throwing path. A real subvec is never nil, so nil +/// is an unambiguous "not handled". +/// PERF: the wrapper otherwise allocated an `->idx` closure and ran ~10 +/// interpreted checks per call (bench/vs_bb vector_assoc). [refs: O-073] +pub fn subvecFastFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("__subvec-fast", args, 3, loc); + const v = args[0]; + if (v.tag() != .vector and v.tag() != .sub_vector) return .nil_val; + if (!args[1].isInt() or !args[2].isInt()) return .nil_val; + const start = args[1].asInteger(); + const end = args[2].asInteger(); + const c: i64 = if (v.tag() == .vector) vector.count(v) else sub_vector.count(v); + if (start < 0 or end < start or c < end) return .nil_val; + return sub_vector.make(rt, v, @intCast(start), @intCast(end)); +} + /// `#queue (e1 e2 …)` data-reader (ADR-0087): build a queue by conj-ing the /// form's elements in order. cljw extension (clj has no `#queue` reader); makes /// the cljw print form reader-round-trippable. @@ -1366,6 +1388,7 @@ const ENTRIES = [_]Entry{ .{ .name = "__queue-pop", .f = &queuePopFn }, .{ .name = "__vector-pop", .f = &vectorPopFn }, .{ .name = "__subvec", .f = &subvecFn }, + .{ .name = "__subvec-fast", .f = &subvecFastFn }, .{ .name = "disj", .f = &disjFn }, .{ .name = "contains?", .f = &containsQFn }, .{ .name = "get", .f = &getFn }, diff --git a/src/lang/primitive/core.zig b/src/lang/primitive/core.zig index 96f2a0c7a..ca2d3c103 100644 --- a/src/lang/primitive/core.zig +++ b/src/lang/primitive/core.zig @@ -945,6 +945,28 @@ pub fn strFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) const s = std.fmt.bufPrint(&dbuf, "{d}", .{args[0].asInteger()}) catch unreachable; return try string_mod.alloc(rt, s); } + // PERF: when every arg is a string / immediate integer / nil, render into + // a stack buffer: no heap writer alloc+free, no per-arg writeStrValue + // switch. Output is byte-identical to writeStrValue for those tags (bare + // string, decimal int, nil → ""). Anything else, or an overflow of the + // buffer, takes the general path below. [refs: O-075] + simple: { + var sbuf: [256]u8 = undefined; + var len: usize = 0; + for (args) |arg| { + if (arg.tag() == .string) { + const s = string_mod.asString(arg); + if (s.len > sbuf.len - len) break :simple; + @memcpy(sbuf[len..][0..s.len], s); + len += s.len; + } else if (arg.isInt()) { + if (sbuf.len - len < 24) break :simple; + const s = std.fmt.bufPrint(sbuf[len..], "{d}", .{arg.asInteger()}) catch unreachable; + len += s.len; + } else if (arg.tag() != .nil) break :simple; + } + return try string_mod.alloc(rt, sbuf[0..len]); + } var aw: std.Io.Writer.Allocating = .init(rt.gpa); defer aw.deinit(); // Per-value str rendering lives in `print.writeStrValue` — the single @@ -1221,6 +1243,10 @@ pub fn formatFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocatio defer stmp.deinit(); if (args[src].tag() == .nil) { try stmp.writer.writeAll("null"); + } else if (args[src].tag() == .big_int or args[src].tag() == .big_decimal) { + // JVM `%s` calls `.toString`: plain digits, no `N`/`M` + // reader suffix (`(format "%s" 1N)` → "1"). + try print_mod.writeStrValue(rt, env, &stmp.writer, args[src]); } else { try writeArgsSpaced(rt, env, &stmp.writer, args[src .. src + 1], false); } diff --git a/src/lang/primitive/edn.zig b/src/lang/primitive/edn.zig index 9906aeba8..deb458095 100644 --- a/src/lang/primitive/edn.zig +++ b/src/lang/primitive/edn.zig @@ -53,7 +53,7 @@ const host_stream = @import("../../runtime/io/host_stream.zig"); /// `clojure.core/read-string` — EOF on empty input is an ERROR (clj: /// `RuntimeException: EOF while reading`). pub fn readStringFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { - return readStringImpl(rt, env, args, loc, .raise); + return readStringImpl(rt, env, args, loc, .raise, .clojure); } /// `clojure.edn/read-string` — EOF on empty input is nil, NOT an error. @@ -61,14 +61,21 @@ pub fn readStringFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLoc /// `(core/read-string "")` throws. The two readers differ on exactly this /// input, which is why they cannot share one entry point (D-581; D-269 was /// discharged against the core behaviour and made edn throw too). +/// It reads clj's EdnReader dialect (ADR-0200 Layer B), so `` ` `` `~` `@` +/// `#'` `#(` `#"` `#=` `#?` `#!` `::kw` `#::{}` are errors, `'a` is a +/// symbol, and `#ns.Rec{}` resolves only through `:readers` / `:default`. +/// A nil source reads as nil (clj: `(when s ...)`). +/// JVM reference: clojure.edn/read-string in clojure/edn.clj, over +/// clojure.lang.EdnReader. pub fn readStringEdnFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { - return readStringImpl(rt, env, args, loc, .nil_on_eof); + if (args.len >= 1 and args.len <= 2 and args[args.len - 1].tag() == .nil) return Value.nil_val; + return readStringImpl(rt, env, args, loc, .nil_on_eof, .edn); } /// What an empty / whitespace-only / comment-only source means to the caller. const EofPolicy = enum { raise, nil_on_eof }; -fn readStringImpl(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation, eof_policy: EofPolicy) anyerror!Value { +fn readStringImpl(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation, eof_policy: EofPolicy, dialect: reader_mod.Dialect) anyerror!Value { try error_catalog.checkArityRange("read-string", args, 1, 2, loc); const opts: ?Value = if (args.len == 2) args[0] else null; // clojure.edn/read-string's single-argument overload uses a nil EOF @@ -86,7 +93,7 @@ fn readStringImpl(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocat }); } const source = string_collection.asString(str_arg); - return readSource(rt, env, opts, source, loc, eof_policy, null); + return readSource(rt, env, opts, source, loc, eof_policy, null, dialect); } /// Read one form and advance the reader by exactly the consumed bytes. @@ -105,13 +112,13 @@ pub fn readEdnFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocati else => return error_catalog.raise(.edn_string_invalid, loc, .{ .reason = "options must be a map" }), }; if (try text_io.remainingReader(rt, input)) |source| - return readSource(rt, env, opts, source, loc, .raise, input); + return readSource(rt, env, opts, source, loc, .raise, input, .edn); if (try host_stream.remainingReader(rt, input)) |source| - return readSource(rt, env, opts, source, loc, .raise, input); + return readSource(rt, env, opts, source, loc, .raise, input, .edn); return error_catalog.raise(.type_arg_invalid, loc, .{ .fn_name = "read", .expected = "a PushbackReader", .actual = @tagName(input.tag()) }); } -fn readSource(rt: *Runtime, env: *Env, opts: ?Value, source: []const u8, loc: SourceLocation, eof_policy: EofPolicy, stream: ?Value) anyerror!Value { +fn readSource(rt: *Runtime, env: *Env, opts: ?Value, source: []const u8, loc: SourceLocation, eof_policy: EofPolicy, stream: ?Value, dialect: reader_mod.Dialect) anyerror!Value { // EOF (no form in `source`): clj THROWS `RuntimeException: EOF while // reading` UNLESS an `:eof` opt is supplied, in which case that value is // returned (D-269 — was a silent nil-default that masked "nothing to read"). @@ -160,13 +167,19 @@ fn readSource(rt: *Runtime, env: *Env, opts: ?Value, source: []const u8, loc: So if (rc == try keyword.intern(rt, null, "allow")) allow_cond = true; } } - var reader = reader_mod.Reader.init(arena.allocator(), source); + var reader = reader_mod.Reader.initDialect(arena.allocator(), source, dialect); reader.allow_reader_cond = allow_cond; // ADR-0200: every form is lifted with formToValueStrict below, which // checks duplicate keys by value and in source order, so the reader's // Form-level approximation stays off. reader.form_checks = false; + // ADR-0200 A4: a `#_` target is lifted (its tag readers run, its + // duplicates raise) the moment it is read, inside this binding frame, + // so the source-order error is the one raised, then dropped. + var discard: DiscardLift = .{ .rt = rt, .env = env, .dialect = dialect }; + reader.discard_hook = .{ .ctx = &discard, .f = DiscardLift.hook }; const form_opt = reader.read() catch |e| { + if (discard.err) |de| return de; // A specific reader diagnostic (e.g. reader_cond_not_allowed, or an // IllegalArgumentException such as a duplicate key) carries its own // catalog Info; re-raise it so the message and kind survive rather @@ -191,9 +204,36 @@ fn readSource(rt: *Runtime, env: *Env, opts: ?Value, source: []const u8, loc: So if (eof_policy == .nil_on_eof and opts == null) return Value.nil_val; return error_catalog.raise(.eof_unexpected, loc, .{}); }; - return try analyzer_mod.formToValueStrict(rt, env, form); + return try liftData(rt, env, form, dialect); } +/// The data lift for `dialect`: EDN never reads `#ns.Rec{}` as a record. +fn liftData(rt: *Runtime, env: *Env, form: reader_mod.ReadForm, dialect: reader_mod.Dialect) anyerror!Value { + return switch (dialect) { + .clojure => analyzer_mod.formToValueStrict(rt, env, form), + .edn => analyzer_mod.formToValueEdn(rt, env, form), + }; +} + +/// Reader discard hook state. The lift can fail with errors outside the +/// reader's error set (a user reader fn's thrown value), so the hook parks +/// the real error here and aborts the read with a placeholder that +/// `readSource` swaps back. +const DiscardLift = struct { + rt: *Runtime, + env: *Env, + dialect: reader_mod.Dialect, + err: ?anyerror = null, + + fn hook(ctx: *anyopaque, form: reader_mod.ReadForm) reader_mod.ReadError!void { + const self: *DiscardLift = @ptrCast(@alignCast(ctx)); + _ = liftData(self.rt, self.env, form, self.dialect) catch |e| { + self.err = e; + return error.SyntaxError; + }; + } +}; + /// Look up `:key` in an opts map, returning the bound value or null when /// absent / nil. Returns null (no error) if `opts` is not a map, so a /// non-map opts is simply treated as "no options". diff --git a/src/lang/primitive/json.zig b/src/lang/primitive/json.zig index 99c9a29d1..4a0950b8a 100644 --- a/src/lang/primitive/json.zig +++ b/src/lang/primitive/json.zig @@ -43,6 +43,7 @@ const map_collection = @import("../../runtime/collection/map.zig"); const sorted_collection = @import("../../runtime/collection/sorted.zig"); const print = @import("../../runtime/print.zig"); const big_int_mod = @import("../../runtime/numeric/big_int.zig"); +const decimal_text = @import("../../runtime/numeric/decimal_text.zig"); const ratio_mod = @import("../../runtime/numeric/ratio.zig"); const keyword_mod = @import("../../runtime/keyword.zig"); @@ -240,9 +241,9 @@ fn cwToJson(v: Value, w: *std.Io.Writer, opts: WriteOpts) anyerror!void { // formatter rather than Zig's `{d}` (D-171, sibling of D-166). .float => try print.printFloat(w, v.asFloat()), // JVM data.json writes a BigInt via `(str x)` → plain digits, NO - // `N` suffix. `Managed.format` (`{f}`) renders just the digits + // `N` suffix. `writeDecimal` renders just the digits, no size ceiling // (printBigInt adds the `N` for pr-str; JSON must not). (D-182) - .big_int => try w.print("{f}", .{big_int_mod.asManaged(v)}), + .big_int => try decimal_text.writeDecimal(w, big_int_mod.asManaged(v)), // JVM data.json writes a Ratio as `(double x)`, its `Ratio.doubleValue` // (1/2 → 0.5, 1/7 → 0.1428571428571429). .ratio => try print.printFloat(w, try ratio_mod.toF64(v)), diff --git a/src/lang/primitive/map_step.zig b/src/lang/primitive/map_step.zig new file mode 100644 index 000000000..79d6cfa2f --- /dev/null +++ b/src/lang/primitive/map_step.zig @@ -0,0 +1,70 @@ +// SPDX-License-Identifier: EPL-2.0 +//! The body of one lazy step of multi-collection `map` (ADR-0205). +//! +//! clj's 2- and 3-coll `map` is deliberately UNCHUNKED: each lazy step seqs +//! every source, calls `f` once on the heads and conses the result onto a lazy +//! tail `(map f (rest s1) (rest s2))`. cljw keeps that shape exactly (one `f` +//! call per realized node, the same order: seq all, f on the firsts, then the +//! rests), but runs the body in Zig instead of interpreting a let / and / seq / +//! first / rest / cons tree per element. The lazy node and the recursion stay +//! in core.clj (`-map-n-lazy`), which passes itself as `self`, so laziness, +//! realization counts and exceptions are unchanged. +//! +//! ## Backend: impl-only (no surface delegation) +//! Impl deps: seqable (Layer 0), list.consHeap, higher_order.invokeCallable +//! Clojure peer: -map-n-lazy (core.clj) + +const Value = @import("../../runtime/value/value.zig").Value; +const Runtime = @import("../../runtime/runtime.zig").Runtime; +const env_mod = @import("../../runtime/env.zig"); +const Env = env_mod.Env; +const error_catalog = @import("../../runtime/error/catalog.zig"); +const SourceLocation = @import("../../runtime/error/info.zig").SourceLocation; +const seqable = @import("../../runtime/seqable.zig"); +const list = @import("../../runtime/collection/list.zig"); +const root_set = @import("../../runtime/gc/root_set.zig"); +const invokeCallable = @import("higher_order.zig").invokeCallable; + +const MAX_COLLS = 3; + +/// `(cljw.internal/__map-step f self c1 c2 [c3])`: nil when any source is +/// empty, else `(cons (f (first s1) ..) (self f (rest s1) ..))`. +/// PERF: the per-element step of multi-coll map in Zig instead of an interpreted body [refs: O-062] +fn mapStepFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + if (args.len < 4 or args.len > 2 + MAX_COLLS) { + return error_catalog.raise(.arity_not_expected, loc, .{ .fn_name = "__map-step", .expected = 4, .got = args.len }); + } + const n = args.len - 2; + // GC-ROOT: A2 reentrant-primitive frame (ADR-0094): slots [s1..sn x1..xn v] + // keep every seq cursor, head, rest and the mapped value alive across the + // user `f`, the lazy forces inside seq/first/rest and the `self` call; the + // args (f, self, colls) ride as constants [ref: .dev/gc_rooting.md]. + var roots: [2 * MAX_COLLS + 1]Value = @splat(Value.nil_val); + var sp: u16 = roots.len; + var frame: root_set.EvalFrame = .{ .stack = &roots, .sp = &sp, .locals = &.{}, .constants = args, .parent = root_set.eval_frame_head }; + root_set.eval_frame_head = &frame; + defer root_set.eval_frame_head = frame.parent; + + const ss = roots[0..n]; + const xs = roots[MAX_COLLS .. MAX_COLLS + n]; + const v_slot = &roots[2 * MAX_COLLS]; + for (args[2..], 0..) |c, i| { + ss[i] = try seqable.seq(rt, env, c, loc); + if (ss[i].isNil()) return .nil_val; + } + for (ss, 0..) |s, i| xs[i] = try seqable.first(rt, env, s, loc); + v_slot.* = try invokeCallable(rt, env, args[0], xs, loc); + + // `(self f (rest s1) ..)`: the rests replace the heads in place (the heads + // are dead once f has run). + for (ss, 0..) |s, i| xs[i] = try seqable.rest(rt, env, s, loc); + var call: [1 + MAX_COLLS]Value = undefined; + call[0] = args[0]; + for (xs, 0..) |r, i| call[1 + i] = r; + ss[0] = try invokeCallable(rt, env, args[1], call[0 .. 1 + n], loc); // the lazy tail, rooted + return try list.consHeap(rt, v_slot.*, ss[0]); +} + +pub fn register(env: *Env, ns: *env_mod.Namespace) !void { + _ = try env.intern(ns, "__map-step", Value.initBuiltinFn(&mapStepFn), null); +} diff --git a/src/lang/primitive/math.zig b/src/lang/primitive/math.zig index 3422c6fa8..c7a3468df 100644 --- a/src/lang/primitive/math.zig +++ b/src/lang/primitive/math.zig @@ -240,52 +240,34 @@ fn rationalize(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation /// which makes the result an integer with no ratio to reduce. Going via a float /// would lose precision that the BigDecimal was chosen to keep. fn bigDecRationalize(rt: *Runtime, v: Value, loc: SourceLocation) anyerror!Value { + _ = loc; const bd = v.decodePtr(*const big_decimal_mod.BigDecimal); - // Unscaled digits, sign included. The bound mirrors the float path's - // buffers; a wider significand than this raises rather than truncating. - var digitbuf: [512]u8 = undefined; - var dw: std.Io.Writer = .fixed(&digitbuf); - dw.print("{f}", .{bd.unscaled.m}) catch - return error_catalog.raise(.arg_value_invalid, loc, .{ .fn_name = "rationalize", .expected = "a BigDecimal within rationalize's digit range", .actual = "too many significand digits" }); - const digits = dw.buffered(); + const Managed = std.math.big.int.Managed; + // 10^|scale| computed exactly, no digit-string round-trip and so no size + // ceiling (CLJW-BIGINT-STR-PLACEHOLDER sweep). + var pow_m = try Managed.initSet(rt.gc.infra, 10); + defer pow_m.deinit(); + const mag: u32 = @intCast(@abs(@as(i64, bd.scale))); + try pow_m.pow(&pow_m, mag); if (bd.scale <= 0) { - const zeros: usize = @intCast(-@as(i64, bd.scale)); - var intbuf: [1040]u8 = undefined; - if (digits.len + zeros > intbuf.len) - return error_catalog.raise(.arg_value_invalid, loc, .{ .fn_name = "rationalize", .expected = "a BigDecimal within rationalize's digit range", .actual = "too many trailing zeros" }); - @memcpy(intbuf[0..digits.len], digits); - @memset(intbuf[digits.len .. digits.len + zeros], '0'); - var int_m = try big_int_mod.parseBase10(rt, intbuf[0 .. digits.len + zeros]); + var int_m = try Managed.init(rt.gc.infra); defer int_m.deinit(); + try int_m.mul(bd.unscaled.m, &pow_m); return promote.wrapManaged(rt, &int_m); } - var num_m = try big_int_mod.parseBase10(rt, digits); - defer num_m.deinit(); - const scale: usize = @intCast(bd.scale); - // denominator = 10^scale, written as "1" followed by `scale` zeros. - var denbuf: [1040]u8 = undefined; - if (1 + scale > denbuf.len) - return error_catalog.raise(.arg_value_invalid, loc, .{ .fn_name = "rationalize", .expected = "a BigDecimal within rationalize's scale range", .actual = "scale too large" }); - denbuf[0] = '1'; - @memset(denbuf[1 .. 1 + scale], '0'); - var den_m = try big_int_mod.parseBase10(rt, denbuf[0 .. 1 + scale]); - defer den_m.deinit(); - - if (try ratio_mod.allocFromManagedPair(rt, &num_m, &den_m)) |ratio_v| return ratio_v; + if (try ratio_mod.allocFromManagedPair(rt, bd.unscaled.m, &pow_m)) |ratio_v| return ratio_v; // Null means the reduction collapsed to an integer, and the caller emits // it. The quotient has to be computed here: `allocFromManagedPair` takes - // its pair as `const`, so `num_m` is still the UNREDUCED numerator, and - // wrapping it answers 10 for 1.0M. A BigDecimal reaches this arm whenever - // its trailing digits are zeros (1.0M, 1.00M, 10.0M), where the float path - // above cannot: a normalised float render carries no trailing zero, so its - // numerator is never divisible by 10^scale. - var quot_m = try std.math.big.int.Managed.init(rt.gc.infra); + // its pair as `const`, so the numerator is still the UNREDUCED unscaled + // value, and wrapping it answers 10 for 1.0M. A BigDecimal reaches this + // arm whenever its trailing digits are zeros (1.0M, 1.00M, 10.0M). + var quot_m = try Managed.init(rt.gc.infra); defer quot_m.deinit(); - var rem_m = try std.math.big.int.Managed.init(rt.gc.infra); + var rem_m = try Managed.init(rt.gc.infra); defer rem_m.deinit(); - try quot_m.divTrunc(&rem_m, &num_m, &den_m); + try quot_m.divTrunc(&rem_m, bd.unscaled.m, &pow_m); return promote.wrapManaged(rt, "_m); } diff --git a/src/lang/primitive/protocol.zig b/src/lang/primitive/protocol.zig index fddfdceac..7ad34eefc 100644 --- a/src/lang/primitive/protocol.zig +++ b/src/lang/primitive/protocol.zig @@ -1054,7 +1054,10 @@ test "__extend-type! appends method_val rows + bumps protocol_generation" { } try testing.expectEqual(@intFromEnum(td_ref), @intFromEnum(result)); - try testing.expectEqual(gen_before +% 1, fix.rt.protocol_generation); + // Two bumps: the method-table write, then addProtocolImpl's membership + // write (ADR-0206: the ProtocolFn cache must not memoise the state + // between them). + try testing.expectEqual(gen_before +% 2, fix.rt.protocol_generation); try testing.expectEqual(@as(usize, 1), td.method_table.len); // addProtocolImpl recorded "P" in the declared-interface list. try testing.expectEqual(@as(usize, 1), td.protocol_impls.len); diff --git a/src/lang/primitive/sequence.zig b/src/lang/primitive/sequence.zig index e3f026a3b..4b7b440c5 100644 --- a/src/lang/primitive/sequence.zig +++ b/src/lang/primitive/sequence.zig @@ -48,6 +48,7 @@ const sorted = @import("../../runtime/collection/sorted.zig"); const set = @import("../../runtime/collection/set.zig"); const string_collection = @import("../../runtime/collection/string.zig"); const chunked_cons = @import("../../runtime/collection/chunked_cons.zig"); +const chunked_seq = @import("../../runtime/collection/chunked_seq.zig"); const chunk_transform = @import("chunk_transform.zig"); const range = @import("../../runtime/collection/range.zig"); const array_seq = @import("../../runtime/collection/array_seq.zig"); @@ -523,28 +524,27 @@ fn chunkedSeqQFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocati _ = rt; _ = env; try error_catalog.checkArity("chunked-seq?", args, 1, loc); - return if (chunked_cons.isChunked(args[0])) Value.true_val else Value.false_val; + return if (chunked_seq.isChunked(args[0])) Value.true_val else Value.false_val; } fn chunkCountFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { _ = rt; _ = env; try error_catalog.checkArity("__chunk-count", args, 1, loc); - return Value.initInteger(@intCast(chunked_cons.currentChunkCount(args[0]))); + return Value.initInteger(@intCast(chunked_seq.chunkCount(args[0]))); } fn chunkNthFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { _ = rt; _ = env; try error_catalog.checkArity("__chunk-nth", args, 2, loc); - return chunked_cons.currentChunkNth(args[0], @intCast(args[1].asInteger())); + return chunked_seq.chunkNth(args[0], @intCast(args[1].asInteger())); } fn chunkRestFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { - _ = rt; _ = env; try error_catalog.checkArity("chunk-rest", args, 1, loc); - return chunked_cons.chunkRest(args[0]); + return try chunked_seq.chunkRest(rt, args[0]); } fn chunkBufferFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { @@ -594,6 +594,7 @@ const ENTRIES = [_]Entry{ // O-032: in-Zig chunk-map/filter drain (chunk_transform.zig) — the // producer-side analogue of reduceFn's O-004 chunk drain. .{ .name = "__chunk-map-step", .f = &chunk_transform.chunkMapStepFn }, + .{ .name = "__dorun", .f = &chunk_transform.dorunFn }, .{ .name = "__chunk-filter-step", .f = &chunk_transform.chunkFilterStepFn }, }; diff --git a/src/lang/primitive/string.zig b/src/lang/primitive/string.zig index 5c949c341..80610fe81 100644 --- a/src/lang/primitive/string.zig +++ b/src/lang/primitive/string.zig @@ -30,6 +30,8 @@ const regex_replace = @import("../../runtime/regex/replace.zig"); const regex_prim = @import("regex.zig"); const print_mod = @import("../../runtime/print.zig"); const char_sequence = @import("../../runtime/char_sequence.zig"); +const seqable = @import("../../runtime/seqable.zig"); +const root_set = @import("../../runtime/gc/root_set.zig"); /// `(clojure.string/upper-case s)` — ASCII upper-case fold. /// Non-ASCII codepoints pass through unchanged; full Unicode case @@ -571,37 +573,71 @@ pub fn splitLines(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocat return result; } -/// `-join` leaf — the vector+string fast path under the public -/// `(clojure.string/join …)` Pattern A `.clj` defn. `coll` must be a -/// `vector` and elements must be strings; other coll/element shapes -/// raise `feature_not_supported` (the `.clj` defn handles the general -/// seq + `str`-coercion arms). Kept as an opt-in fast leaf. +/// `-join` leaf: the whole of `(clojure.string/join coll)` / +/// `(clojure.string/join sep coll)`. clj's body is a StringBuilder loop over +/// `(seq coll)` appending `(str x)` and `(str sep)` between elements; this is +/// that loop in Zig. `coll` is any seqable (vector, list, lazy seq, string, +/// map, set, a custom Seqable); `sep` and every element render through +/// `print.writeStrValue`, the same renderer `str` uses, so nil renders "" and +/// a char / keyword / number renders as `str` would. A non-seqable `coll` +/// raises through `seqable.seq` exactly like `(seq coll)`. +/// +/// PERF: the old Pattern A body `(apply str (interpose sep coll))` built a +/// lazy interpose seq (a closure call + lazy cell per element) and then an +/// `apply` arg spill; that was 87% of bench/vs_bb string_ops. [refs: O-070] pub fn join(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { - _ = env; try error_catalog.checkArityRange("join", args, 1, 2, loc); - const sep: []const u8 = if (args.len == 2) blk: { - if (args[0].tag() != .string) - return error_catalog.raise(.type_arg_not_string, loc, .{ .fn_name = "join", .actual = @tagName(args[0].tag()) }); - break :blk string_collection.asString(args[0]); - } else ""; - const coll = if (args.len == 2) args[1] else args[0]; - - if (coll.tag() != .vector) - return error_catalog.raise(.type_arg_invalid, loc, .{ .fn_name = "clojure.string/join", .expected = "a seqable collection", .actual = @tagName(args[args.len - 1].tag()) }); - - const n = vector_collection.count(coll); - var out: std.ArrayList(u8) = .empty; - defer out.deinit(rt.gc.infra); - - var i: u32 = 0; - while (i < n) : (i += 1) { - const elt = vector_collection.nth(coll, i); - if (elt.tag() != .string) - return error_catalog.raise(.type_arg_invalid, loc, .{ .fn_name = "clojure.string/join", .expected = "a printable element", .actual = "an unprintable element" }); - if (i > 0 and sep.len > 0) try out.appendSlice(rt.gc.infra, sep); - try out.appendSlice(rt.gc.infra, string_collection.asString(elt)); + var out: std.Io.Writer.Allocating = .init(rt.gc.infra); + defer out.deinit(); + + // The separator is rendered once up front (clj `(str separator)` per gap + // gives the same text every time). + var sep_aw: std.Io.Writer.Allocating = .init(rt.gc.infra); + defer sep_aw.deinit(); + if (args.len == 2) try print_mod.writeStrValue(rt, env, &sep_aw.writer, args[0]); + const sep = sep_aw.writer.buffered(); + + // GC-ROOT: A14 — [cursor, element]. `seq`/`next` may build fresh views or + // realize a lazy seq, and rendering an instance's toString re-enters eval; + // either can collect while the cursor is held only here. + // [ref: .dev/gc_rooting.md §A] + var roots = [_]Value{ .nil_val, .nil_val }; + var sp: u16 = roots.len; + var frame: root_set.EvalFrame = .{ .stack = &roots, .sp = &sp, .locals = &.{}, .parent = root_set.eval_frame_head }; + root_set.eval_frame_head = &frame; + defer root_set.eval_frame_head = frame.parent; + + const coll = args[args.len - 1]; + switch (coll.tag()) { + // Indexed fast path: no seq view, no per-element cursor allocation. + .vector => { + const n = vector_collection.count(coll); + var i: u32 = 0; + while (i < n) : (i += 1) { + if (i > 0) try out.writer.writeAll(sep); + const elt = vector_collection.nth(coll, i); + if (elt.tag() == .string) + try out.writer.writeAll(string_collection.asString(elt)) + else + try print_mod.writeStrValue(rt, env, &out.writer, elt); + } + }, + else => { + roots[0] = try seqable.seq(rt, env, coll, loc); + var first_elt = true; + while (roots[0].tag() != .nil) { + if (!first_elt) try out.writer.writeAll(sep); + first_elt = false; + roots[1] = try seqable.first(rt, env, roots[0], loc); + if (roots[1].tag() == .string) + try out.writer.writeAll(string_collection.asString(roots[1])) + else + try print_mod.writeStrValue(rt, env, &out.writer, roots[1]); + roots[0] = try seqable.next(rt, env, roots[0], loc); + } + }, } - return try string_collection.alloc(rt, out.items); + return try string_collection.alloc(rt, out.writer.buffered()); } // --- registration --- diff --git a/src/main.zig b/src/main.zig index e159c07b1..89238d4f4 100644 --- a/src/main.zig +++ b/src/main.zig @@ -104,8 +104,13 @@ test { _ = @import("runtime/gc/arena.zig"); _ = @import("runtime/gc/tag_ops.zig"); _ = @import("runtime/gc/gc_heap.zig"); + _ = @import("runtime/gc/gc_heap_test.zig"); _ = @import("runtime/gc/mark_sweep.zig"); - _ = @import("runtime/gc/free_pool.zig"); + _ = @import("runtime/gc/mark_sweep_test.zig"); + _ = @import("runtime/gc/gc_trigger.zig"); + _ = @import("runtime/gc/gc_types.zig"); + _ = @import("runtime/gc/conservative_scan.zig"); + _ = @import("runtime/gc/page_heap.zig"); _ = @import("runtime/gc/root_set.zig"); _ = @import("runtime/gc/gc_torture.zig"); _ = @import("runtime/concurrency/io_default.zig"); @@ -126,6 +131,7 @@ test { _ = @import("runtime/collection/map.zig"); _ = @import("runtime/collection/set.zig"); _ = @import("runtime/collection/chunked_cons.zig"); + _ = @import("runtime/collection/chunked_seq.zig"); _ = @import("runtime/collection/range.zig"); _ = @import("runtime/collection/persistent_queue.zig"); _ = @import("runtime/collection/transient/transient_vector.zig"); diff --git a/src/runtime/charset.zig b/src/runtime/charset.zig index 3c6b00b05..239462f97 100644 --- a/src/runtime/charset.zig +++ b/src/runtime/charset.zig @@ -69,12 +69,27 @@ pub fn substring(s: []const u8, start: usize, end: usize) Utf8Error![]const u8 { return s[byte_start..byte_end]; } +/// True when every byte is < 0x80 (vectorised by the optimiser). +fn isAllAscii(s: []const u8) bool { + var acc: u8 = 0; + for (s) |c| acc |= c; + return acc < 0x80; +} + /// Allocate the FULL-Unicode uppercase of `s` (JVM String.toUpperCase /// semantics, D-057): SpecialCasing 1:n expansions (ß→SS, fi→FI, ʼn→ʼN) /// then the simple map, via the generated `unicode_case` tables. The /// result may be LONGER than the input (1:n), so it grows an ArrayList. /// Caller owns the returned slice. pub fn upperCaseAlloc(alloc: std.mem.Allocator, s: []const u8) ![]u8 { + // PERF: an all-ASCII input folds byte-for-byte (every ASCII codepoint's + // full Unicode mapping is its ASCII fold, one byte to one byte), so skip + // the per-codepoint table walk. [refs: O-071] + if (isAllAscii(s)) { + const buf = try alloc.alloc(u8, s.len); + for (s, buf) |c, *o| o.* = std.ascii.toUpper(c); + return buf; + } var out: std.ArrayList(u8) = .empty; errdefer out.deinit(alloc); var iter = std.unicode.Utf8Iterator{ .bytes = s, .i = 0 }; @@ -94,6 +109,12 @@ pub fn upperCaseAlloc(alloc: std.mem.Allocator, s: []const u8) ![]u8 { /// the table's has-a-mapping approximation (full General_Category data is /// not carried — covers Greek/Latin, the rule's real population). pub fn lowerCaseAlloc(alloc: std.mem.Allocator, s: []const u8) ![]u8 { + // PERF: all-ASCII fast path; no Σ can occur in ASCII. [refs: O-071] + if (isAllAscii(s)) { + const buf = try alloc.alloc(u8, s.len); + for (s, buf) |c, *o| o.* = std.ascii.toLower(c); + return buf; + } var out: std.ArrayList(u8) = .empty; errdefer out.deinit(alloc); var iter = std.unicode.Utf8Iterator{ .bytes = s, .i = 0 }; diff --git a/src/runtime/cljw/ffi/surface.zig b/src/runtime/cljw/ffi/surface.zig index 8df5ab54e..8d6aa7238 100644 --- a/src/runtime/cljw/ffi/surface.zig +++ b/src/runtime/cljw/ffi/surface.zig @@ -150,6 +150,19 @@ fn addressOf(v: Value) ?usize { return @bitCast(i); } +/// A `:pointer` argument: an address, or a primitive array passed as the +/// address of its storage (no copy; C reads and writes the elements in place, +/// valid for the call). The array is an argument, so it stays rooted, and the +/// collector never moves it. +fn pointerArg(v: Value) ?usize { + if (java_array.isArray(v)) { + const a = java_array.asArray(v); + if (a.kind == .object) return null; + return @intFromPtr(a.data); + } + return addressOf(v); +} + // ── Leaves ────────────────────────────────────────────────────────────────── fn openFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { @@ -231,18 +244,23 @@ fn prepareFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) } /// Scratch copies that must outlive the call and no longer: `:string` and -/// `:bytes` arguments. +/// `:bytes` arguments. Raw allocations: each is overwritten whole at once, +/// so the safe-build `undefined` fill `Allocator.alloc`/`free` add would be +/// two more passes over every byte. const Temps = struct { bufs: [abi.MAX_INT_ARGS][]u8 = undefined, n: usize = 0, - fn keep(t: *Temps, b: []u8) void { - t.bufs[t.n] = b; + fn alloc(t: *Temps, gpa: std.mem.Allocator, n: usize) ![]u8 { + const len = @max(n, 1); + const p = gpa.rawAlloc(len, .@"1", @returnAddress()) orelse return error.OutOfMemory; + t.bufs[t.n] = p[0..len]; t.n += 1; + return p[0..len]; } fn free(t: *Temps, gpa: std.mem.Allocator) void { - for (t.bufs[0..t.n]) |b| gpa.free(b); + for (t.bufs[0..t.n]) |b| gpa.rawFree(b, .@"1", @returnAddress()); } }; @@ -257,7 +275,7 @@ fn marshal(rt: *Runtime, i: usize, t: abi.Type, v: Value, slots: *abi.Slots, tem const x = promote.exactI64(v) catch return failArgType(rt, i, "long"); slots.putInt(@bitCast(x)); }, - .pointer => slots.putInt(addressOf(v) orelse return failArgType(rt, i, "pointer")), + .pointer => slots.putInt(pointerArg(v) orelse return failArgType(rt, i, "pointer")), .double => slots.putDouble(switch (v.tag()) { .float => v.asFloat(), .integer => @floatFromInt(v.asInteger()), @@ -266,20 +284,18 @@ fn marshal(rt: *Runtime, i: usize, t: abi.Type, v: Value, slots: *abi.Slots, tem .string => { if (v.isNil()) return slots.putInt(0); if (!v.isString()) return failArgType(rt, i, "string"); - const z = try rt.gpa.dupeZ(u8, string_mod.asString(v)); - temps.keep(z[0 .. z.len + 1]); + const s = string_mod.asString(v); + const z = try temps.alloc(rt.gpa, s.len + 1); + @memcpy(z[0..s.len], s); + z[s.len] = 0; slots.putInt(@intFromPtr(z.ptr)); }, .bytes => { if (v.isNil()) return slots.putInt(0); - if (!java_array.isArray(v)) return failArgType(rt, i, "bytes"); - const items = java_array.asArray(v).items(); - const buf = try rt.gpa.alloc(u8, @max(items.len, 1)); - temps.keep(buf); - for (items, 0..) |e, k| { - if (!e.isInt()) return failArgType(rt, i, "bytes"); - buf[k] = @truncate(@as(u64, @bitCast(@as(i64, e.asInteger())))); - } + if (!java_array.isKind(v, .byte)) return failArgType(rt, i, "bytes"); + const src = java_array.asArray(v).bytes(); + const buf = try temps.alloc(rt.gpa, src.len); + @memcpy(buf[0..src.len], src); slots.putInt(@intFromPtr(buf.ptr)); }, .void => unreachable, @@ -337,9 +353,7 @@ fn bytesFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) an if (n < 0 or n > std.math.maxInt(u32)) return failArgType(rt, 1, "long"); if (addr == 0) return Value.nil_val; const src: [*]const u8 = @ptrFromInt(addr); - const arr = try java_array.make(rt, @intCast(n), Value.initInteger(0)); - for (java_array.asArray(arr).items(), 0..) |*slot, k| slot.* = Value.initInteger(@as(i8, @bitCast(src[k]))); - return arr; + return java_array.fromBytes(rt, src[0..@intCast(n)]); } /// Create the `cljw.ffi` leaves. Called by `runtime/cljw/_host_api.zig` diff --git a/src/runtime/cljw/net/socket.zig b/src/runtime/cljw/net/socket.zig index 3ef355a70..74294d7d4 100644 --- a/src/runtime/cljw/net/socket.zig +++ b/src/runtime/cljw/net/socket.zig @@ -35,8 +35,9 @@ //! under `safepoint.blocking`, so a `future` waiting on a quiet peer does not //! stall a collection another thread requests. //! -//! Byte arrays are Value-erased (F-004 / F-005), so transfers convert -//! element-wise between the cljw array and a gpa scratch `[]u8`. +//! A byte array is raw `i8` storage (ADR-0204): a read lands in it with one +//! memcpy, a write copies it to a gpa scratch first so the blocking write +//! never reads storage the program can still `aset`. const std = @import("std"); const Runtime = @import("../../runtime.zig").Runtime; const Env = @import("../../env.zig").Env; @@ -108,6 +109,13 @@ fn connectFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) return error_catalog.raise(.net_connect_failed, loc, .{ .host = host, .port = port_i }); }; + return wrapStream(rt, stream); +} + +/// A socket Value over the connected `stream`, in a fresh `SocketBox`. The +/// box's buffers are charged to the collect trigger, since the only cell +/// standing for them is a few words. Closes `stream` if the box cannot be made. +fn wrapStream(rt: *Runtime, stream: std.Io.net.Stream) !Value { const box = rt.gpa.create(SocketBox) catch |e| { stream.close(rt.io); return e; @@ -123,6 +131,7 @@ fn connectFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) }; box.reader = box.stream.reader(box.io, &box.rbuf); box.writer = box.stream.writer(box.io, &box.wbuf); + rt.gc.noteOwnedAlloc(@sizeOf(SocketBox)); return host_instance.alloc(rt, &descriptor, .{ @intFromPtr(box), 0, 0, 0 }); } @@ -132,7 +141,7 @@ fn writeMethod(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation _ = env; try error_catalog.checkArityRange("cljw.net write", args, 2, 3, loc); const box = try expectSocket(args[0], loc); - if (!java_array.isArray(args[1])) return error_catalog.raise(.net_arg_invalid, loc, .{ .detail = "the buffer must be a byte array" }); + if (!java_array.isKind(args[1], .byte)) return error_catalog.raise(.net_arg_invalid, loc, .{ .detail = "the buffer must be a byte array" }); const avail = java_array.alength(args[1]); const n: u32 = if (args.len == 3) blk: { @@ -141,12 +150,9 @@ fn writeMethod(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation break :blk @intCast(want); } else avail; - const scratch = try rt.gpa.alloc(u8, n); + // A copy: the array stays mutable while the blocking write runs. + const scratch = try rt.gpa.dupe(u8, java_array.asArray(args[1]).bytes()[0..n]); defer rt.gpa.free(scratch); - for (0..n) |i| { - const v = try java_array.aget(args[1], @intCast(i), "cljw.net write", loc); - scratch[i] = @truncate(@as(u64, @bitCast(@as(i64, v.asInteger())))); - } safepoint.blocking(std.Io.Writer.writeAll, .{ &box.writer.interface, scratch }) catch return error_catalog.raise(.net_io_failed, loc, .{ .op = "write" }); @@ -162,7 +168,7 @@ fn readMethod(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) _ = env; try error_catalog.checkArity("cljw.net read", args, 2, loc); const box = try expectSocket(args[0], loc); - if (!java_array.isArray(args[1])) return error_catalog.raise(.net_arg_invalid, loc, .{ .detail = "the buffer must be a byte array" }); + if (!java_array.isKind(args[1], .byte)) return error_catalog.raise(.net_arg_invalid, loc, .{ .detail = "the buffer must be a byte array" }); const cap = java_array.alength(args[1]); if (cap == 0) return Value.initInteger(0); @@ -178,9 +184,7 @@ fn readMethod(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) if (bytes.len == 0) return Value.initInteger(-1); const n: u32 = @intCast(@min(bytes.len, cap)); - for (0..n) |i| { - _ = try java_array.aset(args[1], @intCast(i), Value.initInteger(bytes[i]), "cljw.net read", loc); - } + @memcpy(java_array.asArray(args[1]).bytes()[0..n], bytes[0..n]); r.toss(n); return Value.initInteger(@intCast(n)); } @@ -317,23 +321,7 @@ fn acceptMethod(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocatio const stream = safepoint.blocking(std.Io.net.Server.accept, .{ &box.server, box.io }) catch return error_catalog.raise(.net_io_failed, loc, .{ .op = "accept" }); - const sock = rt.gpa.create(SocketBox) catch |e| { - stream.close(rt.io); - return e; - }; - sock.* = .{ - .io = rt.io, - .stream = stream, - .reader = undefined, - .writer = undefined, - .closed = false, - .rbuf = undefined, - .wbuf = undefined, - }; - sock.reader = sock.stream.reader(sock.io, &sock.rbuf); - sock.writer = sock.stream.writer(sock.io, &sock.wbuf); - - return host_instance.alloc(rt, &descriptor, .{ @intFromPtr(sock), 0, 0, 0 }); + return wrapStream(rt, stream); } /// `(.port srv)` — the BOUND port. `listen` with 0 resolves an ephemeral port @@ -437,23 +425,7 @@ fn connectUnixFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocati const stream = safepoint.blocking(std.Io.net.UnixAddress.connect, .{ &ua, rt.io }) catch return error_catalog.raise(.net_unix_connect_failed, loc, .{ .path = path }); - const box = rt.gpa.create(SocketBox) catch |e| { - stream.close(rt.io); - return e; - }; - box.* = .{ - .io = rt.io, - .stream = stream, - .reader = undefined, - .writer = undefined, - .closed = false, - .rbuf = undefined, - .wbuf = undefined, - }; - box.reader = box.stream.reader(box.io, &box.rbuf); - box.writer = box.stream.writer(box.io, &box.wbuf); - - return host_instance.alloc(rt, &descriptor, .{ @intFromPtr(box), 0, 0, 0 }); + return wrapStream(rt, stream); } /// `(cljw.net/listen-unix path)` — bind and listen on a UNIX domain socket. diff --git a/src/runtime/cljw/wasm/marshal.zig b/src/runtime/cljw/wasm/marshal.zig index 701ee44df..ee8c8e734 100644 --- a/src/runtime/cljw/wasm/marshal.zig +++ b/src/runtime/cljw/wasm/marshal.zig @@ -6,6 +6,13 @@ //! wasm type. i32 / i64 / f32 / f64 are covered; v128 / ref are out of P1 //! scope (Phase-16, when wasm refs become first-class cljw Values per F-004). //! +//! An integer is a cljw immediate only inside the i48 NaN-box window; past it +//! (about +/-2^47) it is a heap Long (tag `.big_int`). Both directions go +//! through the numeric tower, as `cljw.ffi`'s `:long` does: `big_int.toInt` +//! reads an arg exactly (the generic form of `promote.exactI64`), and +//! `promote.wrapI64` builds a result. `Value.initInteger` would demote a result past the window to a +//! lossy Double, and `isNumber` / `isInt` see only the immediate. +//! //! Backend: impl-only //! Impl deps: none //! Clojure peer: wasm/load, wasm/call @@ -15,6 +22,9 @@ const ValType = engine.ValType; const ZValue = engine.Value; const value_mod = @import("../../value/value.zig"); const Value = value_mod.Value; +const Runtime = @import("../../runtime.zig").Runtime; +const promote = @import("../../numeric/promote.zig"); +const big_int = @import("../../numeric/big_int.zig"); const error_catalog = @import("../../error/catalog.zig"); const ClojureWasmError = error_catalog.ClojureWasmError; const SourceLocation = @import("../../error/info.zig").SourceLocation; @@ -25,24 +35,25 @@ pub fn toWasm(arg: Value, vt: ValType, loc: SourceLocation) ClojureWasmError!ZVa .i32 => return ZValue.fromI32(try coerceInt(arg, i32, loc)), .i64 => return ZValue.fromI64(try coerceInt(arg, i64, loc)), .f32 => { - if (!arg.isNumber()) return badArg(loc); - const f: f32 = @floatCast(floatOf(arg)); + const f: f32 = @floatCast(try floatOf(arg, loc)); return ZValue.fromF32Bits(@bitCast(f)); }, - .f64 => { - if (!arg.isNumber()) return badArg(loc); - return ZValue.fromF64Bits(@bitCast(floatOf(arg))); - }, + .f64 => return ZValue.fromF64Bits(@bitCast(try floatOf(arg, loc))), .v128 => return error_catalog.raise(.wasm_value_type_unsupported, loc, .{ .what = "v128 arguments" }), .ref => return error_catalog.raise(.wasm_value_type_unsupported, loc, .{ .what = "reference-typed arguments" }), } } -/// Decode one `zwasm.Value` result back to a cljw `Value`. -pub fn fromWasm(res: ZValue, loc: SourceLocation) ClojureWasmError!Value { +/// Decode one `zwasm.Value` result back to a cljw `Value`. An i64 is exact +/// over its whole range: `promote.wrapI64` keeps it an immediate inside the +/// i48 window and makes it a heap Long outside it. wasm i64 is signless, so a +/// guest's u64 with the top bit set reads as the negative Long with the same +/// bits (JVM semantics, as `cljw.ffi`'s `:long`). An i64 result may allocate, +/// so a caller lifting several results brackets them in a no-collect region. +pub fn fromWasm(rt: *Runtime, res: ZValue, loc: SourceLocation) anyerror!Value { switch (res) { .i32 => |v| return Value.initInteger(v), - .i64 => |v| return Value.initInteger(v), + .i64 => |v| return promote.wrapI64(rt, v), .f32 => |bits| return Value.initFloat(@as(f32, @bitCast(bits))), .f64 => |bits| return Value.initFloat(@bitCast(bits)), .v128 => return error_catalog.raise(.wasm_value_type_unsupported, loc, .{ .what = "v128 results" }), @@ -64,24 +75,42 @@ pub fn fromWasm(res: ZValue, loc: SourceLocation) ClojureWasmError!Value { /// - float with a fraction → error (1.5 is not an integer arg; 2.0 is allowed); /// - integer value outside [minInt(T), maxInt(T)] → error; /// otherwise return the in-range value. +/// +/// A heap Long (outside the i48 immediate window) and a BigInt are integers +/// too: `big_int.toInt(T)` reads them exactly, so a WIT `u64` above +/// Long/MAX_VALUE lowers from a BigInt and anything outside `T` is refused. pub fn coerceInt(arg: Value, comptime T: type, loc: SourceLocation) ClojureWasmError!T { - if (!arg.isNumber()) return badArg(loc); - const i: i64 = if (arg.isInt()) arg.asInteger() else blk: { - const f = arg.asFloat(); - if (@floor(f) != f) return error_catalog.raise(.wasm_arg_not_integer, loc, .{}); - // The float is integral; widen-range check before the i64 cast. - if (f < @as(f64, @floatFromInt(std.math.minInt(i64))) or f > @as(f64, @floatFromInt(std.math.maxInt(i64)))) - return error_catalog.raise(.wasm_arg_out_of_range, loc, .{}); - break :blk @intFromFloat(f); + switch (arg.tag()) { + .integer => return std.math.cast(T, @as(i64, arg.asInteger())) orelse outOfRange(loc), + .big_int => return big_int.asManaged(arg).toInt(T) catch outOfRange(loc), + .float => { + const f = arg.asFloat(); + if (@floor(f) != f) return error_catalog.raise(.wasm_arg_not_integer, loc, .{}); + // The float is integral; range-check before the cast. The bounds + // are the exact powers of two, half-open at the top: maxInt(i64) + // as an f64 ROUNDS to 2^63, so a `> maxInt` test let 2^63 itself + // through to `@intFromFloat`, a ReleaseSafe panic on caller data. + if (!(f >= -0x1p63 and f < 0x1p63)) return outOfRange(loc); + const i: i64 = @intFromFloat(f); + return std.math.cast(T, i) orelse outOfRange(loc); + }, + else => return badArg(loc), + } +} + +/// cljw number to f64: a float passes through, an integer widens (a heap Long +/// or BigInt to its nearest double, as `double` does). +fn floatOf(v: Value, loc: SourceLocation) ClojureWasmError!f64 { + return switch (v.tag()) { + .float => v.asFloat(), + .integer => @floatFromInt(@as(i64, v.asInteger())), + .big_int => big_int.asManaged(v).toFloat(f64, .nearest_even)[0], + else => badArg(loc), }; - if (i < std.math.minInt(T) or i > std.math.maxInt(T)) - return error_catalog.raise(.wasm_arg_out_of_range, loc, .{}); - return @intCast(i); } -/// cljw number → f64 (a float passes through; an int widens). -fn floatOf(v: Value) f64 { - return if (v.isFloat()) v.asFloat() else @floatFromInt(v.asInteger()); +fn outOfRange(loc: SourceLocation) ClojureWasmError { + return error_catalog.raise(.wasm_arg_out_of_range, loc, .{}); } fn badArg(loc: SourceLocation) ClojureWasmError { diff --git a/src/runtime/cljw/wasm/memory.zig b/src/runtime/cljw/wasm/memory.zig index 15a145e60..de7bc0bb2 100644 --- a/src/runtime/cljw/wasm/memory.zig +++ b/src/runtime/cljw/wasm/memory.zig @@ -22,8 +22,10 @@ const Runtime = @import("../../runtime.zig").Runtime; const value_mod = @import("../../value/value.zig"); const Value = value_mod.Value; const promote = @import("../../numeric/promote.zig"); +const big_int = @import("../../numeric/big_int.zig"); const vector_mod = @import("../../collection/vector.zig"); const sub_vector_mod = @import("../../collection/sub_vector.zig"); +const java_array = @import("../../collection/java_array.zig"); /// Element type of a linear-memory view. The closed set of wasm-representable /// numeric widths — closed because it is the wasm memory model's, not an @@ -147,20 +149,22 @@ fn exactInt(v: Value) ElemError!i64 { .float => blk: { const f = v.asFloat(); if (@floor(f) != f) return error.NotAnInteger; - if (f < @as(f64, @floatFromInt(std.math.minInt(i64))) or - f > @as(f64, @floatFromInt(std.math.maxInt(i64)))) return error.OutOfRange; + // Half-open at the top: maxInt(i64) as an f64 rounds to 2^63, so a + // `> maxInt` test let 2^63 through to a ReleaseSafe panic. + if (!(f >= -0x1p63 and f < 0x1p63)) return error.OutOfRange; break :blk @intFromFloat(f); }, else => error.NotANumber, }; } -/// Read `v` as an f64. An integer widens; a float passes through. Matches -/// `marshal.zig::toWasm`'s float arms, which accept exactly `isNumber()`. +/// Read `v` as an f64. An integer widens (a heap Long or BigInt to its nearest +/// double); a float passes through. Matches `marshal.zig::toWasm`'s float arms. fn floatOf(v: Value) ElemError!f64 { return switch (v.tag()) { .float => v.asFloat(), .integer => @floatFromInt(@as(i64, v.asInteger())), + .big_int => big_int.asManaged(v).toFloat(f64, .nearest_even)[0], else => error.NotANumber, }; } @@ -188,6 +192,32 @@ pub fn indexedCount(v: Value) ?u32 { }; } +/// The array kind `wasm/mem-read-array` answers for `dt`: the kind of the +/// same width, or the next wider signed kind for the unsigned `:u16` / `:u32` +/// (`:u8` reads as a byte array, the same bits). +pub fn arrayKindOf(dt: Dtype) java_array.Kind { + return switch (dt) { + .i8, .u8 => .byte, + .i16 => .short, + .u16, .i32 => .int, + .u32, .i64 => .long, + .f32 => .float, + .f64 => .double, + }; +} + +/// True when the elements of primitive array `a` have exactly `dt`'s wasm +/// layout, so a range of linear memory and the array's storage are the same +/// bytes and cross with one memcpy (ADR-0204). Wasm memory is little-endian, +/// so a big-endian host always takes the element-wise path. `:u8` matches a +/// byte array (the same bits, read signed on the Clojure side); `:u16` and +/// `:u32` have no array kind. +pub fn sameLayout(dt: Dtype, a: *const java_array.JavaArray) bool { + if (@import("builtin").cpu.arch.endian() != .little) return false; + if (dt == .u16 or dt == .u32) return false; + return a.kind == arrayKindOf(dt); +} + /// Element `i` of an indexed vector. The caller has already established the /// tag via `indexedCount` and that `i` is in range. pub fn indexedNth(v: Value, i: u32) Value { diff --git a/src/runtime/cljw/wasm/memory_array.zig b/src/runtime/cljw/wasm/memory_array.zig new file mode 100644 index 000000000..b621dc23f --- /dev/null +++ b/src/runtime/cljw/wasm/memory_array.zig @@ -0,0 +1,113 @@ +// SPDX-License-Identifier: EPL-2.0 +//! Primitive arrays across a module's linear memory (ADR-0204): the bulk +//! counterpart of `wasm/mem-read` / `wasm/mem-write!`, which box one Value per +//! element. A primitive array is contiguous little-endian storage of its +//! width, so where its kind matches the dtype a crossing is one memcpy. +//! +//! - `(wasm/mem-read-array h :f64 offset n)` answers a primitive array. +//! - `(wasm/mem-write! h :f64 offset arr)` with a primitive array: `surface.zig` +//! routes it here. +//! +//! Backend: impl-only +//! Impl deps: none +//! Clojure peer: wasm/mem-read-array, wasm/mem-write! +const std = @import("std"); +const engine = @import("engine.zig"); +const wasm_memory = @import("memory.zig"); +const surface = @import("surface.zig"); +const Runtime = @import("../../runtime.zig").Runtime; +const Env = @import("../../env.zig").Env; +const Value = @import("../../value/value.zig").Value; +const error_catalog = @import("../../error/catalog.zig"); +const SourceLocation = @import("../../error/info.zig").SourceLocation; +const java_array = @import("../../collection/java_array.zig"); + +fn rangeOf(comptime fn_name: []const u8, mem: engine.Memory, dt: wasm_memory.Dtype, offset: i64, count: i64, loc: SourceLocation) !wasm_memory.Range { + const len = mem.slice().len; + const ok = if (count <= std.math.maxInt(u32)) wasm_memory.byteRange(offset, count, dt, len) else null; + return ok orelse error_catalog.raise(.wasm_memory_range_invalid, loc, .{ + .fn_name = fn_name, + .offset = offset, + .count = count, + .size = len, + }); +} + +/// `(wasm/mem-read-array handle :f64 offset n)`: `n` elements at BYTE `offset` +/// as a primitive array: `:f64` a double-array, `:i64` a long-array, `:u16` +/// an int-array and `:u32` a long-array (widened, so every value fits), `:u8` +/// a byte-array of the same bits. +pub fn wasmMemReadArrayFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("wasm/mem-read-array", args, 4, loc); + const mem = try surface.memoryArg("wasm/mem-read-array", args[0], loc); + const dt = try surface.dtypeArg("wasm/mem-read-array", args[1], loc); + const offset = try surface.indexArg("wasm/mem-read-array", args[2], loc); + const count = try surface.indexArg("wasm/mem-read-array", args[3], loc); + const range = try rangeOf("wasm/mem-read-array", mem, dt, offset, count, loc); + + const n: u32 = @intCast(count); + const out = try java_array.make(rt, wasm_memory.arrayKindOf(dt), n); + const a = java_array.asArray(out); + const bytes = mem.slice(); + if (wasm_memory.sameLayout(dt, a)) { + @memcpy(a.bytes(), bytes[range.start..][0..range.len]); + return out; + } + // `:u16` / `:u32` widen, a big-endian host swaps. `out` is the only live + // allocation and is rooted by the frame below; an i64 box from readElem is + // stored at once. + const w: usize = dt.width(); + for (0..n) |i| { + const at = range.start + i * w; + const v = try wasm_memory.readElem(rt, dt, bytes[at .. at + w]); + try java_array.store(a, i, v, .strict, "wasm/mem-read-array", loc); + } + return out; +} + +/// `wasm/mem-write!` of a primitive array: one memcpy when the array's kind +/// is the dtype's layout, else the element-wise encode with the same per- +/// element refusals as a vector. Returns the element count. +pub fn writeArray(mem: engine.Memory, dt: wasm_memory.Dtype, offset: i64, arr: Value, loc: SourceLocation) anyerror!Value { + const a = java_array.asArray(arr); + if (a.kind == .object) + return error_catalog.raise(.wasm_memory_data_invalid, loc, .{ .fn_name = "wasm/mem-write!" }); + const range = try rangeOf("wasm/mem-write!", mem, dt, offset, a.len, loc); + const bytes = mem.slice(); + if (wasm_memory.sameLayout(dt, a)) { + @memcpy(bytes[range.start..][0..range.len], a.bytes()); + return Value.initInteger(a.len); + } + const w: usize = dt.width(); + for (0..a.len) |i| { + const at = range.start + i * w; + // `peek` boxes without allocating except a long outside i48; take the + // raw i64 for those so this path never allocates. + const v = java_array.peek(a, i) orelse { + const x = a.slice(.long)[i]; + if (dt != .i64) return elemError(i, error.OutOfRange, loc); + std.mem.writeInt(i64, bytes[at..][0..8], x, .little); + continue; + }; + wasm_memory.writeElem(v, dt, bytes[at .. at + w]) catch |e| return elemError(i, e, loc); + } + return Value.initInteger(a.len); +} + +fn elemError(i: usize, e: wasm_memory.ElemError, loc: SourceLocation) anyerror { + return error_catalog.raise(.wasm_memory_element_invalid, loc, .{ + .fn_name = "wasm/mem-write!", + .index = @as(u32, @intCast(i)), + .detail = switch (e) { + error.NotANumber => "is not a number", + error.NotAnInteger => "is not an integer", + error.OutOfRange => "is outside this element type's range", + }, + }); +} + +/// Intern `mem-read-array` into the `wasm` namespace. +pub fn register(env: *Env, ns: anytype) !void { + _ = try env.intern(ns, "mem-read-array", Value.initBuiltinFn(&wasmMemReadArrayFn), null); +} diff --git a/src/runtime/cljw/wasm/surface.zig b/src/runtime/cljw/wasm/surface.zig index ae00e73a9..afcfaa362 100644 --- a/src/runtime/cljw/wasm/surface.zig +++ b/src/runtime/cljw/wasm/surface.zig @@ -185,11 +185,18 @@ fn callExport(rt: *Runtime, loaded: *engine.Loaded, name: []const u8, call_args: }; if (out.len == 0) return Value.nil_val; - if (out.len == 1) return marshal.fromWasm(out[0], loc); + if (out.len == 1) return marshal.fromWasm(rt, out[0], loc); const items = try rt.gpa.alloc(Value, out.len); defer rt.gpa.free(items); - for (out, 0..) |r, i| items[i] = try marshal.fromWasm(r, loc); + + // GC-ROOT: `items` is a gpa slice the collector does NOT trace, and an i64 + // result outside the i48 immediate window is a HEAP Long, so a collect + // during a later result's alloc would sweep the ones already lifted. Same + // bracket as `wasmMemReadFn`. [ref: .dev/gc_rooting.md] + rt.gc.enterFabrication(); + defer rt.gc.exitFabrication(); + for (out, 0..) |r, i| items[i] = try marshal.fromWasm(rt, r, loc); return vector_mod.fromSlice(rt, items); } @@ -476,7 +483,7 @@ pub fn wasmRunFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocati /// Resolve arg 0 to the module's exported linear memory. Raises when the value /// is not a loaded handle, or when the module exports no memory — "exports /// none" is a program error, distinct from "holds nothing yet". -fn memoryArg(comptime fn_name: []const u8, arg: Value, loc: SourceLocation) anyerror!engine.Memory { +pub fn memoryArg(comptime fn_name: []const u8, arg: Value, loc: SourceLocation) anyerror!engine.Memory { if (!wasm_handle.isHandle(arg)) return error_catalog.raise(.wasm_handle_invalid, loc, .{ .fn_name = fn_name }); return wasm_handle.unwrap(arg).memory() orelse @@ -486,7 +493,7 @@ fn memoryArg(comptime fn_name: []const u8, arg: Value, loc: SourceLocation) anye /// Resolve an element-type keyword (`:f64`) to a `Dtype`. The element type is /// the GUEST's layout, so it is stated by the caller and never inferred from /// the data — `[1 2]` and `[1.0 2.0]` denote the same `:f64` buffer. -fn dtypeArg(comptime fn_name: []const u8, arg: Value, loc: SourceLocation) anyerror!wasm_memory.Dtype { +pub fn dtypeArg(comptime fn_name: []const u8, arg: Value, loc: SourceLocation) anyerror!wasm_memory.Dtype { if (arg.tag() != .keyword) return error_catalog.raise(.wasm_memory_dtype_invalid, loc, .{ .fn_name = fn_name }); return wasm_memory.dtypeFromName(keyword_mod.asKeyword(arg).name) orelse @@ -494,7 +501,7 @@ fn dtypeArg(comptime fn_name: []const u8, arg: Value, loc: SourceLocation) anyer } /// Resolve a byte-offset / element-count argument to an exact i64. -fn indexArg(comptime fn_name: []const u8, arg: Value, loc: SourceLocation) anyerror!i64 { +pub fn indexArg(comptime fn_name: []const u8, arg: Value, loc: SourceLocation) anyerror!i64 { return wasm_memory.indexArg(arg) orelse error_catalog.raise(.wasm_memory_index_invalid, loc, .{ .fn_name = fn_name }); } @@ -572,6 +579,7 @@ pub fn wasmMemWriteFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceL const mem = try memoryArg("wasm/mem-write!", args[0], loc); const dt = try dtypeArg("wasm/mem-write!", args[1], loc); const offset = try indexArg("wasm/mem-write!", args[2], loc); + if (args[3].tag() == .array) return @import("memory_array.zig").writeArray(mem, dt, offset, args[3], loc); const n = wasm_memory.indexedCount(args[3]) orelse return error_catalog.raise(.wasm_memory_data_invalid, loc, .{ .fn_name = "wasm/mem-write!" }); @@ -623,6 +631,7 @@ pub fn register(env: *Env) !void { _ = try env.intern(ns, "mem-size", Value.initBuiltinFn(&wasmMemSizeFn), null); _ = try env.intern(ns, "mem-read", Value.initBuiltinFn(&wasmMemReadFn), null); _ = try env.intern(ns, "mem-write!", Value.initBuiltinFn(&wasmMemWriteFn), null); + try @import("memory_array.zig").register(env, ns); // EXPERIMENT (D-404 / ADR-0135, push-suppressed): component introspection + // typed invoke probes — the require-a-component substrate. const component = @import("component.zig"); diff --git a/src/runtime/clojure/lang/LazilyPersistentVector.zig b/src/runtime/clojure/lang/LazilyPersistentVector.zig index 83f6424e6..ab93f400a 100644 --- a/src/runtime/clojure/lang/LazilyPersistentVector.zig +++ b/src/runtime/clojure/lang/LazilyPersistentVector.zig @@ -30,7 +30,7 @@ fn createOwning(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocatio .expected = "a Java array", .actual = @tagName(args[0].tag()), }); - return vector_mod.fromSlice(rt, java_array.asArray(args[0]).items()); + return java_array.toVector(rt, java_array.asArray(args[0])); } /// `(clojure.lang.LazilyPersistentVector/create coll)` — a vector of `coll`'s @@ -39,7 +39,7 @@ fn createOwning(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocatio fn create(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { try error_catalog.checkArity("clojure.lang.LazilyPersistentVector/create", args, 1, loc); if (java_array.isArray(args[0])) - return vector_mod.fromSlice(rt, java_array.asArray(args[0]).items()); + return java_array.toVector(rt, java_array.asArray(args[0])); const core = env.findNs("clojure.core") orelse return error.NoVTable; const vec_var = core.resolve("vec") orelse return error.NoVTable; const vt = rt.vtable orelse return error.NoVTable; diff --git a/src/runtime/clojure/lang/PersistentArrayMap.zig b/src/runtime/clojure/lang/PersistentArrayMap.zig index 222f219be..a9b350a76 100644 --- a/src/runtime/clojure/lang/PersistentArrayMap.zig +++ b/src/runtime/clojure/lang/PersistentArrayMap.zig @@ -20,9 +20,9 @@ const dispatch = @import("../../dispatch.zig"); const print_mod = @import("../../print.zig"); fn expectPairArray(v: Value, fn_name: []const u8, loc: SourceLocation) ![]Value { - if (!java_array.isArray(v)) - return error_catalog.raise(.type_arg_invalid, loc, .{ .fn_name = fn_name, .expected = "a Java array", .actual = @tagName(v.tag()) }); - const items = java_array.asArray(v).items(); + if (!java_array.isKind(v, .object)) + return error_catalog.raise(.type_arg_invalid, loc, .{ .fn_name = fn_name, .expected = "an object array", .actual = @tagName(v.tag()) }); + const items = java_array.asArray(v).objects(); if (items.len % 2 != 0) return error_catalog.raise(.type_arg_invalid, loc, .{ .fn_name = fn_name, .expected = "an even-length array", .actual = "an odd-length array" }); return items; diff --git a/src/runtime/collection/array_seq.zig b/src/runtime/collection/array_seq.zig index d3dd9f3fb..638bf0cf8 100644 --- a/src/runtime/collection/array_seq.zig +++ b/src/runtime/collection/array_seq.zig @@ -87,12 +87,20 @@ fn backingCount(backing: Value) u32 { } /// `backing[i]` for a viewable backing. Callers must satisfy -/// `i < backingCount(backing)`; the ArraySeq invariant guarantees it. -fn elementAt(backing: Value, i: u32) Value { +/// `i < backingCount(backing)`; the ArraySeq invariant guarantees it. Boxing a +/// primitive array's `long` element may allocate. +fn elementAt(rt: *Runtime, backing: Value, i: u32) !Value { + if (backing.tag() == .array) return java_array.get(rt, java_array.asArray(backing), i); + return peekAt(backing, i).?; +} + +/// `elementAt` without allocating: null for the one element that needs a +/// heap box (a primitive `long` outside the immediate range). +fn peekAt(backing: Value, i: u32) ?Value { return switch (backing.tag()) { .vector => vector.nth(backing, i), .sub_vector => sub_vector.nth(backing, i), - .array => java_array.asArray(backing).items()[i], + .array => java_array.peek(java_array.asArray(backing), i), else => Value.nil_val, }; } @@ -129,9 +137,9 @@ pub fn indexOf(v: Value) u32 { } /// `(first as)` — total, by the never-empty invariant. -pub fn first(v: Value) Value { +pub fn first(rt: *Runtime, v: Value) !Value { const as = v.decodePtr(*const ArraySeq); - return elementAt(as.backing, as.index); + return elementAt(rt, as.backing, as.index); } /// `(count as)` — O(1): the backing's count minus the cursor. This is the @@ -142,9 +150,24 @@ pub fn countOf(v: Value) u32 { } /// `(nth as i)` — O(1) into the backing, no walk. -pub fn nth(v: Value, i: u32) Value { +pub fn nth(rt: *Runtime, v: Value, i: u32) !Value { const as = v.decodePtr(*const ArraySeq); - return elementAt(as.backing, as.index + i); + return elementAt(rt, as.backing, as.index + i); +} + +/// `nth` for the rt-free paths (print, key hashing): null where the element +/// needs a heap box (see `peekAt`). +pub fn peekNth(v: Value, i: u32) ?Value { + const as = v.decodePtr(*const ArraySeq); + return peekAt(as.backing, as.index + i); +} + +/// The primitive array an ArraySeq views, or null (a vector backing or an +/// object array). +pub fn primitiveBacking(v: Value) ?*const java_array.JavaArray { + const b = v.decodePtr(*const ArraySeq).backing; + if (b.tag() != .array or java_array.asArray(b).kind == .object) return null; + return java_array.asArray(b); } /// `(rest as)` / `(next as)` — the same view one step along, or nil at the end. @@ -249,15 +272,15 @@ test "make: never mints an empty view; count is backing minus cursor (O-058)" { const s = try make(&fix.rt, v, 0); try testing.expectEqual(Value.Tag.array_seq, s.tag()); try testing.expectEqual(@as(u32, 4), countOf(s)); - try testing.expectEqual(@as(i48, 10), first(s).asInteger()); - try testing.expectEqual(@as(i48, 30), nth(s, 2).asInteger()); + try testing.expectEqual(@as(i48, 10), (try first(&fix.rt, s)).asInteger()); + try testing.expectEqual(@as(i48, 30), (try nth(&fix.rt, s, 2)).asInteger()); // The cursor is subtracted from the backing count at every offset — a view // that reported the BACKING's count would pass at offset 0 and fail here. const s2 = try make(&fix.rt, v, 2); try testing.expectEqual(@as(u32, 2), countOf(s2)); - try testing.expectEqual(@as(i48, 30), first(s2).asInteger()); - try testing.expectEqual(@as(i48, 40), nth(s2, 1).asInteger()); + try testing.expectEqual(@as(i48, 30), (try first(&fix.rt, s2)).asInteger()); + try testing.expectEqual(@as(i48, 40), (try nth(&fix.rt, s2, 1)).asInteger()); // The view shares the backing rather than copying it. try testing.expectEqual(@intFromEnum(v), @intFromEnum(backingOf(s2))); try testing.expectEqual(@as(u32, 2), indexOf(s2)); @@ -272,15 +295,15 @@ test "rest / drop advance the cursor and terminate at the end (O-058)" { const s = try make(&fix.rt, v, 0); const r1 = try rest(&fix.rt, s); - try testing.expectEqual(@as(i48, 2), first(r1).asInteger()); + try testing.expectEqual(@as(i48, 2), (try first(&fix.rt, r1)).asInteger()); const r2 = try rest(&fix.rt, r1); - try testing.expectEqual(@as(i48, 3), first(r2).asInteger()); + try testing.expectEqual(@as(i48, 3), (try first(&fix.rt, r2)).asInteger()); // One step past the last element is nil — the walk terminates exactly, and // `rest` never yields a count-0 view for a caller to `first`. try testing.expect((try rest(&fix.rt, r2)).isNil()); // `drop` reaches the same place in one step as repeated `rest`. - try testing.expectEqual(@as(i48, 3), first(try drop(&fix.rt, s, 2)).asInteger()); + try testing.expectEqual(@as(i48, 3), (try first(&fix.rt, try drop(&fix.rt, s, 2))).asInteger()); try testing.expect((try drop(&fix.rt, s, 3)).isNil()); // A saturating add, so a huge n cannot wrap into a valid-looking cursor. try testing.expect((try drop(&fix.rt, s, std.math.maxInt(u32))).isNil()); @@ -302,7 +325,7 @@ test "withMeta: fresh view, same backing and cursor, meta not shared (O-058)" { try testing.expectEqual(@intFromEnum(v), @intFromEnum(backingOf(tagged))); try testing.expectEqual(@as(u32, 1), indexOf(tagged)); try testing.expectEqual(@as(u32, 2), countOf(tagged)); - try testing.expectEqual(@as(i48, 2), first(tagged).asInteger()); + try testing.expectEqual(@as(i48, 2), (try first(&fix.rt, tagged)).asInteger()); // The receiver is unchanged — with-meta is not a mutation. try testing.expect(metaOf(s).isNil()); // Meta does not ride along to the tail (clj's ASeq.next()). diff --git a/src/runtime/collection/chunked_seq.zig b/src/runtime/collection/chunked_seq.zig new file mode 100644 index 000000000..b7c341b7c --- /dev/null +++ b/src/runtime/collection/chunked_seq.zig @@ -0,0 +1,109 @@ +// SPDX-License-Identifier: EPL-2.0 +//! The IChunkedSeq boundary: one place that answers "is this seq chunked, and +//! what is its current chunk" for every chunked representation (ADR-0205). +//! +//! clj has two native chunked seqs that matter here: `ChunkedCons` (range, +//! chunk-aware map/filter output) and `PersistentVector$ChunkedSeq` (the seq of +//! a vector, whose chunk is the 32-element leaf array holding the cursor). cljw +//! represents the first as `.chunked_cons` and the second as an `.array_seq` +//! view over a `.vector`. Before this module only `.chunked_cons` answered +//! `chunked-seq?`, so `(map f v)` over a vector realized one element per lazy +//! node while the JVM realizes a whole leaf at a time: an observable laziness +//! divergence (side-effecting `f`) and a 5x slowdown. +//! +//! Chunk geometry follows clj exactly: a vector seq at index `i` exposes the +//! leaf containing `i`, from `i` to the leaf end (`(i | 31) + 1`, bounded by +//! count), so the chunks are leaf-aligned whatever index the walk started at. +//! An array_seq over a `.sub_vector` or a Java array stays unchunked: clj's +//! `APersistentVector$Seq` (subvec) and `ArraySeq` (arrays) are not +//! IChunkedSeq. +//! +//! GC: `chunkRest` performs at most one allocation (an array_seq view) and +//! stores only the receiver's backing, which the caller roots through the +//! receiver. No fabrication bracket is needed for a single-alloc builder. + +const Value = @import("../value/value.zig").Value; +const Runtime = @import("../runtime.zig").Runtime; +const chunked_cons = @import("chunked_cons.zig"); +const array_seq = @import("array_seq.zig"); +const vector = @import("vector.zig"); + +/// Elements per vector leaf (PersistentVector branching factor). +const LEAF: u32 = 32; + +fn isVectorView(v: Value) bool { + return v.tag() == .array_seq and array_seq.backingOf(v).tag() == .vector; +} + +/// `chunked-seq?`: true for a chunked_cons and for the seq of a vector. +pub fn isChunked(v: Value) bool { + return v.tag() == .chunked_cons or isVectorView(v); +} + +/// End index (exclusive) of the leaf chunk holding the view's cursor. +fn leafEnd(v: Value) u32 { + const i = array_seq.indexOf(v); + const n = vector.count(array_seq.backingOf(v)); + const end = (i | (LEAF - 1)) + 1; + return if (end < n) end else n; +} + +/// Element count of the current chunk from the read cursor; 0 when `v` is not +/// chunked. +pub fn chunkCount(v: Value) u32 { + if (v.tag() == .chunked_cons) return chunked_cons.currentChunkCount(v); + if (isVectorView(v)) return leafEnd(v) - array_seq.indexOf(v); + return 0; +} + +/// Element `i` (relative to the cursor) of the current chunk. Callers satisfy +/// `i < chunkCount(v)`. +pub fn chunkNth(v: Value, i: u32) Value { + if (v.tag() == .chunked_cons) return chunked_cons.currentChunkNth(v, i); + if (isVectorView(v)) return vector.nth(array_seq.backingOf(v), array_seq.indexOf(v) + i); + return Value.nil_val; +} + +/// The seq after the WHOLE current chunk (clj `chunk-rest` minus the empty +/// list lift: nil at the end, which every caller seqs anyway). +pub fn chunkRest(rt: *Runtime, v: Value) !Value { + if (v.tag() == .chunked_cons) return chunked_cons.chunkRest(v); + if (isVectorView(v)) return try array_seq.make(rt, array_seq.backingOf(v), leafEnd(v)); + return Value.nil_val; +} + +// --- tests --- + +const std = @import("std"); +const testing = std.testing; + +test "vector view chunks are leaf-aligned and bounded by count (ADR-0205)" { + var threaded = std.Io.Threaded.init(testing.allocator, .{}); + defer threaded.deinit(); + var rt = Runtime.init(threaded.io(), testing.allocator); + defer rt.deinit(); + + var items: [70]Value = undefined; + for (&items, 0..) |*it, k| it.* = Value.initInteger(@intCast(k)); + const v = try vector.fromSlice(&rt, &items); + + const s0 = try array_seq.make(&rt, v, 0); + try testing.expect(isChunked(s0)); + try testing.expectEqual(@as(u32, 32), chunkCount(s0)); + try testing.expectEqual(@as(i48, 31), chunkNth(s0, 31).asInteger()); + + // A cursor mid-leaf exposes only the rest of that leaf. + const s5 = try array_seq.make(&rt, v, 5); + try testing.expectEqual(@as(u32, 27), chunkCount(s5)); + try testing.expectEqual(@as(i48, 5), chunkNth(s5, 0).asInteger()); + + const r1 = try chunkRest(&rt, s5); + try testing.expectEqual(@as(u32, 32), array_seq.indexOf(r1)); + const r2 = try chunkRest(&rt, r1); + try testing.expectEqual(@as(u32, 6), chunkCount(r2)); // 64..70 + try testing.expect((try chunkRest(&rt, r2)).isNil()); + + // Non-chunked shapes answer 0 / false. + try testing.expect(!isChunked(Value.nil_val)); + try testing.expectEqual(@as(u32, 0), chunkCount(Value.initInteger(1))); +} diff --git a/src/runtime/collection/java_array.zig b/src/runtime/collection/java_array.zig index 771ead4a5..48eedfd57 100644 --- a/src/runtime/collection/java_array.zig +++ b/src/runtime/collection/java_array.zig @@ -1,19 +1,22 @@ // SPDX-License-Identifier: EPL-2.0 -//! Java array (`.array` tag) — a type-erased, mutable, fixed-size `[]Value` -//! (ADR-0105 / D-287). The neutral impl behind clojure.core's -//! `aget`/`aset`/`make-array`/`*-array`/`alength`/`aclone`/`to-array`. +//! Java array (`.array` tag): a mutable, fixed-size array with an element +//! kind (ADR-0105 / D-287, typed storage per ADR-0204). The neutral impl +//! behind clojure.core's `aget`/`aset`/`make-array`/`*-array`/`alength`/ +//! `aclone`/`to-array`. //! //! Backend: impl-only (keyword: array) //! Impl deps: none //! Clojure peer: clojure.core/aget … (via lang/primitive/array.zig) //! -//! Representation mirrors `type_descriptor.TypedInstance` (ADR-0104): a GC -//! heap object holding a `gc.infra`-owned, traced, in-place-mutable `[]Value`. -//! The element TYPE is erased (F-004 uniform 8-byte Value + F-005 no primitive -//! specialization): `byte-array`/`int-array`/`object-array` all produce this -//! one shape; type hints are advisory (AD-019). Behavioural fidelity to clj is -//! preserved at the *constructor* (per-type init default) and in the byte/ -//! short/char *wrap*, not in the storage. Arrays use identity equality. +//! A GC heap object holding a `gc.infra`-owned element buffer. An `.object` +//! array stores `Value`s and is traced; every other kind stores raw native +//! elements, contiguous and untraced: `byte` i8, `short` i16, `int` i32, +//! `long` i64, `float` f32, `double` f64, `char` u32 (a code point, cljw chars +//! are not UTF-16 units), `boolean` u8. So a primitive array IS a C array, and +//! a bulk crossing (cljw.ffi, a socket write, `String.`) is one memcpy or a +//! pointer. `get` boxes an element, `store` coerces one (a type mismatch is the +//! catalog's `arg_value_invalid`, clj's "argument type mismatch"). Arrays use +//! identity equality. const std = @import("std"); const value_mod = @import("../value/value.zig"); @@ -25,23 +28,111 @@ const gc_heap_mod = @import("../gc/gc_heap.zig"); const mark_sweep = @import("../gc/mark_sweep.zig"); const error_catalog = @import("../error/catalog.zig"); const SourceLocation = @import("../error/info.zig").SourceLocation; +const promote = @import("../numeric/promote.zig"); +const vector_mod = @import("vector.zig"); + +/// The element kind of an array. `object` holds any Value; the rest hold +/// native elements of the named C width. +pub const Kind = enum(u8) { + object, + boolean, + byte, + short, + char, + int, + long, + float, + double, + + pub fn fromName(s: []const u8) ?Kind { + return std.meta.stringToEnum(Kind, s); + } + + /// Bytes per element. + pub fn size(k: Kind) usize { + return switch (k) { + .object => @sizeOf(Value), + .boolean, .byte => 1, + .short => 2, + .char, .int, .float => 4, + .long, .double => 8, + }; + } + + /// The native element type of a primitive kind. + pub fn Elem(comptime k: Kind) type { + return switch (k) { + .object => Value, + .boolean => u8, + .byte => i8, + .short => i16, + .char => u32, + .int => i32, + .long => i64, + .float => f32, + .double => f64, + }; + } + + /// The value a fresh slot holds (clj's per-type default). + fn default(k: Kind) Value { + return switch (k) { + .object => .nil_val, + .boolean => .false_val, + .char => Value.initChar(0), + .float, .double => Value.initFloat(0), + else => Value.initInteger(0), + }; + } +}; + +const ALIGN = std.mem.Alignment.fromByteUnits(8); + +/// The element buffer, through `rawAlloc`: `Allocator.alloc` would first fill +/// it with the safe-build `undefined` pattern, a second pass over every byte +/// that `make` overwrites at once. +fn allocData(infra: std.mem.Allocator, n: usize) ![*]align(8) u8 { + if (n == 0) return @ptrFromInt(ALIGN.toByteUnits()); + const p = infra.rawAlloc(n, ALIGN, @returnAddress()) orelse return error.OutOfMemory; + return @alignCast(p); +} + +fn freeData(infra: std.mem.Allocator, buf: []align(8) u8) void { + if (buf.len == 0) return; + infra.rawFree(buf, ALIGN, @returnAddress()); +} /// A Java array value. `extern struct` so `rt.gc.alloc` accepts it (HeapHeader /// at offset 0) and the GC walker treats the head as a HeapHeader. pub const JavaArray = extern struct { header: HeapHeader, len: u32, - _pad: [4]u8 = .{ 0, 0, 0, 0 }, - /// `gc.infra`-owned `[]Value`; the finaliser releases it. - items_ptr: [*]Value, + kind: Kind, + _pad: [3]u8 = .{ 0, 0, 0 }, + /// `gc.infra`-owned, 8-aligned, `len * kind.size()` bytes; the finaliser + /// releases it. + data: [*]align(8) u8, comptime { std.debug.assert(@alignOf(JavaArray) >= 8); std.debug.assert(@offsetOf(JavaArray, "header") == 0); } - pub fn items(self: *const JavaArray) []Value { - return self.items_ptr[0..self.len]; + /// The raw element storage, `len * kind.size()` bytes. + pub fn bytes(self: *const JavaArray) []align(8) u8 { + return self.data[0 .. self.len * self.kind.size()]; + } + + /// The elements as `T`; the caller has checked `kind`. + pub fn slice(self: *const JavaArray, comptime k: Kind) []k.Elem() { + std.debug.assert(self.kind == k); + const p: [*]k.Elem() = @ptrCast(self.data); + return p[0..self.len]; + } + + /// The Values of an `.object` array. + pub fn objects(self: *const JavaArray) []Value { + return self.slice(.object); } }; @@ -54,55 +145,215 @@ pub fn isArray(v: Value) bool { return v.tag() == .array; } -/// Allocate a `len`-element array, every slot initialised to `init_val`. The -/// per-constructor clj default (0 / 0.0 / false / \space / nil) is the caller's -/// choice (F-011), passed in as `init_val`. -pub fn make(rt: *Runtime, len: u32, init_val: Value) !Value { - const buf = try rt.gc.infra.alloc(Value, len); - errdefer rt.gc.infra.free(buf); - @memset(buf, init_val); +/// True when `v` is an array of kind `k`. +pub fn isKind(v: Value, k: Kind) bool { + return isArray(v) and asArray(v).kind == k; +} + +/// Allocate a `len`-element array of `kind`, every slot at the kind's default +/// (nil, false, 0, 0.0, the NUL char). +pub fn make(rt: *Runtime, kind: Kind, len: u32) !Value { + const buf = (try allocData(rt.gc.infra, len * kind.size()))[0 .. len * kind.size()]; + errdefer freeData(rt.gc.infra, buf); + rt.gc.noteOwnedAlloc(buf.len); + if (kind == .object) { + const vals: [*]Value = @ptrCast(buf.ptr); + @memset(vals[0..len], .nil_val); + } else @memset(buf, 0); const arr = try rt.gc.alloc(JavaArray); arr.* = .{ .header = HeapHeader.init(.array), .len = len, - .items_ptr = buf.ptr, + .kind = kind, + .data = buf.ptr, }; return Value.encodeHeapPtr(.array, arr); } -/// Allocate an array holding a copy of `src` (for `*-array` from a seq, -/// `to-array`, `aclone`). `src` is fully materialised before the alloc, so no -/// half-filled array is ever live across a GC. +/// Allocate an `.object` array holding a copy of `src` (`to-array`, a host +/// method returning `Object[]`). `src` is fully materialised before the +/// alloc, so no half-filled array is ever live across a GC. pub fn fromSlice(rt: *Runtime, src: []const Value) !Value { - const buf = try rt.gc.infra.alloc(Value, src.len); - errdefer rt.gc.infra.free(buf); - std.mem.copyForwards(Value, buf, src); - const arr = try rt.gc.alloc(JavaArray); - arr.* = .{ - .header = HeapHeader.init(.array), - .len = @intCast(src.len), - .items_ptr = buf.ptr, + const v = try make(rt, .object, @intCast(src.len)); + @memcpy(asArray(v).objects(), src); + return v; +} + +/// Allocate a `.byte` array holding a copy of `src`. +pub fn fromBytes(rt: *Runtime, src: []const u8) !Value { + const v = try make(rt, .byte, @intCast(src.len)); + @memcpy(asArray(v).bytes(), src); + return v; +} + +/// Element `i` of `a` as a Value. `i < a.len` is the caller's. Allocates only +/// for a `long` outside the immediate-integer range. +pub fn get(rt: *Runtime, a: *const JavaArray, i: usize) !Value { + return peek(a, i) orelse promote.wrapI64(rt, a.slice(.long)[i]); +} + +/// Element `i` of `a` as a Value without allocating, for the rt-free paths +/// (print, key hashing); null for the one element that needs a heap box, a +/// `long` outside the immediate-integer range. +pub fn peek(a: *const JavaArray, i: usize) ?Value { + return switch (a.kind) { + .object => a.slice(.object)[i], + .boolean => Value.initBoolean(a.slice(.boolean)[i] != 0), + .byte => Value.initInteger(a.slice(.byte)[i]), + .short => Value.initInteger(a.slice(.short)[i]), + .char => Value.initChar(@intCast(a.slice(.char)[i])), + .int => Value.initInteger(a.slice(.int)[i]), + .long => if (promote.inI48(a.slice(.long)[i])) Value.initInteger(a.slice(.long)[i]) else null, + .float => Value.initFloat(widenF32(a.slice(.float)[i])), + .double => Value.initFloat(a.slice(.double)[i]), }; - return Value.encodeHeapPtr(.array, arr); } -/// `(aget array idx)` — bounds-checked element read. -pub fn aget(arr: Value, idx: i64, fn_name: []const u8, loc: SourceLocation) !Value { - const a = asArray(arr); +/// The double a `float` element reads as: the shortest decimal that +/// round-trips the f32, so `(aget (float-array [1.1]) 0)` is 1.1 as clj's +/// boxed Float prints, not the f32's exact 1.100000023841858. cljw has no +/// single-precision Value; the storage is f32 so C sees a `float[]`. +fn widenF32(f: f32) f64 { + if (!std.math.isFinite(f)) return f; + var buf: [64]u8 = undefined; + const s = std.fmt.bufPrint(&buf, "{e}", .{f}) catch return f; + return std.fmt.parseFloat(f64, s) catch f; +} + +/// The elements of `a` as Values, in a `gpa` buffer the caller frees: the +/// shape for host code that walks any array as `[]Value` (`Arrays/asList`, +/// `vec`). Boxing a `long` outside the immediate range allocates, and the +/// buffer does not root those boxes, so the caller brackets the boxing and its +/// use with `rt.gc.enterFabrication()` when it allocates in between. +pub fn boxAll(rt: *Runtime, a: *const JavaArray, gpa: std.mem.Allocator) ![]Value { + const buf = try gpa.alloc(Value, a.len); + errdefer gpa.free(buf); + if (a.kind == .object) { + @memcpy(buf, a.objects()); + } else { + for (buf, 0..) |*slot, i| slot.* = try get(rt, a, i); + } + return buf; +} + +/// A vector of `a`'s elements, in order. +pub fn toVector(rt: *Runtime, a: *const JavaArray) !Value { + if (a.kind == .object) return vector_mod.fromSlice(rt, a.objects()); + rt.gc.enterFabrication(); + defer rt.gc.exitFabrication(); + const items = try boxAll(rt, a, rt.gpa); + defer rt.gpa.free(items); + return vector_mod.fromSlice(rt, items); +} + +/// How `store` turns a Value into an element. `strict` is `aset`: the value +/// must already be of the element's category (an integer in range for an +/// integer kind, a number for a floating kind, a char for `char`, a boolean +/// for `boolean`). `convert` is the constructors' Number conversion (clj +/// `RT.byteArray` and friends): an integer wraps to the width, a float +/// truncates into an integer kind, an integer is a char's code point, any +/// value is a boolean by truthiness. +pub const Coerce = enum { strict, convert }; + +/// The i64 an integer-kind element receives from `v`, or null on a mismatch. +fn integerOf(v: Value, mode: Coerce) ?i64 { + return switch (v.tag()) { + .integer => v.asInteger(), + .big_int => promote.exactI64(v) catch null, + .char => if (mode == .convert) @as(i64, v.asChar()) else null, + .float, .ratio, .big_decimal => if (mode == .convert) blk: { + const f = promote.toF64(v) catch break :blk null; + if (!std.math.isFinite(f)) break :blk null; + break :blk std.math.lossyCast(i64, f); + } else null, + else => null, + }; +} + +fn floatOf(v: Value) ?f64 { + return switch (v.tag()) { + .integer, .float, .big_int, .ratio, .big_decimal => promote.toF64(v) catch null, + else => null, + }; +} + +fn mismatch(a: *const JavaArray, v: Value, fn_name: []const u8, loc: SourceLocation) anyerror { + return error_catalog.raise(.arg_value_invalid, loc, .{ + .fn_name = fn_name, + .expected = switch (a.kind) { + .object => unreachable, + .boolean => "a boolean for a boolean array", + .char => "a char for a char array", + .float, .double => "a number for a floating-point array", + else => "an integer for an integer array", + }, + .actual = @tagName(v.tag()), + }); +} + +/// Store `v` into element `i` of `a` (`i < a.len` is the caller's). GC-safe: +/// an `.object` slot is already traced (non-moving mark-sweep, no barrier) and +/// `v` is a rooted argument; nothing allocates in between. +pub fn store(a: *const JavaArray, i: usize, v: Value, mode: Coerce, fn_name: []const u8, loc: SourceLocation) !void { + switch (a.kind) { + .object => a.slice(.object)[i] = v, + .boolean => a.slice(.boolean)[i] = switch (v.tag()) { + .boolean => @intFromBool(v == Value.true_val), + else => if (mode == .convert) @intFromBool(v.isTruthy()) else return mismatch(a, v, fn_name, loc), + }, + .char => a.slice(.char)[i] = switch (v.tag()) { + .char => v.asChar(), + .integer => if (mode == .convert and v.asInteger() >= 0 and v.asInteger() <= std.math.maxInt(u21)) + @intCast(v.asInteger()) + else + return mismatch(a, v, fn_name, loc), + else => return mismatch(a, v, fn_name, loc), + }, + .float => a.slice(.float)[i] = @floatCast(floatOf(v) orelse return mismatch(a, v, fn_name, loc)), + .double => a.slice(.double)[i] = floatOf(v) orelse return mismatch(a, v, fn_name, loc), + inline .byte, .short, .int, .long => |k| { + const x = integerOf(v, mode) orelse return mismatch(a, v, fn_name, loc); + const T = k.Elem(); + a.slice(k)[i] = std.math.cast(T, x) orelse switch (mode) { + .convert => @bitCast(@as(std.meta.Int(.unsigned, @bitSizeOf(T)), @truncate(@as(u64, @bitCast(x))))), + .strict => return error_catalog.raise(.arg_value_invalid, loc, .{ + .fn_name = fn_name, + .expected = "a value in range for a " ++ @tagName(k) ++ " array", + .actual = "out-of-range integer", + }), + }; + }, + } +} + +/// Set every element of `a` to `v` (`Arrays/fill`, the `(int-array n init)` +/// constructor form). +pub fn fill(a: *const JavaArray, v: Value, mode: Coerce, fn_name: []const u8, loc: SourceLocation) !void { + if (a.len == 0) return; + try store(a, 0, v, mode, fn_name, loc); + const sz = a.kind.size(); + const b = a.bytes(); + var i: usize = 1; + while (i < a.len) : (i += 1) @memcpy(b[i * sz ..][0..sz], b[0..sz]); +} + +fn checkIndex(a: *const JavaArray, idx: i64, fn_name: []const u8, loc: SourceLocation) !usize { if (idx < 0 or idx >= a.len) return error_catalog.raise(.index_out_of_range, loc, .{ .fn_name = fn_name }); - return a.items()[@intCast(idx)]; + return @intCast(idx); +} + +/// `(aget array idx)`: bounds-checked element read. +pub fn aget(rt: *Runtime, arr: Value, idx: i64, fn_name: []const u8, loc: SourceLocation) !Value { + const a = asArray(arr); + return get(rt, a, try checkIndex(a, idx, fn_name, loc)); } -/// `(aset array idx val)` — in-place write, returns `val` (clj semantics). -/// GC-safe: the slot is already traced (non-moving mark-sweep, no barrier) and -/// `val` is an already-rooted argument; no allocation between, so no collection -/// runs mid-write (same reasoning as `TypedInstance.setField`). +/// `(aset array idx val)`: bounds-checked, strictly coerced in-place write; +/// returns `val` (clj semantics). pub fn aset(arr: Value, idx: i64, val: Value, fn_name: []const u8, loc: SourceLocation) !Value { const a = asArray(arr); - if (idx < 0 or idx >= a.len) - return error_catalog.raise(.index_out_of_range, loc, .{ .fn_name = fn_name }); - a.items_ptr[@intCast(idx)] = val; + try store(a, try checkIndex(a, idx, fn_name, loc), val, .strict, fn_name, loc); return val; } @@ -110,33 +361,45 @@ pub fn alength(arr: Value) u32 { return asArray(arr).len; } -/// `(aclone array)` — a fresh array with copied contents (not `identical?`). +/// `(aclone array)`: a fresh array of the same kind with copied contents +/// (not `identical?`). pub fn aclone(rt: *Runtime, arr: Value) !Value { - return fromSlice(rt, asArray(arr).items()); + const a = asArray(arr); + const out = try make(rt, a.kind, a.len); + @memcpy(asArray(out).bytes(), a.bytes()); + return out; } -/// Trace fn for `.array` — mark every element's heap reference. Mirrors -/// `traceTypedInstance`. +/// Copy `n` elements from `src[src_pos..]` to `dst[dst_pos..]`, overlap-safe +/// (`System/arraycopy`). The kinds must match, the range is the caller's. +pub fn copy(src: *const JavaArray, src_pos: usize, dst: *const JavaArray, dst_pos: usize, n: usize) void { + std.debug.assert(src.kind == dst.kind); + const sz = src.kind.size(); + @memmove(dst.bytes()[dst_pos * sz ..][0 .. n * sz], src.bytes()[src_pos * sz ..][0 .. n * sz]); +} + +/// Trace fn for `.array`: mark every element's heap reference of an +/// `.object` array; a primitive array holds no references. fn traceJavaArray(gc_ptr: *anyopaque, header: *HeapHeader) void { const gc: *gc_heap_mod.GcHeap = @ptrCast(@alignCast(gc_ptr)); const arr: *JavaArray = @ptrCast(@alignCast(header)); - var i: u32 = 0; - while (i < arr.len) : (i += 1) { - if (arr.items_ptr[i].heapHeader()) |hdr| mark_sweep.mark(gc, hdr); + if (arr.kind != .object) return; + for (arr.objects()) |v| { + if (v.heapHeader()) |hdr| mark_sweep.mark(gc, hdr); } } -/// Finaliser for `.array` — release the `gc.infra`-owned element slice. +/// Finaliser for `.array`: release the `gc.infra`-owned element buffer. fn finaliseJavaArray(gc_ptr: *anyopaque, header: *HeapHeader) void { const gc: *gc_heap_mod.GcHeap = @ptrCast(@alignCast(gc_ptr)); const arr: *JavaArray = @ptrCast(@alignCast(header)); - gc.infra.free(arr.items_ptr[0..arr.len]); + freeData(gc.infra, arr.bytes()); } /// D-573: the element storage finaliseJavaArray frees. fn ownedBytes(header: *HeapHeader) usize { const arr: *JavaArray = @ptrCast(@alignCast(header)); - return arr.len * @sizeOf(@TypeOf(arr.items_ptr[0])); + return arr.bytes().len; } pub fn registerGcHooks() void { @@ -155,13 +418,13 @@ test "make + aget/aset + alength" { var rt = Runtime.init(th.io(), testing.allocator); defer rt.deinit(); - const a = try make(&rt, 3, Value.nil_val); + const a = try make(&rt, .object, 3); try testing.expectEqual(@as(u32, 3), alength(a)); - try testing.expect((try aget(a, 0, "aget", .{})).isNil()); + try testing.expect((try aget(&rt, a, 0, "aget", .{})).isNil()); _ = try aset(a, 1, Value.initInteger(42), "aset", .{}); - try testing.expectEqual(@as(i48, 42), (try aget(a, 1, "aget", .{})).asInteger()); - try testing.expect((try aget(a, 0, "aget", .{})).isNil()); + try testing.expectEqual(@as(i48, 42), (try aget(&rt, a, 1, "aget", .{})).asInteger()); + try testing.expect((try aget(&rt, a, 0, "aget", .{})).isNil()); } test "aget/aset out of range raises" { @@ -170,9 +433,9 @@ test "aget/aset out of range raises" { var rt = Runtime.init(th.io(), testing.allocator); defer rt.deinit(); - const a = try make(&rt, 2, Value.nil_val); - try testing.expectError(error.IndexError, aget(a, 5, "aget", .{})); - try testing.expectError(error.IndexError, aget(a, -1, "aget", .{})); + const a = try make(&rt, .long, 2); + try testing.expectError(error.IndexError, aget(&rt, a, 5, "aget", .{})); + try testing.expectError(error.IndexError, aget(&rt, a, -1, "aget", .{})); try testing.expectError(error.IndexError, aset(a, 2, Value.initInteger(1), "aset", .{})); } @@ -182,10 +445,54 @@ test "aclone copies, not identical" { var rt = Runtime.init(th.io(), testing.allocator); defer rt.deinit(); - const a = try make(&rt, 2, Value.initInteger(7)); + const a = try make(&rt, .int, 2); + try fill(asArray(a), Value.initInteger(7), .strict, "fill", .{}); const b = try aclone(&rt, a); - try testing.expect(a != b); // distinct Values (different heap ptrs) - try testing.expectEqual(@as(i48, 7), (try aget(b, 0, "aget", .{})).asInteger()); + try testing.expect(a != b); + try testing.expectEqual(Kind.int, asArray(b).kind); + try testing.expectEqual(@as(i48, 7), (try aget(&rt, b, 0, "aget", .{})).asInteger()); _ = try aset(a, 0, Value.initInteger(99), "aset", .{}); - try testing.expectEqual(@as(i48, 7), (try aget(b, 0, "aget", .{})).asInteger()); // clone unaffected + try testing.expectEqual(@as(i48, 7), (try aget(&rt, b, 0, "aget", .{})).asInteger()); +} + +test "a primitive array is contiguous native storage of its width" { + var th = std.Io.Threaded.init(testing.allocator, .{}); + defer th.deinit(); + var rt = Runtime.init(th.io(), testing.allocator); + defer rt.deinit(); + + const b = try make(&rt, .byte, 3); + _ = try aset(b, 0, Value.initInteger(-1), "aset", .{}); + try testing.expectError(error.ValueError, aset(b, 1, Value.initInteger(300), "aset", .{})); + try store(asArray(b), 1, Value.initInteger(300), .convert, "byte-array", .{}); // wraps to 44 + try testing.expectEqualSlices(u8, &.{ 0xFF, 44, 0 }, asArray(b).bytes()); + + const f = try make(&rt, .float, 1); + _ = try aset(f, 0, Value.initFloat(1.1), "aset", .{}); + try testing.expectEqual(@as(f64, 1.1), (try aget(&rt, f, 0, "aget", .{})).asFloat()); + + const l = try make(&rt, .long, 1); + _ = try aset(l, 0, try promote.wrapI64(&rt, std.math.maxInt(i64)), "aset", .{}); + try testing.expectEqual(std.math.maxInt(i64), asArray(l).slice(.long)[0]); + try testing.expectEqual(std.math.maxInt(i64), try promote.exactI64(try aget(&rt, l, 0, "aget", .{}))); + + const d = try make(&rt, .double, 1); + _ = try aset(d, 0, Value.initInteger(2), "aset", .{}); + try testing.expectEqual(@as(f64, 2), asArray(d).slice(.double)[0]); +} + +test "aset rejects a value outside the element category; the ctor mode converts" { + var th = std.Io.Threaded.init(testing.allocator, .{}); + defer th.deinit(); + var rt = Runtime.init(th.io(), testing.allocator); + defer rt.deinit(); + + const l = try make(&rt, .long, 1); + try testing.expectError(error.ValueError, aset(l, 0, Value.initFloat(1.5), "aset", .{})); + try testing.expectError(error.ValueError, aset(l, 0, .nil_val, "aset", .{})); + try store(asArray(l), 0, Value.initFloat(1.9), .convert, "long-array", .{}); + try testing.expectEqual(@as(i64, 1), asArray(l).slice(.long)[0]); + + const z = try make(&rt, .boolean, 1); + try testing.expectError(error.ValueError, aset(z, 0, Value.initInteger(1), "aset", .{})); } diff --git a/src/runtime/collection/string.zig b/src/runtime/collection/string.zig index 67a78fa48..272fe4f1b 100644 --- a/src/runtime/collection/string.zig +++ b/src/runtime/collection/string.zig @@ -56,6 +56,7 @@ pub const String = extern struct { pub fn alloc(rt: *Runtime, byte_view: []const u8) !Value { const owned = try rt.gc.infra.dupe(u8, byte_view); errdefer rt.gc.infra.free(owned); + rt.gc.noteOwnedAlloc(owned.len); const s = try rt.gc.alloc(String); s.* = .{ .header = HeapHeader.init(.string), diff --git a/src/runtime/collection/transient/transient_vector.zig b/src/runtime/collection/transient/transient_vector.zig index d29ed86d7..f3d187672 100644 --- a/src/runtime/collection/transient/transient_vector.zig +++ b/src/runtime/collection/transient/transient_vector.zig @@ -96,6 +96,7 @@ pub fn fromVector(rt: *Runtime, source: Value) !Value { const buf = try rt.gc.infra.alloc(Value, initial_cap); errdefer rt.gc.infra.free(buf); + rt.gc.noteOwnedAlloc(buf.len * @sizeOf(Value)); var i: u32 = 0; while (i < src_count) : (i += 1) { buf[i] = vector.nth(source, i); @@ -273,7 +274,7 @@ fn ensureCapacity(rt: *Runtime, tv: *TransientVector, needed: u32) !void { // OOM-kills the process before `persistent!` (gc.alloc) would trip the cap // (the D-361 Linux symptom). Refuse here so the cap surfaces as the // catchable/uncatchable eval_heap_exceeded during the build, on every host. - try rt.gc.checkInfraCap(new_cap * @sizeOf(Value)); + try rt.gc.chargeInfraBytes(new_cap * @sizeOf(Value)); const new_buf = try rt.gc.infra.realloc(old_buf, new_cap); tv.items_ptr = new_buf.ptr; tv.capacity = new_cap; diff --git a/src/runtime/collection/vector.zig b/src/runtime/collection/vector.zig index 971b94f58..2ddb358b3 100644 --- a/src/runtime/collection/vector.zig +++ b/src/runtime/collection/vector.zig @@ -129,13 +129,35 @@ pub fn empty() Value { return Value.encodeHeapPtr(.vector, &EMPTY); } +/// A fresh HamtNode holding a copy of `slots`. Writes the header and the 32 +/// slots once; `.{ .header = … }` + `@memcpy` wrote every slot twice (nil +/// default, then the copy). PERF: [refs: O-074] +inline fn copyHamt(rt: *Runtime, slots: *const [BRANCH_FACTOR]Value) !*HamtNode { + const c = try rt.gc.alloc(HamtNode); + c.header = HeapHeader.init(.hamt_node); + c._pad = .{ 0, 0, 0, 0, 0, 0 }; + c.slots = slots.*; + return c; +} + +/// A fresh TailNode of length `len` whose live prefix is `prefix` (len >= +/// prefix.len). Slots past `len` stay nil so the node matches a `.{}`-built +/// one; slots in `[prefix.len, len)` are the caller's to fill. PERF: [refs: O-074] +inline fn newTail(rt: *Runtime, len: u32, prefix: []const Value) !*TailNode { + const t = try rt.gc.alloc(TailNode); + t.header = HeapHeader.init(.tail_node); + t._pad = .{ 0, 0 }; + t.len = len; + @memcpy(t.slots[0..prefix.len], prefix); + @memset(t.slots[len..], Value.nil_val); + return t; +} + /// HamtNode.dupe — shallow copy with a fresh header. Used inside /// popTail's recursion when descending into a slot that already /// exists but isn't being modified at this level. fn dupeHamt(parent: *const HamtNode, rt: *Runtime) !*HamtNode { - const c = try rt.gc.alloc(HamtNode); - c.* = .{ .header = HeapHeader.init(.hamt_node) }; - @memcpy(&c.slots, &parent.slots); + const c = try copyHamt(rt, &parent.slots); return c; } @@ -206,11 +228,8 @@ pub fn conj(rt: *Runtime, v: Value, x: Value) !Value { // Fast path: tail has space (< BRANCH_FACTOR elements). const tail_size = if (old.tail) |t| t.len else 0; if (tail_size < BRANCH_FACTOR) { - const new_tail = try rt.gc.alloc(TailNode); - new_tail.* = .{ .header = HeapHeader.init(.tail_node), .len = tail_size + 1 }; - if (old.tail) |t| { - @memcpy(new_tail.slots[0..tail_size], t.slots[0..tail_size]); - } + const prefix: []const Value = if (old.tail) |t| t.slots[0..tail_size] else &.{}; + const new_tail = try newTail(rt, tail_size + 1, prefix); new_tail.slots[tail_size] = x; return try newVector(rt, old.count + 1, old.shift, old.root, new_tail, old.meta); } @@ -218,9 +237,7 @@ pub fn conj(rt: *Runtime, v: Value, x: Value) !Value { // Tail-full path: promote the old tail to a leaf HamtNode under // root; new tail holds just `x`. The promoted leaf goes into the // root trie via pushTail, possibly growing the trie depth. - const tail_as_leaf = try rt.gc.alloc(HamtNode); - tail_as_leaf.* = .{ .header = HeapHeader.init(.hamt_node) }; - @memcpy(&tail_as_leaf.slots, &old.tail.?.slots); + const tail_as_leaf = try copyHamt(rt, &old.tail.?.slots); const old_count = old.count; var new_shift = old.shift; @@ -323,9 +340,7 @@ pub fn assoc(rt: *Runtime, v: Value, i: u32, x: Value) !Value { if (i >= tailoff(old.count)) { const tail_size = if (old.tail) |t| t.len else 0; - const new_tail = try rt.gc.alloc(TailNode); - new_tail.* = .{ .header = HeapHeader.init(.tail_node), .len = tail_size }; - @memcpy(new_tail.slots[0..tail_size], old.tail.?.slots[0..tail_size]); + const new_tail = try newTail(rt, tail_size, old.tail.?.slots[0..tail_size]); new_tail.slots[i - tailoff(old.count)] = x; return try newVector(rt, old.count, old.shift, old.root, new_tail, old.meta); } @@ -339,9 +354,7 @@ pub fn assoc(rt: *Runtime, v: Value, i: u32, x: Value) !Value { /// duplicate the node at each level, overwrite the slot on the path /// to `i`, recurse until shift==0 then overwrite the leaf slot. fn assocInRoot(rt: *Runtime, level: u32, node: *const HamtNode, i: u32, x: Value) !*HamtNode { - const dup = try rt.gc.alloc(HamtNode); - dup.* = .{ .header = HeapHeader.init(.hamt_node) }; - @memcpy(&dup.slots, &node.slots); + const dup = try copyHamt(rt, &node.slots); if (level == 0) { dup.slots[i & MASK] = x; @@ -410,9 +423,7 @@ pub fn fromSlice(rt: *Runtime, items: []const Value) !Value { defer rt.gc.infra.free(level_nodes); var li: u32 = 0; while (li < leaf_count) : (li += 1) { - const leaf = try rt.gc.alloc(HamtNode); - leaf.* = .{ .header = HeapHeader.init(.hamt_node) }; - @memcpy(&leaf.slots, items[li * BRANCH_FACTOR ..][0..BRANCH_FACTOR]); + const leaf = try copyHamt(rt, items[li * BRANCH_FACTOR ..][0..BRANCH_FACTOR]); level_nodes[li] = leaf; } @@ -487,9 +498,7 @@ fn pushTail( tail_leaf: *HamtNode, ) !*HamtNode { const sub_index = (tail_offset >> @as(u5, @intCast(level))) & MASK; - const new_parent = try rt.gc.alloc(HamtNode); - new_parent.* = .{ .header = HeapHeader.init(.hamt_node) }; - @memcpy(&new_parent.slots, &parent.slots); + const new_parent = try copyHamt(rt, &parent.slots); if (level == SHIFT_BITS) { // Parent is one above leaf level — slot it directly. @@ -530,16 +539,12 @@ fn popTail(rt: *Runtime, level: u32, parent: *const HamtNode, new_count: u32) !? if (new_child == null and sub_index == 0) { return null; // whole subtree gone } - const new_parent = try rt.gc.alloc(HamtNode); - new_parent.* = .{ .header = HeapHeader.init(.hamt_node) }; - @memcpy(&new_parent.slots, &parent.slots); + const new_parent = try copyHamt(rt, &parent.slots); new_parent.slots[sub_index] = if (new_child) |c| Value.encodeHeapPtr(.hamt_node, c) else Value.nil_val; return new_parent; } if (sub_index == 0) return null; - const new_parent = try rt.gc.alloc(HamtNode); - new_parent.* = .{ .header = HeapHeader.init(.hamt_node) }; - @memcpy(&new_parent.slots, &parent.slots); + const new_parent = try copyHamt(rt, &parent.slots); new_parent.slots[sub_index] = Value.nil_val; return new_parent; } diff --git a/src/runtime/dispatch.zig b/src/runtime/dispatch.zig index 58e7ebf9c..dd1ec201b 100644 --- a/src/runtime/dispatch.zig +++ b/src/runtime/dispatch.zig @@ -130,21 +130,75 @@ pub fn dispatch( args: []const Value, loc: SourceLocation, ) anyerror!Value { - if (try dispatchOrNull(rt, env, cs, receiver, protocol_name, method_name, args, loc)) |v| return v; + const td = try resolveDescriptor(rt, receiver); + const mv = try resolveProtocolMethod(rt, cs, receiver, td, protocol_name, method_name, loc); + const vt = rt.vtable orelse return error.NoVTable; + return try vt.callFn(rt, env, mv, args, loc); +} + +/// `dispatch` through a per-`ProtocolFn` `DispatchCache` (ADR-0206). The +/// cache memoises the WHOLE `resolveProtocolMethod` chain per receiver +/// descriptor, so a hit skips every table scan and string compare. The +/// result is a pure function of `(td, protocol_generation)`; every input to +/// it bumps the generation when it changes, so a hit is always current. +pub fn dispatchCached( + rt: *Runtime, + env: *Env, + cache: *method_table.DispatchCache, + receiver: Value, + protocol_name: []const u8, + method_name: []const u8, + args: []const Value, + loc: SourceLocation, +) anyerror!Value { + const td = try resolveDescriptor(rt, receiver); + const gen = rt.protocol_generation; + // PERF: per-protocol-fn (td, generation) -> method memo; naive form is + // `dispatch` with a fresh CallSite per call, which re-ran the method-table + // scan + fallback chain on every protocol call [refs: O-081] + const mv = cache.lookup(td, gen) orelse blk: { + var cs: CallSite = .{}; + const resolved = try resolveProtocolMethod(rt, &cs, receiver, td, protocol_name, method_name, loc); + cache.fill(td, gen, resolved); + break :blk resolved; + }; + const vt = rt.vtable orelse return error.NoVTable; + return try vt.callFn(rt, env, mv, args, loc); +} + +/// Resolve the method Value a protocol call on `receiver` (descriptor `td`) +/// runs, in clj's precedence order, or raise `protocol_no_satisfies`. Pure in +/// `(td, rt.protocol_generation)`: it reads only `td`'s method tables, the +/// ADR-0187 side table and the IRecord / Object class descriptors, and every +/// writer of those bumps the generation. `receiver` is consulted only for its +/// tag and its descriptor's kind, both functions of `td`. +pub fn resolveProtocolMethod( + rt: *Runtime, + cs: *CallSite, + receiver: Value, + td: *const td_mod.TypeDescriptor, + protocol_name: []const u8, + method_name: []const u8, + loc: SourceLocation, +) anyerror!Value { + if (cs.lookupWithCache(td, protocol_name, method_name, rt.protocol_generation)) |me| { + if (me.method_val.tag() == .nil) return error_catalog.raise(.feature_not_supported, loc, .{ + .name = "protocol method with nil method_val (declare-only)", + }); + return me.method_val; + } // ADR-0187: `(extend-protocol P some.ns.Q …)` where Q is a PROTOCOL means // "every type satisfying Q". Consulted BEFORE the Object default because on // the JVM an interface impl is a real per-type impl and outranks an Object // fallback. Membership is answered from the receiver's descriptor // (`protocol_impls` is the SSOT, D-190), so no receiver value is needed. if (rt.protocol_target_exts.get(protocol_name)) |exts| { - const rtd = try resolveDescriptor(rt, receiver); for (exts) |ext| { - if (!protocol_mod.satisfiesName(ext.target_proto, rtd)) continue; + if (!protocol_mod.satisfiesName(ext.target_proto, td)) continue; for (ext.impls) |me| { if (!std.mem.eql(u8, me.method_name, method_name)) continue; if (me.method_val.tag() == .nil) break; - const vt = rt.vtable orelse return error.NoVTable; - return try vt.callFn(rt, env, me.method_val, args, loc); + return me.method_val; } } } @@ -152,15 +206,10 @@ pub fn dispatch( // `(extend-protocol P clojure.lang.IRecord …)` impl (registered on the // IRecord class descriptor) serves any record with no impl of its own. // Like an interface impl on the JVM it outranks the Object default. - if (receiver.tag() == .typed_instance and - receiver.decodePtr(*const td_mod.TypedInstance).descriptor.kind == .defrecord) - { + if (receiver.tag() == .typed_instance and td.kind == .defrecord) { if (rt.class_descriptors.get("IRecord")) |rec_td| { if (rec_td.lookupMethod(protocol_name, method_name)) |me| { - if (me.method_val.tag() != .nil) { - const vt = rt.vtable orelse return error.NoVTable; - return try vt.callFn(rt, env, me.method_val, args, loc); - } + if (me.method_val.tag() != .nil) return me.method_val; } } } @@ -172,14 +221,10 @@ pub fn dispatch( if (receiver.tag() != .nil) { if (rt.class_descriptors.get("Object")) |obj_td| { if (obj_td.lookupMethod(protocol_name, method_name)) |me| { - if (me.method_val.tag() != .nil) { - const vt = rt.vtable orelse return error.NoVTable; - return try vt.callFn(rt, env, me.method_val, args, loc); - } + if (me.method_val.tag() != .nil) return me.method_val; } } } - const td = try resolveDescriptor(rt, receiver); return error_catalog.raise(.protocol_no_satisfies, loc, .{ .protocol = protocol_name, .method = method_name, diff --git a/src/runtime/dispatch/method_table.zig b/src/runtime/dispatch/method_table.zig index 99c515e8a..ca2c565bb 100644 --- a/src/runtime/dispatch/method_table.zig +++ b/src/runtime/dispatch/method_table.zig @@ -15,6 +15,7 @@ //! (D-040, discharged). const std = @import("std"); +const builtin = @import("builtin"); const TypeDescriptor = @import("../type_descriptor.zig").TypeDescriptor; const Value = @import("../value/value.zig").Value; @@ -81,6 +82,77 @@ pub const CallSite = struct { } }; +/// Per-protocol-method resolution cache, embedded in every `ProtocolFn` +/// (ADR-0206). Maps the receiver's TypeDescriptor to the FINAL resolved +/// method Value of the whole `dispatch.resolveProtocolMethod` chain +/// (per-type impl, then the ADR-0187 protocol-target, IRecord and Object +/// fallbacks), all of which are a pure function of `(td, protocol_generation)`. +/// Every mutation of a method table or of the fallback side tables bumps +/// `protocol_generation`, so a generation mismatch is the whole +/// invalidation story: extend-type, extend-protocol, extend, a re-extend that +/// replaces a row, and a redefined record (a NEW descriptor address) all miss. +/// Descriptors are never freed while the runtime lives (D-587 retirement), so +/// a stale `td` address cannot be reused by another type (no ABA). +/// Host-surface `init`s (`appendMethodEntries`, ADR-0174) write rows with +/// `protocol_name = ""` and no bump: interop members that a protocol lookup +/// never matches, so they cannot stale a memo. A failed resolution raises and +/// is never filled, so a `class_descriptors` insert (no bump) cannot either. +/// The cached `method_val` is not traced: its row roots it, and a replaced row +/// bumps the generation, so it is only returned while still rooted (a reify +/// descriptor's row lives until process exit, `trackHeap`). +/// Known gap (D-600): `protocol_generation` is a plain `+%= 1`, so two extends +/// racing on different threads can lose a bump. +/// +/// Unlike `CallSite` (analyzer-arena, one owner), a `ProtocolFn` is shared by +/// every thread that calls it, so the slot is a seqlock: a reader that +/// observes an odd or changed `seq` treats the read as a miss, and a writer +/// that loses the even-to-odd race simply skips the fill. A torn +/// `(td, val)` pair therefore never dispatches. +pub const DispatchCache = extern struct { + seq: u32 = 0, + generation: u32 = 0, + /// `@intFromPtr` of the cached receiver descriptor; 0 = empty. + td_addr: usize = 0, + method_val: Value = .nil_val, + + pub fn lookup(self: *const DispatchCache, td: *const TypeDescriptor, generation: u32) ?Value { + const s1 = @atomicLoad(u32, &self.seq, .acquire); + if (s1 & 1 != 0) return null; + const addr = @atomicLoad(usize, &self.td_addr, .acquire); + const gen = @atomicLoad(u32, &self.generation, .acquire); + const val = loadMethodVal(&self.method_val); + if (@atomicLoad(u32, &self.seq, .acquire) != s1) return null; + if (addr != @intFromPtr(td) or gen != generation) return null; + return val; + } + + pub fn fill(self: *DispatchCache, td: *const TypeDescriptor, generation: u32, method_val: Value) void { + const s0 = @atomicLoad(u32, &self.seq, .monotonic); + if (s0 & 1 != 0) return; + if (@cmpxchgStrong(u32, &self.seq, s0, s0 +% 1, .acquire, .monotonic) != null) return; + @atomicStore(usize, &self.td_addr, @intFromPtr(td), .release); + @atomicStore(u32, &self.generation, generation, .release); + storeMethodVal(&self.method_val, method_val); + @atomicStore(u32, &self.seq, s0 +% 2, .release); + } +}; + +/// A `Value` is 64 bits wide, and 32-bit wasm has no 64-bit atomics. The +/// single-threaded targets (wasm32-wasi, ADR-0193) have no concurrent reader, +/// so a plain access is the whole seqlock there. +inline fn loadMethodVal(p: *const Value) Value { + if (comptime builtin.single_threaded) return p.*; + return @atomicLoad(Value, p, .acquire); +} + +inline fn storeMethodVal(p: *Value, v: Value) void { + if (comptime builtin.single_threaded) { + p.* = v; + return; + } + @atomicStore(Value, p, v, .release); +} + // --- tests --- const testing = std.testing; @@ -197,3 +269,18 @@ test "CallSite cache slots accept TypeDescriptor + MethodEntry pointers" { try testing.expect(cs.last_type.? == &td); try testing.expectEqualStrings("first", cs.last_method.?.method_name); } + +test "DispatchCache: hit on same (td, generation), miss on generation bump or another td" { + const td_a: TypeDescriptor = .{ .fqcn = "A", .kind = .deftype, .field_layout = null, .protocol_impls = &.{}, .method_table = &.{}, .parent = null, .meta = Value.nil_val }; + const td_b: TypeDescriptor = .{ .fqcn = "B", .kind = .deftype, .field_layout = null, .protocol_impls = &.{}, .method_table = &.{}, .parent = null, .meta = Value.nil_val }; + var c: DispatchCache = .{}; + try testing.expect(c.lookup(&td_a, 0) == null); + c.fill(&td_a, 3, Value.initInteger(7)); + try testing.expectEqual(Value.initInteger(7), c.lookup(&td_a, 3).?); + try testing.expect(c.lookup(&td_a, 4) == null); + try testing.expect(c.lookup(&td_b, 3) == null); + c.fill(&td_b, 4, Value.initInteger(9)); + try testing.expectEqual(Value.initInteger(9), c.lookup(&td_b, 4).?); + try testing.expect(c.lookup(&td_a, 3) == null); + try testing.expectEqual(@as(u32, 4), c.seq); +} diff --git a/src/runtime/equal.zig b/src/runtime/equal.zig index 042cc2e14..9650bd78f 100644 --- a/src/runtime/equal.zig +++ b/src/runtime/equal.zig @@ -58,6 +58,7 @@ const local_date_time_value_mod = @import("time/local_date_time_value.zig"); const local_date_value_mod = @import("time/local_date_value.zig"); const local_time_value_mod = @import("time/local_time_value.zig"); const instant_mod = @import("time/instant.zig"); +const zone_offset_mod = @import("time/zone_offset.zig"); const dispatch_mod = @import("dispatch.zig"); const root_set = @import("gc/root_set.zig"); const ClojureWasmError = @import("error/info.zig").ClojureWasmError; @@ -193,7 +194,7 @@ const Cursor = union(enum) { }, .aseq => |*s| { if (s.i >= s.n) return null; - const e = array_seq.nth(s.v, s.i); + const e = try array_seq.nth(rt, s.v, s.i); s.i += 1; return e; }, @@ -740,6 +741,7 @@ const temporal_kinds = [_]TemporalKind{ .{ .is = local_date_time_value_mod.isLocalDateTime, .eq = temporalEq.localDateTime, .hash = temporalHash.localDateTime }, .{ .is = local_date_value_mod.isLocalDate, .eq = temporalEq.localDate, .hash = temporalHash.localDate }, .{ .is = local_time_value_mod.isLocalTime, .eq = temporalEq.localTime, .hash = temporalHash.localTime }, + .{ .is = zone_offset_mod.isZone, .eq = temporalEq.zone, .hash = temporalHash.zone }, }; /// The `temporal_kinds` row for `v`, or null. Only a `.native` (host) @@ -783,6 +785,10 @@ const temporalEq = struct { fn localTime(a: Value, b: Value) bool { return local_time_value_mod.nanoOfDayOf(a) == local_time_value_mod.nanoOfDayOf(b); } + // ZoneOffset / ZoneRegion: equal by id, so the class (prefix) must match too. + fn zone(a: Value, b: Value) bool { + return std.meta.eql(zone_offset_mod.zoneOf(a), zone_offset_mod.zoneOf(b)); + } }; /// The JDK hashCode formulas, one per type (java.util.Date, java.sql.Timestamp, @@ -825,6 +831,15 @@ const temporalHash = struct { fn localTime(v: Value) u32 { return long(local_time_value_mod.nanoOfDayOf(v)); } + // ZoneOffset: `totalSeconds`; ZoneRegion: `String.hashCode` of its id. + fn zone(v: Value) u32 { + const z = zone_offset_mod.zoneOf(v); + if (!z.isRegion()) return @bitCast(z.seconds); + var buf: [24]u8 = undefined; + var h: u32 = 0; + for (zone_offset_mod.formatId(&buf, z)) |c| h = h *% 31 +% c; + return h; + } // LocalDateTime: `date.hashCode() ^ time.hashCode()`. fn localDateTime(v: Value) u32 { return civilDay(local_date_time_value_mod.epochDayOf(v)) ^ @@ -1088,7 +1103,12 @@ const SeqKeyCursor = struct { }, .aseq => |*s| { if (s.i >= s.n) return null; - const e = array_seq.nth(s.v, s.i); + const e = array_seq.peekNth(s.v, s.i) orelse { + // A primitive long that needs a heap box: not realizable + // rt-free, so the key falls back to identity. + self.truncated = true; + return null; + }; s.i += 1; return e; }, diff --git a/src/runtime/error/catalog.zig b/src/runtime/error/catalog.zig index e9af3e6b7..e1a4e0f41 100644 --- a/src/runtime/error/catalog.zig +++ b/src/runtime/error/catalog.zig @@ -176,6 +176,36 @@ pub const Code = enum { future_cancel_abort, executor_rejected, stm_no_transaction, + // --- java.time (DateTimeException family, keyword date_time_format) --- + /// args: `.{ .detail = "..." }` — a java.time value or resolve step is + /// invalid (out-of-range offset, field value outside its range, a resolve + /// conflict). The engine composes the JVM-parity detail text. + date_time_value_invalid, + /// args: `.{ .text = "...", .detail = "..." }` — a DateTimeFormatter could not + /// parse `text`; `detail` is " at index N", ", unparsed text found at index + /// N" or ": " (JVM DateTimeParseException wording). + date_time_text_unparseable, + /// args: `.{ .field = "YearOfEra" }` — a temporal value does not carry the + /// field a formatter or getter asked for. + temporal_field_unsupported, + /// args: `.{ .target = "LocalDate", .temporal = "...", .class = "..." }` — + /// `X/from` got a TemporalAccessor that lacks the parts X needs. + temporal_accessor_unconvertible, + /// args: `.{ .class = "java.util.Date" }` — a formatter was handed a value + /// that is not a TemporalAccessor at all. + temporal_accessor_expected, + /// args: `.{ .detail = "..." }` — a DateTimeFormatter pattern or builder call + /// is malformed (JVM IllegalArgumentException wording). + date_time_pattern_invalid, + /// args: `.{ .letter = "w" }` — a valid JVM pattern letter cljw does not + /// implement (week-based fields, localized zone names, padding). + date_time_pattern_letter_unsupported, + /// args: `.{ .detail = "..." }` — a DateTimeFormatterBuilder call out of + /// order (`optionalEnd` with no open `optionalStart`): IllegalStateException. + date_time_builder_state_invalid, + /// args: `.{ .id = "Europe/Paris" }` — a region zone id. cljw ships no + /// time-zone database; only fixed offsets and UTC / GMT / UT resolve. + zone_id_unknown, stm_retry_limit, locking_needs_object, locking_nest_overflow, @@ -1093,6 +1123,53 @@ pub fn entry(comptime code: Code) Entry { .phase = .eval, .template = "executor rejected task submission ({[reason]s})", }, + .date_time_value_invalid => .{ + .kind = .date_time_error, + .phase = .eval, + .template = "{[detail]s}", + }, + .date_time_text_unparseable => .{ + .kind = .date_time_parse_error, + .phase = .eval, + .template = "Text '{[text]s}' could not be parsed{[detail]s}", + }, + .temporal_field_unsupported => .{ + .kind = .unsupported_temporal_error, + .phase = .eval, + .template = "Unsupported field: {[field]s}", + }, + .temporal_accessor_unconvertible => .{ + .kind = .date_time_error, + .phase = .eval, + .template = "Unable to obtain {[target]s} from TemporalAccessor: {[temporal]s} of type {[class]s}", + }, + .temporal_accessor_expected => .{ + .kind = .type_error, + .phase = .eval, + .template = "Cannot cast {[class]s} to java.time.temporal.TemporalAccessor", + }, + .date_time_pattern_invalid => .{ + .kind = .value_error, + .phase = .eval, + .template = "{[detail]s}", + }, + // Loud, uncatchable: a pattern using one of these letters would format + // wrong text if it were skipped, so the gap must surface. + .date_time_pattern_letter_unsupported => .{ + .kind = .not_implemented, + .phase = .eval, + .template = "DateTimeFormatter pattern letter '{[letter]s}' is not supported by cljw", + }, + .date_time_builder_state_invalid => .{ + .kind = .state_error, + .phase = .eval, + .template = "{[detail]s}", + }, + .zone_id_unknown => .{ + .kind = .date_time_error, + .phase = .eval, + .template = "Unknown time-zone ID: {[id]s} (cljw has no time-zone database: only fixed offsets and UTC, GMT, UT resolve)", + }, .stm_no_transaction => .{ // clj: IllegalStateException "No transaction running" — a STATE error // (a ref op outside dosync), catchable as IllegalStateException. diff --git a/src/runtime/error/host_class.zig b/src/runtime/error/host_class.zig index b72f76b0d..2bb24982b 100644 --- a/src/runtime/error/host_class.zig +++ b/src/runtime/error/host_class.zig @@ -120,6 +120,10 @@ pub const ENTRIES = [_]Entry{ .{ .name = "IndexOutOfBoundsException", .parent = "RuntimeException" }, .{ .name = "NullPointerException", .parent = "RuntimeException" }, .{ .name = "UnsupportedOperationException", .parent = "RuntimeException" }, + // java.time: DateTimeException and its two thrown subclasses. + .{ .name = "DateTimeException", .parent = "RuntimeException" }, + .{ .name = "DateTimeParseException", .parent = "DateTimeException" }, + .{ .name = "UnsupportedTemporalTypeException", .parent = "DateTimeException" }, .{ .name = "ExceptionInfo", .parent = "RuntimeException" }, }; @@ -158,6 +162,9 @@ const FQCN_MAP = std.StaticStringMap([]const u8).initComptime(.{ .{ "java.lang.ReflectiveOperationException", "ReflectiveOperationException" }, .{ "java.lang.ClassNotFoundException", "ClassNotFoundException" }, .{ "java.lang.NoSuchMethodException", "NoSuchMethodException" }, + .{ "java.time.DateTimeException", "DateTimeException" }, + .{ "java.time.format.DateTimeParseException", "DateTimeParseException" }, + .{ "java.time.temporal.UnsupportedTemporalTypeException", "UnsupportedTemporalTypeException" }, .{ "clojure.lang.ExceptionInfo", "ExceptionInfo" }, }); @@ -333,6 +340,9 @@ pub fn kindToHostClass(kind: Kind) ?[]const u8 { .state_error => "IllegalStateException", .cancellation_error => "CancellationException", .rejected_execution_error => "RejectedExecutionException", + .date_time_error => "DateTimeException", + .date_time_parse_error => "DateTimeParseException", + .unsupported_temporal_error => "UnsupportedTemporalTypeException", .arity_error => "ArityException", .number_error => "NumberFormatException", .name_error, .syntax_error, .string_error => "RuntimeException", @@ -445,6 +455,14 @@ test "normalizeClassName: unknown FQCN passes through unchanged (caller decides) try testing.expectEqualStrings("", normalizeClassName("")); } +test "java.time exception family: both leaves ⊂ DateTimeException ⊂ RuntimeException" { + try testing.expect(isSubclassOf("java.time.format.DateTimeParseException", "java.time.DateTimeException")); + try testing.expect(isSubclassOf("UnsupportedTemporalTypeException", "DateTimeException")); + try testing.expect(isSubclassOf("DateTimeException", "RuntimeException")); + try testing.expect(!isSubclassOf("DateTimeException", "IllegalArgumentException")); + try testing.expectEqualStrings("DateTimeParseException", kindToHostClass(.date_time_parse_error).?); +} + test "CancellationException hierarchy (D-442): ⊂ IllegalStateException ⊂ RuntimeException, ≠ IllegalArgumentException" { try testing.expect(isKnownException("CancellationException")); try testing.expect(isKnownException("java.util.concurrent.CancellationException")); diff --git a/src/runtime/error/info.zig b/src/runtime/error/info.zig index 5099a166d..350a43dc5 100644 --- a/src/runtime/error/info.zig +++ b/src/runtime/error/info.zig @@ -62,6 +62,16 @@ pub const Kind = enum { /// A saturated or shut-down executor rejected a submission. Catchable as /// java.util.concurrent.RejectedExecutionException. rejected_execution_error, + /// A java.time value or computation is invalid: `java.time.DateTimeException` + /// (an out-of-range offset, a TemporalAccessor that cannot become the asked + /// type, a resolve conflict). Catchable, a RuntimeException. + date_time_error, + /// Text a DateTimeFormatter could not parse: + /// `java.time.format.DateTimeParseException` (⊂ DateTimeException). + date_time_parse_error, + /// A temporal value asked for a field it does not carry (a LocalTime's year): + /// `java.time.temporal.UnsupportedTemporalTypeException` (⊂ DateTimeException). + unsupported_temporal_error, /// An operation called in the wrong STATE — clj's `IllegalStateException` /// (e.g. `(re-groups m)`/`(.group m)` before a match → "No match found"; a /// transaction op outside `dosync`). Distinct from `value_error` @@ -221,6 +231,7 @@ fn kindToError(kind: Kind) ClojureWasmError { .index_error => ClojureWasmError.IndexError, .cancellation_error => ClojureWasmError.CancellationError, .rejected_execution_error => ClojureWasmError.ValueError, + .date_time_error, .date_time_parse_error, .unsupported_temporal_error => ClojureWasmError.ValueError, .state_error => ClojureWasmError.StateError, .io_error => ClojureWasmError.IoError, .file_not_found => ClojureWasmError.FileNotFound, @@ -302,7 +313,10 @@ pub fn argLoc(i: usize, fallback: SourceLocation) SourceLocation { /// Set the threadlocal error payload and return the matching Zig error. /// On message overflow the buffer is truncated with a trailing "...". -pub fn setErrorFmt( +/// `noinline`: every instance is an error path, and an inlined one puts its +/// formatter temporaries in the caller's frame. In `vm.eval` and +/// `treeWalkCall` that frame is paid per native eval re-entry (ADR-0195). +pub noinline fn setErrorFmt( phase: Phase, kind: Kind, location: SourceLocation, diff --git a/src/runtime/gc/conservative_scan.zig b/src/runtime/gc/conservative_scan.zig new file mode 100644 index 000000000..2bdaa4937 --- /dev/null +++ b/src/runtime/gc/conservative_scan.zig @@ -0,0 +1,66 @@ +// SPDX-License-Identifier: EPL-2.0 +//! D-556: conservative scan of the collecting thread's native stack for live +//! heap Values (tree_walk backend only; the vm's operand stack is the A1 root). +//! +//! Range: from the collector's own frame (deep) up to +//! `root_set.conservative_stack_top` (the shallowest tree_walk eval entry; 0 +//! means tree_walk never ran on this thread, so skip). Each 8-byte word is +//! decoded via the `Value.heapHeader()` membrane (tag check only, no deref) +//! and pinned IFF the pointer matches a live allocation record. A stale word +//! that happens to look like a heap tag can only pin a still-live object (one +//! delayed free), never deref a dangling pointer. The live-pointer set is +//! built in one pass over `allocations` on `gc.infra` (never `gc.alloc`: +//! collect must not re-enter the heap). +//! +//! Compiled only under `-Dbackend=tree_walk`; vm builds get the no-op branch. + +const std = @import("std"); +const build_options = @import("build_options"); +const GcHeap = @import("gc_heap.zig").GcHeap; +const root_set_mod = @import("root_set.zig"); +const mark = @import("mark_sweep.zig").mark; + +pub const ConservativeScan = if (build_options.backend == .tree_walk) struct { + /// libc setjmp used ONLY as a register spill (Boehm GC's `GC_push_regs` + /// idiom): it writes every callee-saved register into the buffer, so a + /// heap Value living only in a callee-saved register becomes scannable. + /// We never longjmp. + extern fn setjmp(env: *anyopaque) c_int; + + /// Decode one candidate word and pin it if it is a live heap Value. + inline fn pinIfLiveWord(gc: *GcHeap, live: *const std.AutoHashMapUnmanaged(usize, void), word: u64) void { + const Value = @import("../value/value.zig").Value; + const nb = @import("../value/nan_box.zig"); + // Pre-filter a zero address: `heapHeader`'s decodePtr would cast it to + // a null pointer (a ReleaseSafe panic). + if (word & nb.NB_ADDR_SHIFTED_MASK == 0) return; + const v: Value = @enumFromInt(word); + const hdr = v.heapHeader() orelse return; + if (live.contains(@intFromPtr(hdr))) mark(gc, hdr); + } + + pub fn run(gc: *GcHeap) void { + const top = root_set_mod.conservative_stack_top; + if (top == 0) return; + const bottom = @frameAddress(); + if (bottom >= top) return; + var live: std.AutoHashMapUnmanaged(usize, void) = .empty; + defer live.deinit(gc.infra); + live.ensureTotalCapacity(gc.infra, @intCast(gc.allocations.items.len)) catch return; + for (gc.allocations.items) |rec| live.putAssumeCapacity(@intFromPtr(rec.header), {}); + // Registers first: 128 u64s comfortably hold any ABI's jmp_buf; + // unwritten tail words stay zero and are filtered. + var jb: [128]u64 align(16) = [_]u64{0} ** 128; + _ = setjmp(@ptrCast(&jb)); + for (jb) |word| pinIfLiveWord(gc, &live, word); + // Then the native stack from the collector's frame up to the anchor. + var addr = std.mem.alignForward(usize, bottom, @alignOf(u64)); + while (addr + @sizeOf(u64) <= top) : (addr += @sizeOf(u64)) { + pinIfLiveWord(gc, &live, @as(*const u64, @ptrFromInt(addr)).*); + } + } +} else struct { + pub fn run(_: *GcHeap) void { + // vm backend: precise root_set only; no conservative native-stack scan. + } +}; diff --git a/src/runtime/gc/free_pool.zig b/src/runtime/gc/free_pool.zig deleted file mode 100644 index 8aa43be8e..000000000 --- a/src/runtime/gc/free_pool.zig +++ /dev/null @@ -1,197 +0,0 @@ -// SPDX-License-Identifier: EPL-2.0 -//! Free-pool recycling for cw v1 mark-sweep GC per ADR-0028 §3. -//! -//! Struct shapes: -//! - `FreeNode { next: ?*FreeNode }` — intrusive linked-list node -//! overlaid in the freed object's payload **at offset 8** (per -//! ADR-0028 §3 DIVERGENCE from cw v0 which overlaid at offset 0 -//! and clobbered the header). -//! - `FreePoolKey { size, alignment }` — hash key for the per- -//! (size, alignment) pool head map. -//! - `FreePoolMap` — `AutoHashMapUnmanaged(FreePoolKey, ?*FreeNode)` -//! wrapper with `initMap` / `deinit` / `push` / `pop`, all landed. -//! -//! A future size-class optimisation (quantise sizes to N power-of-2 -//! classes + replace the HashMap with a flat `[N]?*FreeNode` array) is -//! left open per ADR-0028 §6 — picked, if ever, against a measured -//! size-class distribution. -//! -//! Minimum allocation size: **16 bytes** (per ADR-0028 §3) so the -//! freed payload can host `@sizeOf(FreeNode) = 8` bytes after the -//! 8-byte HeapHeader. Allocations under 16 bytes round up. - -const std = @import("std"); -const testing = std.testing; - -/// Intrusive free-list node, overlaid in the freed object's payload -/// at offset 8 (after the 8-byte HeapHeader). The freed block's -/// payload must be ≥ `@sizeOf(FreeNode) = 8` bytes — combined with -/// the 8-byte header this gives the 16-byte minimum allocation that -/// ADR-0028 §3 enforces. -pub const FreeNode = struct { - next: ?*FreeNode = null, -}; - -/// Per-(size, alignment) free pool key. Alignment is always 8 per -/// ADR-0027 §1 `align(8)` invariant, so an optimised future shape -/// can collapse to a size-only key + per-size-class array (the -/// Devil's-advocate Wildcard Alt 3 candidate captured in ADR-0028 §3 -/// + §6's F-003 deferral). -pub const FreePoolKey = struct { - size: usize, - alignment: std.mem.Alignment, - - pub fn eql(self: FreePoolKey, other: FreePoolKey) bool { - return self.size == other.size and self.alignment == other.alignment; - } -}; - -/// Per-(size, alignment) free pool head map. -/// Push lands freed memory at the head of the matching pool (intrusive -/// overlay at offset 8 per ADR-0028 §3); pop returns the memory base -/// to caller for reuse. Deinit walks every pool and `rawFree`s each -/// node's backing memory back to `infra`. -/// -/// Memory layout convention (per ADR-0028 §3): the freed block's -/// bytes 0..7 hold the (preserved-but-stale) HeapHeader; bytes 8..15 -/// host the `FreeNode { next }` overlay. The returned `[*]u8` from -/// `pop` points at offset 0 (the HeapHeader position) so the caller -/// can cast to `*T` with HeapHeader at offset 0. -pub const FreePoolMap = struct { - /// HashMap from key to pool head. `null` head = pool exists with - /// no free nodes; absent key = pool not yet observed at any size. - /// Devil's-advocate Wildcard Alt 3 (size-class quantised flat - /// array) defers to a future ADR per ADR-0028 §3 + §6 F-003. - map: std.AutoHashMap(FreePoolKey, ?*FreeNode) = undefined, - - pub const empty: FreePoolMap = .{ .map = undefined }; - - /// Two-step init pattern: create the map with the infra allocator - /// at GcHeap.init time. `empty` literal defers map construction - /// to a follow-up `initMap(infra)` call so the `FreePoolMap` can - /// live as a default-initialised field on `GcHeap`. - pub fn initMap(self: *FreePoolMap, infra: std.mem.Allocator) void { - self.map = std.AutoHashMap(FreePoolKey, ?*FreeNode).init(infra); - } - - pub fn deinit(self: *FreePoolMap, infra: std.mem.Allocator) void { - var it = self.map.iterator(); - while (it.next()) |entry| { - const key = entry.key_ptr.*; - var maybe_node = entry.value_ptr.*; - while (maybe_node) |node| { - const next = node.next; - // Recover the memory base (offset 0, HeapHeader - // position) from the FreeNode address (offset 8). - const mem_base: [*]u8 = @ptrFromInt(@intFromPtr(node) - 8); - infra.rawFree(mem_base[0..key.size], key.alignment, @returnAddress()); - maybe_node = next; - } - } - self.map.deinit(); - } - - /// Push a freed memory block onto the matching (size, alignment) - /// pool. `mem` must be the memory base (offset 0, HeapHeader - /// position) — the FreeNode is overlaid at offset 8. - /// Requires `key.size >= 16` so the payload can host the FreeNode. - pub fn push(self: *FreePoolMap, key: FreePoolKey, mem: [*]u8) !void { - std.debug.assert(key.size >= 16); - const node_ptr: *FreeNode = @ptrCast(@alignCast(mem + 8)); - const gop = try self.map.getOrPut(key); - if (!gop.found_existing) gop.value_ptr.* = null; - node_ptr.* = .{ .next = gop.value_ptr.* }; - gop.value_ptr.* = node_ptr; - } - - /// Pop a freed memory block from the matching pool, or return - /// null if the pool is empty / absent. Returns the memory base - /// (offset 0, HeapHeader position) suitable for caller cast. - pub fn pop(self: *FreePoolMap, key: FreePoolKey) ?[*]u8 { - const head_entry = self.map.getPtr(key) orelse return null; - const node = head_entry.* orelse return null; - head_entry.* = node.next; - const mem_base: [*]u8 = @ptrFromInt(@intFromPtr(node) - 8); - return mem_base; - } -}; - -// --- tests --- - -test "FreeNode is at most 16 bytes (intrusive overlay budget)" { - // @sizeOf(FreeNode) ≤ 8 ensures it fits in the payload of the - // 16-byte minimum allocation after the 8-byte HeapHeader. On 64- - // bit targets this is exactly 8 bytes (single `?*FreeNode`). - try testing.expect(@sizeOf(FreeNode) <= 8); -} - -test "FreePoolKey eql" { - const k1 = FreePoolKey{ .size = 32, .alignment = .@"8" }; - const k2 = FreePoolKey{ .size = 32, .alignment = .@"8" }; - const k3 = FreePoolKey{ .size = 64, .alignment = .@"8" }; - - try testing.expect(k1.eql(k2)); - try testing.expect(!k1.eql(k3)); -} - -test "FreePoolMap init / deinit empty" { - var pool: FreePoolMap = .empty; - pool.initMap(testing.allocator); - defer pool.deinit(testing.allocator); - - try testing.expectEqual(@as(u32, 0), pool.map.count()); -} - -test "FreePoolMap push then pop returns the same memory base (LIFO)" { - var pool: FreePoolMap = .empty; - pool.initMap(testing.allocator); - defer pool.deinit(testing.allocator); - - const align_t: std.mem.Alignment = .@"8"; - const key = FreePoolKey{ .size = 16, .alignment = align_t }; - - // Manually allocate a 16-byte block (header + payload) and push. - const raw_a = testing.allocator.rawAlloc(16, align_t, @returnAddress()) orelse - return error.OutOfMemory; - try pool.push(key, raw_a); - - const popped = pool.pop(key) orelse return error.PopMissed; - try testing.expectEqual(raw_a, popped); - try testing.expect(pool.pop(key) == null); - - testing.allocator.rawFree(popped[0..16], align_t, @returnAddress()); -} - -test "FreePoolMap push twice + pop twice (LIFO order)" { - var pool: FreePoolMap = .empty; - pool.initMap(testing.allocator); - defer pool.deinit(testing.allocator); - - const align_t: std.mem.Alignment = .@"8"; - const key = FreePoolKey{ .size = 16, .alignment = align_t }; - - const raw_a = testing.allocator.rawAlloc(16, align_t, @returnAddress()) orelse - return error.OutOfMemory; - const raw_b = testing.allocator.rawAlloc(16, align_t, @returnAddress()) orelse - return error.OutOfMemory; - try pool.push(key, raw_a); - try pool.push(key, raw_b); - - // LIFO: pop returns most-recently pushed first. - const first_pop = pool.pop(key) orelse return error.PopMissed; - const second_pop = pool.pop(key) orelse return error.PopMissed; - try testing.expectEqual(raw_b, first_pop); - try testing.expectEqual(raw_a, second_pop); - - testing.allocator.rawFree(raw_a[0..16], align_t, @returnAddress()); - testing.allocator.rawFree(raw_b[0..16], align_t, @returnAddress()); -} - -test "FreePoolMap.pop on absent key returns null" { - var pool: FreePoolMap = .empty; - pool.initMap(testing.allocator); - defer pool.deinit(testing.allocator); - - const key = FreePoolKey{ .size = 32, .alignment = .@"8" }; - try testing.expect(pool.pop(key) == null); -} diff --git a/src/runtime/gc/gc_heap.zig b/src/runtime/gc/gc_heap.zig index e81fc4d98..d3c646ecd 100644 --- a/src/runtime/gc/gc_heap.zig +++ b/src/runtime/gc/gc_heap.zig @@ -1,295 +1,164 @@ // SPDX-License-Identifier: EPL-2.0 -//! Mark-sweep GC heap for cw v1 — `gc_alloc` layer of the 3-layer +//! Mark-sweep GC heap for cw v1: the `gc_alloc` layer of the 3-layer //! allocator boundary per ADR-0028 §2 + F-006. //! //! Struct shape: -//! - `allocations` — side-table of live heap-object records (header -//! + size + alignment); see the `GcHeap` docstring for why a -//! side-table beats an intrusive next-pointer. -//! - `permanent_roots` — embedder-pinned root Values (`pin` / `unpin`). -//! - `free_pools` — per-(size, alignment) free pool head map, owned -//! by `runtime/gc/free_pool.zig`. -//! - `stats` — bytes_allocated / bytes_freed / alloc/collect/sweep -//! counts + last_live_bytes. -//! - `infra` — backing GPA allocator (per F-006 §2 layer 1) for the -//! raw heap pages this `GcHeap` operates over. -//! - `bytes_since_last_gc` + `threshold_bytes` — drive the adaptive -//! `collect()` trigger per ADR-0028 §1. +//! - `allocations`: side-table of live heap-object records (header + size + +//! alignment); see the `GcHeap` docstring for why a side-table beats an +//! intrusive next-pointer. +//! - `permanent_roots`: embedder-pinned root Values (`pin` / `unpin`). +//! - `pages`: the size-classed page heap small cells live in, owned by +//! `page_heap.zig` (empty pages go back to the OS after a collect). +//! - `stats`: counters, shapes in `gc_types.zig`. +//! - `infra`: backing GPA allocator (F-006 §2 layer 1). +//! - the pressure counters + `threshold_bytes`: the collect trigger, whose +//! policy lives in `gc_trigger.zig` (re-exposed here as methods). //! -//! `alloc` (free-pool fast path → infra slow path), `pin` / `unpin`, -//! and `deinit` (per-tag finaliser → rawFree drain) are all landed. -//! The `collect()` orchestrator lives in `mark_sweep.zig` to keep the -//! import graph acyclic (it reaches `root_set.zig`, which imports this -//! file). +//! The `collect()` orchestrator lives in `mark_sweep.zig` to keep the import +//! graph acyclic (it reaches `root_set.zig`, which imports this file). Unit +//! tests are in `gc_heap_test.zig`. //! -//! Thread-safety (ADR-0090 §2, F-006): `gc_mutex` (a `std.Io.Mutex`, -//! locked through the `io_default` singleton because the allocator API -//! takes no `io` arg) serializes `alloc` / `pin` / `unpin` and the whole -//! `collect()` cycle (the latter brackets in `mark_sweep.collect`). It is -//! uncontended today (real threads arrive with `future` / `pmap`); the -//! global alloc lock is the foundation the `ThreadGcContext` -//! root-publication handshake (ADR-0090) builds on for collection safety. -//! Not reentrant: `alloc` never calls `collect`, and `collect` never -//! allocates, so the two lock-takers never nest. +//! Thread-safety (ADR-0090 §2, F-006): `gc_mutex` (a `std.Io.Mutex`, locked +//! through the `io_default` singleton because the allocator API takes no `io` +//! arg) serializes `alloc` / `pin` / `unpin` and the whole `collect()` cycle. +//! Not reentrant: `alloc` never calls `collect` under the lock, and `collect` +//! never allocates through `alloc`, so the two lock-takers never nest. const std = @import("std"); -const testing = std.testing; const heap_header = @import("../value/heap_header.zig"); -const free_pool_mod = @import("free_pool.zig"); +const page_heap_mod = @import("page_heap.zig"); const value_mod = @import("../value/value.zig"); const io_default = @import("../concurrency/io_default.zig"); const safepoint = @import("../concurrency/safepoint.zig"); const root_set_mod = @import("root_set.zig"); +const gc_types = @import("gc_types.zig"); +const trigger = @import("gc_trigger.zig"); -/// Reentrancy guard for the alloc-driven GC torture (D-386): a torture collect -/// must not re-enter itself if the collector's own bookkeeping reaches `alloc`. -/// Threadlocal — each thread's torture cadence is independent. Inert (false) -/// unless `CLJW_GC_TORTURE_ALLOC` is armed. -threadlocal var in_alloc_torture: bool = false; +/// Reentrancy guard for the alloc-driven GC torture (D-386) and the trigger's +/// collects: a collect must not re-enter itself if the collector's own +/// bookkeeping reaches `alloc`. Threadlocal: each thread's cadence is +/// independent. +pub threadlocal var in_alloc_torture: bool = false; // Fabrication no-collect region depth (D-244 #4, ADR-0150): the canonical -// docs + the variable itself live in `root_set.zig` (`fabrication_depth`, -// moved there at ADR-0150 am1 so `safepoint.park`/`enterBlocked` can assert -// against it directly). This file's consumers: `enterFabrication`/ -// `exitFabrication` (the bracket API), the alloc-prologue park gate (D-559), -// the alloc-torture gate, and `maybeAutoCollect`. +// docs + the variable live in `root_set.zig` (`fabrication_depth`). const HeapHeader = heap_header.HeapHeader; -const FreePoolMap = free_pool_mod.FreePoolMap; +const PageHeap = page_heap_mod.PageHeap; const Value = value_mod.Value; -/// Default GC trigger threshold (bytes since last collection), and the default -/// floor of the per-heap adaptive threshold. Adaptive at runtime: -/// `threshold = max(threshold_floor_bytes, last_live_bytes * 2)` per ADR-0028 §1. -/// Raised 1MB→4MB by ADR-0164/D-519: for a churn workload `last_live≈0` so the -/// floor IS the operative cadence; 4MB ≈ ¼ the collects (same total sweep work, -/// fewer STW fences) at ~3MB extra peak. Tune via `CLJW_GC_THRESHOLD_MB`. -pub const default_gc_threshold_bytes: usize = 4 * 1024 * 1024; - -/// Ceiling for the ramping floor (D-573). 64 MB is where the measured curve -/// knees: on the Rush Hour BFS, 4 MB -> 17.7 s, 16 MB -> 9.6 s, 64 MB -> 7.6 s, -/// 256 MB -> 7.1 s. Past 64 MB the remaining 6% costs 4x the resident heap, -/// which is the wrong trade for a runtime whose selling point is a small -/// footprint. `CLJW_GC_THRESHOLD_MB` still overrides the STARTING floor for a -/// caller who knows their workload. -pub const max_threshold_floor_bytes: usize = 64 * 1024 * 1024; - -/// Target share of wall clock spent collecting, as a percent (D-573). -/// -/// The floor is steered by the MEASURED ratio rather than by a byte constant or -/// a collection count, because that ratio is the thing anyone actually cares -/// about, and because it is the quantity the theory is written in: for a -/// mark-sweep collector, GC share is `mark_cost(live) / bytes_between_collects`, -/// so raising the trigger lowers the share directly and is independent of how -/// big the live set happens to be. -/// -/// The JVM exposes exactly this as `-XX:GCTimeRatio`; its default asks for -/// about 1 % in the throughput collector. 10 % is deliberately laxer: cljw's -/// collector is non-generational, so it re-marks the whole live set every time -/// and cannot reach 1 % without letting the heap grow far past what a small -/// runtime should hold. +pub const default_gc_threshold_bytes = trigger.default_gc_threshold_bytes; +pub const max_threshold_floor_bytes = trigger.max_threshold_floor_bytes; +pub const gc_time_target_percent = trigger.gc_time_target_percent; +pub const min_retained_bytes = trigger.min_retained_bytes; +pub const min_alloc_bytes = gc_types.min_alloc_bytes; +pub const PersistentMark = gc_types.PersistentMark; +pub const Stats = gc_types.Stats; +pub const AllocRecord = gc_types.AllocRecord; + +/// Mark-sweep GC heap. Owns a list of tracked heap objects + the page heap; +/// the trigger threshold adapts to the last live-set measurement. /// -/// Measured motivation: a Rush Hour BFS with a 747 KB live set spent 10.1 s of -/// 17.9 s collecting, because the 4 MB floor fired 755 times and each collect -/// re-marked the same live graph. -pub const gc_time_target_percent: u64 = 10; - -/// Minimum allocation size (bytes) per ADR-0028 §3: the freed payload -/// must host the FreeNode overlay at offset 8 (8 bytes header + -/// ≥ 8 bytes payload = 16 bytes minimum). Allocations of types -/// smaller than 16 bytes round up; the extra bytes are unused while -/// live but become the FreeNode region on free. -pub const min_alloc_bytes: usize = 16; - -/// Comptime invariant: every `T` passed to `GcHeap.alloc` must have -/// `HeapHeader` as its first field. The returned `*T` and the -/// live-list `*HeapHeader` are pointer-aliases — caster relies on -/// offset 0 holding the header so mark/sweep can read `header.tag` -/// without knowing T. A struct with a different first field would -/// silently misinterpret the first bytes as a tag enum value, and -/// the mis-link would surface only under a debugger or on the -/// downstream sweep path (where the tag's `tag_finaliser_table` -/// entry runs against the wrong memory). The check below fires at -/// compile time so the bug never reaches a debugger. -fn assertHeaderAtOffsetZero(comptime T: type) void { - const info = @typeInfo(T); - if (info != .@"struct") { - @compileError("GcHeap.alloc requires a struct type; got " ++ @typeName(T)); - } - const fields = info.@"struct".fields; - if (fields.len == 0) { - @compileError("GcHeap.alloc requires T to have at least one field; " ++ @typeName(T) ++ " has none"); - } - if (fields[0].type != HeapHeader) { - @compileError("GcHeap.alloc requires T to have HeapHeader as its first field; " ++ @typeName(T) ++ "'s first field is " ++ @typeName(fields[0].type)); - } - if (@offsetOf(T, fields[0].name) != 0) { - @compileError("GcHeap.alloc requires HeapHeader at offset 0 of T; " ++ @typeName(T) ++ " has it at a different offset"); - } -} - -/// A process-lifetime mark-waypoint (D-251) + the bytes it owns (D-573): -/// the size feeds `persistent_bytes`, which sweep folds into -/// `last_live_bytes` because these objects are re-traced every cycle. -pub const PersistentMark = struct { - obj: *anyopaque, - size: usize, -}; - -/// Allocation + collection statistics. -pub const Stats = struct { - bytes_allocated: usize = 0, - bytes_freed: usize = 0, - alloc_count: u64 = 0, - collect_count: u64 = 0, - sweep_count: u64 = 0, - last_live_bytes: usize = 0, - /// Diagnostic (gc_alloc_rate lever, D-450): allocs served by reusing a freed - /// block from the free pool (vs a fresh `rawAlloc`/malloc). pool_hits/alloc_count - /// = the reuse rate — a low rate means same-size churn is re-malloc'ing. - pool_hits: u64 = 0, - /// D-573: nanoseconds spent inside `collectStopTheWorld`, and the wall clock - /// at the first collection. Their ratio is the GC time fraction the floor - /// is steered by — the same quantity the JVM exposes as `-XX:GCTimeRatio`. - gc_ns_total: u64 = 0, - first_collect_ns: i64 = 0, -}; - -/// Per-allocation record on the GcHeap live list. Stores the type- -/// erased `*HeapHeader` plus the original (size, alignment) so deinit -/// / 5.3.c sweep can `rawFree` with the matching metadata back to -/// `infra`. Without the per-record size + alignment, debug allocators -/// trip "Invalid free" canaries because the type-erased *HeapHeader -/// alone would imply an 8-byte destroy of a (typically larger) full -/// allocation. -pub const AllocRecord = struct { - header: *HeapHeader, - size: usize, - alignment: std.mem.Alignment, -}; - -/// Mark-sweep GC heap. Owns a list of tracked heap objects + per-size -/// free pools; trigger threshold adapts based on the last live-set -/// measurement. -/// -/// Live-list shape (per 5.3.b.1 design decision, depth 1): -/// **side-table** `ArrayListUnmanaged(AllocRecord)` on GcHeap rather -/// than an intrusive `next: ?*HeapHeader` field on HeapHeader. Reason: -/// (1) preserves ADR-0009's 8-byte HeapHeader invariant (avoiding -/// header-grow surgery); (2) matches cw v0's path (refined per -/// ADR-0028 audit bullet #5 — only the mark BIT goes inline per -/// ADR-0028 §6, not the live-list link); (3) sweep walk stays -/// sequential through ArrayList memory (cache-friendly), which is -/// what the cw v0 perf inheritance language in ADR-0028 §3 cares -/// about. Each record carries (size, alignment) so the deinit / -/// sweep `rawFree` paths match what `rawAlloc` originally requested. +/// Live-list shape (5.3.b.1, depth 1): a **side-table** +/// `ArrayList(AllocRecord)` rather than an intrusive `next: ?*HeapHeader` +/// field. It (1) preserves ADR-0009's 8-byte HeapHeader invariant; (2) +/// matches cw v0 (only the mark BIT goes inline per ADR-0028 §6); (3) keeps +/// the sweep walk sequential through ArrayList memory. Each record carries +/// (size, alignment) so the free paths match what was allocated. pub const GcHeap = struct { - /// Backing allocator for raw heap pages. Per F-006 §2 this is the - /// process-lifetime GPA (`infra_alloc`). + /// Backing allocator for raw heap memory: the process-lifetime GPA + /// (`infra_alloc`, F-006 §2). infra: std.mem.Allocator, /// Side-table of live heap-object records. Append at alloc time; - /// swap-remove during sweep for dead objects. Per the docstring - /// design decision: side-table chosen over intrusive next-pointer - /// to preserve the 8-byte HeapHeader invariant. - allocations: std.ArrayList(AllocRecord) = .empty, - /// Embedder-pinned root Values per ADR-0028 §5 row 10. Holds - /// `Value` entries that the embedder (FFI / test fixture / future - /// `cljw -e` REPL prompt buffer) wants to keep alive across - /// `collect()` cycles. `pin` appends, `unpin` swap-removes the - /// first match. The root walker yields each entry's - /// `Value.heapHeader()` (skipping immediates). + /// swap-remove during sweep for dead objects. + allocations: std.ArrayList(gc_types.AllocRecord) = .empty, + /// Embedder-pinned root Values per ADR-0028 §5 row 10 (FFI, test + /// fixtures, the REPL buffer). The root walker yields each entry's + /// `Value.heapHeader()`, skipping immediates. permanent_roots: std.ArrayList(Value) = .empty, - /// D-556: analysis-produced Values persisted past their bracket — the - /// literal pool of every analyzed form whose Nodes outlive the form (fn - /// bodies held by `Function.body` in the heap-lifetime analyzer arena). - /// Appended by `root_set.endAnalysisPersist`; walked right after - /// `permanent_roots`. Grows with the arena — the same lifetime the - /// Nodes themselves already have. + /// D-556: analysis-produced Values persisted past their bracket (the + /// literal pool of every analyzed form whose Nodes outlive the form). + /// Appended by `root_set.endAnalysisPersist`; walked after + /// `permanent_roots`. persisted_analysis_roots: std.ArrayList(Value) = .empty, /// Process-lifetime mark-waypoints (D-251): headers of `trackHeap`'d - /// objects (`Function` / `ProtocolDescriptor` / `TypeDescriptorRef`) that - /// are NOT in `allocations` and so are never swept. The mark bit doubles as - /// the per-collect cycle/visited flag; for objects that sweep never clears, - /// a stale bit makes `mark()` short-circuit on the SECOND collect, so their - /// per-tag trace never re-runs and any GC child reachable ONLY through them - /// (a `Function`'s `closure_bindings`) is swept. `collect()` clears these - /// bits at mark-phase start so the waypoint is re-traced every cycle. - persistent_marks: std.ArrayList(PersistentMark) = .empty, - /// Total bytes registered with the persistent marks (D-573): part of every - /// collection's marking cost, so part of `last_live_bytes`. + /// objects (`Function` / `ProtocolDescriptor` / `TypeDescriptorRef`) NOT + /// in `allocations` and so never swept. Sweep never clears their mark + /// bit, so `collect()` clears it at mark-phase start; otherwise `mark()` + /// short-circuits from the 2nd collect on and strands their children. + persistent_marks: std.ArrayList(gc_types.PersistentMark) = .empty, + /// Total bytes registered with the persistent marks (D-573). persistent_bytes: usize = 0, - /// Per-(size, alignment) free pool heads. Sweep pushes freed blocks - /// here (intrusive FreeNode at offset 8); `alloc` pops them as its - /// fast path before falling back to `infra`. - free_pools: FreePoolMap = .empty, - /// Gray worklist for the mark phase (ADR-0028 amendment 3): `mark()` - /// pushes newly-bitted non-leaf headers here instead of descending - /// recursively, so an arbitrarily deep object graph (a 1M-long cons - /// chain) marks with O(1) native stack. `collect()` reserves capacity - /// up front (bit-set-at-push ⇒ each object enters at most once) and - /// `mark_sweep.drainGray` empties it. Capacity is retained across - /// collects; STW-only state — a future parallel marker replaces it. + /// Size-classed pages small cells live in (`page_heap.zig`). Sweep frees + /// a dead small cell onto its page; after sweep, `collect` hands empty + /// pages beyond `retainBytesAfterCollect` back to the OS. + pages: PageHeap = .{}, + /// Bytes of live cells too large (or too aligned) for the page heap, held + /// on `infra`. With `pages.mappedBytes()` this is the committed cell heap. + large_live_bytes: usize = 0, + /// Bytes of non-cell payload (a String's bytes, an array's elements, a + /// transient vector's buffer, a BigInt's limbs) allocated on `infra` since + /// the last collect, charged through `noteOwnedAlloc`. Counts toward the + /// trigger beside `bytes_since_last_gc`: a 24 MB buffer behind a 40-byte + /// cell is 24 MB of pressure. Atomic: a payload is allocated outside + /// `gc_mutex`. + owned_since_last_gc: std.atomic.Value(usize) = .init(0), + /// Set by `requestCollect` (`System/gc`) so its collect releases empty + /// pages down to `min_retained_bytes`. `collect` clears it. + release_requested: bool = false, + /// Gray worklist for the mark phase (ADR-0028 amendment 3): an explicit + /// worklist so an arbitrarily deep graph marks with O(1) native stack. + /// `collect()` reserves capacity up front; retained across collects. mark_worklist: std.ArrayList(*HeapHeader) = .empty, /// Allocation + collection counters. - stats: Stats = .{}, - /// Adaptive GC trigger threshold (bytes since last collect). - /// Recomputed at end of each `collect()` cycle. - threshold_bytes: usize = default_gc_threshold_bytes, - /// D-519 (ADR-0164): persistent floor for the adaptive `threshold_bytes`, so a - /// `CLJW_GC_THRESHOLD_MB` knob survives each collect's recompute (which would - /// otherwise reset the floor to the module default). Defaults to the module const. - threshold_floor_bytes: usize = default_gc_threshold_bytes, - /// The floor the steering may not go BELOW — the default, or whatever - /// CLJW_GC_THRESHOLD_MB set. Steering moves `threshold_floor_bytes` - /// between this and `max_threshold_floor_bytes`. - base_threshold_floor_bytes: usize = default_gc_threshold_bytes, - - /// Bytes allocated since the last `collect()` invocation. Trips - /// collection when it exceeds `threshold_bytes`. + stats: gc_types.Stats = .{}, + /// Adaptive GC trigger threshold (pressure bytes since last collect). + /// Recomputed at the end of each `collect()` (`gc_trigger.afterSweep`). + threshold_bytes: usize = trigger.default_gc_threshold_bytes, + /// D-519 (ADR-0164): persistent floor for the adaptive `threshold_bytes`, + /// so a `CLJW_GC_THRESHOLD_MB` knob survives each collect's recompute. + threshold_floor_bytes: usize = trigger.default_gc_threshold_bytes, + /// The floor the steering may not go BELOW (default or the env knob). + /// Steering moves `threshold_floor_bytes` between this and + /// `max_threshold_floor_bytes`. + base_threshold_floor_bytes: usize = trigger.default_gc_threshold_bytes, + /// Cell bytes allocated since the last `collect()`. bytes_since_last_gc: usize = 0, - /// True iff this heap is the `gc` field embedded in a `Runtime` — set by - /// `Runtime.init`. `collect()` then reaches the Runtime via - /// `@fieldParentPtr` to walk the descriptor registries (rt.types / - /// native_descriptors / class_descriptors) as roots (ADR-0184 B). - /// Bare `GcHeap.init` test heaps leave it false and skip that walk. + /// True iff this heap is the `gc` field embedded in a `Runtime` (set by + /// `Runtime.init`). `collect()` then reaches the Runtime via + /// `@fieldParentPtr` to walk the descriptor registries (ADR-0184 B). runtime_embedded: bool = false, - /// ADR-0125 / D-352 (isolation dim (b)): per-eval LIVE-heap ceiling in bytes. - /// When non-null, `alloc` REFUSES (never merely triggers a collect) any - /// allocation that would push live bytes (`bytes_allocated - bytes_freed`) - /// past the cap, bounding untrusted code's memory in-process. `null` (default) - /// = unmetered. Set by `runner.runSource` from `CLJW_EVAL_MAX_HEAP_MB`. + /// ADR-0125 / D-352: per-eval LIVE-heap ceiling in bytes. When non-null, + /// `alloc` REFUSES any allocation that would push live bytes past it. + /// Set by `runner.runSource` from `CLJW_EVAL_MAX_HEAP_MB`. heap_ceiling: ?usize = null, /// Installed by a higher layer (eval_budget, which may import the error - /// catalog — gc_heap may not, big_int→gc_heap cycle) to SET the catalog - /// `eval_heap_exceeded` (resource_exhausted, uncatchable) Info before the - /// breach surfaces. Vtable pattern (zone_deps low→high). Returns void — - /// `alloc` always returns `error.OutOfMemory` after it, so alloc's error set - /// is unchanged (a returning-`anyerror` hook would poison every caller's - /// inferred set). When null, the bare `error.OutOfMemory` surfaces. + /// catalog; gc_heap may not) to SET the catalog `eval_heap_exceeded` Info + /// before the breach surfaces as `error.OutOfMemory`. heap_exceeded_hook: ?*const fn (cap: usize) void = null, - /// Global heap lock (ADR-0090 §2). Serializes `alloc` / `pin` / - /// `unpin` and the whole `collect()` cycle so allocation is - /// thread-safe under F-006. Locked via the `io_default` singleton - /// (the allocator API has no `io` arg). Uncontended until real - /// threads land (`future` / `pmap`). + /// Global heap lock (ADR-0090 §2). gc_mutex: std.Io.Mutex = .init, - /// The heap's owning main thread (D-566, ADR-0175 Consequences): the OS - /// thread that ran `init`. Production builds the Runtime (and so this heap) - /// on the `runtime_thread.runMain` entry thread, the one unregistered - /// mutator the STW collect walks directly. Every other thread must be a - /// registered worker before it allocates; `allocSized` asserts it. + /// The heap's owning main thread (D-566, ADR-0175): the OS thread that ran + /// `init`. Every other thread must be a registered worker before it + /// allocates; `allocSized` asserts it. owner_thread_id: std.Thread.Id, + pub const noteOwnedAlloc = trigger.noteOwnedAlloc; + pub const pressureBytes = trigger.pressureBytes; + pub const chargeInfraBytes = trigger.chargeInfraBytes; + pub const retainBytesAfterCollect = trigger.retainBytesAfterCollect; + pub const maybeAutoCollect = trigger.maybeAutoCollect; + pub const requestCollect = trigger.requestCollect; + pub const collectBeforeIdle = trigger.collectBeforeIdle; + pub fn init(infra: std.mem.Allocator) GcHeap { const owner = root_set_mod.currentThreadId(); root_set_mod.noteHeapOwner(owner); var g = GcHeap{ .infra = infra, .owner_thread_id = owner }; - g.free_pools.initMap(infra); - // D-519 (ADR-0164): CLJW_GC_THRESHOLD_MB tunes the auto-collect floor — the - // wall-clock GO gate raises it until every won fastest-script bench holds. - // Invalid / unset / 0 → the 4MB default. Process env is published at startup - // (cli.zig) before the runtime + its GcHeap are built, so the read is live. + // D-519 (ADR-0164): CLJW_GC_THRESHOLD_MB tunes the auto-collect floor. + // Invalid / unset / 0 keeps the 4MB default. Process env is published + // at startup (cli.zig) before the runtime is built. if (@import("../process_env.zig").get("CLJW_GC_THRESHOLD_MB")) |raw| { if (std.fmt.parseInt(usize, raw, 10) catch null) |mb| { if (mb > 0) { @@ -303,49 +172,43 @@ pub const GcHeap = struct { } pub fn deinit(self: *GcHeap) void { - // Diagnostic (gc_alloc_rate lever, D-450): CLJW_GC_STATS=1 prints the - // alloc/reuse/collect tallies to stderr at teardown — load-INDEPENDENT - // (counts, not timing), so the free-pool reuse rate is measurable even - // under host load. Off by default = one null-check. + // Diagnostic (D-450): CLJW_GC_STATS=1 prints the alloc/collect tallies + // at teardown. Load-independent counts. Off by default. if (@import("../process_env.zig").get("CLJW_GC_STATS") != null) { const s = self.stats; - std.debug.print("[gc-stats] allocs={d} pool_hits={d} mallocs={d} reuse={d}% collects={d} sweeps={d} bytes_alloc={d} last_live={d} floor_mb={d} gc_ms={d}\n", .{ - s.alloc_count, s.pool_hits, s.alloc_count - s.pool_hits, - if (s.alloc_count > 0) s.pool_hits * 100 / s.alloc_count else 0, s.collect_count, s.sweep_count, - s.bytes_allocated, s.last_live_bytes, self.threshold_floor_bytes / (1024 * 1024), - s.gc_ns_total / std.time.ns_per_ms, + std.debug.print("[gc-stats] allocs={d} small={d} large={d} collects={d} sweeps={d} bytes_alloc={d} last_live={d} floor_mb={d} gc_ms={d}\n", .{ + s.alloc_count, s.small_allocs, s.alloc_count - s.small_allocs, + s.collect_count, s.sweep_count, s.bytes_allocated, + s.last_live_bytes, self.threshold_floor_bytes / (1024 * 1024), s.gc_ns_total / std.time.ns_per_ms, }); + std.debug.print("[gc-stats] pages_now={d} pages_mapped={d} pages_released={d} large_live={d}\n", .{ self.pages.page_count, self.pages.pages_mapped_total, self.pages.pages_released_total, self.large_live_bytes }); std.debug.print("[gc-stats] persistent_marks={d} persistent_bytes={d}\n", .{ self.persistent_marks.items.len, self.persistent_bytes }); } - // Drain every live allocation back to infra. Calls the per-tag - // finaliser before rawFree so types that own non-GC resources - // (String.bytes / BigInt limbs / wasm_module references) get - // their cleanup chance — matches the sweep contract per - // ADR-0028 §4. + // Finalise every live allocation (types owning non-GC resources get + // their cleanup, the sweep contract of ADR-0028 §4), then free large + // cells. Small cells need no per-cell free: their pages go back whole + // in `pages.deinit`. const tag_ops = @import("tag_ops.zig"); for (self.allocations.items) |rec| { if (tag_ops.tag_finaliser_table[rec.header.tag]) |finaliser| { finaliser(@ptrCast(self), rec.header); } - const mem = @as([*]u8, @ptrCast(rec.header))[0..rec.size]; - self.infra.rawFree(mem, rec.alignment, @returnAddress()); + if (!page_heap_mod.isSmall(rec.size, rec.alignment)) { + const mem = @as([*]u8, @ptrCast(rec.header))[0..rec.size]; + self.infra.rawFree(mem, rec.alignment, @returnAddress()); + } } self.allocations.deinit(self.infra); self.permanent_roots.deinit(self.infra); self.persisted_analysis_roots.deinit(self.infra); self.persistent_marks.deinit(self.infra); - self.free_pools.deinit(self.infra); + self.pages.deinit(); self.mark_worklist.deinit(self.infra); } - /// Register a process-lifetime mark-waypoint header (D-251). Called by + /// Register a process-lifetime mark-waypoint header (D-251), called by /// `Runtime.trackHeap` for every gpa-created, never-swept object whose - /// per-tag trace may reach GC-managed children (e.g. a `Function`'s - /// `closure_bindings`). `collect()` clears the mark bit on each before the - /// root walk so the waypoint is re-traced every cycle (a never-cleared bit - /// would short-circuit `mark()` from the 2nd collect on, stranding the - /// children). Best-effort: a registration OOM is swallowed — the worst case - /// is the pre-fix behaviour for that one object, not a crash. + /// trace may reach GC children. Header must be at offset 0 of `obj`. // GC-ROOT: D4 — process-lifetime trackHeap'd mark-waypoints (off gc.allocations); highest-risk moving-GC site [ref: .dev/gc_rooting.md §D] pub fn registerPersistentMark(self: *GcHeap, obj: *anyopaque, size: usize) !void { io_default.lockMutex(&self.gc_mutex); @@ -354,20 +217,17 @@ pub const GcHeap = struct { self.persistent_bytes += size; } - /// Roll back the most recent `registerPersistentMark` (for `trackHeap`'s - /// two-list atomicity: if the `heap_objects` append fails after the mark was - /// registered, the dangling header must not stay in `persistent_marks`). + /// Roll back the most recent `registerPersistentMark` (`trackHeap`'s + /// two-list atomicity on a failed second append). pub fn unregisterLastPersistentMark(self: *GcHeap) void { io_default.lockMutex(&self.gc_mutex); defer io_default.unlockMutex(&self.gc_mutex); if (self.persistent_marks.pop()) |m| self.persistent_bytes -= m.size; } - /// Clear the mark bit on every registered process-lifetime waypoint. - /// `collect()` calls this at mark-phase start (D-251). The cast to - /// `*HeapHeader` (header-at-offset-0 per `registerPersistentMark`'s - /// precondition) happens here, not at registration, so a registration - /// never alignment-checks a non-Value pointer (e.g. a unit-test box). + /// Clear the mark bit on every registered process-lifetime waypoint, at + /// mark-phase start (D-251). The cast happens here, not at registration, + /// so a registration never alignment-checks a non-Value pointer. pub fn clearPersistentMarks(self: *GcHeap) void { for (self.persistent_marks.items) |m| { const hdr: *HeapHeader = @ptrCast(@alignCast(m.obj)); @@ -375,20 +235,15 @@ pub const GcHeap = struct { } } - /// Pin a Value so it stays alive across `collect()` cycles. Returns - /// after appending to `permanent_roots`. Callers (FFI / test - /// fixtures / future REPL prompt buffer per ADR-0028 §5 row 10) - /// must pair every `pin` with an `unpin` to avoid steady-state - /// leaks. Immediates can be pinned too — the walker filters them. + /// Pin a Value so it stays alive across `collect()` cycles. Every `pin` + /// pairs with an `unpin`. Immediates may be pinned; the walker filters. pub fn pin(self: *GcHeap, v: Value) !void { io_default.lockMutex(&self.gc_mutex); defer io_default.unlockMutex(&self.gc_mutex); try self.permanent_roots.append(self.infra, v); } - /// Unpin the first matching Value entry. Returns `true` on a hit, - /// `false` if the Value was not pinned (treated as a programming - /// error by callers — typically wrapped in `std.debug.assert`). + /// Unpin the first matching Value. `false` if it was not pinned. pub fn unpin(self: *GcHeap, v: Value) bool { io_default.lockMutex(&self.gc_mutex); defer io_default.unlockMutex(&self.gc_mutex); @@ -401,134 +256,83 @@ pub const GcHeap = struct { return false; } - /// Enter a fabrication no-collect region (D-244 #4). A multi-alloc - /// collection builder calls this before its alloc sequence and - /// `exitFabrication` (via `defer`) after — see `fabrication_depth`. - /// Nests: a builder calling a wrapped builder balances. Cheap (one - /// threadlocal increment, no lock). + /// Enter a fabrication no-collect region (D-244 #4): a multi-alloc + /// builder brackets its alloc sequence with this and `exitFabrication`. + /// Nests. One threadlocal increment. pub fn enterFabrication(_: *GcHeap) void { root_set_mod.fabrication_depth += 1; } - /// Leave a fabrication no-collect region (pairs with `enterFabrication`). + /// Leave a fabrication no-collect region. pub fn exitFabrication(_: *GcHeap) void { root_set_mod.fabrication_depth -= 1; } - /// Allocate a typed heap object on the GC heap: free-pool fast path - /// → infra slow path, with a comptime HeapHeader-at-offset-0 check. - /// Caller initialises the value (HeapHeader and payload fields). - /// The prologue CAN collect BEFORE the new object exists: it parks for - /// a peer's stop-the-world, runs the alloc-driven torture collect - /// (D-386) and the D-519 threshold auto-collect, all outside a - /// fabrication bracket. So a value held only in a Zig local across an - /// `alloc` must be published as a root (`.dev/gc_rooting.md`). - /// - /// Allocation size is rounded up to `min_alloc_bytes = 16` per - /// ADR-0028 §3 so freed memory can host the FreeNode overlay at - /// offset 8. The extra payload bytes (for T smaller than 16) are - /// unused while live but become the FreeNode region on free. - /// - /// Convention: `T` must have `HeapHeader` as its first field so - /// the returned `*T` and `*HeapHeader` are pointer-aliases. The - /// caller-side existing pattern (`pub const BigInt = struct { - /// header: HeapHeader, ... }`) satisfies this; the comptime check - /// below rejects mis-typed `T` at compile time before the live- - /// list mis-link can land. + /// Allocate a typed heap object: page heap for a small cell, infra for a + /// large one, with a comptime HeapHeader-at-offset-0 check. The caller + /// initialises the value. The prologue CAN collect BEFORE the new object + /// exists (peer STW park, D-386 torture, D-519 auto-collect), so a value + /// held only in a Zig local across an `alloc` must be published as a root + /// (`.dev/gc_rooting.md`). Size rounds up to `min_alloc_bytes`. pub fn alloc(self: *GcHeap, comptime T: type) !*T { return self.allocSized(T, @sizeOf(T)); } - /// ADR-0184: allocate `T` plus `tail_bytes` of trailing storage as ONE - /// GC cell — the pattern for a heap object whose owned arrays live - /// inline after the struct (Function: methods + closure_bindings), so - /// sweep frees a single allocation and no finaliser is needed. The - /// caller lays out the tail itself; `@sizeOf(T)` must be 8-aligned so - /// any Value/pointer-bearing tail array starts aligned. + /// ADR-0184: allocate `T` plus `tail_bytes` of trailing storage as ONE GC + /// cell (a Function's methods + closure_bindings inline after the + /// struct), so sweep frees a single allocation. `@sizeOf(T)` must be + /// 8-aligned so a Value-bearing tail starts aligned. pub fn allocWithTail(self: *GcHeap, comptime T: type, tail_bytes: usize) !*T { comptime std.debug.assert(@sizeOf(T) % 8 == 0); return self.allocSized(T, @sizeOf(T) + tail_bytes); } - /// True iff the calling thread may mutate this heap (D-566): it is a - /// registered GC worker, or it is the heap's owning main thread. Anything - /// else is an unregistered mutator, invisible to the STW rendezvous and the - /// root walk (the ADR-0175 corruption class). + /// True iff the calling thread may mutate this heap (D-566): a registered + /// GC worker or the heap's owning main thread. pub fn mutatorAllowed(self: *const GcHeap) bool { return root_set_mod.is_registered_worker or root_set_mod.currentThreadId() == self.owner_thread_id; } fn allocSized(self: *GcHeap, comptime T: type, size: usize) !*T { - comptime assertHeaderAtOffsetZero(T); - // D-566: an unregistered, non-owner allocator is a caught bug here - // instead of a rare GP fault in a peer's root walk. + comptime gc_types.assertHeaderAtOffsetZero(T); + // D-566: an unregistered, non-owner allocator is caught here instead + // of as a rare GP fault in a peer's root walk. std.debug.assert(self.mutatorAllowed()); - // Worker safe point (ADR-0090 Alt B / D-244 #4): if a peer is collecting, - // park HERE — BEFORE contending on `gc_mutex` — so the collector counts - // us parked. A thread that blocked on `gc_mutex` first would never be - // counted (`stopWorld` would hang waiting for it). At this point our live - // Values are already published (operand stack / eval frames / bindings / - // gc_self_guard) and the about-to-be-allocated obj does not exist yet, so - // the collection sees this thread's roots quiescent. Runtime-inert until - // a collector arms `gc_requested` (#4 wires the VM-safe-point trigger). - // - // D-559 (ADR-0150 amendment 1): the park HONORS the fabrication region. - // Mid-bracket, this thread's in-progress builder NODES (*TailNode/ - // *HamtNode/*Cons — raw pointers, not Values) live in unrooted Zig - // locals, so parking here would hand the peer's collect a stack it - // cannot see → the intermediate is swept → the resumed builder reads - // recycled memory (the D-559 `@memcpy arguments alias` crash). ADR-0150's - // contract ("a mid-builder STW collect is DEFERRED") previously covered - // only the SAME-thread torture/auto collect; this extends it to a peer's - // stopWorld, JVM-GCLocker-style: the bracket completes, and the thread - // parks at its next safepoint outside (back-edge poll / non-bracket - // alloc / blocking-lock / unregister — all published-roots sites). The - // collector just waits: no deadlock, because brackets are bounded pure - // builders (no eval, no locks) and their allocs only need `gc_mutex`, - // which `stopWorld` does not hold while it waits. + // Worker safe point (ADR-0090 Alt B / D-244 #4): if a peer is + // collecting, park HERE, before contending on `gc_mutex`, so the + // collector counts us parked. Our live Values are published and the + // new obj does not exist yet. D-559 (ADR-0150 am1): the park honors + // the fabrication region, since mid-bracket builder nodes live in + // unrooted Zig locals; the thread parks at its next safepoint outside. if (root_set_mod.fabrication_depth == 0 and safepoint.gc_requested.load(.acquire)) safepoint.park(); // D-386 alloc-driven GC torture (validation only; inert unless - // CLJW_GC_TORTURE_ALLOC is armed → `alloc_period != 0`, one global load + - // predicted-not-taken branch otherwise). Force a STW collect HERE — at the - // mid-op alloc boundary, BEFORE contending on `gc_mutex`, and BEFORE the new - // obj exists (the roots are quiescent per the park comment above) — so a VM - // op that did not publish its operand watermark / fabrication before - // allocating surfaces as a deterministic UAF, not a rare crash. Scoped to - // the main (unregistered) thread WITH a live `active_env`; a reentrancy - // guard stops the collect's own bookkeeping from re-triggering. + // CLJW_GC_TORTURE_ALLOC is armed). Forces a STW collect at the + // mid-op alloc boundary so a VM op that did not publish its roots + // before allocating surfaces as a deterministic UAF. torture: { const gc_torture = @import("gc_torture.zig"); if (gc_torture.alloc_period == 0) break :torture; - // Inside a multi-alloc builder's no-collect region (D-244 #4): an - // intermediate node is live-but-unrooted, so defer the collect to - // just after the builder. The same gate guards a future ADR-0028 - // alloc-driven auto-collect (it would land right here). if (root_set_mod.fabrication_depth > 0) break :torture; - const root_set = @import("root_set.zig"); - if (root_set.is_registered_worker or in_alloc_torture) break :torture; - const e = root_set.active_env orelse break :torture; + if (root_set_mod.is_registered_worker or in_alloc_torture) break :torture; + const e = root_set_mod.active_env orelse break :torture; if (!gc_torture.allocTick()) break :torture; const mark_sweep = @import("mark_sweep.zig"); in_alloc_torture = true; defer in_alloc_torture = false; mark_sweep.collectStopTheWorld(self, .{ .envs = &.{e}, .gc = self }, false); } - // D-519 (ADR-0164): threshold-driven auto-collect — the completeness floor - // for a bulk Zig-primitive allocation that never crosses a VM back-edge (the - // same boundary + roots-quiescent rationale as the `heap_ceiling` check - // below). Idempotent with the back-edge site via `bytes_since_last_gc`. A - // no-op if CLJW_GC_TORTURE_ALLOC just collected (it reset the byte counter). + // D-519 (ADR-0164): threshold-driven auto-collect, the completeness + // floor for a bulk Zig-primitive allocation that never crosses a VM + // back-edge. self.maybeAutoCollect(); io_default.lockMutex(&self.gc_mutex); defer io_default.unlockMutex(&self.gc_mutex); const align_t: std.mem.Alignment = .fromByteUnits(@alignOf(T)); - const effective_size: usize = @max(size, min_alloc_bytes); + const effective_size: usize = @max(size, gc_types.min_alloc_bytes); - // D-352: per-eval LIVE-heap ceiling. One predicted-not-taken branch when - // unmetered (heap_ceiling == null). Checked HERE (the alloc boundary) — - // not the back-edge poll — because a Zig primitive (e.g. a bulk seq - // realization) can allocate megabytes without crossing an eval back-edge. - // `live = allocated - freed` stays correct if auto-collect ever lands. + // D-352: per-eval LIVE-heap ceiling, checked at the alloc boundary + // because a Zig primitive can allocate megabytes without an eval + // back-edge. if (self.heap_ceiling) |cap| { const live = self.stats.bytes_allocated - self.stats.bytes_freed; if (live + effective_size > cap) { @@ -537,17 +341,8 @@ pub const GcHeap = struct { } } - const key = free_pool_mod.FreePoolKey{ .size = effective_size, .alignment = align_t }; - - const raw: [*]u8 = if (self.free_pools.pop(key)) |reused| reuse: { - self.stats.pool_hits += 1; // diagnostic: served from the free pool (no malloc) - break :reuse reused; - } else blk: { - const fresh = self.infra.rawAlloc(effective_size, align_t, @returnAddress()) orelse - return error.OutOfMemory; - break :blk fresh; - }; - errdefer self.infra.rawFree(raw[0..effective_size], align_t, @returnAddress()); + const raw = try self.cellAlloc(effective_size, align_t); + errdefer self.cellFree(raw, effective_size, align_t); const obj: *T = @ptrCast(@alignCast(raw)); const hdr: *HeapHeader = @ptrCast(@alignCast(raw)); @@ -562,250 +357,33 @@ pub const GcHeap = struct { return obj; } - /// D-519 (ADR-0164): threshold-driven auto-collect during eval. Run a STW - /// collect when the heap has grown past `threshold_bytes` since the last GC, - /// at a quiescent-roots boundary. Called from BOTH the `alloc` boundary (the - /// completeness floor for a bulk Zig-primitive allocation that never crosses a - /// VM back-edge — the same boundary + rationale as the `heap_ceiling` REFUSE - /// check, its collecting twin) AND the VM back-edge poll (the cheap tight-loop - /// path: a `recur` loop back-edges every iteration but allocs once). The shared - /// `bytes_since_last_gc` reset (a collect sets it to 0) keeps the two sites - /// idempotent — no double collect. The guard set + collect ARE the - /// `CLJW_GC_TORTURE_ALLOC` path verbatim, gated on the byte threshold instead of - /// a torture period; torture stays in the gate as the stronger frequent rooting - /// probe (this 4MB cadence can step over a 2-alloc unrooted window for years). - pub fn maybeAutoCollect(self: *GcHeap) void { - if (self.bytes_since_last_gc <= self.threshold_bytes) return; - // A multi-alloc builder's intermediate node is live-but-unrooted here; defer - // the collect to just after the builder (the gate the torture path uses). - if (root_set_mod.fabrication_depth > 0) return; - const root_set = @import("root_set.zig"); - if (root_set.is_registered_worker or in_alloc_torture) return; - const e = root_set.active_env orelse return; - const mark_sweep = @import("mark_sweep.zig"); - // Reentrancy guard reuse: the collect's own bookkeeping must not re-trigger. - in_alloc_torture = true; - defer in_alloc_torture = false; - mark_sweep.collectStopTheWorld(self, .{ .envs = &.{e}, .gc = self }, false); - } - - /// `(System/gc)` — a user-REQUESTED collect (ADR-0174 D5). Same safety - /// gates as `maybeAutoCollect` but without the byte threshold: if a safe - /// stop-the-world collect is possible right now, run it; otherwise no-op. - /// The no-op arm is JVM-faithful — `System.gc()` is specified as a hint - /// ("suggests that the JVM expend effort"), so a skipped collect inside a - /// fabrication region / off-worker context is correct, not a lie. - pub fn requestCollect(self: *GcHeap) void { - if (root_set_mod.fabrication_depth > 0) return; - const root_set = @import("root_set.zig"); - if (root_set.is_registered_worker or in_alloc_torture) return; - const e = root_set.active_env orelse return; - const mark_sweep = @import("mark_sweep.zig"); - in_alloc_torture = true; - defer in_alloc_torture = false; - mark_sweep.collectStopTheWorld(self, .{ .envs = &.{e}, .gc = self }, false); + /// Backing memory for one cell: a page-heap slot when the cell is small, + /// an `infra` block otherwise. Caller holds `gc_mutex`. + fn cellAlloc(self: *GcHeap, size: usize, alignment: std.mem.Alignment) error{OutOfMemory}![*]u8 { + if (page_heap_mod.isSmall(size, alignment)) { + self.stats.small_allocs += 1; + return self.pages.alloc(size) orelse error.OutOfMemory; + } + const mem = self.infra.rawAlloc(size, alignment, @returnAddress()) orelse return error.OutOfMemory; + self.large_live_bytes += size; + return mem; } - /// D-361: enforce the per-eval live-heap ceiling against a BULK `infra` - /// allocation that bypasses `alloc` — the transient vector's element buffer - /// grows via `gc.infra.realloc` and can reach hundreds of MB before the - /// build finishes (`persistent!`, which DOES go through `alloc`, only trips - /// the cap afterwards). On a memory-tight host that uncapped buffer - /// OS-OOM-kills the process first (the D-361 Linux symptom) instead of - /// surfacing the catchable `eval_heap_exceeded`. A bulk-infra site calls - /// this BEFORE allocating `infra_bytes`; it refuses (hook + OutOfMemory) - /// when that plus the current live gc bytes would exceed the cap. No-op when - /// unmetered. (Per-eval is single-threaded, so the read needs no gc_mutex.) - pub fn checkInfraCap(self: *GcHeap, infra_bytes: usize) error{OutOfMemory}!void { - if (self.heap_ceiling) |cap| { - const live = self.stats.bytes_allocated - self.stats.bytes_freed; - if (live + infra_bytes > cap) { - if (self.heap_exceeded_hook) |hook| hook(cap); - return error.OutOfMemory; - } + /// Give one dead cell's memory back: to its page, or to `infra`. Called by + /// sweep (inside the stop-the-world collect) and by `allocSized`'s errdefer. + pub fn cellFree(self: *GcHeap, mem: [*]u8, size: usize, alignment: std.mem.Alignment) void { + if (page_heap_mod.isSmall(size, alignment)) { + self.pages.free(mem); + } else { + self.large_live_bytes -= size; + self.infra.rawFree(mem[0..size], alignment, @returnAddress()); } } - // The `collect()` orchestrator lives in `mark_sweep.zig` — it - // imports `root_set.zig` which itself imports `gc_heap.zig`, so the - // natural cycle-free place for the entry point is - // `mark_sweep.collect(gc, ctx)`. Callers reach the entry point - // through `mark_sweep.collect`, not through a method on this struct. -}; - -// --- tests --- - -test "GcHeap.init / deinit on an empty heap" { - var gc = GcHeap.init(testing.allocator); - defer gc.deinit(); - - try testing.expectEqual(@as(usize, 0), gc.allocations.items.len); - try testing.expectEqual(@as(usize, 0), gc.permanent_roots.items.len); - try testing.expectEqual(@as(usize, 0), gc.stats.bytes_allocated); - try testing.expectEqual(@as(u64, 0), gc.stats.alloc_count); - try testing.expectEqual(@as(usize, default_gc_threshold_bytes), gc.threshold_bytes); -} - -test "D-352: heap_ceiling refuses allocations past the live-byte cap" { - var gc = GcHeap.init(testing.allocator); - defer gc.deinit(); - const Cell = extern struct { header: HeapHeader, payload: u64 = 0 }; - gc.heap_ceiling = 200; // bytes — a handful of cells, then refuse - var refused = false; - var i: u32 = 0; - while (i < 1000) : (i += 1) { - // No hook installed in this unit test, so the bare error.OutOfMemory - // surfaces once live bytes would pass the cap. - _ = gc.alloc(Cell) catch { - refused = true; - break; - }; + /// The committed cell heap: mapped pages (live cells, free slots and + /// retained empty pages) plus live large cells. `(.totalMemory r)` reports + /// it; it shrinks when a collect releases pages. + pub fn committedBytes(self: *const GcHeap) usize { + return self.pages.mappedBytes() + self.large_live_bytes; } - try testing.expect(refused); - // An unmetered heap (cap cleared) keeps allocating past the old ceiling. - gc.heap_ceiling = null; - _ = try gc.alloc(Cell); -} - -test "GcHeap.pin appends to permanent_roots; unpin removes first match" { - var gc = GcHeap.init(testing.allocator); - defer gc.deinit(); - - const a = Value.initInteger(42); - const b = Value.initInteger(7); - try gc.pin(a); - try gc.pin(b); - try testing.expectEqual(@as(usize, 2), gc.permanent_roots.items.len); - - try testing.expect(gc.unpin(a)); - try testing.expectEqual(@as(usize, 1), gc.permanent_roots.items.len); - try testing.expect(gc.unpin(b)); - try testing.expectEqual(@as(usize, 0), gc.permanent_roots.items.len); - - try testing.expect(!gc.unpin(a)); // already removed -} - -test "GcHeap.alloc tracks bytes + count + bytes_since_last_gc" { - // Use a HeapHeader-prefixed test type to satisfy the 5.3.b.1 - // "HeapHeader at offset 0" convention. - const Cell = extern struct { header: HeapHeader, payload: u64 = 0 }; - - var gc = GcHeap.init(testing.allocator); - defer gc.deinit(); - - const c1 = try gc.alloc(Cell); - c1.* = .{ .header = HeapHeader.init(.symbol) }; - try testing.expectEqual(@as(usize, 1), gc.allocations.items.len); - try testing.expectEqual(@as(usize, @sizeOf(Cell)), gc.stats.bytes_allocated); - try testing.expectEqual(@as(u64, 1), gc.stats.alloc_count); - try testing.expectEqual(@as(usize, @sizeOf(Cell)), gc.bytes_since_last_gc); - - const c2 = try gc.alloc(Cell); - c2.* = .{ .header = HeapHeader.init(.vector) }; - try testing.expectEqual(@as(usize, 2), gc.allocations.items.len); - try testing.expectEqual(@as(usize, 2 * @sizeOf(Cell)), gc.stats.bytes_allocated); - try testing.expectEqual(@as(u64, 2), gc.stats.alloc_count); -} - -test "concurrent alloc through the global heap lock is race-free (ADR-0090 §2)" { - // The gc_mutex (locked via io_default) must serialize alloc across real - // OS threads so the `allocations` ArrayList append + `infra` rawAlloc - // never race. Set io_default to a THREADED io for real mutex blocking, - // then restore it (the singleton is process-wide; tests run serially). - const Cell = extern struct { header: HeapHeader, payload: u64 = 0 }; - const per_thread = 500; - const n_threads = 4; - - const saved_io = io_default.get(); - var threaded = std.Io.Threaded.init(testing.allocator, .{}); - defer threaded.deinit(); - defer io_default.set(saved_io); // runs before threaded.deinit (LIFO) - io_default.set(threaded.io()); - - var gc = GcHeap.init(testing.allocator); - defer gc.deinit(); - - const Worker = struct { - fn run(g: *GcHeap) void { - // D-566: a non-owner thread allocates only as a registered worker. - var tx: ?*anyopaque = null; - var ctx = root_set_mod.workerContext(&tx); - root_set_mod.registerThread(&ctx) catch return; - defer root_set_mod.unregisterThread(&ctx); - var i: usize = 0; - while (i < per_thread) : (i += 1) { - const c = g.alloc(Cell) catch return; - c.* = .{ .header = HeapHeader.init(.symbol) }; - } - } - }; - - var threads: [n_threads]std.Thread = undefined; - for (&threads) |*t| t.* = try std.Thread.spawn(.{}, Worker.run, .{&gc}); - for (&threads) |t| t.join(); - - // Every alloc landed exactly once — no lost append, no double-count. - try testing.expectEqual(@as(usize, n_threads * per_thread), gc.allocations.items.len); - try testing.expectEqual(@as(u64, n_threads * per_thread), gc.stats.alloc_count); -} - -test "mutatorAllowed: the stamping thread and registered workers may alloc, a foreign unregistered thread may not (D-566)" { - if (@import("builtin").single_threaded) return error.SkipZigTest; - var gc = GcHeap.init(testing.allocator); - defer gc.deinit(); - // The thread that built the heap is its owning main thread. - try testing.expect(gc.mutatorAllowed()); - try testing.expect(root_set_mod.analysisMutatorAllowed()); - - const Probe = struct { - var unregistered: bool = true; - var unregistered_analysis: bool = true; - var registered: bool = false; - fn run(g: *const GcHeap) void { - unregistered = g.mutatorAllowed(); - unregistered_analysis = root_set_mod.analysisMutatorAllowed(); - var tx: ?*anyopaque = null; - var ctx = root_set_mod.workerContext(&tx); - root_set_mod.registerThread(&ctx) catch return; - defer root_set_mod.unregisterThread(&ctx); - registered = g.mutatorAllowed(); - } - }; - Probe.unregistered = true; - Probe.unregistered_analysis = true; - Probe.registered = false; - const t = try std.Thread.spawn(.{}, Probe.run, .{&gc}); - t.join(); - try testing.expect(!Probe.unregistered); - try testing.expect(!Probe.unregistered_analysis); - try testing.expect(Probe.registered); -} - -test "GcHeap.alloc returned pointer aliases the live-list HeapHeader" { - // The HeapHeader-at-offset-0 convention means the returned *T and - // the live-list *HeapHeader are pointer-aliases (same address); - // 5.3.b mark / 5.3.c sweep both rely on this. - const Cell = extern struct { header: HeapHeader, payload: u64 = 0 }; - - var gc = GcHeap.init(testing.allocator); - defer gc.deinit(); - - const c = try gc.alloc(Cell); - c.* = .{ .header = HeapHeader.init(.list) }; - - const rec = gc.allocations.items[0]; - const hdr_via_cast: *HeapHeader = @ptrCast(c); - try testing.expectEqual(rec.header, hdr_via_cast); - try testing.expectEqual(@as(u8, @intFromEnum(value_mod.HeapTag.list)), rec.header.tag); - try testing.expectEqual(@sizeOf(Cell), rec.size); -} - -test "Stats struct shape" { - const s = Stats{}; - try testing.expectEqual(@as(usize, 0), s.bytes_allocated); - try testing.expectEqual(@as(usize, 0), s.bytes_freed); - try testing.expectEqual(@as(u64, 0), s.alloc_count); - try testing.expectEqual(@as(u64, 0), s.collect_count); - try testing.expectEqual(@as(u64, 0), s.sweep_count); - try testing.expectEqual(@as(usize, 0), s.last_live_bytes); -} +}; diff --git a/src/runtime/gc/gc_heap_test.zig b/src/runtime/gc/gc_heap_test.zig new file mode 100644 index 000000000..85ac8ca4c --- /dev/null +++ b/src/runtime/gc/gc_heap_test.zig @@ -0,0 +1,178 @@ +// SPDX-License-Identifier: EPL-2.0 +//! Unit tests for `gc_heap.zig`: init/deinit, the live-heap ceiling, pinning, +//! allocation accounting, the global heap lock under real threads, and the +//! D-566 mutator check. + +const std = @import("std"); +const testing = std.testing; + +const gc_heap_mod = @import("gc_heap.zig"); +const heap_header = @import("../value/heap_header.zig"); +const value_mod = @import("../value/value.zig"); +const io_default = @import("../concurrency/io_default.zig"); +const root_set_mod = @import("root_set.zig"); + +const GcHeap = gc_heap_mod.GcHeap; +const HeapHeader = heap_header.HeapHeader; +const Value = value_mod.Value; +const Cell = extern struct { header: HeapHeader, payload: u64 = 0 }; + +test "GcHeap.init / deinit on an empty heap" { + var gc = GcHeap.init(testing.allocator); + defer gc.deinit(); + + try testing.expectEqual(@as(usize, 0), gc.allocations.items.len); + try testing.expectEqual(@as(usize, 0), gc.permanent_roots.items.len); + try testing.expectEqual(@as(usize, 0), gc.stats.bytes_allocated); + try testing.expectEqual(@as(u64, 0), gc.stats.alloc_count); + try testing.expectEqual(@as(usize, gc_heap_mod.default_gc_threshold_bytes), gc.threshold_bytes); +} + +test "D-352: heap_ceiling refuses allocations past the live-byte cap" { + var gc = GcHeap.init(testing.allocator); + defer gc.deinit(); + gc.heap_ceiling = 200; // bytes: a handful of cells, then refuse + var refused = false; + var i: u32 = 0; + while (i < 1000) : (i += 1) { + // No hook installed, so the bare error.OutOfMemory surfaces. + _ = gc.alloc(Cell) catch { + refused = true; + break; + }; + } + try testing.expect(refused); + // An unmetered heap keeps allocating past the old ceiling. + gc.heap_ceiling = null; + _ = try gc.alloc(Cell); +} + +test "GcHeap.pin appends to permanent_roots; unpin removes first match" { + var gc = GcHeap.init(testing.allocator); + defer gc.deinit(); + + const a = Value.initInteger(42); + const b = Value.initInteger(7); + try gc.pin(a); + try gc.pin(b); + try testing.expectEqual(@as(usize, 2), gc.permanent_roots.items.len); + + try testing.expect(gc.unpin(a)); + try testing.expectEqual(@as(usize, 1), gc.permanent_roots.items.len); + try testing.expect(gc.unpin(b)); + try testing.expectEqual(@as(usize, 0), gc.permanent_roots.items.len); + + try testing.expect(!gc.unpin(a)); // already removed +} + +test "GcHeap.alloc tracks bytes + count + bytes_since_last_gc" { + var gc = GcHeap.init(testing.allocator); + defer gc.deinit(); + + const c1 = try gc.alloc(Cell); + c1.* = .{ .header = HeapHeader.init(.symbol) }; + try testing.expectEqual(@as(usize, 1), gc.allocations.items.len); + try testing.expectEqual(@as(usize, @sizeOf(Cell)), gc.stats.bytes_allocated); + try testing.expectEqual(@as(u64, 1), gc.stats.alloc_count); + try testing.expectEqual(@as(usize, @sizeOf(Cell)), gc.bytes_since_last_gc); + + const c2 = try gc.alloc(Cell); + c2.* = .{ .header = HeapHeader.init(.vector) }; + try testing.expectEqual(@as(usize, 2), gc.allocations.items.len); + try testing.expectEqual(@as(usize, 2 * @sizeOf(Cell)), gc.stats.bytes_allocated); + try testing.expectEqual(@as(u64, 2), gc.stats.alloc_count); +} + +test "concurrent alloc through the global heap lock is race-free (ADR-0090 §2)" { + // gc_mutex must serialize alloc across real OS threads. Set io_default to + // a THREADED io for real mutex blocking, then restore it. + const per_thread = 500; + const n_threads = 4; + + const saved_io = io_default.get(); + var threaded = std.Io.Threaded.init(testing.allocator, .{}); + defer threaded.deinit(); + defer io_default.set(saved_io); // runs before threaded.deinit (LIFO) + io_default.set(threaded.io()); + + var gc = GcHeap.init(testing.allocator); + defer gc.deinit(); + + const Worker = struct { + fn run(g: *GcHeap) void { + // D-566: a non-owner thread allocates only as a registered worker. + var tx: ?*anyopaque = null; + var ctx = root_set_mod.workerContext(&tx); + root_set_mod.registerThread(&ctx) catch return; + defer root_set_mod.unregisterThread(&ctx); + var i: usize = 0; + while (i < per_thread) : (i += 1) { + const c = g.alloc(Cell) catch return; + c.* = .{ .header = HeapHeader.init(.symbol) }; + } + } + }; + + var threads: [n_threads]std.Thread = undefined; + for (&threads) |*t| t.* = try std.Thread.spawn(.{}, Worker.run, .{&gc}); + for (&threads) |t| t.join(); + + // Every alloc landed exactly once. + try testing.expectEqual(@as(usize, n_threads * per_thread), gc.allocations.items.len); + try testing.expectEqual(@as(u64, n_threads * per_thread), gc.stats.alloc_count); +} + +test "mutatorAllowed: the stamping thread and registered workers may alloc, a foreign unregistered thread may not (D-566)" { + if (@import("builtin").single_threaded) return error.SkipZigTest; + var gc = GcHeap.init(testing.allocator); + defer gc.deinit(); + try testing.expect(gc.mutatorAllowed()); + try testing.expect(root_set_mod.analysisMutatorAllowed()); + + const Probe = struct { + var unregistered: bool = true; + var unregistered_analysis: bool = true; + var registered: bool = false; + fn run(g: *const GcHeap) void { + unregistered = g.mutatorAllowed(); + unregistered_analysis = root_set_mod.analysisMutatorAllowed(); + var tx: ?*anyopaque = null; + var ctx = root_set_mod.workerContext(&tx); + root_set_mod.registerThread(&ctx) catch return; + defer root_set_mod.unregisterThread(&ctx); + registered = g.mutatorAllowed(); + } + }; + Probe.unregistered = true; + Probe.unregistered_analysis = true; + Probe.registered = false; + const t = try std.Thread.spawn(.{}, Probe.run, .{&gc}); + t.join(); + try testing.expect(!Probe.unregistered); + try testing.expect(!Probe.unregistered_analysis); + try testing.expect(Probe.registered); +} + +test "GcHeap.alloc returned pointer aliases the live-list HeapHeader" { + var gc = GcHeap.init(testing.allocator); + defer gc.deinit(); + + const c = try gc.alloc(Cell); + c.* = .{ .header = HeapHeader.init(.list) }; + + const rec = gc.allocations.items[0]; + const hdr_via_cast: *HeapHeader = @ptrCast(c); + try testing.expectEqual(rec.header, hdr_via_cast); + try testing.expectEqual(@as(u8, @intFromEnum(value_mod.HeapTag.list)), rec.header.tag); + try testing.expectEqual(@sizeOf(Cell), rec.size); +} + +test "Stats struct shape" { + const s = gc_heap_mod.Stats{}; + try testing.expectEqual(@as(usize, 0), s.bytes_allocated); + try testing.expectEqual(@as(usize, 0), s.bytes_freed); + try testing.expectEqual(@as(u64, 0), s.alloc_count); + try testing.expectEqual(@as(u64, 0), s.collect_count); + try testing.expectEqual(@as(u64, 0), s.sweep_count); + try testing.expectEqual(@as(usize, 0), s.last_live_bytes); +} diff --git a/src/runtime/gc/gc_trigger.zig b/src/runtime/gc/gc_trigger.zig new file mode 100644 index 000000000..ca2b7a657 --- /dev/null +++ b/src/runtime/gc/gc_trigger.zig @@ -0,0 +1,228 @@ +// SPDX-License-Identifier: EPL-2.0 +//! Collect-trigger policy for the mark-sweep heap: WHEN a collection runs and +//! how much of the freed heap it keeps mapped afterwards (ADR-0028 §1, +//! ADR-0164, D-573, CLJW-GC-HEAP-RETURN). +//! +//! `gc_heap.zig` owns the allocator state and `mark_sweep.zig` owns the +//! collection itself; this file owns the numbers that connect them: +//! - the pressure counters (`noteOwnedAlloc`, `chargeInfraBytes`, +//! `pressureBytes`) the trigger compares to `threshold_bytes`, +//! - the two collect entry points the runtime calls (`maybeAutoCollect` at +//! the alloc boundary and the VM back-edge, `requestCollect` for +//! `System/gc`), +//! - the post-sweep threshold recompute and the time-share floor steering, +//! - the empty-page retention budget handed to `PageHeap.release`. +//! +//! `GcHeap` re-exposes the mutator-facing functions as methods (decl aliases), +//! so callers write `rt.gc.noteOwnedAlloc(n)`. + +const std = @import("std"); +const builtin = @import("builtin"); +const gc_heap_mod = @import("gc_heap.zig"); +const root_set_mod = @import("root_set.zig"); + +const GcHeap = gc_heap_mod.GcHeap; + +/// Default GC trigger threshold (bytes since last collection), and the default +/// floor of the per-heap adaptive threshold. Adaptive at runtime: +/// `threshold = max(threshold_floor_bytes, last_live_bytes * 2)` per ADR-0028 §1. +/// Raised 1MB to 4MB by ADR-0164/D-519: for a churn workload `last_live` is +/// about 0, so the floor IS the operative cadence; 4MB is a quarter of the +/// collects (same total sweep work, fewer STW fences) at ~3MB extra peak. Tune +/// via `CLJW_GC_THRESHOLD_MB`. +pub const default_gc_threshold_bytes: usize = 4 * 1024 * 1024; + +/// Ceiling for the ramping floor (D-573). 64 MB is where the measured curve +/// knees: on the Rush Hour BFS, 4 MB -> 17.7 s, 16 MB -> 9.6 s, 64 MB -> 7.6 s, +/// 256 MB -> 7.1 s. Past 64 MB the remaining 6% costs 4x the resident heap, +/// which is the wrong trade for a runtime whose selling point is a small +/// footprint. `CLJW_GC_THRESHOLD_MB` still overrides the STARTING floor. +pub const max_threshold_floor_bytes: usize = 64 * 1024 * 1024; + +/// Target share of wall clock spent collecting, as a percent (D-573). +/// +/// The floor is steered by the MEASURED ratio rather than by a byte constant or +/// a collection count, because that ratio is the thing anyone cares about, and +/// because it is the quantity the theory is written in: for a mark-sweep +/// collector, GC share is `mark_cost(live) / bytes_between_collects`, so +/// raising the trigger lowers the share directly, independent of live size. +/// +/// The JVM exposes exactly this as `-XX:GCTimeRatio` (about 1 % by default in +/// the throughput collector). 10 % is deliberately laxer: cljw's collector is +/// non-generational, so it re-marks the whole live set every time and cannot +/// reach 1 % without letting the heap grow far past what a small runtime +/// should hold. Measured motivation: a Rush Hour BFS with a 747 KB live set +/// spent 10.1 s of 17.9 s collecting, the 4 MB floor firing 755 times. +pub const gc_time_target_percent: u64 = 10; + +/// Empty pages a collect always keeps mapped (CLJW-GC-HEAP-RETURN): 1 MiB, 16 +/// pages, so a program that allocates a little after every `System/gc` does +/// not map and unmap the same pages each time. +pub const min_retained_bytes: usize = 1024 * 1024; + +/// Charge `n` bytes of a cell's payload, allocated on `infra` outside the +/// cell, to the collect trigger (see `GcHeap.owned_since_last_gc`). An owner +/// whose payload can be large calls this where it allocates the payload. One +/// relaxed atomic add, no lock. +pub fn noteOwnedAlloc(gc: *GcHeap, n: usize) void { + _ = gc.owned_since_last_gc.fetchAdd(n, .monotonic); +} + +/// Allocation pressure since the last collect: cell bytes plus charged payload +/// bytes. The quantity the trigger compares to `threshold_bytes`. +pub fn pressureBytes(gc: *const GcHeap) usize { + return gc.bytes_since_last_gc +| gc.owned_since_last_gc.load(.monotonic); +} + +/// D-361: enforce the per-eval live-heap ceiling against a BULK `infra` +/// allocation that bypasses `alloc` (the transient vector's element buffer +/// grows via `gc.infra.realloc` and can reach hundreds of MB before +/// `persistent!` runs). On a memory-tight host that uncapped buffer +/// OS-OOM-kills the process first (the D-361 Linux symptom) instead of +/// surfacing the catchable `eval_heap_exceeded`. A bulk-infra site calls this +/// BEFORE allocating `infra_bytes`; it refuses (hook + OutOfMemory) when that +/// plus the current live gc bytes would exceed the cap. Per-eval is +/// single-threaded, so the read needs no gc_mutex. Admitted bytes are charged +/// to the collect trigger. +pub fn chargeInfraBytes(gc: *GcHeap, infra_bytes: usize) error{OutOfMemory}!void { + if (gc.heap_ceiling) |cap| { + const live = gc.stats.bytes_allocated - gc.stats.bytes_freed; + if (live + infra_bytes > cap) { + if (gc.heap_exceeded_hook) |hook| hook(cap); + return error.OutOfMemory; + } + } + noteOwnedAlloc(gc, infra_bytes); +} + +/// Empty-page budget a collect keeps mapped after its sweep, the hysteresis +/// against map/unmap thrash. An automatic collect keeps up to one threshold's +/// worth: the next cycle allocates at most `threshold_bytes` before it +/// collects again, so that is the most fresh space it can use, and a page +/// beyond it would sit idle until the next release anyway. A steady-state +/// burst therefore refills its predecessor's pages instead of faulting in new +/// ones. A requested collect (`System/gc`) keeps only `min_retained_bytes`: +/// the program asked for the memory back. +pub fn retainBytesAfterCollect(gc: *const GcHeap, requested: bool) usize { + if (requested) return min_retained_bytes; + return @max(min_retained_bytes, gc.threshold_bytes); +} + +/// D-519 (ADR-0164): threshold-driven auto-collect during eval. Runs a STW +/// collect when pressure has passed `threshold_bytes`, at a quiescent-roots +/// boundary. Called from BOTH the `alloc` boundary (the completeness floor for +/// a bulk Zig-primitive allocation that never crosses a VM back-edge) AND the +/// VM back-edge poll (a `recur` loop back-edges every iteration but allocs +/// once). A collect resets the pressure counters, which keeps the two sites +/// idempotent. The guard set + collect are the `CLJW_GC_TORTURE_ALLOC` path, +/// gated on bytes instead of a torture period. +pub fn maybeAutoCollect(gc: *GcHeap) void { + if (pressureBytes(gc) <= gc.threshold_bytes) return; + collectIfSafe(gc); +} + +/// `(System/gc)`: a user-REQUESTED collect (ADR-0174 D5). Same safety gates +/// as `maybeAutoCollect` without the byte threshold. The no-op arm is +/// JVM-faithful: `System.gc()` is specified as a hint, so a skipped collect +/// inside a fabrication region or off-worker context is correct. A requested +/// collect also returns the empty pages an automatic one keeps warm +/// (`retainBytesAfterCollect`); `collect` consumes the flag. +pub fn requestCollect(gc: *GcHeap) void { + if (!collectSafeHere()) return; + gc.release_requested = true; + collectIfSafe(gc); +} + +/// Allocation pressure below which `collectBeforeIdle` does nothing, so an +/// idle loop that allocates little pays no mark. +pub const idle_collect_min_bytes: usize = 1024 * 1024; + +/// Collect while the mutator waits for input (a server between requests), +/// then hand freed malloc memory back to the OS. A no-op below +/// `idle_collect_min_bytes` of pressure. Same safety gates and page +/// release as `requestCollect`. +pub fn collectBeforeIdle(gc: *GcHeap) void { + if (pressureBytes(gc) < idle_collect_min_bytes) return; + requestCollect(gc); + trimProcessHeap(); +} + +extern "c" fn malloc_trim(pad: usize) c_int; + +/// Return glibc's free arena memory to the OS. Other allocators unmap large +/// frees themselves, so this is a no-op off glibc. +fn trimProcessHeap() void { + if (comptime builtin.link_libc and builtin.target.abi.isGnu()) _ = malloc_trim(0); +} + +/// True iff a stop-the-world collect may run on this thread right now: no +/// multi-alloc builder is mid-bracket (its intermediate nodes are +/// live-but-unrooted), this is not a registered worker, the collector is not +/// already running, and an env exists to root from. +fn collectSafeHere() bool { + if (root_set_mod.fabrication_depth > 0) return false; + if (root_set_mod.is_registered_worker or gc_heap_mod.in_alloc_torture) return false; + return root_set_mod.active_env != null; +} + +// `noinline`: maybeAutoCollect is inlined into the VM's per-op loop, and an +// inlined collect would put its whole frame on every `eval` re-entry's native +// stack (ADR-0195 budget). The call happens only past the threshold. +noinline fn collectIfSafe(gc: *GcHeap) void { + if (!collectSafeHere()) return; + const e = root_set_mod.active_env.?; + const mark_sweep = @import("mark_sweep.zig"); + // Reentrancy guard: the collect's own bookkeeping must not re-trigger. + gc_heap_mod.in_alloc_torture = true; + defer gc_heap_mod.in_alloc_torture = false; + mark_sweep.collectStopTheWorld(gc, .{ .envs = &.{e}, .gc = gc }, false); +} + +/// End-of-collect bookkeeping, after sweep set `last_live_bytes`: recompute +/// the threshold, reset the pressure counters, release empty pages past the +/// retention budget (after the recompute, since the budget is one threshold's +/// worth). +/// +/// The threshold is `max(threshold_floor_bytes, last_live * 2)`. The floor is +/// the per-heap value (default, or `CLJW_GC_THRESHOLD_MB`) so the knob +/// survives this recompute; `steerFloor` moves it between collects. For a +/// churn workload `last_live` is about 0 and the floor governs (ADR-0164); +/// once the live set grows past half the floor, the adaptive arm takes over. +/// Measured on a Rush Hour BFS (3.17 GB over 26.2M allocations): 4 MB floor +/// 755 collections 17.7 s, 64 MB floor 47 collections 7.6 s, about 13 ms of +/// fixed cost per collect paid to reclaim a heap that was nearly all garbage. +pub fn afterSweep(gc: *GcHeap) void { + gc.threshold_bytes = @max(gc.threshold_floor_bytes, gc.stats.last_live_bytes *| 2); + gc.bytes_since_last_gc = 0; + gc.owned_since_last_gc.store(0, .monotonic); + gc.stats.collect_count += 1; + _ = gc.pages.release(retainBytesAfterCollect(gc, gc.release_requested)); + gc.release_requested = false; +} + +/// Move `threshold_floor_bytes` toward whatever keeps the measured GC share at +/// `gc_time_target_percent` (D-573). +/// +/// The share is `gc_ns_total / (now - first_collect_ns)`: wall clock since the +/// program first had to collect, the window the floor can influence. Over +/// target, double the floor, so twice as much garbage accumulates per +/// collection and the share roughly halves. Under target with room to spare, +/// ease back down, so a program whose live set shrank stops holding a floor it +/// no longer needs. Doubling/halving rather than solving: the share is a +/// lagging average, so a single-step solve would oscillate. +pub fn steerFloor(gc: *GcHeap, now_ns: i64) void { + const elapsed = now_ns - gc.stats.first_collect_ns; + // Below this there is no meaningful ratio yet: a first collection is 100 % + // of the time since the first collection. + if (elapsed < 50 * std.time.ns_per_ms) return; + const share_pct = gc.stats.gc_ns_total * 100 / @as(u64, @intCast(elapsed)); + if (share_pct > gc_time_target_percent) { + if (gc.threshold_floor_bytes < max_threshold_floor_bytes) { + gc.threshold_floor_bytes = @min(max_threshold_floor_bytes, gc.threshold_floor_bytes *| 2); + } + } else if (share_pct * 4 < gc_time_target_percent and + gc.threshold_floor_bytes > gc.base_threshold_floor_bytes) + { + gc.threshold_floor_bytes = @max(gc.base_threshold_floor_bytes, gc.threshold_floor_bytes / 2); + } +} diff --git a/src/runtime/gc/gc_types.zig b/src/runtime/gc/gc_types.zig new file mode 100644 index 000000000..a30264d74 --- /dev/null +++ b/src/runtime/gc/gc_types.zig @@ -0,0 +1,74 @@ +// SPDX-License-Identifier: EPL-2.0 +//! Plain data shapes of the mark-sweep heap (ADR-0028): the live-list record, +//! the persistent mark-waypoint, the statistics block, and the comptime +//! HeapHeader-at-offset-0 check every `GcHeap.alloc` type passes. No state, +//! no behaviour: `gc_heap.zig` holds the heap that uses them. + +const std = @import("std"); +const HeapHeader = @import("../value/heap_header.zig").HeapHeader; + +/// Minimum allocation size (bytes) per ADR-0028 §3: the freed payload must +/// host the free-list link at offset 8 (8 bytes header + at least 8 bytes +/// payload = 16 bytes minimum). Smaller types round up; the extra bytes are +/// unused while live. +pub const min_alloc_bytes: usize = 16; + +/// Comptime invariant: every `T` passed to `GcHeap.alloc` has `HeapHeader` as +/// its first field, at offset 0. The returned `*T` and the live-list +/// `*HeapHeader` are pointer-aliases; mark/sweep read `header.tag` without +/// knowing T. A different first field would silently misread its first bytes +/// as a tag and run the wrong `tag_finaliser_table` entry at sweep, so the +/// check fires at compile time. +pub fn assertHeaderAtOffsetZero(comptime T: type) void { + const info = @typeInfo(T); + if (info != .@"struct") { + @compileError("GcHeap.alloc requires a struct type; got " ++ @typeName(T)); + } + const fields = info.@"struct".fields; + if (fields.len == 0) { + @compileError("GcHeap.alloc requires T to have at least one field; " ++ @typeName(T) ++ " has none"); + } + if (fields[0].type != HeapHeader) { + @compileError("GcHeap.alloc requires T to have HeapHeader as its first field; " ++ @typeName(T) ++ "'s first field is " ++ @typeName(fields[0].type)); + } + if (@offsetOf(T, fields[0].name) != 0) { + @compileError("GcHeap.alloc requires HeapHeader at offset 0 of T; " ++ @typeName(T) ++ " has it at a different offset"); + } +} + +/// A process-lifetime mark-waypoint (D-251) + the bytes it owns (D-573): the +/// size feeds `persistent_bytes`, the cost the time-share arm absorbs because +/// these objects are re-traced every cycle. +pub const PersistentMark = struct { + obj: *anyopaque, + size: usize, +}; + +/// Allocation + collection statistics. +pub const Stats = struct { + bytes_allocated: usize = 0, + bytes_freed: usize = 0, + alloc_count: u64 = 0, + collect_count: u64 = 0, + sweep_count: u64 = 0, + last_live_bytes: usize = 0, + /// Diagnostic (gc_alloc_rate lever, D-450): allocs served from the page + /// heap (vs a large cell's `infra.rawAlloc`). small_allocs/alloc_count is + /// the share of the allocation stream the page heap carries. + small_allocs: u64 = 0, + /// D-573: nanoseconds spent inside `collectStopTheWorld`, and the wall + /// clock at the first collection. Their ratio is the GC time fraction the + /// floor is steered by (the JVM's `-XX:GCTimeRatio` quantity). + gc_ns_total: u64 = 0, + first_collect_ns: i64 = 0, +}; + +/// Per-allocation record on the GcHeap live list: the type-erased +/// `*HeapHeader` plus the original (size, alignment), so sweep and deinit free +/// with the metadata `rawAlloc` was given. Without them a debug allocator +/// trips "Invalid free": the header alone would imply an 8-byte block. +pub const AllocRecord = struct { + header: *HeapHeader, + size: usize, + alignment: std.mem.Alignment, +}; diff --git a/src/runtime/gc/mark_sweep.zig b/src/runtime/gc/mark_sweep.zig index 950343261..9250614dd 100644 --- a/src/runtime/gc/mark_sweep.zig +++ b/src/runtime/gc/mark_sweep.zig @@ -2,119 +2,91 @@ //! Mark + sweep phases for cw v1 mark-sweep GC per ADR-0028 §1 + §4 + §5. //! //! Two stop-the-world phases plus the `collect()` orchestrator: -//! - `mark(gc, header)` — shades an object gray: sets -//! `HeapHeader.gc_and_lock.gc_mark` (bit 0) and pushes non-leaf -//! headers onto `gc.mark_worklist` (ADR-0028 amendment 3 — the -//! prior recursive descent overflowed the native stack on deep -//! cons chains). The bit doubles as the visited flag — -//! cycle-mark invariant per ADR-0028 §5. -//! - `drainGray(gc)` — blackens the worklist: pops headers and runs -//! their per-tag trace fn (`tag_ops.tag_trace_table`) until empty. -//! - `sweep(gc)` — walks the live list. For every object with -//! `mark == 0`: call per-tag finaliser via -//! `tag_ops.tag_finaliser_table` (no-alloc invariant per ADR-0028 -//! §4), push onto the matching free pool (rawFree fallback), -//! swap-remove from the live list. For every object with -//! `mark == 1`: clear the bit and keep. -//! - `collect(gc, ctx)` — enumerates roots via `root_set.enumerate`, -//! shades each gray, drains the worklist (blacken), sweeps, then -//! recomputes the adaptive `threshold_bytes` per ADR-0028 §1. +//! - `mark(gc, header)`: shades an object gray: sets +//! `HeapHeader.gc_and_lock.gc_mark` (bit 0) and pushes non-leaf headers +//! onto `gc.mark_worklist` (ADR-0028 amendment 3: the prior recursive +//! descent overflowed the native stack on deep cons chains). The bit +//! doubles as the visited flag (cycle-mark invariant, ADR-0028 §5). +//! - `drainGray(gc)`: blackens the worklist via the per-tag trace fns. +//! - `sweep(gc)`: finalises and frees every unmarked object +//! (`GcHeap.cellFree`: its page, or infra), clears the bit on the rest. +//! - `collect(gc, ctx)`: roots, mark, drain, sweep, then the trigger +//! bookkeeping (`gc_trigger.afterSweep`: threshold recompute, counter +//! reset, empty-page release). //! -//! `clearMarks(gc)` resets every live object's mark bit (used by tests -//! and reserved for an explicit clear pass). All four are landed. +//! The conservative native-stack scan is `conservative_scan.zig`; the trigger +//! policy is `gc_trigger.zig`; unit tests are `mark_sweep_test.zig`. const std = @import("std"); -const build_options = @import("build_options"); const clock = @import("../clock.zig"); -const max_threshold_floor_bytes = @import("gc_heap.zig").max_threshold_floor_bytes; -const gc_time_target_percent = @import("gc_heap.zig").gc_time_target_percent; -const testing = std.testing; const gc_heap_mod = @import("gc_heap.zig"); const heap_header = @import("../value/heap_header.zig"); const tag_ops = @import("tag_ops.zig"); -const free_pool_mod = @import("free_pool.zig"); const root_set_mod = @import("root_set.zig"); const io_default = @import("../concurrency/io_default.zig"); const safepoint = @import("../concurrency/safepoint.zig"); const lock_tx = @import("../concurrency/lock_tx.zig"); +const trigger = @import("gc_trigger.zig"); +const ConservativeScan = @import("conservative_scan.zig").ConservativeScan; + +const GcHeap = gc_heap_mod.GcHeap; +const HeapHeader = heap_header.HeapHeader; /// Adapter so `lock_tx.markRoots` can call back into `mark` with the GcHeap as -/// an opaque context (avoids a lock_tx → mark_sweep import). +/// an opaque context (avoids a lock_tx -> mark_sweep import). fn markTxHeader(ctx: *anyopaque, header: *HeapHeader) void { - const gc: *gc_heap_mod.GcHeap = @ptrCast(@alignCast(ctx)); + const gc: *GcHeap = @ptrCast(@alignCast(ctx)); mark(gc, header); } -/// Adapter for `root_set.markRegisteredTxs`: cast the opaque `current_tx` value -/// back to a transaction and mark its in-txn roots. Called once per registered +/// Adapter for `root_set.markRegisteredTxs`: cast the opaque `current_tx` +/// back to a transaction and mark its in-txn roots, once per registered /// worker thread during a collect (#4a' in-txn-map rooting). fn markWorkerTx(ctx: *anyopaque, tx_opaque: ?*anyopaque) void { - const gc: *gc_heap_mod.GcHeap = @ptrCast(@alignCast(ctx)); + const gc: *GcHeap = @ptrCast(@alignCast(ctx)); const tx_any = tx_opaque orelse return; const tx: *lock_tx.LockingTransaction = @ptrCast(@alignCast(tx_any)); lock_tx.markRoots(tx, @ptrCast(gc), &markTxHeader); } -const GcHeap = gc_heap_mod.GcHeap; -const HeapHeader = heap_header.HeapHeader; - -// CLJW_GC_STATS=3 — cumulative per-phase collect time, printed per collect. +// CLJW_GC_STATS=3: cumulative per-phase collect time, printed per collect. // The diagnostic that attributed D-450's ~24ms-per-collect fixed cost to the -// prep phase (clearPersistentMarks over 15.5M immortal closures); kept, like -// the =2 owned-bytes breakdown, because the next attribution will want it. -// Accumulation is unconditional: ~5 clock reads per collect, noise next to a -// single collect's work. +// prep phase; kept because the next attribution will want it. ~5 clock reads +// per collect. var phase_prep_ns: u64 = 0; var phase_roots_ns: u64 = 0; var phase_scan_ns: u64 = 0; var phase_drain_ns: u64 = 0; var phase_sweep_ns: u64 = 0; -/// Shade a heap object gray: set `header.gc_and_lock.gc_mark` bit 0 and -/// push the header onto `gc.mark_worklist` for `drainGray` to blacken. -/// Recursive per-child descent was retired by ADR-0028 amendment 3 — a -/// deep object graph (a ≥400k-long cons chain from -/// `(count (repeat 1000000 1))`) overflowed the native stack during a -/// mid-walk collect; the explicit worklist keeps mark depth O(1). +/// Shade a heap object gray: set mark bit 0 and push the header onto +/// `gc.mark_worklist` for `drainGray` to blacken. /// -/// Cycle invariant (per ADR-0028 §5 Mark cycle invariant): if the -/// header's mark bit is already set, return immediately — the bit -/// doubles as a visited flag during the mark phase. Cycles -/// (e.g. `LazySeq` whose `thunk` captures another `LazySeq` whose -/// `seq_cache` points back) terminate at the second visit. +/// Cycle invariant (ADR-0028 §5): an already-set bit returns immediately. /// /// LOAD-BEARING ordering: the bit is set BEFORE the push. Push-before-bit -/// would let a failed push leave a marked-looking-but-untraced object, -/// so a live child could be swept (UAF). Bit-before-push also bounds the -/// worklist: each object enters at most once, so `collect()` can reserve -/// `allocations + persistent_marks` capacity up front and the in-collect -/// append never allocates. +/// would let a failed push leave a marked-looking-but-untraced object, so a +/// live child could be swept (UAF). Bit-before-push also bounds the worklist: +/// each object enters at most once, so `collect()` reserves capacity up front +/// and the in-collect append never allocates. /// -/// Per-tag trace dispatch: `tag_ops.tag_trace_table[hdr.tag]` returns -/// the type-specific outgoing-pointer walker, registered per-tag via -/// `tag_ops.registerTrace`. A `null` entry means a genuine leaf node -/// (e.g. `string`, `big_int` limb data) — mark sets the bit and never -/// pushes it. +/// A `null` `tag_trace_table` entry is a leaf (`string`, `big_int`): the bit +/// is set and nothing is pushed. pub fn mark(gc: *GcHeap, header: *HeapHeader) void { if (header.gc_and_lock.gc_mark & 1 == 1) return; // cycle invariant header.gc_and_lock.gc_mark |= 1; if (tag_ops.tag_trace_table[header.tag] == null) return; // leaf - // In-collect this never allocates (capacity reserved up front); a - // direct `mark` caller outside collect (unit tests) may grow the - // worklist on `gc.infra`. True infra OOM here is unrecoverable — - // continuing would strand unmarked live children and sweep them. + // In-collect this never allocates (capacity reserved). A direct caller + // outside collect (unit tests) may grow it; infra OOM here is + // unrecoverable, since continuing would sweep unmarked live children. gc.mark_worklist.append(gc.infra, header) catch @panic("GC mark: gray worklist allocation failed"); } -/// Blacken the gray worklist: pop headers and run their per-tag trace -/// fn (which shades children gray via `mark`) until the worklist is -/// empty. The explicit drain is a named phase of `collect()` — publish -/// roots (gray) → drainGray (blacken) → sweep — the textbook tri-color -/// structure a future incremental / parallel marker inherits. Direct -/// `mark()` callers outside collect (unit tests) must call this to get -/// transitive marking. +/// Blacken the gray worklist: pop headers and run their per-tag trace fn +/// (which shades children gray via `mark`) until empty. Direct `mark()` +/// callers outside collect must call this to get transitive marking. pub fn drainGray(gc: *GcHeap) void { while (gc.mark_worklist.pop()) |header| { // Only non-leaf headers are ever pushed, so the table entry is set. @@ -122,29 +94,21 @@ pub fn drainGray(gc: *GcHeap) void { } } -/// Reset the mark bit on every live allocation. `collect()` already -/// clears marks on surviving objects inline during `sweep`, so this -/// stands alone for tests and an explicit clear-pass caller. +/// Reset the mark bit on every live allocation. `sweep` already clears it on +/// survivors; this stands alone for tests and an explicit clear pass. pub fn clearMarks(gc: *GcHeap) void { for (gc.allocations.items) |rec| { rec.header.gc_and_lock.gc_mark &= ~@as(u30, 1); } } -/// Walk the live list, finalise + recycle unreached objects, clear -/// marks on reached ones. Iterates `gc.allocations` backward (so -/// swap-remove indices stay valid). For each record: -/// - mark bit 0 == 0 (dead): call per-tag finaliser via -/// `tag_ops.tag_finaliser_table` (no-alloc invariant per -/// ADR-0028 §4), push the backing memory onto the matching free -/// pool (`FreePoolMap.push`; rawFree fallback on push failure), -/// swap-remove from allocations, bump `bytes_freed` + `sweep_count`. -/// - mark bit 0 == 1 (live): clear the bit, sum into -/// `last_live_bytes` for the next adaptive-threshold cycle. +/// Walk the live list backward (swap-remove indices stay valid). Dead +/// (bit 0 clear): per-tag finaliser (no-alloc invariant, ADR-0028 §4), free +/// the cell, swap-remove, bump `bytes_freed`. Live: clear the bit and sum its +/// record + owned bytes into `last_live_bytes`. pub fn sweep(gc: *GcHeap) void { - // CLJW_GC_STATS=2 — per-tag owned-bytes breakdown of the LIVE set, printed - // per sweep. The diagnostic that found the D-573 numbers; kept because the - // next accounting question will want it too. + // CLJW_GC_STATS=2: per-tag owned-bytes breakdown of the LIVE set, printed + // per sweep (the diagnostic that found the D-573 numbers). const debug_owned = if (@import("../process_env.zig").get("CLJW_GC_STATS")) |v| std.mem.eql(u8, v, "2") else false; var owned_by_tag: [64]usize = @splat(0); var count_by_tag: [64]usize = @splat(0); @@ -153,28 +117,20 @@ pub fn sweep(gc: *GcHeap) void { while (i > 0) { i -= 1; const rec = gc.allocations.items[i]; - const mark_bit: u30 = rec.header.gc_and_lock.gc_mark & 1; - if (mark_bit == 0) { + if (rec.header.gc_and_lock.gc_mark & 1 == 0) { if (tag_ops.tag_finaliser_table[rec.header.tag]) |finaliser| { finaliser(@ptrCast(gc), rec.header); } - const mem: [*]u8 = @ptrCast(rec.header); - const key = free_pool_mod.FreePoolKey{ .size = rec.size, .alignment = rec.alignment }; - // Push onto the matching free pool for reuse; fall through - // to direct rawFree only if push fails (e.g. infra OOM - // during HashMap getOrPut for a never-before-seen key). - gc.free_pools.push(key, mem) catch { - gc.infra.rawFree(mem[0..rec.size], rec.alignment, @returnAddress()); - }; + // Never fails: a page free is a list push into memory the page + // already owns. + gc.cellFree(@ptrCast(rec.header), rec.size, rec.alignment); gc.stats.bytes_freed += rec.size; _ = gc.allocations.swapRemove(i); } else { rec.header.gc_and_lock.gc_mark &= ~@as(u30, 1); live_bytes += rec.size; - // D-573: count what the object OWNS, not just its record block — - // a String's bytes, a BigInt's limbs, an array's elements. The - // trigger steers on this number; an invisible payload made the - // adaptive arm read as inert on exactly the heaviest heaps. + // D-573: count what the object OWNS (a String's bytes, a BigInt's + // limbs, an array's elements), not just its record block. if (tag_ops.tag_owned_bytes_table[rec.header.tag]) |owned| { const n = owned(rec.header); live_bytes += n; @@ -192,63 +148,36 @@ pub fn sweep(gc: *GcHeap) void { std.debug.print("[gc-owned] persistent_bytes={d}\n", .{gc.persistent_bytes}); } gc.stats.sweep_count += 1; - // Deliberately EXCLUDES `gc.persistent_bytes`. The trigger has two arms - // with separate jobs: this byte arm sizes the threshold against the SWEPT - // heap (allocate ~2x what is live before collecting again), and the - // time-share arm (ADR-0183 steerFloor) absorbs the cost of everything a - // collection actually traces — including the process-lifetime waypoints, - // which grow with every closure created and are re-traced every cycle. - // Folding them in here was measured to regress gc_alloc_rate ~11%: it - // inflates the threshold from bootstrap onward, collections stop - // refilling the free pool, and the alloc path degrades to malloc - // (ADR-0164's finding). The waypoints' cost is real, and the arm that - // measures real cost is the one that owns it. + // Deliberately EXCLUDES `gc.persistent_bytes`. The byte arm sizes the + // threshold against the SWEPT heap; the time-share arm (ADR-0183 + // steerFloor) absorbs the cost of the re-traced process-lifetime + // waypoints. Folding them in was measured to regress gc_alloc_rate ~11% + // (ADR-0164's finding). gc.stats.last_live_bytes = live_bytes; } -/// Full mark-sweep collection cycle: enumerate every root via -/// `root_set.enumerate(ctx)`, mark each transitively (per-tag trace -/// dispatch handles outgoing pointers), sweep the live list -/// (per-tag finaliser → push onto free pool / direct rawFree -/// fallback), then update the adaptive threshold per ADR-0028 §1 -/// (`threshold_bytes = max(default, last_live_bytes * 2)`) and -/// reset `bytes_since_last_gc`. -/// -/// Callers invoke this directly rather than through a method on -/// `GcHeap` because the cycle traverses `root_set.zig` which itself -/// imports `gc_heap.zig` — the entry point lives here in -/// `mark_sweep.zig` to keep the import graph acyclic. +/// Full mark-sweep collection cycle: enumerate every root, mark +/// transitively, sweep, then `gc_trigger.afterSweep` (threshold +/// `max(floor, last_live * 2)`, counter reset, empty-page release). /// -/// Adaptive-threshold semantics (per ADR-0028 §1 Load-bearing -/// concern #2 disposition): after sweep updates `stats.last_live_bytes`, -/// the next-trigger threshold is recomputed so it doubles on each -/// growth cycle. Prevents the "first def triggers a full GC mid- -/// load on a 4 MiB Clojure source" failure mode. +/// The entry point lives here rather than on `GcHeap` because the cycle +/// traverses `root_set.zig`, which imports `gc_heap.zig`. pub fn collect(gc: *GcHeap, ctx: root_set_mod.WalkContext) void { - // Whole collection runs under the global heap lock (ADR-0090 §2): no - // mutator can allocate while mark+sweep run. Not reentrant — mark/sweep - // never allocate THROUGH `gc.alloc` (so no torture re-entry and no - // `gc_mutex` re-take); the gray worklist's capacity is reserved here on - // `gc.infra`, before any mark bit is set, so the mark phase itself - // performs no allocation at all. + // The whole collection runs under the global heap lock (ADR-0090 §2). + // Mark/sweep never allocate through `gc.alloc`; the gray worklist's + // capacity is reserved here before any mark bit is set. const t_entry = clock.nanoTime(io_default.get()); io_default.lockMutex(&gc.gc_mutex); defer io_default.unlockMutex(&gc.gc_mutex); - // Bit-set-at-push ⇒ each object enters the gray worklist at most once, - // so `allocations + persistent_marks` bounds it (infra-lifetime - // singletons like the interned empty list add a handful; the fallible - // append in `mark` absorbs them). A failed reserve aborts this collect - // BEFORE any bit is set — a safe degradation (heap intact; the pressure - // resurfaces as OOM at the next `gc.alloc`), unlike a mid-mark OOM. + // Bit-set-at-push bounds the worklist by `allocations + persistent_marks` + // (infra-lifetime singletons add a handful; `mark`'s fallible append + // absorbs them). A failed reserve aborts BEFORE any bit is set: heap + // intact, the pressure resurfaces as OOM at the next `gc.alloc`. gc.mark_worklist.ensureTotalCapacity( gc.infra, gc.allocations.items.len + gc.persistent_marks.items.len, ) catch return; - // D-251: clear the mark bit on process-lifetime trackHeap'd waypoints - // (Functions etc.) so each is re-traced this cycle. They are never swept - // (not in `allocations`), so sweep never clears their bit; an un-cleared - // bit short-circuits `mark()` and strands their GC children (a closure's - // `closure_bindings`) from the 2nd collect onward. + // D-251: re-trace the never-swept waypoints this cycle. gc.clearPersistentMarks(); const t_prep = clock.nanoTime(io_default.get()); phase_prep_ns +|= @intCast(@max(0, t_prep - t_entry)); @@ -257,36 +186,23 @@ pub fn collect(gc: *GcHeap, ctx: root_set_mod.WalkContext) void { mark(gc, root_header); } // #4a' in-txn-map rooting: the collecting thread's `dosync` body may hold - // live Values in `current_tx`'s gpa `vals`/`commutes` maps that are on no - // operand stack — mark them so a mid-transaction collect does not sweep - // them. (Parked WORKER threads' transactions are a follow-up — the registry - // tx-slot exposure; today's collects are quiescent/single-tx.) + // live Values only in `current_tx`'s maps. if (lock_tx.current_tx) |tx| lock_tx.markRoots(tx, @ptrCast(gc), &markTxHeader); - // ...and every registered WORKER thread's transaction (parked at a safepoint - // during STW, so its tx is quiescent). Empty registry in single-thread → no-op. + // ...and every registered WORKER thread's transaction (parked, quiescent). root_set_mod.markRegisteredTxs(@ptrCast(gc), &markWorkerTx); - // ADR-0184 B: descriptor-registry roots (rt.types / native_descriptors / - // class_descriptors method tables + meta). Only when this heap is the one - // embedded in a Runtime — bare test heaps have no registries. + // ADR-0184 B: descriptor-registry roots, only on a Runtime-embedded heap. if (gc.runtime_embedded) { const rt: *@import("../runtime.zig").Runtime = @alignCast(@fieldParentPtr("gc", gc)); @import("../type_descriptor.zig").markRegistryRoots(rt, gc); } - // D-556: conservative native-stack scan (tree_walk backend only — the vm's - // operand stack is the A1 root). Pins every stack word that decodes to a - // live heap Value, covering evaluation intermediates the explicit brackets - // miss. Runs on the COLLECTING thread's stack only. Parked workers are safe - // WITHOUT a scan because a park can no longer fire inside a fabrication - // bracket (D-559 / ADR-0150 am1 — the alloc-prologue park honors - // `fabrication_depth`), so a parked worker's builder intermediates are never - // exposed; its Values are published (eval frames / bindings / operand - // stack). Residual: tree_walk eval RUNNING ON A WORKER holds C-stack Value - // intermediates this main-only scan does not cover (the D-556 residual). + // D-556: conservative native-stack scan of the COLLECTING thread (tree_walk + // only). Parked workers need none: a park cannot fire inside a fabrication + // bracket (D-559), so their Values are published. Residual: tree_walk eval + // running ON A WORKER (the D-556 residual). const t_scan0 = clock.nanoTime(io_default.get()); phase_roots_ns +|= @intCast(@max(0, t_scan0 - t_prep)); ConservativeScan.run(gc); const t_scan1 = clock.nanoTime(io_default.get()); - // All roots are shaded gray; blacken transitively (ADR-0028 amendment 3). drainGray(gc); const t_drain = clock.nanoTime(io_default.get()); sweep(gc); @@ -298,129 +214,18 @@ pub fn collect(gc: *GcHeap, ctx: root_set_mod.WalkContext) void { if (std.mem.eql(u8, v, "3")) std.debug.print("[gc-phases] prep_ms={d} roots_ms={d} scan_ms={d} drain_ms={d} sweep_ms={d} (cum)\n", .{ phase_prep_ns / 1_000_000, phase_roots_ns / 1_000_000, phase_scan_ns / 1_000_000, phase_drain_ns / 1_000_000, phase_sweep_ns / 1_000_000 }); } - // D-573: ramp the FLOOR when a collection turns out to have been mostly - // garbage. - // - // ADR-0164 set a fixed 4 MB floor and noted, correctly, that for a churn - // workload `last_live ~ 0` makes the adaptive `last_live*2` arm inert — so - // the constant becomes the operative number. It was tuned against short - // benchmark scripts. On a long allocation-heavy program a constant is the - // wrong shape, because the cost of a collection is mostly FIXED (root scan - // + sweep of the side table), not proportional to what it reclaims. - // - // Measured on a Rush Hour BFS (cw-arcade), same 3.17 GB allocated over - // 26.2M allocations either way: - // - // floor collections wall - // 4 MB 755 17.7 s - // 64 MB 47 7.6 s - // - // 708 collections avoided, 10.1 s saved — about 13 ms of fixed cost each, - // paid to reclaim a heap that was nearly all garbage. - // - // The condition is exactly the situation ADR-0164 named: the adaptive arm is - // INERT (`last_live*2` does not reach the floor), so the constant is what - // governs — and we collected anyway. That is the case where collecting more - // often buys nothing, so double the floor, capped. Once the live set grows - // enough for `last_live*2` to exceed the floor, the adaptive arm takes over - // and the floor stops mattering, which is why the ramp does not need to - // come back down. - // - // A short script is unaffected: the ramp needs collections to happen, and - // one that never crosses 4 MB never collects. `string_ops` — ADR-0164's - // named canary, continuous string allocation — ramps, so it collects LESS - // than before, not more. - // The floor is the per-heap `threshold_floor_bytes` (default = the module - // const, overridable by CLJW_GC_THRESHOLD_MB) so the knob survives this - // recompute; the adaptive `last_live*2` arm is unchanged (growth workloads). - gc.threshold_bytes = @max( - gc.threshold_floor_bytes, - gc.stats.last_live_bytes *| 2, - ); - gc.bytes_since_last_gc = 0; - gc.stats.collect_count += 1; + trigger.afterSweep(gc); } -/// D-556: conservatively scan the collecting thread's native stack for live -/// heap Values (tree_walk backend). Range: from the collector's own frame -/// (deep) up to `root_set.conservative_stack_top` (the shallowest tree_walk -/// eval entry; 0 = tree_walk never ran on this thread → skip). Each 8-byte -/// word is decoded via the `Value.heapHeader()` membrane (tag check only, no -/// deref) and pinned IFF the pointer matches a live allocation record — a -/// stale/garbage word that happens to look like a heap tag can only pin a -/// still-live object (false positive = one delayed free), never deref a -/// dangling pointer. The live-pointer set is built in one pass over the -/// `allocations` side-table using `gc.infra` (never `gc.alloc` — collect must -/// not re-enter the heap). -/// -/// Compiled only under `-Dbackend=tree_walk`; vm builds get the no-op branch. -const ConservativeScan = if (build_options.backend == .tree_walk) struct { - /// libc setjmp — used ONLY as a register spill (Boehm GC's `GC_push_regs` - /// idiom): it writes every callee-saved register into the buffer, so a heap - /// Value living ONLY in a callee-saved register (never spilled on the paths - /// the collector happened to take) becomes scannable. We never longjmp. - extern fn setjmp(env: *anyopaque) c_int; - - /// Decode one candidate word and pin it if it is a live heap Value. - inline fn pinIfLiveWord(gc: *GcHeap, live: *const std.AutoHashMapUnmanaged(usize, void), word: u64) void { - const Value = @import("../value/value.zig").Value; - const nb = @import("../value/nan_box.zig"); - // Pre-filter a zero address: `heapHeader`'s decodePtr would cast it to a - // null pointer (a ReleaseSafe panic) — a heap-TAGGED word with address - // bits 0 is garbage, never a real Value. - if (word & nb.NB_ADDR_SHIFTED_MASK == 0) return; - const v: Value = @enumFromInt(word); - const hdr = v.heapHeader() orelse return; - if (live.contains(@intFromPtr(hdr))) mark(gc, hdr); - } - - fn run(gc: *GcHeap) void { - const top = root_set_mod.conservative_stack_top; - if (top == 0) return; - const bottom = @frameAddress(); - if (bottom >= top) return; - var live: std.AutoHashMapUnmanaged(usize, void) = .empty; - defer live.deinit(gc.infra); - live.ensureTotalCapacity(gc.infra, @intCast(gc.allocations.items.len)) catch return; - for (gc.allocations.items) |rec| live.putAssumeCapacity(@intFromPtr(rec.header), {}); - // Registers first (see `setjmp` above): 128 u64s comfortably hold any - // ABI's jmp_buf; unwritten tail words stay zero and are filtered. - var jb: [128]u64 align(16) = [_]u64{0} ** 128; - _ = setjmp(@ptrCast(&jb)); - for (jb) |word| pinIfLiveWord(gc, &live, word); - // Then the native stack from the collector's frame up to the anchor. - var addr = std.mem.alignForward(usize, bottom, @alignOf(u64)); - while (addr + @sizeOf(u64) <= top) : (addr += @sizeOf(u64)) { - pinIfLiveWord(gc, &live, @as(*const u64, @ptrFromInt(addr)).*); - } - } -} else struct { - fn run(_: *GcHeap) void { - // vm backend: precise root_set only; no conservative native-stack scan. - } -}; - /// Stop-the-world collect (ADR-0090 Alt B / D-244 #4): pause every other -/// registered worker at a safe point (its `alloc`-prologue park or the VM -/// back-edge poll), run a full `collect` over the union root set (every parked -/// worker's published roots are quiescent), then resume the workers. This is -/// the entry point a real-threading collector calls instead of bare `collect`. +/// registered worker at a safe point, run `collect` over the union root set, +/// resume them, then steer the floor by the measured GC share (D-573). /// -/// The caller MUST NOT hold `gc_mutex`: `collect` re-takes it (`Io.Mutex` is -/// not reentrant), so a holder would self-deadlock. At the VM safe point that -/// triggers a collection, `gc_mutex` is free — `alloc` releases it before the -/// trigger runs (#4 wire-up). `self_registered` excludes the calling thread -/// from the park target (true when the collector is itself a registered worker -/// that crossed the threshold, false for the main / unregistered collector). -/// -/// With no other registered workers (single-threaded, or every peer already at -/// a safe point) `stopWorld` returns immediately and this is just `collect` -/// with a no-op fence — so it is correct (if heavier) even single-threaded. +/// The caller MUST NOT hold `gc_mutex` (`collect` re-takes it; `Io.Mutex` is +/// not reentrant). `self_registered` excludes the calling thread from the +/// park target. With no other registered workers this is `collect` with a +/// no-op fence. pub fn collectStopTheWorld(gc: *GcHeap, ctx: root_set_mod.WalkContext, self_registered: bool) void { - // D-573: time the collection so the floor can be steered by the MEASURED - // GC share rather than by a byte constant. The process-wide io clock — - // every collect site can reach it without an io parameter, and `.awake` is - // monotonic, which is what a duration needs. const started = clock.nanoTime(io_default.get()); safepoint.stopWorld(self_registered); collect(gc, ctx); @@ -428,358 +233,5 @@ pub fn collectStopTheWorld(gc: *GcHeap, ctx: root_set_mod.WalkContext, self_regi const ended = clock.nanoTime(io_default.get()); gc.stats.gc_ns_total +|= @intCast(@max(0, ended - started)); if (gc.stats.first_collect_ns == 0) gc.stats.first_collect_ns = started; - steerFloor(gc, ended); -} - -/// Move `threshold_floor_bytes` toward whatever keeps the measured GC share at -/// `gc_time_target_percent`. -/// -/// The share is `gc_ns_total / (now - first_collect_ns)` — wall clock since the -/// program first had to collect, which is the window the floor can influence. -/// Over target, the collector is running too often for what it costs to mark -/// this program's live set: double the floor, so twice as much garbage -/// accumulates per collection and the share roughly halves. Under target with -/// room to spare, ease back down, so a program whose live set shrank stops -/// holding a floor it no longer needs. -/// -/// Doubling/halving rather than solving for a target: the share is a lagging -/// average, so a single-step solve would oscillate. It converges in a handful -/// of collections either way, which is fast against the hundreds a program has -/// to make before the floor matters at all. -fn steerFloor(gc: *GcHeap, now_ns: i64) void { - const elapsed = now_ns - gc.stats.first_collect_ns; - // Below this there is no meaningful ratio yet — a first collection is 100 % - // of the time since the first collection, which says nothing. - if (elapsed < 50 * std.time.ns_per_ms) return; - const share_pct = gc.stats.gc_ns_total * 100 / @as(u64, @intCast(elapsed)); - if (share_pct > gc_time_target_percent) { - if (gc.threshold_floor_bytes < max_threshold_floor_bytes) { - gc.threshold_floor_bytes = @min( - max_threshold_floor_bytes, - gc.threshold_floor_bytes *| 2, - ); - } - } else if (share_pct * 4 < gc_time_target_percent and - gc.threshold_floor_bytes > gc.base_threshold_floor_bytes) - { - gc.threshold_floor_bytes = @max( - gc.base_threshold_floor_bytes, - gc.threshold_floor_bytes / 2, - ); - } -} - -// --- tests --- - -const Cell = extern struct { header: HeapHeader, payload: u64 = 0 }; - -test "mark sets gc_mark bit 0 on a leaf header" { - var gc = GcHeap.init(testing.allocator); - defer gc.deinit(); - - const c = try gc.alloc(Cell); - c.* = .{ .header = HeapHeader.init(.symbol) }; - try testing.expectEqual(@as(u30, 0), c.header.gc_and_lock.gc_mark); - - mark(&gc, &c.header); - try testing.expectEqual(@as(u30, 1), c.header.gc_and_lock.gc_mark & 1); -} - -test "mark cycle invariant: double-mark is idempotent" { - var gc = GcHeap.init(testing.allocator); - defer gc.deinit(); - - const c = try gc.alloc(Cell); - c.* = .{ .header = HeapHeader.init(.list) }; - - mark(&gc, &c.header); - const after_first = c.header.gc_and_lock.gc_mark; - mark(&gc, &c.header); - const after_second = c.header.gc_and_lock.gc_mark; - try testing.expectEqual(after_first, after_second); -} - -test "clearMarks resets bit 0 on every live allocation" { - var gc = GcHeap.init(testing.allocator); - defer gc.deinit(); - - const a = try gc.alloc(Cell); - a.* = .{ .header = HeapHeader.init(.symbol) }; - const b = try gc.alloc(Cell); - b.* = .{ .header = HeapHeader.init(.vector) }; - - mark(&gc, &a.header); - mark(&gc, &b.header); - try testing.expect(a.header.gc_and_lock.gc_mark & 1 == 1); - try testing.expect(b.header.gc_and_lock.gc_mark & 1 == 1); - - clearMarks(&gc); - try testing.expectEqual(@as(u30, 0), a.header.gc_and_lock.gc_mark & 1); - try testing.expectEqual(@as(u30, 0), b.header.gc_and_lock.gc_mark & 1); -} - -test "clearMarks preserves bits 1..29 (only bit 0 is the mark)" { - // Bits 1..29 are reserved per ADR-0028 §6 for tri-colour upgrade + - // size_class (5.3 owner picks split). clearMarks must not touch - // them; 5.3.b.2 only resets bit 0. - var gc = GcHeap.init(testing.allocator); - defer gc.deinit(); - - const c = try gc.alloc(Cell); - c.* = .{ .header = HeapHeader.init(.symbol) }; - c.header.gc_and_lock.gc_mark = 0b101010101; // arbitrary non-bit-0 pattern + bit 0 - - clearMarks(&gc); - try testing.expectEqual(@as(u30, 0b101010100), c.header.gc_and_lock.gc_mark); -} - -test "sweep removes unmarked allocations + frees their backing memory" { - var gc = GcHeap.init(testing.allocator); - defer gc.deinit(); - - const a = try gc.alloc(Cell); - a.* = .{ .header = HeapHeader.init(.symbol) }; - const b = try gc.alloc(Cell); - b.* = .{ .header = HeapHeader.init(.vector) }; - const c = try gc.alloc(Cell); - c.* = .{ .header = HeapHeader.init(.list) }; - - // Mark only `a` and `c` as reachable; `b` is dead. - mark(&gc, &a.header); - mark(&gc, &c.header); - - sweep(&gc); - - try testing.expectEqual(@as(usize, 2), gc.allocations.items.len); - try testing.expectEqual(@as(u64, 1), gc.stats.sweep_count); - try testing.expectEqual(@as(usize, @sizeOf(Cell)), gc.stats.bytes_freed); - try testing.expectEqual(@as(usize, 2 * @sizeOf(Cell)), gc.stats.last_live_bytes); - - // Survivors retain their tag and have mark bit reset to 0 (ready - // for the next mark phase per the sweep contract). - for (gc.allocations.items) |rec| { - try testing.expectEqual(@as(u30, 0), rec.header.gc_and_lock.gc_mark & 1); - } -} - -test "sweep on empty heap is a no-op (bumps sweep_count, no panics)" { - var gc = GcHeap.init(testing.allocator); - defer gc.deinit(); - - sweep(&gc); - try testing.expectEqual(@as(u64, 1), gc.stats.sweep_count); - try testing.expectEqual(@as(usize, 0), gc.stats.last_live_bytes); -} - -test "collect: pinned roots survive; unpinned allocations get swept" { - var gc = GcHeap.init(testing.allocator); - defer gc.deinit(); - - const cell_kept = try gc.alloc(Cell); - cell_kept.* = .{ .header = HeapHeader.init(.symbol) }; - const cell_dropped = try gc.alloc(Cell); - cell_dropped.* = .{ .header = HeapHeader.init(.vector) }; - - const value_mod_test = @import("../value/value.zig"); - try gc.pin(value_mod_test.Value.encodeHeapPtr(.string, cell_kept)); - - const before_alloc_count = gc.allocations.items.len; - try testing.expectEqual(@as(usize, 2), before_alloc_count); - - collect(&gc, .{ .envs = &.{}, .gc = &gc }); - try testing.expectEqual(@as(usize, 1), gc.allocations.items.len); - try testing.expectEqual(@as(u64, 1), gc.stats.collect_count); - try testing.expectEqual(@as(usize, @sizeOf(Cell)), gc.stats.last_live_bytes); - try testing.expectEqual(@as(usize, 0), gc.bytes_since_last_gc); - try testing.expectEqual(@as(*HeapHeader, @ptrCast(cell_kept)), gc.allocations.items[0].header); -} - -test "collect: in-transaction vals values survive (#4a' self-tx rooting)" { - var gc = GcHeap.init(testing.allocator); - defer gc.deinit(); - const value_mod_test = @import("../value/value.zig"); - const lock_tx_mod = @import("../concurrency/lock_tx.zig"); - const Ref = @import("../stm/ref.zig").Ref; - - // A heap value that lives ONLY in the transaction's vals cache — on no - // operand stack, not pinned. Without the #4a' tx-rooting it would be swept. - const in_tx_cell = try gc.alloc(Cell); - in_tx_cell.* = .{ .header = HeapHeader.init(.vector) }; - const in_tx_val = value_mod_test.Value.encodeHeapPtr(.vector, in_tx_cell); - - var tx: lock_tx_mod.LockingTransaction = .{ .read_point = 0, .gpa = testing.allocator }; - defer tx.vals.deinit(testing.allocator); - // A fake (never-dereferenced) Ref key — markRoots reads only the VALUES. - const fake_ref: *Ref = @ptrFromInt(@alignOf(Ref) * 4096); - try tx.vals.put(testing.allocator, fake_ref, in_tx_val); - - lock_tx_mod.current_tx = &tx; - defer lock_tx_mod.current_tx = null; - - collect(&gc, .{ .envs = &.{}, .gc = &gc }); - - var found = false; - for (gc.allocations.items) |a| { - if (a.header == @as(*HeapHeader, @ptrCast(in_tx_cell))) found = true; - } - try testing.expect(found); // survived ONLY via the in-transaction rooting -} - -test "collect: a registered worker's transaction roots survive (#4a' worker-tx)" { - var gc = GcHeap.init(testing.allocator); - defer gc.deinit(); - const value_mod_test = @import("../value/value.zig"); - const lock_tx_mod = @import("../concurrency/lock_tx.zig"); - const root_set_test = @import("root_set.zig"); - const env_mod_test = @import("../env.zig"); - const Ref = @import("../stm/ref.zig").Ref; - - const in_tx_cell = try gc.alloc(Cell); - in_tx_cell.* = .{ .header = HeapHeader.init(.vector) }; - const in_tx_val = value_mod_test.Value.encodeHeapPtr(.vector, in_tx_cell); - - var worker_tx: lock_tx_mod.LockingTransaction = .{ .read_point = 0, .gpa = testing.allocator }; - defer worker_tx.vals.deinit(testing.allocator); - const fake_ref: *Ref = @ptrFromInt(@alignOf(Ref) * 4096); - try worker_tx.vals.put(testing.allocator, fake_ref, in_tx_val); - - // A registered worker whose current_tx is worker_tx (the MAIN thread's - // current_tx stays null, so only the worker-tx pass can root in_tx_cell). - var worker_current_tx: ?*lock_tx_mod.LockingTransaction = &worker_tx; - var ctx: root_set_test.ThreadGcContext = .{ - .frame_slot = &env_mod_test.current_frame, - .analysis_frame_slot = &root_set_test.analysis_frame_head, - .eval_frame_slot = &root_set_test.eval_frame_head, - .self_guard_slot = &root_set_test.gc_self_guard, - .tx_slot = @ptrCast(&worker_current_tx), - }; - try root_set_test.registerThread(&ctx); - defer root_set_test.unregisterThread(&ctx); - - collect(&gc, .{ .envs = &.{}, .gc = &gc }); - - var found = false; - for (gc.allocations.items) |a| { - if (a.header == @as(*HeapHeader, @ptrCast(in_tx_cell))) found = true; - } - try testing.expect(found); // survived via the registered worker's tx rooting -} - -test "collect: adaptive threshold = max(default, last_live_bytes * 2)" { - var gc = GcHeap.init(testing.allocator); - defer gc.deinit(); - - // No live allocations → last_live_bytes = 0 → threshold stays at default. - collect(&gc, .{ .envs = &.{}, .gc = &gc }); - try testing.expectEqual(gc_heap_mod.default_gc_threshold_bytes, gc.threshold_bytes); - - // Pin a heap Value so it survives the cycle. last_live_bytes = sizeOf(Cell); - // since 2 * sizeOf(Cell) is far below default, threshold still stays at default. - const cell = try gc.alloc(Cell); - cell.* = .{ .header = HeapHeader.init(.list) }; - const value_mod_test = @import("../value/value.zig"); - try gc.pin(value_mod_test.Value.encodeHeapPtr(.list, cell)); - - collect(&gc, .{ .envs = &.{}, .gc = &gc }); - try testing.expectEqual(@as(usize, @sizeOf(Cell)), gc.stats.last_live_bytes); - try testing.expectEqual(gc_heap_mod.default_gc_threshold_bytes, gc.threshold_bytes); -} - -test "sweep iterating backward keeps swap-remove indices valid" { - // Allocate 5 cells; mark only the middle one (index 2). Sweep - // should leave exactly that one allocation in place. - var gc = GcHeap.init(testing.allocator); - defer gc.deinit(); - - var survivor_addr: *HeapHeader = undefined; - for (0..5) |idx| { - const cell = try gc.alloc(Cell); - cell.* = .{ .header = HeapHeader.init(.symbol) }; - if (idx == 2) { - mark(&gc, &cell.header); - survivor_addr = &cell.header; - } - } - - sweep(&gc); - try testing.expectEqual(@as(usize, 1), gc.allocations.items.len); - try testing.expectEqual(survivor_addr, gc.allocations.items[0].header); -} - -test "mark bounds native stack on a deep cons chain (explicit worklist)" { - // A ~1M-deep cons chain: per-child recursive descent would need ~1M - // native frames and SIGSEGVs (the `(count (repeat 1000000 1))` crash); - // the explicit worklist keeps mark's stack depth O(1). - var gc = GcHeap.init(testing.allocator); - defer gc.deinit(); - const list_mod = @import("../collection/list.zig"); - const value_mod = @import("../value/value.zig"); - const Value = value_mod.Value; - list_mod.registerGcHooks(); - - var tail: Value = .nil_val; - var i: u32 = 0; - while (i < 1_000_000) : (i += 1) { - const cell = try gc.alloc(list_mod.Cons); - cell.* = .{ - .header = HeapHeader.init(.list), - .first = .nil_val, - .rest = tail, - .meta = .nil_val, - .count = i + 1, - }; - tail = Value.encodeHeapPtr(.list, cell); - } - - mark(&gc, tail.heapHeader().?); - drainGray(&gc); - // The deep end (first-allocated cell) must have been reached. - try testing.expectEqual(@as(u30, 1), gc.allocations.items[0].header.gc_and_lock.gc_mark & 1); -} - -test "sweep counts finaliser-owned bytes via the per-tag hook (D-573)" { - // Register the real string hook, as Runtime.init does. - const string_mod = @import("../collection/string.zig"); - string_mod.registerGcHooks(); - - var gc = GcHeap.init(testing.allocator); - defer gc.deinit(); - - const payload_len = 1000; - const payload = try gc.infra.alloc(u8, payload_len); - const s = try gc.alloc(string_mod.String); - s.* = .{ - .header = HeapHeader.init(.string), - .bytes_ptr = payload.ptr, - .bytes_len = payload_len, - }; - - mark(&gc, &s.header); - sweep(&gc); - - // The record block alone was the OLD number — the 4.6x under-report. The - // honest number includes the byte payload the finaliser would free. - try testing.expectEqual(@sizeOf(string_mod.String) + payload_len, gc.stats.last_live_bytes); -} - -test "persistent bytes are measured but NOT folded into last_live (D-573)" { - var gc = GcHeap.init(testing.allocator); - defer gc.deinit(); - - var waypoint: extern struct { header: HeapHeader = HeapHeader.init(.symbol), payload: u64 = 0 } = .{}; - try gc.registerPersistentMark(@ptrCast(&waypoint), 4096); - try testing.expectEqual(@as(usize, 4096), gc.persistent_bytes); - - const a = try gc.alloc(Cell); - a.* = .{ .header = HeapHeader.init(.symbol) }; - mark(&gc, &a.header); - sweep(&gc); - - // The byte arm sizes against the SWEPT heap only; the waypoints' cost - // belongs to the time-share arm (see sweep()'s comment — folding them in - // was measured to starve the free pool and regress gc_alloc_rate ~11%). - try testing.expectEqual(@sizeOf(Cell), gc.stats.last_live_bytes); - - gc.unregisterLastPersistentMark(); - try testing.expectEqual(@as(usize, 0), gc.persistent_bytes); + trigger.steerFloor(gc, ended); } diff --git a/src/runtime/gc/mark_sweep_test.zig b/src/runtime/gc/mark_sweep_test.zig new file mode 100644 index 000000000..ccc2f2094 --- /dev/null +++ b/src/runtime/gc/mark_sweep_test.zig @@ -0,0 +1,377 @@ +// SPDX-License-Identifier: EPL-2.0 +//! Unit tests for `mark_sweep.zig` and the collect-side of `gc_trigger.zig`: +//! mark bits, sweep, collect rooting (pins, transactions), the adaptive +//! threshold, page release, payload charging, and deep-graph marking. + +const std = @import("std"); +const testing = std.testing; + +const gc_heap_mod = @import("gc_heap.zig"); +const page_heap_mod = @import("page_heap.zig"); +const heap_header = @import("../value/heap_header.zig"); +const mark_sweep = @import("mark_sweep.zig"); + +const GcHeap = gc_heap_mod.GcHeap; +const HeapHeader = heap_header.HeapHeader; +const mark = mark_sweep.mark; +const drainGray = mark_sweep.drainGray; +const clearMarks = mark_sweep.clearMarks; +const sweep = mark_sweep.sweep; +const collect = mark_sweep.collect; + +const Cell = extern struct { header: HeapHeader, payload: u64 = 0 }; + +test "mark sets gc_mark bit 0 on a leaf header" { + var gc = GcHeap.init(testing.allocator); + defer gc.deinit(); + + const c = try gc.alloc(Cell); + c.* = .{ .header = HeapHeader.init(.symbol) }; + try testing.expectEqual(@as(u30, 0), c.header.gc_and_lock.gc_mark); + + mark(&gc, &c.header); + try testing.expectEqual(@as(u30, 1), c.header.gc_and_lock.gc_mark & 1); +} + +test "mark cycle invariant: double-mark is idempotent" { + var gc = GcHeap.init(testing.allocator); + defer gc.deinit(); + + const c = try gc.alloc(Cell); + c.* = .{ .header = HeapHeader.init(.list) }; + + mark(&gc, &c.header); + const after_first = c.header.gc_and_lock.gc_mark; + mark(&gc, &c.header); + const after_second = c.header.gc_and_lock.gc_mark; + try testing.expectEqual(after_first, after_second); +} + +test "clearMarks resets bit 0 on every live allocation" { + var gc = GcHeap.init(testing.allocator); + defer gc.deinit(); + + const a = try gc.alloc(Cell); + a.* = .{ .header = HeapHeader.init(.symbol) }; + const b = try gc.alloc(Cell); + b.* = .{ .header = HeapHeader.init(.vector) }; + + mark(&gc, &a.header); + mark(&gc, &b.header); + try testing.expect(a.header.gc_and_lock.gc_mark & 1 == 1); + try testing.expect(b.header.gc_and_lock.gc_mark & 1 == 1); + + clearMarks(&gc); + try testing.expectEqual(@as(u30, 0), a.header.gc_and_lock.gc_mark & 1); + try testing.expectEqual(@as(u30, 0), b.header.gc_and_lock.gc_mark & 1); +} + +test "clearMarks preserves bits 1..29 (only bit 0 is the mark)" { + // Bits 1..29 are reserved per ADR-0028 §6 for tri-colour upgrade + + // size_class (5.3 owner picks split). clearMarks must not touch + // them; 5.3.b.2 only resets bit 0. + var gc = GcHeap.init(testing.allocator); + defer gc.deinit(); + + const c = try gc.alloc(Cell); + c.* = .{ .header = HeapHeader.init(.symbol) }; + c.header.gc_and_lock.gc_mark = 0b101010101; // arbitrary non-bit-0 pattern + bit 0 + + clearMarks(&gc); + try testing.expectEqual(@as(u30, 0b101010100), c.header.gc_and_lock.gc_mark); +} + +test "sweep removes unmarked allocations + frees their backing memory" { + var gc = GcHeap.init(testing.allocator); + defer gc.deinit(); + + const a = try gc.alloc(Cell); + a.* = .{ .header = HeapHeader.init(.symbol) }; + const b = try gc.alloc(Cell); + b.* = .{ .header = HeapHeader.init(.vector) }; + const c = try gc.alloc(Cell); + c.* = .{ .header = HeapHeader.init(.list) }; + + // Mark only `a` and `c` as reachable; `b` is dead. + mark(&gc, &a.header); + mark(&gc, &c.header); + + sweep(&gc); + + try testing.expectEqual(@as(usize, 2), gc.allocations.items.len); + try testing.expectEqual(@as(u64, 1), gc.stats.sweep_count); + try testing.expectEqual(@as(usize, @sizeOf(Cell)), gc.stats.bytes_freed); + try testing.expectEqual(@as(usize, 2 * @sizeOf(Cell)), gc.stats.last_live_bytes); + + // Survivors retain their tag and have mark bit reset to 0 (ready + // for the next mark phase per the sweep contract). + for (gc.allocations.items) |rec| { + try testing.expectEqual(@as(u30, 0), rec.header.gc_and_lock.gc_mark & 1); + } +} + +test "sweep on empty heap is a no-op (bumps sweep_count, no panics)" { + var gc = GcHeap.init(testing.allocator); + defer gc.deinit(); + + sweep(&gc); + try testing.expectEqual(@as(u64, 1), gc.stats.sweep_count); + try testing.expectEqual(@as(usize, 0), gc.stats.last_live_bytes); +} + +test "collect: pinned roots survive; unpinned allocations get swept" { + var gc = GcHeap.init(testing.allocator); + defer gc.deinit(); + + const cell_kept = try gc.alloc(Cell); + cell_kept.* = .{ .header = HeapHeader.init(.symbol) }; + const cell_dropped = try gc.alloc(Cell); + cell_dropped.* = .{ .header = HeapHeader.init(.vector) }; + + const value_mod_test = @import("../value/value.zig"); + try gc.pin(value_mod_test.Value.encodeHeapPtr(.string, cell_kept)); + + const before_alloc_count = gc.allocations.items.len; + try testing.expectEqual(@as(usize, 2), before_alloc_count); + + collect(&gc, .{ .envs = &.{}, .gc = &gc }); + try testing.expectEqual(@as(usize, 1), gc.allocations.items.len); + try testing.expectEqual(@as(u64, 1), gc.stats.collect_count); + try testing.expectEqual(@as(usize, @sizeOf(Cell)), gc.stats.last_live_bytes); + try testing.expectEqual(@as(usize, 0), gc.bytes_since_last_gc); + try testing.expectEqual(@as(*HeapHeader, @ptrCast(cell_kept)), gc.allocations.items[0].header); +} + +test "collect: in-transaction vals values survive (#4a' self-tx rooting)" { + var gc = GcHeap.init(testing.allocator); + defer gc.deinit(); + const value_mod_test = @import("../value/value.zig"); + const lock_tx_mod = @import("../concurrency/lock_tx.zig"); + const Ref = @import("../stm/ref.zig").Ref; + + // A heap value that lives ONLY in the transaction's vals cache: on no + // operand stack, not pinned. Without the #4a' tx-rooting it would be swept. + const in_tx_cell = try gc.alloc(Cell); + in_tx_cell.* = .{ .header = HeapHeader.init(.vector) }; + const in_tx_val = value_mod_test.Value.encodeHeapPtr(.vector, in_tx_cell); + + var tx: lock_tx_mod.LockingTransaction = .{ .read_point = 0, .gpa = testing.allocator }; + defer tx.vals.deinit(testing.allocator); + // A fake (never-dereferenced) Ref key: markRoots reads only the VALUES. + const fake_ref: *Ref = @ptrFromInt(@alignOf(Ref) * 4096); + try tx.vals.put(testing.allocator, fake_ref, in_tx_val); + + lock_tx_mod.current_tx = &tx; + defer lock_tx_mod.current_tx = null; + + collect(&gc, .{ .envs = &.{}, .gc = &gc }); + + var found = false; + for (gc.allocations.items) |a| { + if (a.header == @as(*HeapHeader, @ptrCast(in_tx_cell))) found = true; + } + try testing.expect(found); // survived ONLY via the in-transaction rooting +} + +test "collect: a registered worker's transaction roots survive (#4a' worker-tx)" { + var gc = GcHeap.init(testing.allocator); + defer gc.deinit(); + const value_mod_test = @import("../value/value.zig"); + const lock_tx_mod = @import("../concurrency/lock_tx.zig"); + const root_set_test = @import("root_set.zig"); + const env_mod_test = @import("../env.zig"); + const Ref = @import("../stm/ref.zig").Ref; + + const in_tx_cell = try gc.alloc(Cell); + in_tx_cell.* = .{ .header = HeapHeader.init(.vector) }; + const in_tx_val = value_mod_test.Value.encodeHeapPtr(.vector, in_tx_cell); + + var worker_tx: lock_tx_mod.LockingTransaction = .{ .read_point = 0, .gpa = testing.allocator }; + defer worker_tx.vals.deinit(testing.allocator); + const fake_ref: *Ref = @ptrFromInt(@alignOf(Ref) * 4096); + try worker_tx.vals.put(testing.allocator, fake_ref, in_tx_val); + + // A registered worker whose current_tx is worker_tx (the MAIN thread's + // current_tx stays null, so only the worker-tx pass can root in_tx_cell). + var worker_current_tx: ?*lock_tx_mod.LockingTransaction = &worker_tx; + var ctx: root_set_test.ThreadGcContext = .{ + .frame_slot = &env_mod_test.current_frame, + .analysis_frame_slot = &root_set_test.analysis_frame_head, + .eval_frame_slot = &root_set_test.eval_frame_head, + .self_guard_slot = &root_set_test.gc_self_guard, + .tx_slot = @ptrCast(&worker_current_tx), + }; + try root_set_test.registerThread(&ctx); + defer root_set_test.unregisterThread(&ctx); + + collect(&gc, .{ .envs = &.{}, .gc = &gc }); + + var found = false; + for (gc.allocations.items) |a| { + if (a.header == @as(*HeapHeader, @ptrCast(in_tx_cell))) found = true; + } + try testing.expect(found); // survived via the registered worker's tx rooting +} + +test "collect: adaptive threshold = max(default, last_live_bytes * 2)" { + var gc = GcHeap.init(testing.allocator); + defer gc.deinit(); + + // No live allocations → last_live_bytes = 0 → threshold stays at default. + collect(&gc, .{ .envs = &.{}, .gc = &gc }); + try testing.expectEqual(gc_heap_mod.default_gc_threshold_bytes, gc.threshold_bytes); + + // Pin a heap Value so it survives the cycle. last_live_bytes = sizeOf(Cell); + // since 2 * sizeOf(Cell) is far below default, threshold still stays at default. + const cell = try gc.alloc(Cell); + cell.* = .{ .header = HeapHeader.init(.list) }; + const value_mod_test = @import("../value/value.zig"); + try gc.pin(value_mod_test.Value.encodeHeapPtr(.list, cell)); + + collect(&gc, .{ .envs = &.{}, .gc = &gc }); + try testing.expectEqual(@as(usize, @sizeOf(Cell)), gc.stats.last_live_bytes); + try testing.expectEqual(gc_heap_mod.default_gc_threshold_bytes, gc.threshold_bytes); +} + +test "collect: dead cells' pages go back to the OS past the retention budget (CLJW-GC-HEAP-RETURN)" { + var gc = GcHeap.init(testing.allocator); + defer gc.deinit(); + // 200k unrooted 16-byte cells: ~49 pages, past the 16-page minimum budget. + var i: usize = 0; + while (i < 200_000) : (i += 1) { + const c = try gc.alloc(Cell); + c.* = .{ .header = HeapHeader.init(.symbol) }; + } + const mapped = gc.pages.page_count; + try testing.expect(mapped * page_heap_mod.page_bytes > gc_heap_mod.min_retained_bytes); + try testing.expect(gc.committedBytes() >= 200_000 * @sizeOf(Cell)); + + // An automatic collect keeps one threshold's worth of empty pages warm. + collect(&gc, .{ .envs = &.{}, .gc = &gc }); + try testing.expectEqual(@as(usize, 0), gc.allocations.items.len); + try testing.expect(gc.pages.mappedBytes() <= @max(gc.threshold_bytes, gc_heap_mod.min_retained_bytes)); + + // A requested collect gives back all but the minimum. + gc.release_requested = true; + collect(&gc, .{ .envs = &.{}, .gc = &gc }); + try testing.expect(!gc.release_requested); + try testing.expect(gc.pages.mappedBytes() <= gc_heap_mod.min_retained_bytes); + try testing.expect(gc.pages.pages_released_total >= mapped - gc_heap_mod.min_retained_bytes / page_heap_mod.page_bytes); + + // The retained pages serve the next allocations without mapping more. + const kept = gc.pages.page_count; + const c = try gc.alloc(Cell); + c.* = .{ .header = HeapHeader.init(.symbol) }; + try testing.expectEqual(kept, gc.pages.page_count); +} + +test "collect trigger counts charged payload bytes, and collect resets them" { + var gc = GcHeap.init(testing.allocator); + defer gc.deinit(); + try testing.expectEqual(@as(usize, 0), gc.pressureBytes()); + gc.noteOwnedAlloc(gc.threshold_bytes + 1); + try testing.expect(gc.pressureBytes() > gc.threshold_bytes); + try gc.chargeInfraBytes(8); + try testing.expectEqual(gc.threshold_bytes + 9, gc.pressureBytes()); + collect(&gc, .{ .envs = &.{}, .gc = &gc }); + try testing.expectEqual(@as(usize, 0), gc.pressureBytes()); +} + +test "sweep iterating backward keeps swap-remove indices valid" { + // Allocate 5 cells; mark only the middle one (index 2). Sweep + // should leave exactly that one allocation in place. + var gc = GcHeap.init(testing.allocator); + defer gc.deinit(); + + var survivor_addr: *HeapHeader = undefined; + for (0..5) |idx| { + const cell = try gc.alloc(Cell); + cell.* = .{ .header = HeapHeader.init(.symbol) }; + if (idx == 2) { + mark(&gc, &cell.header); + survivor_addr = &cell.header; + } + } + + sweep(&gc); + try testing.expectEqual(@as(usize, 1), gc.allocations.items.len); + try testing.expectEqual(survivor_addr, gc.allocations.items[0].header); +} + +test "mark bounds native stack on a deep cons chain (explicit worklist)" { + // A ~1M-deep cons chain: per-child recursive descent would need ~1M + // native frames and SIGSEGVs (the `(count (repeat 1000000 1))` crash); + // the explicit worklist keeps mark's stack depth O(1). + var gc = GcHeap.init(testing.allocator); + defer gc.deinit(); + const list_mod = @import("../collection/list.zig"); + const value_mod = @import("../value/value.zig"); + const Value = value_mod.Value; + list_mod.registerGcHooks(); + + var tail: Value = .nil_val; + var i: u32 = 0; + while (i < 1_000_000) : (i += 1) { + const cell = try gc.alloc(list_mod.Cons); + cell.* = .{ + .header = HeapHeader.init(.list), + .first = .nil_val, + .rest = tail, + .meta = .nil_val, + .count = i + 1, + }; + tail = Value.encodeHeapPtr(.list, cell); + } + + mark(&gc, tail.heapHeader().?); + drainGray(&gc); + // The deep end (first-allocated cell) must have been reached. + try testing.expectEqual(@as(u30, 1), gc.allocations.items[0].header.gc_and_lock.gc_mark & 1); +} + +test "sweep counts finaliser-owned bytes via the per-tag hook (D-573)" { + // Register the real string hook, as Runtime.init does. + const string_mod = @import("../collection/string.zig"); + string_mod.registerGcHooks(); + + var gc = GcHeap.init(testing.allocator); + defer gc.deinit(); + + const payload_len = 1000; + const payload = try gc.infra.alloc(u8, payload_len); + const s = try gc.alloc(string_mod.String); + s.* = .{ + .header = HeapHeader.init(.string), + .bytes_ptr = payload.ptr, + .bytes_len = payload_len, + }; + + mark(&gc, &s.header); + sweep(&gc); + + // The record block alone was the OLD number: the 4.6x under-report. The + // honest number includes the byte payload the finaliser would free. + try testing.expectEqual(@sizeOf(string_mod.String) + payload_len, gc.stats.last_live_bytes); +} + +test "persistent bytes are measured but NOT folded into last_live (D-573)" { + var gc = GcHeap.init(testing.allocator); + defer gc.deinit(); + + var waypoint: extern struct { header: HeapHeader = HeapHeader.init(.symbol), payload: u64 = 0 } = .{}; + try gc.registerPersistentMark(@ptrCast(&waypoint), 4096); + try testing.expectEqual(@as(usize, 4096), gc.persistent_bytes); + + const a = try gc.alloc(Cell); + a.* = .{ .header = HeapHeader.init(.symbol) }; + mark(&gc, &a.header); + sweep(&gc); + + // The byte arm sizes against the SWEPT heap only; the waypoints' cost + // belongs to the time-share arm (see sweep()'s comment: folding them in + // was measured to starve the free pool and regress gc_alloc_rate ~11%). + try testing.expectEqual(@sizeOf(Cell), gc.stats.last_live_bytes); + + gc.unregisterLastPersistentMark(); + try testing.expectEqual(@as(usize, 0), gc.persistent_bytes); +} diff --git a/src/runtime/gc/page_heap.zig b/src/runtime/gc/page_heap.zig new file mode 100644 index 000000000..f4ffc96ef --- /dev/null +++ b/src/runtime/gc/page_heap.zig @@ -0,0 +1,355 @@ +// SPDX-License-Identifier: EPL-2.0 +//! Size-classed page heap: the backing store for small GC cells. +//! +//! Replaces the per-exact-size free pools (ADR-0028 §3, `free_pool.zig`), +//! which had two defects that made a long-lived cljw process only grow +//! (CLJW-GC-HEAP-RETURN): +//! +//! 1. **No return.** A swept cell was pushed onto a pool and kept for the +//! life of the process. The pool never handed a byte back to `infra`, so +//! the OS never got it back either: RSS was a high-water mark. +//! 2. **No sharing.** A pool was keyed by the exact (size, alignment), so a +//! freed 264-byte vector node could only ever become another 264-byte +//! vector node. Every size a program had ever used held its own peak. +//! +//! Shape: memory comes from the OS in `page_bytes` (64 KiB) pages, aligned to +//! their size, so the page owning any cell is `addr & ~(page_bytes - 1)` with no +//! lookup. Each page serves one size class and carries its own header (free +//! list, bump offset, live count) at offset 0. A cell freed by sweep goes onto +//! its page's free list, so the moment a page's live count reaches zero the +//! whole page is reclaimable, which a global per-size free list can never tell. +//! +//! After every sweep `release` rebuilds the per-class available lists and +//! unmaps empty pages beyond a retention budget (the hysteresis: see +//! `GcHeap.retainBytesAfterCollect`). Cells larger than `max_small_bytes`, or +//! over-aligned, are not served here; `GcHeap` sends them to `infra`, whose +//! large-block path (mmap on every allocator cljw links) already returns them. +//! +//! The freed-slot link is overlaid at offset 8, after the stale HeapHeader, +//! exactly as the old pools did (ADR-0028 §3 divergence from cw v0): offset 0 +//! keeps the header bytes so a stray read of a dead cell sees a header, not a +//! pointer. Hence `min_alloc_bytes = 16`. +//! +//! Not thread-safe on its own: every call happens under `GcHeap.gc_mutex` +//! (alloc) or inside the stop-the-world collect (sweep, release). + +const std = @import("std"); +const testing = std.testing; + +/// Page size and alignment. 64 KiB: large enough that the 64-byte header and +/// the tail waste stay small for the 4 KiB class (15 slots, 6 % waste), small +/// enough that one long-lived cell pins little memory. +pub const page_bytes: usize = 64 * 1024; + +/// Largest cell served from pages. A larger cell goes to `infra`. +pub const max_small_bytes: usize = 4096; + +/// Largest alignment served from pages. Slot sizes are multiples of 8 and the +/// first slot sits at a 64-byte offset, so every slot is 8-aligned. +pub const max_small_alignment: std.mem.Alignment = .@"8"; + +/// Size classes: 8-byte steps to 128, 16-byte steps to 512, then four per +/// power of two to 4096. A vector/map trie node (264 bytes) lands in the 272 +/// class, 3 % waste; the worst case past 512 is 25 %, on cells that are rare. +pub const class_sizes = blk: { + var sizes: [51]u32 = undefined; + var n: usize = 0; + var s: u32 = 16; + while (s <= 128) : (s += 8) { + sizes[n] = s; + n += 1; + } + s = 144; + while (s <= 512) : (s += 16) { + sizes[n] = s; + n += 1; + } + var base: u32 = 512; + while (base < max_small_bytes) : (base *= 2) { + var q: u32 = 1; + while (q <= 4) : (q += 1) { + sizes[n] = base + base / 4 * q; + n += 1; + } + } + if (n != sizes.len) @compileError("class_sizes table length mismatch"); + if (sizes[n - 1] != max_small_bytes) @compileError("class_sizes must end at max_small_bytes"); + break :blk sizes; +}; + +pub const class_count = class_sizes.len; + +/// `(size + 7) / 8` -> class index, for every size up to 512. +const small_lookup = blk: { + var table: [512 / 8 + 1]u8 = undefined; + var c: usize = 0; + for (&table, 0..) |*slot, i| { + const size = @max(i * 8, 16); + while (class_sizes[c] < size) c += 1; + slot.* = @intCast(c); + } + break :blk table; +}; + +/// The class serving `size` bytes, or null when `size` is above +/// `max_small_bytes`. +pub fn classOf(size: usize) ?u8 { + if (size <= 512) return small_lookup[(size + 7) / 8]; + if (size > max_small_bytes) return null; + var c: usize = small_lookup[512 / 8]; + while (class_sizes[c] < size) c += 1; + return @intCast(c); +} + +/// True iff a cell of this size and alignment is served from pages. +pub fn isSmall(size: usize, alignment: std.mem.Alignment) bool { + return size <= max_small_bytes and alignment.compare(.lte, max_small_alignment); +} + +const FreeSlot = struct { + next: ?*FreeSlot, +}; + +/// Header at offset 0 of every page. +const Page = struct { + /// Every mapped page, for `release` and `deinit`. + next_all: ?*Page, + prev_all: ?*Page, + /// The class's available list: pages with a free slot or unbumped space. + next_avail: ?*Page, + /// Freed slots of this page (link at slot offset 8). + free: ?*FreeSlot, + /// Offset of the next never-used slot. + bump: u32, + slot_size: u32, + /// Live slots. + used: u32, + capacity: u32, + class: u8, + in_avail: bool, + + fn base(self: *Page) [*]u8 { + return @ptrCast(self); + } +}; + +/// First slot offset. 64 keeps every slot 8-aligned for any class size. +const first_slot: u32 = 64; + +comptime { + std.debug.assert(@sizeOf(Page) <= first_slot); +} + +fn pageOf(ptr: [*]u8) *Page { + return @ptrFromInt(@intFromPtr(ptr) & ~(page_bytes - 1)); +} + +pub const PageHeap = struct { + /// Source of pages. `std.heap.page_allocator` in production (mmap/munmap, + /// VirtualAlloc, or wasm memory); injectable so a test can count maps. + source: std.mem.Allocator = std.heap.page_allocator, + avail: [class_count]?*Page = @splat(null), + all: ?*Page = null, + /// Pages currently mapped. + page_count: usize = 0, + /// Lifetime totals, for `CLJW_GC_STATS`. + pages_mapped_total: u64 = 0, + pages_released_total: u64 = 0, + + pub fn mappedBytes(self: *const PageHeap) usize { + return self.page_count * page_bytes; + } + + /// Allocate one slot for a cell of `size` bytes (`isSmall` must hold). + /// Returns null only when the OS refuses a new page. + pub fn alloc(self: *PageHeap, size: usize) ?[*]u8 { + const c = classOf(size).?; + while (true) { + const page = self.avail[c] orelse self.newPage(c) orelse return null; + if (page.free) |slot| { + page.free = slot.next; + page.used += 1; + return @as([*]u8, @ptrCast(slot)) - 8; + } + if (page.bump + page.slot_size <= page_bytes) { + const p = page.base() + page.bump; + page.bump += page.slot_size; + page.used += 1; + return p; + } + // Full: drop it from the available list. `free` re-adds it. + self.avail[c] = page.next_avail; + page.next_avail = null; + page.in_avail = false; + } + } + + /// Return a slot to its page. The page stays mapped; `release` decides. + pub fn free(self: *PageHeap, ptr: [*]u8) void { + const page = pageOf(ptr); + std.debug.assert(page.used > 0); + const slot: *FreeSlot = @ptrCast(@alignCast(ptr + 8)); + slot.next = page.free; + page.free = slot; + page.used -= 1; + if (!page.in_avail) { + page.next_avail = self.avail[page.class]; + self.avail[page.class] = page; + page.in_avail = true; + } + } + + /// After a sweep: unmap empty pages beyond `retain_bytes` of them, and + /// rebuild every available list so partially used pages are filled before + /// a retained empty one is touched (the empties then stay empty and are + /// the first released next time). Returns the pages released. + pub fn release(self: *PageHeap, retain_bytes: usize) usize { + self.avail = @splat(null); + var retained: usize = 0; + var released: usize = 0; + // Pass 1: empty pages. Keep up to the budget, unmap the rest. + var it = self.all; + while (it) |page| { + it = page.next_all; + if (page.used != 0) continue; + if (retained + page_bytes <= retain_bytes) { + retained += page_bytes; + page.free = null; + page.bump = first_slot; + self.pushAvail(page); + } else { + self.unlinkAll(page); + self.source.rawFree(page.base()[0..page_bytes], .fromByteUnits(page_bytes), @returnAddress()); + self.page_count -= 1; + released += 1; + } + } + // Pass 2: partially used pages go on top, so they are filled first. + it = self.all; + while (it) |page| { + it = page.next_all; + if (page.used == 0) continue; + if (page.used < page.capacity) { + self.pushAvail(page); + } else { + page.next_avail = null; + page.in_avail = false; + } + } + self.pages_released_total += released; + return released; + } + + pub fn deinit(self: *PageHeap) void { + var it = self.all; + while (it) |page| { + it = page.next_all; + self.source.rawFree(page.base()[0..page_bytes], .fromByteUnits(page_bytes), @returnAddress()); + } + self.* = .{ .source = self.source }; + } + + fn pushAvail(self: *PageHeap, page: *Page) void { + page.next_avail = self.avail[page.class]; + self.avail[page.class] = page; + page.in_avail = true; + } + + fn unlinkAll(self: *PageHeap, page: *Page) void { + if (page.prev_all) |p| p.next_all = page.next_all else self.all = page.next_all; + if (page.next_all) |n| n.prev_all = page.prev_all; + } + + fn newPage(self: *PageHeap, c: u8) ?*Page { + const mem = self.source.rawAlloc(page_bytes, .fromByteUnits(page_bytes), @returnAddress()) orelse return null; + const page: *Page = @ptrCast(@alignCast(mem)); + const slot_size = class_sizes[c]; + page.* = .{ + .next_all = self.all, + .prev_all = null, + .next_avail = null, + .free = null, + .bump = first_slot, + .slot_size = slot_size, + .used = 0, + .capacity = @intCast((page_bytes - first_slot) / slot_size), + .class = c, + .in_avail = false, + }; + if (self.all) |head| head.prev_all = page; + self.all = page; + self.pushAvail(page); + self.page_count += 1; + self.pages_mapped_total += 1; + return page; + } +}; + +// --- tests --- + +test "classOf: exact fit at every class, null past max_small_bytes" { + for (class_sizes, 0..) |s, i| { + try testing.expectEqual(@as(?u8, @intCast(i)), classOf(s)); + if (s > 16) try testing.expect(class_sizes[classOf(s - 1).?] >= s - 1); + } + try testing.expectEqual(@as(?u8, 0), classOf(1)); + try testing.expectEqual(@as(?u8, 0), classOf(16)); + try testing.expectEqual(@as(u32, 272), class_sizes[classOf(264).?]); + try testing.expectEqual(@as(?u8, null), classOf(max_small_bytes + 1)); + // Monotone and always large enough. + var size: usize = 1; + while (size <= max_small_bytes) : (size += 1) { + try testing.expect(class_sizes[classOf(size).?] >= size); + } +} + +test "PageHeap: freed slots are reused before a new page is mapped" { + var ph: PageHeap = .{ .source = testing.allocator }; + defer ph.deinit(); + var ptrs: [1000][*]u8 = undefined; + for (&ptrs) |*p| p.* = ph.alloc(264).?; + const pages = ph.page_count; + for (ptrs) |p| ph.free(p); + for (&ptrs) |*p| p.* = ph.alloc(264).?; + try testing.expectEqual(pages, ph.page_count); +} + +test "PageHeap: release unmaps empty pages beyond the budget and keeps partial ones" { + var ph: PageHeap = .{ .source = testing.allocator }; + defer ph.deinit(); + var ptrs: [2000][*]u8 = undefined; + for (&ptrs) |*p| p.* = ph.alloc(64).?; + const pages = ph.page_count; + try testing.expect(pages >= 2); + // Keep one cell alive: its page is partial, every other page empties. + for (ptrs[1..]) |p| ph.free(p); + try testing.expectEqual(pages - 1, ph.release(0)); + try testing.expectEqual(@as(usize, 1), ph.page_count); + // The survivor's page serves the next allocation; no new page. + _ = ph.alloc(64).?; + try testing.expectEqual(@as(usize, 1), ph.page_count); +} + +test "PageHeap: release keeps empty pages up to the retention budget" { + var ph: PageHeap = .{ .source = testing.allocator }; + defer ph.deinit(); + var ptrs: [4000][*]u8 = undefined; + for (&ptrs) |*p| p.* = ph.alloc(128).?; + const pages = ph.page_count; + for (ptrs) |p| ph.free(p); + _ = ph.release(2 * page_bytes); + try testing.expectEqual(@as(usize, 2), ph.page_count); + try testing.expect(pages > 2); + // Retained pages are reset to bump allocation and serve the next burst. + for (ptrs[0..100]) |*p| p.* = ph.alloc(128).?; + try testing.expectEqual(@as(usize, 2), ph.page_count); +} + +test "PageHeap: classes do not share a page" { + var ph: PageHeap = .{ .source = testing.allocator }; + defer ph.deinit(); + const a = ph.alloc(16).?; + const b = ph.alloc(4096).?; + try testing.expect(pageOf(a) != pageOf(b)); + try testing.expectEqual(@as(usize, 2), ph.page_count); +} diff --git a/src/runtime/gc/root_set.zig b/src/runtime/gc/root_set.zig index f20280371..12f4ea622 100644 --- a/src/runtime/gc/root_set.zig +++ b/src/runtime/gc/root_set.zig @@ -29,8 +29,10 @@ //! `tag_ops.tag_trace_table` from the owning module — `tree_walk.zig` //! / `lazy_seq.zig` / `type_descriptor.zig`) and reach via the //! transitive trace, not via root enumeration. Sources 5 / 9 yield -//! nothing in cw v1 by construction (ProtocolFn caches don't exist; -//! CallSite holds namespace-owned pointers, not GC edges). Source 6 +//! nothing in cw v1 by construction (a ProtocolFn's dispatch cache +//! is a memo of a value its descriptor's method_table already roots, read +//! only under a matching generation, ADR-0206; CallSite holds +//! namespace-owned pointers, not GC edges). Source 6 //! (`refer` borrows) is closed-at-construction at `env.zig:229`'s //! `referAll` (the dupe lifts the borrowed name onto `infra_alloc`), //! so no GC walker is needed. The RootIterator's enum slot for each diff --git a/src/runtime/host_enum.zig b/src/runtime/host_enum.zig index 001b37bbc..6b04a1c0a 100644 --- a/src/runtime/host_enum.zig +++ b/src/runtime/host_enum.zig @@ -1,8 +1,8 @@ // SPDX-License-Identifier: EPL-2.0 //! General host-enum mechanism (ADR-0161, D-510) — ONE comptime registry + ONE -//! flat cache for the four JVM host enums (`java.math.RoundingMode`, -//! `java.time.temporal.ChronoUnit`, `java.time.DayOfWeek`, `java.time.Month`), -//! folding the former one-off `rounding_mode.zig` / `chrono_unit.zig` / +//! flat cache for the JVM host enums (`java.math.RoundingMode`, +//! `java.time.temporal.ChronoUnit`, `java.time.DayOfWeek`, `java.time.Month`, +//! `java.util.concurrent.TimeUnit`, `java.time.temporal.ChronoField`), folding the former one-off `rounding_mode.zig` / `chrono_unit.zig` / //! `time/day_of_week_value.zig` / `time/month_value.zig`. //! //! Neutral (keyword `host_enum`): the analyzer's static-field resolve + the @@ -17,7 +17,7 @@ //! `singleton` is the SOLE minting point, so `=` / `identical?` parity is //! structural — no getter can re-mint a fresh value (the bug the former //! getter-minted DayOfWeek/Month had). The cache size is the comptime sum of the -//! registry counts (8+16+7+12 = 43), so a fifth host enum is a one-row edit with +//! registry counts (8+16+7+12+7+30 = 80), so another host enum is a one-row edit with //! zero new `rt` fields. const Value = @import("value/value.zig").Value; @@ -39,6 +39,7 @@ pub const Idx = enum(u8) { day_of_week = 2, month = 3, time_unit = 4, + chrono_field = 5, }; // Canonical per-enum tables. `to_strings` == `names` except ChronoUnit, whose @@ -48,6 +49,10 @@ const CHRONO_NAMES = [_][]const u8{ "NANOS", "MICROS", "MILLIS", "SECONDS", "MIN const CHRONO_DISPLAY = [_][]const u8{ "Nanos", "Micros", "Millis", "Seconds", "Minutes", "Hours", "HalfDays", "Days", "Weeks", "Months", "Years", "Decades", "Centuries", "Millennia", "Eras", "Forever" }; const DOW_NAMES = [_][]const u8{ "MONDAY", "TUESDAY", "WEDNESDAY", "THURSDAY", "FRIDAY", "SATURDAY", "SUNDAY" }; const MONTH_NAMES = [_][]const u8{ "JANUARY", "FEBRUARY", "MARCH", "APRIL", "MAY", "JUNE", "JULY", "AUGUST", "SEPTEMBER", "OCTOBER", "NOVEMBER", "DECEMBER" }; +// ChronoField: JVM declaration order (the ordinal); `toString` is the display +// name. `time/date_time_format.zig::Field` mirrors this order (comptime-pinned). +const CHRONO_FIELD_NAMES = [_][]const u8{ "NANO_OF_SECOND", "NANO_OF_DAY", "MICRO_OF_SECOND", "MICRO_OF_DAY", "MILLI_OF_SECOND", "MILLI_OF_DAY", "SECOND_OF_MINUTE", "SECOND_OF_DAY", "MINUTE_OF_HOUR", "MINUTE_OF_DAY", "HOUR_OF_AMPM", "CLOCK_HOUR_OF_AMPM", "HOUR_OF_DAY", "CLOCK_HOUR_OF_DAY", "AMPM_OF_DAY", "DAY_OF_WEEK", "ALIGNED_DAY_OF_WEEK_IN_MONTH", "ALIGNED_DAY_OF_WEEK_IN_YEAR", "DAY_OF_MONTH", "DAY_OF_YEAR", "EPOCH_DAY", "ALIGNED_WEEK_OF_MONTH", "ALIGNED_WEEK_OF_YEAR", "MONTH_OF_YEAR", "PROLEPTIC_MONTH", "YEAR_OF_ERA", "YEAR", "ERA", "INSTANT_SECONDS", "OFFSET_SECONDS" }; +const CHRONO_FIELD_DISPLAY = [_][]const u8{ "NanoOfSecond", "NanoOfDay", "MicroOfSecond", "MicroOfDay", "MilliOfSecond", "MilliOfDay", "SecondOfMinute", "SecondOfDay", "MinuteOfHour", "MinuteOfDay", "HourOfAmPm", "ClockHourOfAmPm", "HourOfDay", "ClockHourOfDay", "AmPmOfDay", "DayOfWeek", "AlignedDayOfWeekInMonth", "AlignedDayOfWeekInYear", "DayOfMonth", "DayOfYear", "EpochDay", "AlignedWeekOfMonth", "AlignedWeekOfYear", "MonthOfYear", "ProlepticMonth", "YearOfEra", "Year", "Era", "InstantSeconds", "OffsetSeconds" }; const TIME_UNIT_NAMES = [_][]const u8{ "NANOSECONDS", "MICROSECONDS", "MILLISECONDS", "SECONDS", "MINUTES", "HOURS", "DAYS" }; pub const EnumDef = struct { @@ -72,6 +77,7 @@ pub const defs = build: { .{ .fqcn = "java.time.DayOfWeek", .names = &DOW_NAMES, .to_strings = &DOW_NAMES, .value_base = 1, .cache_base = 0 }, .{ .fqcn = "java.time.Month", .names = &MONTH_NAMES, .to_strings = &MONTH_NAMES, .value_base = 1, .cache_base = 0 }, .{ .fqcn = "java.util.concurrent.TimeUnit", .names = &TIME_UNIT_NAMES, .to_strings = &TIME_UNIT_NAMES, .value_base = null, .cache_base = 0 }, + .{ .fqcn = "java.time.temporal.ChronoField", .names = &CHRONO_FIELD_NAMES, .to_strings = &CHRONO_FIELD_DISPLAY, .value_base = null, .cache_base = 0 }, }; var base: u16 = 0; for (&d) |*e| { @@ -81,7 +87,7 @@ pub const defs = build: { break :build d; }; -/// Flat-cache size = sum of all enum counts (8+16+7+12 = 43). +/// Flat-cache size = sum of all enum counts (8+16+7+12+7+30 = 80). pub const TOTAL = blk: { var n: u16 = 0; for (defs) |e| n += @intCast(e.names.len); @@ -203,8 +209,12 @@ const std = @import("std"); const testing = std.testing; test "host_enum registry: cache_base accumulation + TOTAL + name/toString/value" { - // Counts: 8 + 16 + 7 + 12 + 7 = 50, non-overlapping flat-cache windows. - try testing.expectEqual(@as(u16, 50), TOTAL); + // Counts: 8 + 16 + 7 + 12 + 7 + 30 = 80, non-overlapping flat-cache windows. + try testing.expectEqual(@as(u16, 80), TOTAL); + try testing.expectEqual(@as(usize, 30), count(.chrono_field)); + try testing.expectEqual(@as(u16, 50), defs[@intFromEnum(Idx.chrono_field)].cache_base); + try testing.expectEqualStrings("HOUR_OF_DAY", name(.chrono_field, 12)); + try testing.expectEqualStrings("HourOfDay", toStringOf(.chrono_field, 12)); try testing.expectEqual(@as(usize, 8), count(.rounding_mode)); try testing.expectEqual(@as(usize, 16), count(.chrono_unit)); try testing.expectEqual(@as(usize, 7), count(.day_of_week)); diff --git a/src/runtime/io/host_stream.zig b/src/runtime/io/host_stream.zig index 490cd6bb7..d220834b1 100644 --- a/src/runtime/io/host_stream.zig +++ b/src/runtime/io/host_stream.zig @@ -115,6 +115,7 @@ fn readByte(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) a /// stdin mode: append one blocking chunk from process stdin to `data`. /// A zero-byte read (or error) latches EOF. Mirrors text_io's `*in*` fill. fn stdinFill(rt: *Runtime, st: *StreamState) !void { + stdin.beforeFill(rt.gc.infra, &st.data, &st.pos, &rt.gc); var chunk: [4096]u8 = undefined; const n = stdin.readChunk(rt.io, &chunk) catch { st.stdin_eof = true; diff --git a/src/runtime/io/stdin.zig b/src/runtime/io/stdin.zig index 75a26e8be..d4ef97ce8 100644 --- a/src/runtime/io/stdin.zig +++ b/src/runtime/io/stdin.zig @@ -3,9 +3,89 @@ //! `*in*` readers (text_io, host_stream), via `std.Io.File`. const std = @import("std"); +const builtin = @import("builtin"); /// Read up to `buf.len` bytes from stdin into `buf`. Returns bytes read /// (0 = EOF). pub fn readChunk(io: std.Io, buf: []u8) std.Io.File.ReadStreamingError!usize { return std.Io.File.stdin().readStreaming(io, &.{buf}); } + +/// How long a stdin fill waits for input before running an idle collect. +pub const idle_poll_ms: i32 = 1000; + +/// Wait up to `timeout_ms` for stdin to become readable (data, EOF or +/// hang-up). True when a read will not block. Answers true at once where +/// there is no `poll`, so callers never spin. +pub fn waitReadable(timeout_ms: i32) bool { + if (comptime builtin.os.tag == .wasi or builtin.os.tag == .windows) return true; + var fds = [_]std.posix.pollfd{.{ + .fd = std.posix.STDIN_FILENO, + .events = std.posix.POLL.IN, + .revents = 0, + }}; + const n = std.posix.poll(&fds, timeout_ms) catch return true; + return n > 0; +} + +/// Ready a demand-filled stdin buffer for its next fill: drop its consumed +/// prefix, then wait for input, calling `gc.collectBeforeIdle()` each time +/// `idle_poll_ms` passes with none, so a process blocked on stdin still +/// collects and returns memory. +pub fn beforeFill(infra: std.mem.Allocator, data: *std.ArrayList(u8), pos: *usize, gc: anytype) void { + dropConsumed(infra, data, pos); + while (!waitReadable(idle_poll_ms)) gc.collectBeforeIdle(); +} + +/// Capacity up to which `dropConsumed` never frees a stdin buffer's excess, so +/// a steady stream of small lines does not realloc per fill. +pub const retained_capacity: usize = 64 * 1024; + +/// Drop the consumed prefix `data[0..pos]` of a demand-filled stdin buffer +/// and reset `pos` to 0, so a long-running reader holds only its unread tail. +/// Frees the excess allocation once it is over 4x the tail and +/// `retained_capacity`. +/// Call before appending a fill: the append may move `data` anyway, so no +/// caller holds a slice or offset across it. +pub fn dropConsumed(infra: std.mem.Allocator, data: *std.ArrayList(u8), pos: *usize) void { + const consumed = pos.*; + if (consumed == 0) return; + const rest = data.items.len - consumed; + std.mem.copyForwards(u8, data.items[0..rest], data.items[consumed..]); + data.shrinkRetainingCapacity(rest); + pos.* = 0; + if (data.capacity > retained_capacity and data.capacity > rest * 4) + data.shrinkAndFree(infra, rest); +} + +test "dropConsumed keeps the unread tail and resets pos" { + const a = std.testing.allocator; + var data: std.ArrayList(u8) = .empty; + defer data.deinit(a); + try data.appendSlice(a, "line one\nline two"); + var pos: usize = 9; + dropConsumed(a, &data, &pos); + try std.testing.expectEqualStrings("line two", data.items); + try std.testing.expectEqual(@as(usize, 0), pos); +} + +test "dropConsumed releases a buffer that grew far past its tail" { + const a = std.testing.allocator; + var data: std.ArrayList(u8) = .empty; + defer data.deinit(a); + try data.appendNTimes(a, 'x', 1024 * 1024); + var pos: usize = data.items.len - 3; + dropConsumed(a, &data, &pos); + try std.testing.expectEqual(@as(usize, 3), data.items.len); + try std.testing.expect(data.capacity <= retained_capacity); +} + +test "dropConsumed is a no-op at pos 0" { + const a = std.testing.allocator; + var data: std.ArrayList(u8) = .empty; + defer data.deinit(a); + try data.appendSlice(a, "abc"); + var pos: usize = 0; + dropConsumed(a, &data, &pos); + try std.testing.expectEqualStrings("abc", data.items); +} diff --git a/src/runtime/io/text_io.zig b/src/runtime/io/text_io.zig index 1528b261d..d9f9a6370 100644 --- a/src/runtime/io/text_io.zig +++ b/src/runtime/io/text_io.zig @@ -174,6 +174,7 @@ const ReaderState = struct { /// TTY is the correct clj-parity behaviour (read-line waits for input). fn fillFromStdin(rt: *Runtime, st: *ReaderState) bool { if (st.stdin_eof) return false; + stdin.beforeFill(rt.gc.infra, &st.data, &st.pos, &rt.gc); var chunk: [4096]u8 = undefined; const n = stdin.readChunk(rt.io, &chunk) catch { st.stdin_eof = true; diff --git a/src/runtime/java/_host_api.zig b/src/runtime/java/_host_api.zig index 7bbfe8b11..5e5243cd0 100644 --- a/src/runtime/java/_host_api.zig +++ b/src/runtime/java/_host_api.zig @@ -108,6 +108,11 @@ const java_surfaces = [_]type{ @import("time/LocalDateTime.zig"), @import("time/LocalTime.zig"), @import("time/Duration.zig"), + @import("time/ZoneId.zig"), // fixed-offset zones only (no tz database) + @import("time/ZoneOffset.zig"), + @import("time/temporal/ChronoField.zig"), // host-enum (ADR-0161 registry) + @import("time/format/DateTimeFormatter.zig"), + @import("time/format/DateTimeFormatterBuilder.zig"), // ZonedDateTime backing impl under runtime/time/ is unbuilt // (method_table empty) — D-105. @import("time/ZonedDateTime.zig"), diff --git a/src/runtime/java/lang/Character.zig b/src/runtime/java/lang/Character.zig index e3420348f..fcd80f7c5 100644 --- a/src/runtime/java/lang/Character.zig +++ b/src/runtime/java/lang/Character.zig @@ -176,7 +176,9 @@ fn toChars(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) an if (cp < 0 or cp > 0x10FFFF) { return error_catalog.raise(.type_arg_invalid, loc, .{ .fn_name = "Character/toChars", .expected = "valid codepoint", .actual = "out-of-range int" }); } - return java_array.fromSlice(rt, &.{Value.initChar(@intCast(cp))}); + const arr = try java_array.make(rt, .char, 1); + java_array.asArray(arr).slice(.char)[0] = @intCast(cp); + return arr; } /// Strict codepoint from a `.char` OR an in-range `.integer` — for the @@ -617,7 +619,7 @@ pub const ___HOST_EXTENSION: host_api.Extension = .{ // Static fields (ADR-0061) — comptime-const. `Character/TYPE` (the // primitive Class object) is NOT carried: cljw has no primitive Class -// values (ADR-0059 no-JVM; arrays are type-erased per AD-019). +// values (ADR-0059 no-JVM; a class argument names no array kind, AD-019). const character_static_fields = [_]type_descriptor.TypeDescriptor.StaticField{ .{ .name = "MIN_VALUE", .value = .{ .char = 0x0000 } }, .{ .name = "MAX_VALUE", .value = .{ .char = 0xFFFF } }, diff --git a/src/runtime/java/lang/Runtime.zig b/src/runtime/java/lang/Runtime.zig index 3e9a76506..58d1ac0e8 100644 --- a/src/runtime/java/lang/Runtime.zig +++ b/src/runtime/java/lang/Runtime.zig @@ -18,12 +18,12 @@ //! that reads a made-up denominator is worse than one that cannot read at all. //! The mapping onto the JVM's vocabulary (divergence AD-063): //! -//! - `.totalMemory` — the heap cljw currently commits to, i.e. the live set -//! or the adaptive collect threshold, whichever is larger. cljw keeps no -//! reserved-but-unused pool, so the threshold is the honest analogue of -//! "heap the collector will let you grow into". -//! - `.freeMemory` — `totalMemory` minus the live set: headroom before the -//! next collection, which is what the JVM reading also means. +//! - `.totalMemory`: the cell heap cljw currently commits: the mapped GC +//! pages (live cells, free slots, retained empty pages) plus live large +//! cells (`GcHeap.committedBytes`). It grows with the heap and shrinks +//! when a collect hands empty pages back to the OS, as the JVM's does. +//! - `.freeMemory`: `totalMemory` minus the live cells: committed space +//! the next allocations fill before the heap has to grow. //! - `.maxMemory` — `gc.heap_ceiling` (set by `CLJW_EVAL_MAX_HEAP_MB`, //! ADR-0125), or `Long/MAX_VALUE` when the heap is unmetered. An unmetered //! cljw genuinely has no ceiling, so a used/max pressure ratio reads ~0. @@ -94,7 +94,7 @@ fn asLong(rt: *Runtime, n: usize) !Value { fn totalMemory(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { _ = env; try error_catalog.checkArity("Runtime/totalMemory", args, 1, loc); - return asLong(rt, @max(liveBytes(rt), rt.gc.threshold_bytes)); + return asLong(rt, @max(liveBytes(rt), rt.gc.committedBytes())); } /// `(.freeMemory r)` — headroom inside `totalMemory` before the next collect. @@ -102,7 +102,7 @@ fn freeMemory(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) _ = env; try error_catalog.checkArity("Runtime/freeMemory", args, 1, loc); const live = liveBytes(rt); - return asLong(rt, @max(live, rt.gc.threshold_bytes) - live); + return asLong(rt, @max(live, rt.gc.committedBytes()) - live); } /// `(.maxMemory r)` — the per-eval heap ceiling, or `Long/MAX_VALUE` when the diff --git a/src/runtime/java/lang/String.zig b/src/runtime/java/lang/String.zig index 8ea5173d8..7cd46d9e7 100644 --- a/src/runtime/java/lang/String.zig +++ b/src/runtime/java/lang/String.zig @@ -517,19 +517,15 @@ fn toCharArray(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation } /// Implements `(.getBytes s)` / `(.getBytes s charset)` — the UTF-8 bytes of `s` -/// as a byte array (cljw `.array` of SIGNED-byte ints, matching clj's `byte[]`: -/// a byte > 127 is negative, e.g. "é" → (-61 -87)). cljw is UTF-8-only, so a +/// as a byte array (signed, matching clj's `byte[]`: a byte > 127 reads +/// negative, e.g. "é" → (-61 -87)). cljw is UTF-8-only, so a /// charset arg (a name String — or any object) is accepted and the encoding is /// always UTF-8 (documented limitation; the common `(.getBytes s "UTF-8")` works). fn getBytes(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { _ = env; if (args.len < 1 or args.len > 2) return error_catalog.raise(.arity_out_of_range, loc, .{ .fn_name = ".getBytes", .got = args.len, .min = 1, .max = 2 }); - const s = string_collection.asString(args[0]); - const buf = try rt.gpa.alloc(Value, s.len); - defer rt.gpa.free(buf); - for (s, 0..) |b, i| buf[i] = Value.initInteger(@as(i8, @bitCast(b))); // signed byte (clj byte[]) - return java_array.fromSlice(rt, buf); + return java_array.fromBytes(rt, string_collection.asString(args[0])); } /// `(.intern s)` → the canonical interned instance. cw v1 has no string pool @@ -603,7 +599,7 @@ fn valueOf(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) an // Java String.valueOf(char[]) / (char[], offset, count) — the char-array // overloads build the string FROM the characters, not a render of the // array object (D-526; was falling into the generic "#" arm). - if (java_array.isArray(args[0])) return charsToString(rt, args, "String/valueOf", loc); + if (java_array.isKind(args[0], .char)) return charsToString(rt, args, "String/valueOf", loc); try error_catalog.checkArity("String/valueOf", args, 1, loc); var aw: std.Io.Writer.Allocating = .init(rt.gpa); defer aw.deinit(); @@ -613,10 +609,12 @@ fn valueOf(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) an /// Shared body of the char-array string builders: `(f arr)` uses the whole /// array, `(f arr offset count)` a validated window (JVM -/// StringIndexOutOfBounds → cljw bounds error). Non-char elements raise a -/// type error (a Java char[] cannot hold anything else). +/// StringIndexOutOfBounds → cljw bounds error). Any array but a char array +/// raises a type error. fn charsToString(rt: *Runtime, args: []const Value, comptime fn_name: []const u8, loc: SourceLocation) anyerror!Value { - const items = java_array.asArray(args[0]).items(); + if (!java_array.isKind(args[0], .char)) + return error_catalog.raise(.type_arg_invalid, loc, .{ .fn_name = fn_name, .expected = "a char array", .actual = @tagName(java_array.asArray(args[0]).kind) }); + const items = java_array.asArray(args[0]).slice(.char); var start: usize = 0; var count: usize = items.len; if (args.len == 3) { @@ -629,11 +627,9 @@ fn charsToString(rt: *Runtime, args: []const Value, comptime fn_name: []const u8 } var aw: std.Io.Writer.Allocating = .init(rt.gpa); defer aw.deinit(); - for (items[start .. start + count]) |e| { - if (e.tag() != .char) - return error_catalog.raise(.type_arg_invalid, loc, .{ .fn_name = fn_name, .expected = "a char array", .actual = @tagName(e.tag()) }); + for (items[start .. start + count]) |cp| { var buf: [4]u8 = undefined; - const n = std.unicode.utf8Encode(e.asChar(), &buf) catch 0; + const n = std.unicode.utf8Encode(@intCast(cp), &buf) catch 0; try aw.writer.writeAll(buf[0..n]); } return string_collection.alloc(rt, aw.writer.buffered()); @@ -728,32 +724,24 @@ fn stringCtor(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) start = @intCast(off); count = @intCast(len_i); } - const elems = arr.items()[start .. start + count]; // A CHAR array → the chars' codepoints, UTF-8 encoded (clj `(String. - // char[])`); a BYTE/INT array → each element's low 8 bits as a byte (the - // `.getBytes` inverse). The first element of the window selects the path; - // an empty window takes the byte path and yields "". - if (elems.len > 0 and elems[0].tag() == .char) { - var buf: std.ArrayList(u8) = .empty; - defer buf.deinit(rt.gpa); - for (elems) |v| { - if (v.tag() != .char) - return error_catalog.raise(.type_arg_invalid, loc, .{ .fn_name = "java.lang.String.", .expected = "char array", .actual = @tagName(v.tag()) }); - var cbuf: [4]u8 = undefined; - const n = std.unicode.utf8Encode(v.asChar(), &cbuf) catch - return error_catalog.raise(.type_arg_invalid, loc, .{ .fn_name = "java.lang.String.", .expected = "valid char", .actual = "unencodable codepoint" }); - try buf.appendSlice(rt.gpa, cbuf[0..n]); - } - return string_collection.alloc(rt, buf.items); - } - const buf = try rt.gpa.alloc(u8, elems.len); - defer rt.gpa.free(buf); - for (elems, 0..) |v, i| { - if (v.tag() != .integer) - return error_catalog.raise(.type_arg_invalid, loc, .{ .fn_name = "java.lang.String.", .expected = "byte array", .actual = @tagName(v.tag()) }); - buf[i] = @intCast(v.asInteger() & 0xFF); // low 8 bits → byte (signed or unsigned in) + // char[])`); a BYTE array → its bytes as UTF-8, one memcpy (the + // `.getBytes` inverse). Any other kind has no String constructor. + switch (arr.kind) { + .byte => return string_collection.alloc(rt, arr.bytes()[start .. start + count]), + .char => { + var buf: std.ArrayList(u8) = .empty; + defer buf.deinit(rt.gpa); + for (arr.slice(.char)[start .. start + count]) |cp| { + var cbuf: [4]u8 = undefined; + const n = std.unicode.utf8Encode(@intCast(cp), &cbuf) catch + return error_catalog.raise(.type_arg_invalid, loc, .{ .fn_name = "java.lang.String.", .expected = "valid char", .actual = "unencodable codepoint" }); + try buf.appendSlice(rt.gpa, cbuf[0..n]); + } + return string_collection.alloc(rt, buf.items); + }, + else => return error_catalog.raise(.type_arg_invalid, loc, .{ .fn_name = "java.lang.String.", .expected = "a byte or char array", .actual = @tagName(arr.kind) }), } - return string_collection.alloc(rt, buf); } return error_catalog.raise(.type_arg_invalid, loc, .{ .fn_name = "java.lang.String.", .expected = "string or byte array", .actual = @tagName(a.tag()) }); } diff --git a/src/runtime/java/lang/StringBuilder.zig b/src/runtime/java/lang/StringBuilder.zig index c2a8970bb..688f362b3 100644 --- a/src/runtime/java/lang/StringBuilder.zig +++ b/src/runtime/java/lang/StringBuilder.zig @@ -50,7 +50,15 @@ fn appendStr(rt: *Runtime, env: *Env, lp: *ByteList, v: Value) !void { var aw: std.Io.Writer.Allocating = .init(rt.gpa); defer aw.deinit(); try print_mod.writeStrValue(rt, env, &aw.writer, v); - try lp.appendSlice(rt.gc.infra, aw.writer.buffered()); + try appendCharged(rt, lp, aw.writer.buffered()); +} + +/// Append `bytes` to `lp`, charging any buffer growth to the collect trigger: +/// the builder's cell is a few words however large the buffer gets. +fn appendCharged(rt: *Runtime, lp: *ByteList, bytes: []const u8) !void { + const before = lp.capacity; + try lp.appendSlice(rt.gc.infra, bytes); + if (lp.capacity > before) rt.gc.noteOwnedAlloc(lp.capacity - before); } /// `(StringBuilder.)` / `(StringBuilder. "seed")` / `(StringBuilder. n)` — @@ -90,7 +98,7 @@ fn append(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) any const to = byteOffsetOfCodepoint(s, to_i); if (from_i < 0 or to_i < from_i or from == null or to == null) return error_catalog.raise(.index_out_of_range, loc, .{ .fn_name = "java.lang.StringBuilder/append" }); - try listOf(args[0]).appendSlice(rt.gc.infra, s[from.?..to.?]); + try appendCharged(rt, listOf(args[0]), s[from.?..to.?]); return args[0]; } diff --git a/src/runtime/java/lang/System.zig b/src/runtime/java/lang/System.zig index 21d0b639a..4944ee1fc 100644 --- a/src/runtime/java/lang/System.zig +++ b/src/runtime/java/lang/System.zig @@ -327,7 +327,8 @@ fn exit(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyer /// Implements `(java.lang.System/arraycopy src srcPos dest destPos length)`. /// Spec: copies `length` elements from `src[srcPos..]` into `dest[destPos..]`. -/// Both are cljw Java arrays (ADR-0105 type-erased []Value). A same-array +/// Both are cljw Java arrays of the same element kind (ADR-0204; a kind +/// mismatch is JVM ArrayStoreException, here `arg_value_invalid`). A same-array /// overlapping copy is correct (JVM semantics: as if through a temp — @memmove). /// Out-of-range positions/length raise an index error (JVM throws /// ArrayIndexOutOfBoundsException; cljw's Kind differs per AD-007). @@ -353,8 +354,9 @@ fn arraycopy(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) if (length < 0 or src_pos < 0 or dst_pos < 0 or src_pos + length > src.len or dst_pos + length > dst.len) return error_catalog.raise(.index_out_of_range, loc, .{ .fn_name = "java.lang.System/arraycopy" }); - const n: usize = @intCast(length); - @memmove(dst.items_ptr[@intCast(dst_pos)..][0..n], src.items_ptr[@intCast(src_pos)..][0..n]); + if (src.kind != dst.kind) + return error_catalog.raise(.arg_value_invalid, loc, .{ .fn_name = "java.lang.System/arraycopy", .expected = "two arrays of the same element kind", .actual = @tagName(dst.kind) }); + java_array.copy(src, @intCast(src_pos), dst, @intCast(dst_pos), @intCast(length)); return Value.nil_val; } diff --git a/src/runtime/java/lang/Thread.zig b/src/runtime/java/lang/Thread.zig index 9e4af87e6..b02f80f82 100644 --- a/src/runtime/java/lang/Thread.zig +++ b/src/runtime/java/lang/Thread.zig @@ -189,7 +189,7 @@ fn getStackTraceFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLoca _ = env; try error_catalog.checkArity(".getStackTrace", args, 1, loc); try expectThread(args, ".getStackTrace", loc); - return java_array.make(rt, 0, Value.nil_val); + return java_array.make(rt, .object, 0); } /// `(Thread/yield)` — a scheduling hint (JVM-exact semantics: a hint). diff --git a/src/runtime/java/lang/reflect/Array.zig b/src/runtime/java/lang/reflect/Array.zig index 7c40a7d3e..03330b337 100644 --- a/src/runtime/java/lang/reflect/Array.zig +++ b/src/runtime/java/lang/reflect/Array.zig @@ -30,7 +30,7 @@ fn newInstance(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation .expected = "a non-negative length", .actual = "a negative length", }); - return java_array.make(rt, @intCast(len), .nil_val); + return java_array.make(rt, .object, @intCast(len)); } /// `(java.lang.reflect.Array/getLength arr)` — the array's slot count. diff --git a/src/runtime/java/math/BigInteger.zig b/src/runtime/java/math/BigInteger.zig index f332a9947..485d91f32 100644 --- a/src/runtime/java/math/BigInteger.zig +++ b/src/runtime/java/math/BigInteger.zig @@ -44,16 +44,12 @@ fn initBigInteger(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocat } if (java_array.isArray(a)) { const arr = java_array.asArray(a); + if (arr.kind != .byte) + return error_catalog.raise(.type_arg_invalid, loc, .{ .fn_name = "java.math.BigInteger.", .expected = "byte array", .actual = @tagName(arr.kind) }); const n = arr.len; if (n == 0) return error_catalog.raise(.number_format_invalid, loc, .{ .fn_name = "java.math.BigInteger.", .text = "" }); - const bytes = try rt.gpa.alloc(u8, n); - defer rt.gpa.free(bytes); - for (arr.items(), 0..) |item, i| { - if (item.tag() != .integer) - return error_catalog.raise(.type_arg_invalid, loc, .{ .fn_name = "java.math.BigInteger.", .expected = "byte array", .actual = @tagName(item.tag()) }); - bytes[i] = @intCast(item.asInteger() & 0xFF); - } + const bytes = arr.bytes(); const bit_count = n * 8; var m = try std.math.big.int.Managed.init(rt.gc.infra); defer m.deinit(); diff --git a/src/runtime/java/time/Instant.zig b/src/runtime/java/time/Instant.zig index 1dc257483..49f2080c5 100644 --- a/src/runtime/java/time/Instant.zig +++ b/src/runtime/java/time/Instant.zig @@ -2,15 +2,16 @@ //! Java surface for `java.time.Instant`. //! //! Backend: impl-only -//! Impl deps: instant +//! Impl deps: instant, date_time_format //! Clojure peer: none (clojure.instant has its own parser) //! //! Java 8+ canonical time class. The static factories (now / ofEpochSecond / //! ofEpochMilli / parse) populate `method_table` via the `init` callback (D-462) //! and mint a `.typed_instance` Instant VALUE (`runtime/time/instant_value.zig`) //! whose per-Runtime descriptor carries the instance methods -//! (getEpochSecond / getNano / toEpochMilli). The parse/format impl lives in the -//! sibling `runtime/time/instant.zig` (F-009 neutral home). +//! (getEpochSecond / getNano / toEpochMilli). The ISO print lives in the +//! sibling `runtime/time/instant.zig` (F-009 neutral home); `parse` and `from` +//! go through the java.time.format engine, like the JVM's `ISO_INSTANT`. //! //! method_table is populated in `initInstant` (runtime) rather than at module //! scope because `Value.initBuiltinFn(&fn)` calls `@intFromPtr(&fn)` which is @@ -26,7 +27,7 @@ const SourceLocation = @import("../../error/info.zig").SourceLocation; const error_catalog = @import("../../error/catalog.zig"); const instant = @import("../../time/instant.zig"); const instant_value = @import("../../time/instant_value.zig"); -const string_collection = @import("../../collection/string.zig"); +const dtf_value = @import("../../time/date_time_formatter_value.zig"); /// Mint an Instant from a whole-second epoch + a nanos fraction. Shared by /// `ofEpochSecond` (1- and 2-arg) and the milli/parse/now factories. Stores the @@ -76,20 +77,21 @@ fn ofEpochMilli(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocatio return fromSecondNanos(rt, secs, nanos); } -/// `(java.time.Instant/parse s)` — parse an ISO-8601 instant string. Shares the -/// `instant.parseInstantFields` grammar with `#inst` / Date (F-009). JVM throws -/// `DateTimeParseException`; cljw raises the same `instant_string_invalid` the -/// `#inst` reader uses. +/// `(java.time.Instant/parse s)` — JVM `Instant.parse` = `parse(s, +/// DateTimeFormatter/ISO_INSTANT)`: a bad string is the DateTimeParseException. fn parse(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { _ = env; try error_catalog.checkArity("java.time.Instant/parse", args, 1, loc); - if (args[0].tag() != .string) - return error_catalog.raise(.type_arg_not_string, loc, .{ .fn_name = "java.time.Instant/parse", .actual = @tagName(args[0].tag()) }); - const s = string_collection.asString(args[0]); - const fields = instant.parseInstantFields(s) catch - return error_catalog.raise(.inst_string_invalid, loc, .{ .s = s }); - // `epoch_ms` folds the offset + ms; re-split to second-aligned + full nanos. - return instant_value.make(rt, @divFloor(fields.epoch_ms, 1000) * 1000, fields.nanos); + return dtf_value.parseIso(rt, .instant, args[0], "ISO_INSTANT", "java.time.Instant/parse", loc); +} + +/// `(java.time.Instant/from temporal)` — JVM `Instant.from`: an Instant +/// itself, or any TemporalAccessor carrying InstantSeconds + NanoOfSecond +/// (a parse result with an offset). +fn from(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("java.time.Instant/from", args, 1, loc); + return dtf_value.convert(rt, .instant, args[0], loc); } fn initInstant(td: *type_descriptor.TypeDescriptor, gpa: std.mem.Allocator) anyerror!void { @@ -102,6 +104,7 @@ fn initInstant(td: *type_descriptor.TypeDescriptor, gpa: std.mem.Allocator) anye .{ "ofEpochSecond", &ofEpochSecond }, .{ "ofEpochMilli", &ofEpochMilli }, .{ "parse", &parse }, + .{ "from", &from }, }); } td.temporal_print = .iso_instant; diff --git a/src/runtime/java/time/LocalDate.zig b/src/runtime/java/time/LocalDate.zig index a34f7e7aa..f8e939870 100644 --- a/src/runtime/java/time/LocalDate.zig +++ b/src/runtime/java/time/LocalDate.zig @@ -10,8 +10,10 @@ //! `.typed_instance` LocalDate VALUE (`runtime/time/local_date_value.zig`) //! whose per-Runtime descriptor carries the instance methods (getYear / //! getMonthValue / getDayOfMonth / plusDays). The civil↔epoch-day conversions -//! + the date parser are reused from the sibling `runtime/time/instant.zig` -//! (F-009 neutral home). +//! are reused from the sibling `runtime/time/instant.zig` (F-009 neutral +//! home); `parse` / `from` / `.format` go through the java.time.format engine +//! (`runtime/time/date_time_formatter_value.zig`), so a bad string is the +//! JVM DateTimeParseException. //! //! method_table is populated in `initLocalDate` (runtime) rather than at //! module scope because `Value.initBuiltinFn(&fn)` calls `@intFromPtr(&fn)` @@ -27,7 +29,7 @@ const SourceLocation = @import("../../error/info.zig").SourceLocation; const error_catalog = @import("../../error/catalog.zig"); const instant = @import("../../time/instant.zig"); const ld_value = @import("../../time/local_date_value.zig"); -const string_collection = @import("../../collection/string.zig"); +const dtf_value = @import("../../time/date_time_formatter_value.zig"); const MS_PER_DAY: i64 = 86_400_000; @@ -52,18 +54,30 @@ fn now(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerr return ld_value.make(rt, @divFloor(ms, MS_PER_DAY)); } -/// `(java.time.LocalDate/parse s)` — parse an ISO local date `yyyy-MM-dd`. JVM -/// throws `DateTimeParseException`; cljw raises the same `inst_string_invalid` -/// the `#inst` reader uses. +/// `(java.time.LocalDate/parse s)` = `parse(s, DateTimeFormatter/ISO_LOCAL_DATE)`, and +/// `(java.time.LocalDate/parse s formatter)` (JVM `LocalDate.parse`). A failure is +/// the JVM DateTimeParseException. fn parse(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { _ = env; - try error_catalog.checkArity("java.time.LocalDate/parse", args, 1, loc); - if (args[0].tag() != .string) - return error_catalog.raise(.type_arg_not_string, loc, .{ .fn_name = "java.time.LocalDate/parse", .actual = @tagName(args[0].tag()) }); - const s = string_collection.asString(args[0]); - const epoch_day = instant.parseLocalDate(s) catch - return error_catalog.raise(.inst_string_invalid, loc, .{ .s = s }); - return ld_value.make(rt, epoch_day); + try error_catalog.checkArityRange("java.time.LocalDate/parse", args, 1, 2, loc); + if (args.len == 2) return dtf_value.parseTo(rt, .local_date, args[0], args[1], "java.time.LocalDate/parse", loc); + return dtf_value.parseIso(rt, .local_date, args[0], "ISO_LOCAL_DATE", "java.time.LocalDate/parse", loc); +} + +/// `(LocalDate/from temporal)` — JVM `LocalDate.from(TemporalAccessor)`. +fn from(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("java.time.LocalDate/from", args, 1, loc); + return dtf_value.convert(rt, .local_date, args[0], loc); +} + +/// `(.format x formatter)` — JVM `LocalDate.format(DateTimeFormatter)`. +fn formatFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("format", args, 2, loc); + if (!dtf_value.isFormatter(args[1])) + return error_catalog.raise(.type_arg_invalid, loc, .{ .fn_name = ".format", .expected = "a DateTimeFormatter", .actual = dtf_value.classNameOf(args[1]) }); + return dtf_value.formatValue(rt, args[1], args[0], loc); } /// `(java.time.LocalDate/ofEpochDay n)` — the date `n` days after 1970-01-01 @@ -87,6 +101,8 @@ fn initLocalDate(td: *type_descriptor.TypeDescriptor, gpa: std.mem.Allocator) an .{ "now", &now }, .{ "parse", &parse }, .{ "ofEpochDay", &ofEpochDay }, + .{ "from", &from }, + .{ "format", &formatFn }, }); } td.temporal_print = .iso_local_date; diff --git a/src/runtime/java/time/LocalDateTime.zig b/src/runtime/java/time/LocalDateTime.zig index 95d771754..c6f2334d5 100644 --- a/src/runtime/java/time/LocalDateTime.zig +++ b/src/runtime/java/time/LocalDateTime.zig @@ -11,7 +11,9 @@ //! whose per-Runtime descriptor carries the instance methods (getYear / //! getMonthValue / getDayOfMonth / getHour / getMinute / getSecond / getNano). //! The civil↔epoch-day conversions are reused from the sibling -//! `runtime/time/instant.zig` (F-009 neutral home). +//! `runtime/time/instant.zig` (F-009 neutral home); `parse` / `from` / +//! `.format` go through the java.time.format engine +//! (`runtime/time/date_time_formatter_value.zig`). //! //! method_table is populated in `initLocalDateTime` (runtime) rather than at //! module scope because `Value.initBuiltinFn(&fn)` calls `@intFromPtr(&fn)` @@ -27,7 +29,7 @@ const SourceLocation = @import("../../error/info.zig").SourceLocation; const error_catalog = @import("../../error/catalog.zig"); const instant = @import("../../time/instant.zig"); const ldt_value = @import("../../time/local_date_time_value.zig"); -const string_collection = @import("../../collection/string.zig"); +const dtf_value = @import("../../time/date_time_formatter_value.zig"); const MS_PER_DAY: i64 = 86_400_000; @@ -68,76 +70,30 @@ fn now(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerr return ldt_value.make(rt, epoch_day, nano_of_day); } -/// `(java.time.LocalDateTime/parse s)` — parse an ISO local date-time -/// `yyyy-MM-ddTHH:mm[:ss[.fraction]]` (NO offset/Z). JVM throws -/// `DateTimeParseException`; cljw raises the same `inst_string_invalid` the -/// `#inst` reader uses. +/// `(java.time.LocalDateTime/parse s)` = `parse(s, DateTimeFormatter/ISO_LOCAL_DATE_TIME)`, and +/// `(java.time.LocalDateTime/parse s formatter)` (JVM `LocalDateTime.parse`). A failure is +/// the JVM DateTimeParseException. fn parse(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { _ = env; - try error_catalog.checkArity("java.time.LocalDateTime/parse", args, 1, loc); - if (args[0].tag() != .string) - return error_catalog.raise(.type_arg_not_string, loc, .{ .fn_name = "java.time.LocalDateTime/parse", .actual = @tagName(args[0].tag()) }); - const s = string_collection.asString(args[0]); - const f = parseLocalDateTime(s) catch - return error_catalog.raise(.inst_string_invalid, loc, .{ .s = s }); - return ldt_value.make(rt, instant.daysFromCivil(f.y, f.m, f.d), nanoOfDay(f.h, f.mi, f.s, f.n)); + try error_catalog.checkArityRange("java.time.LocalDateTime/parse", args, 1, 2, loc); + if (args.len == 2) return dtf_value.parseTo(rt, .local_date_time, args[0], args[1], "java.time.LocalDateTime/parse", loc); + return dtf_value.parseIso(rt, .local_date_time, args[0], "ISO_LOCAL_DATE_TIME", "java.time.LocalDateTime/parse", loc); } -const ParseError = error{Invalid}; -const Fields = struct { y: i64, m: i64, d: i64, h: i64, mi: i64, s: i64, n: i64 }; - -fn readN(s: []const u8, i: *usize, n: usize) ParseError!i64 { - if (i.* + n > s.len) return error.Invalid; - var acc: i64 = 0; - var k: usize = 0; - while (k < n) : (k += 1) { - const c = s[i.* + k]; - if (c < '0' or c > '9') return error.Invalid; - acc = acc * 10 + (c - '0'); - } - i.* += n; - return acc; +/// `(LocalDateTime/from temporal)` — JVM `LocalDateTime.from(TemporalAccessor)`. +fn from(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("java.time.LocalDateTime/from", args, 1, loc); + return dtf_value.convert(rt, .local_date_time, args[0], loc); } -/// Parse `yyyy-MM-ddTHH:mm[:ss[.fraction]]` (no offset). The year is read as a -/// positive 4-digit field (the e2e only needs positive years). -fn parseLocalDateTime(s: []const u8) ParseError!Fields { - var i: usize = 0; - const y = try readN(s, &i, 4); - if (i >= s.len or s[i] != '-') return error.Invalid; - i += 1; - const m = try readN(s, &i, 2); - if (i >= s.len or s[i] != '-') return error.Invalid; - i += 1; - const d = try readN(s, &i, 2); - if (i >= s.len or s[i] != 'T') return error.Invalid; - i += 1; - const h = try readN(s, &i, 2); - if (i >= s.len or s[i] != ':') return error.Invalid; - i += 1; - const mi = try readN(s, &i, 2); - var sec: i64 = 0; - var nanos: i64 = 0; - if (i < s.len and s[i] == ':') { - i += 1; - sec = try readN(s, &i, 2); - if (i < s.len and s[i] == '.') { - i += 1; - // Fractional seconds: 1-9 digits scaled to nanos (pad / truncate to 9). - var digits: usize = 0; - var acc: i64 = 0; - while (i < s.len and s[i] >= '0' and s[i] <= '9') : (i += 1) { - if (digits < 9) acc = acc * 10 + (s[i] - '0'); - digits += 1; - } - if (digits == 0) return error.Invalid; - while (digits < 9) : (digits += 1) acc *= 10; - nanos = acc; - } - } - if (i != s.len) return error.Invalid; - if (m < 1 or m > 12 or d < 1 or d > 31 or h > 23 or mi > 59 or sec > 59) return error.Invalid; - return .{ .y = y, .m = m, .d = d, .h = h, .mi = mi, .s = sec, .n = nanos }; +/// `(.format x formatter)` — JVM `LocalDateTime.format(DateTimeFormatter)`. +fn formatFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("format", args, 2, loc); + if (!dtf_value.isFormatter(args[1])) + return error_catalog.raise(.type_arg_invalid, loc, .{ .fn_name = ".format", .expected = "a DateTimeFormatter", .actual = dtf_value.classNameOf(args[1]) }); + return dtf_value.formatValue(rt, args[1], args[0], loc); } fn initLocalDateTime(td: *type_descriptor.TypeDescriptor, gpa: std.mem.Allocator) anyerror!void { @@ -149,6 +105,8 @@ fn initLocalDateTime(td: *type_descriptor.TypeDescriptor, gpa: std.mem.Allocator .{ "of", &of }, .{ "now", &now }, .{ "parse", &parse }, + .{ "from", &from }, + .{ "format", &formatFn }, }); } td.temporal_print = .iso_local_date_time; @@ -179,22 +137,3 @@ var descriptor: type_descriptor.TypeDescriptor = .{ .meta = .nil_val, .temporal_print = .iso_local_date_time, }; - -// --- tests --- - -const testing = std.testing; - -test "parseLocalDateTime: with / without seconds + fraction" { - const a = try parseLocalDateTime("2024-01-01T12:30"); - try testing.expectEqual(@as(i64, 2024), a.y); - try testing.expectEqual(@as(i64, 30), a.mi); - try testing.expectEqual(@as(i64, 0), a.s); - try testing.expectEqual(@as(i64, 0), a.n); - const b = try parseLocalDateTime("2024-01-01T12:30:45"); - try testing.expectEqual(@as(i64, 45), b.s); - const c = try parseLocalDateTime("2024-01-01T12:30:45.5"); - try testing.expectEqual(@as(i64, 500_000_000), c.n); - try testing.expectError(error.Invalid, parseLocalDateTime("2024-01-01")); - try testing.expectError(error.Invalid, parseLocalDateTime("not-a-date")); - try testing.expectError(error.Invalid, parseLocalDateTime("2024-13-01T00:00")); -} diff --git a/src/runtime/java/time/LocalTime.zig b/src/runtime/java/time/LocalTime.zig index da4014573..4211bf827 100644 --- a/src/runtime/java/time/LocalTime.zig +++ b/src/runtime/java/time/LocalTime.zig @@ -9,8 +9,9 @@ //! parse) populate `method_table` via the `init` callback (D-462) and mint a //! `.typed_instance` LocalTime VALUE (`runtime/time/local_time_value.zig`) //! whose per-Runtime descriptor carries the instance methods (getHour / -//! getMinute / getSecond / getNano). The time parser is reused from the sibling -//! `runtime/time/instant.zig` (F-009 neutral home). +//! getMinute / getSecond / getNano). `parse` / `from` / `.format` go through +//! the java.time.format engine (`runtime/time/date_time_formatter_value.zig`), +//! so a bad string is the JVM DateTimeParseException. //! //! method_table is populated in `initLocalTime` (runtime) rather than at //! module scope because `Value.initBuiltinFn(&fn)` calls `@intFromPtr(&fn)` @@ -26,7 +27,7 @@ const SourceLocation = @import("../../error/info.zig").SourceLocation; const error_catalog = @import("../../error/catalog.zig"); const instant = @import("../../time/instant.zig"); const lt_value = @import("../../time/local_time_value.zig"); -const string_collection = @import("../../collection/string.zig"); +const dtf_value = @import("../../time/date_time_formatter_value.zig"); const MS_PER_DAY: i64 = 86_400_000; const NANO_PER_SECOND: i64 = 1_000_000_000; @@ -62,18 +63,30 @@ fn now(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerr return lt_value.make(rt, @mod(ms, MS_PER_DAY) * 1_000_000); } -/// `(java.time.LocalTime/parse s)` — parse an ISO local time -/// `HH:mm[:ss[.fraction]]`. JVM throws `DateTimeParseException`; cljw raises the -/// same `inst_string_invalid` the `#inst` reader uses. +/// `(java.time.LocalTime/parse s)` = `parse(s, DateTimeFormatter/ISO_LOCAL_TIME)`, and +/// `(java.time.LocalTime/parse s formatter)` (JVM `LocalTime.parse`). A failure is +/// the JVM DateTimeParseException. fn parse(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { _ = env; - try error_catalog.checkArity("java.time.LocalTime/parse", args, 1, loc); - if (args[0].tag() != .string) - return error_catalog.raise(.type_arg_not_string, loc, .{ .fn_name = "java.time.LocalTime/parse", .actual = @tagName(args[0].tag()) }); - const s = string_collection.asString(args[0]); - const nod = instant.parseLocalTime(s) catch - return error_catalog.raise(.inst_string_invalid, loc, .{ .s = s }); - return lt_value.make(rt, nod); + try error_catalog.checkArityRange("java.time.LocalTime/parse", args, 1, 2, loc); + if (args.len == 2) return dtf_value.parseTo(rt, .local_time, args[0], args[1], "java.time.LocalTime/parse", loc); + return dtf_value.parseIso(rt, .local_time, args[0], "ISO_LOCAL_TIME", "java.time.LocalTime/parse", loc); +} + +/// `(LocalTime/from temporal)` — JVM `LocalTime.from(TemporalAccessor)`. +fn from(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("java.time.LocalTime/from", args, 1, loc); + return dtf_value.convert(rt, .local_time, args[0], loc); +} + +/// `(.format x formatter)` — JVM `LocalTime.format(DateTimeFormatter)`. +fn formatFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("format", args, 2, loc); + if (!dtf_value.isFormatter(args[1])) + return error_catalog.raise(.type_arg_invalid, loc, .{ .fn_name = ".format", .expected = "a DateTimeFormatter", .actual = dtf_value.classNameOf(args[1]) }); + return dtf_value.formatValue(rt, args[1], args[0], loc); } const NANO_PER_DAY: i64 = 86_400_000_000_000; @@ -117,6 +130,8 @@ fn initLocalTime(td: *type_descriptor.TypeDescriptor, gpa: std.mem.Allocator) an .{ "parse", &parse }, .{ "ofSecondOfDay", &ofSecondOfDay }, .{ "ofNanoOfDay", &ofNanoOfDay }, + .{ "from", &from }, + .{ "format", &formatFn }, }); } td.temporal_print = .iso_local_time; diff --git a/src/runtime/java/time/ZoneId.zig b/src/runtime/java/time/ZoneId.zig new file mode 100644 index 000000000..04b93da4d --- /dev/null +++ b/src/runtime/java/time/ZoneId.zig @@ -0,0 +1,68 @@ +// SPDX-License-Identifier: EPL-2.0 +//! Java surface for `java.time.ZoneId` — the abstract zone class: the statics +//! `of` and `systemDefault`. Instances are the ZoneOffset / ZoneRegion values +//! of `runtime/time/zone_offset.zig`, which list `java.time.ZoneId` as their +//! host supertype so `(instance? ZoneId z)` holds. +//! +//! Backend: impl-only +//! Impl deps: zone_offset +//! Clojure peer: none +//! +//! cljw has no time-zone database: `of` resolves offset ids and UTC / GMT / UT +//! (with an optional offset suffix) and refuses a region id such as +//! `Europe/Paris` with `zone_id_unknown`. `systemDefault` is the `UTC` region, +//! matching the UTC-based `LocalDate/now` and friends. + +const std = @import("std"); +const host_api = @import("../_host_api.zig"); +const type_descriptor = @import("../../type_descriptor.zig"); +const Value = @import("../../value/value.zig").Value; +const Runtime = @import("../../runtime.zig").Runtime; +const Env = @import("../../env.zig").Env; +const SourceLocation = @import("../../error/info.zig").SourceLocation; +const error_catalog = @import("../../error/catalog.zig"); +const string_collection = @import("../../collection/string.zig"); +const zone_offset = @import("../../time/zone_offset.zig"); + +/// `(ZoneId/of "UTC")` — JVM `ZoneId.of(String)`. +fn of(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("java.time.ZoneId/of", args, 1, loc); + if (args[0].tag() != .string) + return error_catalog.raise(.type_arg_not_string, loc, .{ .fn_name = "java.time.ZoneId/of", .actual = @tagName(args[0].tag()) }); + const id = string_collection.asString(args[0]); + var diag: zone_offset.Diag = .{}; + const zone = zone_offset.parseZoneId(id, &diag) catch return zone_offset.raiseDiag(&diag, id, loc); + return zone_offset.make(rt, zone); +} + +/// `(ZoneId/systemDefault)` — the `UTC` region (cljw's clock is UTC). +fn systemDefault(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("java.time.ZoneId/systemDefault", args, 0, loc); + return zone_offset.make(rt, .{ .seconds = 0, .prefix = .utc }); +} + +fn initZoneId(td: *type_descriptor.TypeDescriptor, gpa: std.mem.Allocator) anyerror!void { + if (td.method_table.len != 0) return; // idempotent + try type_descriptor.appendMethodEntries(td, gpa, .{ + .{ "of", &of }, + .{ "systemDefault", &systemDefault }, + }); +} + +pub const ___HOST_EXTENSION: host_api.Extension = .{ + .cljw_ns = "cljw.java.time.ZoneId", + .descriptor = &descriptor, + .init = &initZoneId, +}; + +var descriptor: type_descriptor.TypeDescriptor = .{ + .fqcn = "java.time.ZoneId", + .kind = .native, + .field_layout = null, + .protocol_impls = &.{}, + .method_table = &.{}, + .parent = null, + .meta = .nil_val, +}; diff --git a/src/runtime/java/time/ZoneOffset.zig b/src/runtime/java/time/ZoneOffset.zig new file mode 100644 index 000000000..620106731 --- /dev/null +++ b/src/runtime/java/time/ZoneOffset.zig @@ -0,0 +1,117 @@ +// SPDX-License-Identifier: EPL-2.0 +//! Java surface for `java.time.ZoneOffset` — a fixed offset from UTC. +//! +//! Backend: impl-only +//! Impl deps: zone_offset +//! Clojure peer: none +//! +//! The statics (of / ofHours / ofHoursMinutes / ofHoursMinutesSeconds / +//! ofTotalSeconds, UTC / MIN / MAX) mint the `.typed_instance` zone VALUE of +//! `runtime/time/zone_offset.zig`, whose descriptor (this one, ADR-0174 merge) +//! carries the instance methods getId / getTotalSeconds / normalized and the +//! `str` form (the id). Validation texts are the JVM DateTimeException ones. + +const std = @import("std"); +const host_api = @import("../_host_api.zig"); +const type_descriptor = @import("../../type_descriptor.zig"); +const Value = @import("../../value/value.zig").Value; +const Runtime = @import("../../runtime.zig").Runtime; +const Env = @import("../../env.zig").Env; +const SourceLocation = @import("../../error/info.zig").SourceLocation; +const error_catalog = @import("../../error/catalog.zig"); +const string_collection = @import("../../collection/string.zig"); +const zone_offset = @import("../../time/zone_offset.zig"); + +fn intArg(v: Value, fn_name: []const u8, loc: SourceLocation) !i64 { + if (v.tag() != .integer) + return error_catalog.raise(.type_arg_invalid, loc, .{ .fn_name = fn_name, .expected = "integer", .actual = @tagName(v.tag()) }); + return v.asInteger(); +} + +/// Mint from validated h/m/s; a validation failure raises its DateTimeException. +fn fromHms(rt: *Runtime, h: i64, m: i64, s: i64, loc: SourceLocation) !Value { + var diag: zone_offset.Diag = .{}; + const secs = zone_offset.ofHoursMinutesSeconds(h, m, s, &diag) catch + return error_catalog.raise(.date_time_value_invalid, loc, .{ .detail = diag.msg() }); + return zone_offset.make(rt, .{ .seconds = secs }); +} + +/// `(ZoneOffset/of "+02:00")` — JVM `ZoneOffset.of(String)`. +fn of(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("java.time.ZoneOffset/of", args, 1, loc); + if (args[0].tag() != .string) + return error_catalog.raise(.type_arg_not_string, loc, .{ .fn_name = "java.time.ZoneOffset/of", .actual = @tagName(args[0].tag()) }); + const id = string_collection.asString(args[0]); + var diag: zone_offset.Diag = .{}; + const secs = zone_offset.parseOffsetId(id, &diag) catch return zone_offset.raiseDiag(&diag, id, loc); + return zone_offset.make(rt, .{ .seconds = secs }); +} + +fn ofHours(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("java.time.ZoneOffset/ofHours", args, 1, loc); + return fromHms(rt, try intArg(args[0], "java.time.ZoneOffset/ofHours", loc), 0, 0, loc); +} + +fn ofHoursMinutes(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("java.time.ZoneOffset/ofHoursMinutes", args, 2, loc); + const name = "java.time.ZoneOffset/ofHoursMinutes"; + return fromHms(rt, try intArg(args[0], name, loc), try intArg(args[1], name, loc), 0, loc); +} + +fn ofHoursMinutesSeconds(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("java.time.ZoneOffset/ofHoursMinutesSeconds", args, 3, loc); + const name = "java.time.ZoneOffset/ofHoursMinutesSeconds"; + return fromHms(rt, try intArg(args[0], name, loc), try intArg(args[1], name, loc), try intArg(args[2], name, loc), loc); +} + +fn ofTotalSeconds(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("java.time.ZoneOffset/ofTotalSeconds", args, 1, loc); + var diag: zone_offset.Diag = .{}; + const secs = zone_offset.ofTotalSeconds(try intArg(args[0], "java.time.ZoneOffset/ofTotalSeconds", loc), &diag) catch + return error_catalog.raise(.date_time_value_invalid, loc, .{ .detail = diag.msg() }); + return zone_offset.make(rt, .{ .seconds = secs }); +} + +fn initZoneOffset(td: *type_descriptor.TypeDescriptor, gpa: std.mem.Allocator) anyerror!void { + if (td.lookupMethod(null, "ofHours") == null) { + try type_descriptor.appendMethodEntries(td, gpa, .{ + .{ "of", &of }, + .{ "ofHours", &ofHours }, + .{ "ofHoursMinutes", &ofHoursMinutes }, + .{ "ofHoursMinutesSeconds", &ofHoursMinutesSeconds }, + .{ "ofTotalSeconds", &ofTotalSeconds }, + }); + } + try zone_offset.ensureOffsetMethods(td, gpa); +} + +pub const ___HOST_EXTENSION: host_api.Extension = .{ + .cljw_ns = "cljw.java.time.ZoneOffset", + .descriptor = &descriptor, + .init = &initZoneOffset, +}; + +const static_fields = [_]type_descriptor.TypeDescriptor.StaticField{ + .{ .name = "UTC", .value = .{ .singleton = .time_zone_offset_utc } }, + .{ .name = "MIN", .value = .{ .singleton = .time_zone_offset_min } }, + .{ .name = "MAX", .value = .{ .singleton = .time_zone_offset_max } }, +}; + +const supertypes = [_][]const u8{"java.time.ZoneId"}; + +var descriptor: type_descriptor.TypeDescriptor = .{ + .fqcn = zone_offset.OFFSET_FQCN, // the ONE canonical key (ADR-0174) + .kind = .native, + .field_layout = null, + .protocol_impls = &.{}, + .method_table = &.{}, + .static_fields = &static_fields, + .host_supertypes = &supertypes, + .parent = null, + .meta = .nil_val, +}; diff --git a/src/runtime/java/time/format/DateTimeFormatter.zig b/src/runtime/java/time/format/DateTimeFormatter.zig new file mode 100644 index 000000000..a63232b99 --- /dev/null +++ b/src/runtime/java/time/format/DateTimeFormatter.zig @@ -0,0 +1,72 @@ +// SPDX-License-Identifier: EPL-2.0 +//! Java surface for `java.time.format.DateTimeFormatter`. +//! +//! Backend: impl-only +//! Impl deps: date_time_format +//! Clojure peer: none +//! +//! Statics: `ofPattern` (a SMART formatter compiled from a pattern string; the +//! 2-arity Locale form is accepted, cljw's only locale being English) and the +//! STRICT `ISO_*` / `BASIC_ISO_DATE` constants. Instances are the +//! `.host_instance` formatters of `runtime/time/date_time_formatter_value.zig`, +//! which carry the instance methods (format / parse / withZone / getZone / +//! toString) on this same descriptor (ADR-0174 merge). The engine is +//! `runtime/time/date_time_format.zig`. + +const std = @import("std"); +const host_api = @import("../../_host_api.zig"); +const type_descriptor = @import("../../../type_descriptor.zig"); +const Value = @import("../../../value/value.zig").Value; +const Runtime = @import("../../../runtime.zig").Runtime; +const Env = @import("../../../env.zig").Env; +const SourceLocation = @import("../../../error/info.zig").SourceLocation; +const error_catalog = @import("../../../error/catalog.zig"); +const string_collection = @import("../../../collection/string.zig"); +const engine = @import("../../../time/date_time_format.zig"); +const dtf_value = @import("../../../time/date_time_formatter_value.zig"); + +/// `(DateTimeFormatter/ofPattern "yyyy-MM-dd")` / `(… pattern locale)`. +fn ofPattern(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArityRange("java.time.format.DateTimeFormatter/ofPattern", args, 1, 2, loc); + if (args[0].tag() != .string) + return error_catalog.raise(.type_arg_not_string, loc, .{ .fn_name = "java.time.format.DateTimeFormatter/ofPattern", .actual = @tagName(args[0].tag()) }); + var diag: engine.Diag = .{}; + const p = engine.compilePattern(rt.gc.infra, string_collection.asString(args[0]), &diag) catch |e| switch (e) { + error.OutOfMemory => return error.OutOfMemory, + error.DateTime => return dtf_value.raiseDiag(&diag, loc), + }; + return dtf_value.wrapProgram(rt, p); +} + +fn initDescriptor(td: *type_descriptor.TypeDescriptor, gpa: std.mem.Allocator) anyerror!void { + if (td.lookupMethod(null, "ofPattern") == null) + try type_descriptor.appendMethodEntries(td, gpa, .{.{ "ofPattern", &ofPattern }}); + try dtf_value.ensureFormatterMethods(td, gpa); +} + +pub const ___HOST_EXTENSION: host_api.Extension = .{ + .cljw_ns = "cljw.java.time.format.DateTimeFormatter", + .descriptor = &descriptor, + .init = &initDescriptor, +}; + +/// The ISO constants, generated from the engine's table (index = arm payload). +const static_fields = build: { + var arr: [engine.ISO_CONSTANTS.len]type_descriptor.TypeDescriptor.StaticField = undefined; + for (&arr, engine.ISO_CONSTANTS, 0..) |*sf, c, i| { + sf.* = .{ .name = c.name, .value = .{ .date_time_formatter = @intCast(i) } }; + } + break :build arr; +}; + +var descriptor: type_descriptor.TypeDescriptor = .{ + .fqcn = dtf_value.FORMATTER_FQCN, // the ONE canonical key (ADR-0174) + .kind = .native, + .field_layout = null, + .protocol_impls = &.{}, + .method_table = &.{}, + .static_fields = &static_fields, + .parent = null, + .meta = .nil_val, +}; diff --git a/src/runtime/java/time/format/DateTimeFormatterBuilder.zig b/src/runtime/java/time/format/DateTimeFormatterBuilder.zig new file mode 100644 index 000000000..c194f654a --- /dev/null +++ b/src/runtime/java/time/format/DateTimeFormatterBuilder.zig @@ -0,0 +1,51 @@ +// SPDX-License-Identifier: EPL-2.0 +//! Java surface for `java.time.format.DateTimeFormatterBuilder` — the fluent +//! builder `(DateTimeFormatterBuilder.)`; each `append…` / `optional…` / +//! `parse…` call mutates the builder and returns it, `toFormatter` freezes a +//! SMART `DateTimeFormatter`. +//! +//! Backend: impl-only +//! Impl deps: date_time_format +//! Clojure peer: none +//! +//! The instance methods live in `runtime/time/date_time_formatter_value.zig` +//! (registered onto this descriptor by `init`); this file owns ``. + +const std = @import("std"); +const host_api = @import("../../_host_api.zig"); +const type_descriptor = @import("../../../type_descriptor.zig"); +const Value = @import("../../../value/value.zig").Value; +const Runtime = @import("../../../runtime.zig").Runtime; +const Env = @import("../../../env.zig").Env; +const SourceLocation = @import("../../../error/info.zig").SourceLocation; +const error_catalog = @import("../../../error/catalog.zig"); +const dtf_value = @import("../../../time/date_time_formatter_value.zig"); + +/// `(DateTimeFormatterBuilder.)` — an empty builder. +fn initBuilder(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("DateTimeFormatterBuilder", args, 0, loc); + return dtf_value.newBuilder(rt); +} + +fn initDescriptor(td: *type_descriptor.TypeDescriptor, gpa: std.mem.Allocator) anyerror!void { + if (td.lookupMethod(null, "") == null) + try type_descriptor.appendMethodEntries(td, gpa, .{.{ "", &initBuilder }}); + try dtf_value.ensureBuilderMethods(td, gpa); +} + +pub const ___HOST_EXTENSION: host_api.Extension = .{ + .cljw_ns = "cljw.java.time.format.DateTimeFormatterBuilder", + .descriptor = &descriptor, + .init = &initDescriptor, +}; + +var descriptor: type_descriptor.TypeDescriptor = .{ + .fqcn = dtf_value.BUILDER_FQCN, // the ONE canonical key (ADR-0174) + .kind = .native, + .field_layout = null, + .protocol_impls = &.{}, + .method_table = &.{}, + .parent = null, + .meta = .nil_val, +}; diff --git a/src/runtime/java/time/temporal/ChronoField.zig b/src/runtime/java/time/temporal/ChronoField.zig new file mode 100644 index 000000000..0712f32f9 --- /dev/null +++ b/src/runtime/java/time/temporal/ChronoField.zig @@ -0,0 +1,84 @@ +// SPDX-License-Identifier: EPL-2.0 +//! Java surface for `java.time.temporal.ChronoField` — the 30 enum constants +//! (NANO_OF_SECOND..OFFSET_SECONDS) as host-enum singletons + `.toString` (the +//! display name, "HourOfDay") / `.name` (the enum name, "HOUR_OF_DAY"). The +//! field argument of `DateTimeFormatterBuilder` (`appendValue`, `appendFraction`, +//! `parseDefaulting`) and of `Parsed.getLong` / `isSupported`. +//! +//! Backend: impl-only +//! Impl deps: host_enum +//! Clojure peer: none. +//! +//! Singletons + the ordinal↔name↔display mapping live in the neutral host-enum +//! registry `runtime/host_enum.zig`; the per-field ranges and the parse/format +//! semantics live in `runtime/time/date_time_format.zig::Field`, which mirrors +//! the registry order. + +const std = @import("std"); +const host_api = @import("../../_host_api.zig"); +const type_descriptor = @import("../../../type_descriptor.zig"); +const Value = @import("../../../value/value.zig").Value; +const Runtime = @import("../../../runtime.zig").Runtime; +const Env = @import("../../../env.zig").Env; +const SourceLocation = @import("../../../error/info.zig").SourceLocation; +const error_catalog = @import("../../../error/catalog.zig"); +const host_instance = @import("../../../host_instance.zig"); +const string_collection = @import("../../../collection/string.zig"); +const host_enum = @import("../../../host_enum.zig"); + +/// `(str f)` / `(.toString f)` — the DISPLAY name ("HourOfDay"). +fn toString(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("toString", args, 1, loc); + const ord: u8 = @intCast(host_instance.asHostInstance(args[0]).state[0]); + return string_collection.alloc(rt, host_enum.toStringOf(.chrono_field, ord)); +} + +/// `(.name f)` — the enum-constant name ("HOUR_OF_DAY"). +fn nameFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("name", args, 1, loc); + const ord: u8 = @intCast(host_instance.asHostInstance(args[0]).state[0]); + return string_collection.alloc(rt, host_enum.name(.chrono_field, ord)); +} + +fn initDescriptor(td: *type_descriptor.TypeDescriptor, gpa: std.mem.Allocator) anyerror!void { + if (td.method_table.len != 0) return; // idempotent + const statics = host_enum.Statics(.chrono_field); + try type_descriptor.appendMethodEntries(td, gpa, .{ + .{ "toString", &toString }, + .{ "name", &nameFn }, + .{ "values", &statics.values }, + .{ "valueOf", &statics.valueOf }, + }); +} + +/// The 30 enum constants, generated from the canonical `host_enum` registry. +const static_fields = build: { + var arr: [host_enum.count(.chrono_field)]type_descriptor.TypeDescriptor.StaticField = undefined; + for (&arr, 0..) |*sf, i| { + sf.* = .{ + .name = host_enum.name(.chrono_field, @intCast(i)), + .value = .{ .host_enum = .{ .enum_idx = @intFromEnum(host_enum.Idx.chrono_field), .ordinal = @intCast(i) } }, + }; + } + break :build arr; +}; + +pub const ___HOST_EXTENSION: host_api.Extension = .{ + .cljw_ns = "cljw.java.time.temporal.ChronoField", + .descriptor = &descriptor, + .init = &initDescriptor, +}; + +var descriptor: type_descriptor.TypeDescriptor = .{ + .fqcn = "java.time.temporal.ChronoField", + .kind = .native, + .field_layout = null, + .protocol_impls = &.{}, + .method_table = &.{}, + .static_fields = &static_fields, + .parent = null, + .meta = .nil_val, + .host_enum_idx = @intFromEnum(host_enum.Idx.chrono_field), +}; diff --git a/src/runtime/java/util/Arrays.zig b/src/runtime/java/util/Arrays.zig index d2325cc17..3a2c616d7 100644 --- a/src/runtime/java/util/Arrays.zig +++ b/src/runtime/java/util/Arrays.zig @@ -5,13 +5,14 @@ //! Impl deps: java_array //! Clojure peer: none //! -//! The common static surface over cljw's `.array` tag (ADR-0105 JavaArray — -//! Object[] semantics; Value slots): `asList` (an ArrayList host instance, -//! the D-294 surface) / `toString` ("[e1, e2, …]") / `copyOf` (nil-padded -//! like Java's null pad) / `sort` (in-place, natural `compare` order) / -//! `equals` (element value equality) / `fill`. `binarySearch` / `hashCode` / -//! the typed-primitive-array overload family are intentionally omitted until -//! a real consumer needs them (dead-interop lesson). +//! The common static surface over cljw's `.array` tag (ADR-0105 JavaArray, +//! typed per ADR-0204): `asList` (an ArrayList host instance, the D-294 +//! surface) / `toString` ("[e1, e2, …]") / `copyOf` (padded with the kind's +//! default, Java's null / 0 / false) / `sort` (in place: natural `compare` +//! order for an object array, numeric order for a primitive one, `-0.0` +//! before `0.0` and NaN last as `Double.compare`) / `equals` (same kind, same +//! length, equal elements) / `fill`. `binarySearch` / `hashCode` are +//! intentionally omitted until a real consumer needs them (dead-interop lesson). const std = @import("std"); const value_sort = @import("../../collection/value_sort.zig"); @@ -43,7 +44,7 @@ fn expectArray(v: Value, fn_name: []const u8, loc: SourceLocation) !void { fn asList(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { try error_catalog.checkArity("java.util.Arrays/asList", args, 1, loc); try expectArray(args[0], "java.util.Arrays/asList", loc); - const vec = try vector_mod.fromSlice(rt, java_array.asArray(args[0]).items()); + const vec = try java_array.toVector(rt, java_array.asArray(args[0])); return array_list_surface.initArrayList(rt, env, &.{vec}, loc); } @@ -56,7 +57,9 @@ fn toStringFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) var aw: std.Io.Writer.Allocating = .init(rt.gpa); defer aw.deinit(); try aw.writer.writeByte('['); - for (java_array.asArray(args[0]).items(), 0..) |e, i| { + const arr = java_array.asArray(args[0]); + for (0..arr.len) |i| { + const e = try java_array.get(rt, arr, i); if (i > 0) try aw.writer.writeAll(", "); if (e.tag() == .nil) { try aw.writer.writeAll("null"); // Java String.valueOf(null) @@ -68,8 +71,9 @@ fn toStringFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) return string_mod.alloc(rt, aw.writer.buffered()); } -/// `(java.util.Arrays/copyOf arr n)` — a fresh array of length `n`: truncated -/// or nil-padded (Java pads Object[] with null). JVM ref: Arrays#copyOf. +/// `(java.util.Arrays/copyOf arr n)` — a fresh array of the same kind and +/// length `n`: truncated, or padded with the kind's default (Java's null / 0 / +/// false). JVM ref: Arrays#copyOf. fn copyOf(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { _ = env; try error_catalog.checkArity("java.util.Arrays/copyOf", args, 2, loc); @@ -78,11 +82,9 @@ fn copyOf(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) any if (n_i64 < 0) return error_catalog.raise(.type_arg_invalid, loc, .{ .fn_name = "java.util.Arrays/copyOf", .expected = "a non-negative length", .actual = "negative int" }); const n: u32 = @intCast(n_i64); - const out = try java_array.make(rt, n, .nil_val); - const src = java_array.asArray(args[0]).items(); - const dst = java_array.asArray(out).items(); - const keep = @min(src.len, dst.len); - @memcpy(dst[0..keep], src[0..keep]); + const src = java_array.asArray(args[0]); + const out = try java_array.make(rt, src.kind, n); + java_array.copy(src, 0, java_array.asArray(out), 0, @min(src.len, n)); return out; } @@ -102,20 +104,55 @@ fn arrayLessThan(opaque_ctx: *anyopaque, a: Value, b: Value) bool { return ord == .lt; } -/// `(java.util.Arrays/sort arr)` — IN-PLACE natural-order sort (cljw -/// `compare`), returning nil like the void Java method. Stable block sort. +/// `Double.compare` order: `-0.0` before `0.0`, every NaN after +Inf. +fn floatLessThan(comptime T: type) fn (void, T, T) bool { + return struct { + fn lt(_: void, a: T, b: T) bool { + if (std.math.isNan(a)) return false; + if (std.math.isNan(b)) return true; + if (a == b) return std.math.signbit(a) and !std.math.signbit(b); + return a < b; + } + }.lt; +} + +/// `(java.util.Arrays/sort arr)` — IN-PLACE sort returning nil like the void +/// Java method. An object array sorts by cljw `compare` (stable block sort), +/// a primitive one by element value. fn sortFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { _ = env; try error_catalog.checkArity("java.util.Arrays/sort", args, 1, loc); try expectArray(args[0], "java.util.Arrays/sort", loc); - var ctx: SortCtx = .{ .rt = rt, .loc = loc }; - value_sort.sort(java_array.asArray(args[0]).items(), @ptrCast(&ctx), &arrayLessThan); - if (ctx.err) |e| return e; + const arr = java_array.asArray(args[0]); + switch (arr.kind) { + .object => { + var ctx: SortCtx = .{ .rt = rt, .loc = loc }; + value_sort.sort(arr.objects(), @ptrCast(&ctx), &arrayLessThan); + if (ctx.err) |e| return e; + }, + inline .float, .double => |k| std.mem.sort(k.Elem(), arr.slice(k), {}, floatLessThan(k.Elem())), + inline else => |k| std.mem.sort(k.Elem(), arr.slice(k), {}, std.sort.asc(k.Elem())), + } return .nil_val; } -/// `(java.util.Arrays/equals a b)` — same length + element value equality -/// (nil-safe: two nils true, one nil false). JVM ref: Arrays#equals. +/// Element equality of two primitive arrays of kind `k`; a float compares as +/// `Double.equals` (bit equality with every NaN equal). +fn primEqual(comptime k: java_array.Kind, a: *const java_array.JavaArray, b: *const java_array.JavaArray) bool { + const T = k.Elem(); + if (comptime T == f32 or T == f64) { + const U = std.meta.Int(.unsigned, @bitSizeOf(T)); + for (a.slice(k), b.slice(k)) |x, y| { + if (std.math.isNan(x) and std.math.isNan(y)) continue; + if (@as(U, @bitCast(x)) != @as(U, @bitCast(y))) return false; + } + return true; + } else return std.mem.eql(u8, a.bytes(), b.bytes()); +} + +/// `(java.util.Arrays/equals a b)` — same kind, same length, element value +/// equality (nil-safe: two nils true, one nil false). Arrays of different +/// kinds are unequal (no Java overload takes two). JVM ref: Arrays#equals. fn equalsFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { try error_catalog.checkArity("java.util.Arrays/equals", args, 2, loc); const a = args[0]; @@ -124,11 +161,14 @@ fn equalsFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) a if (a.tag() == .nil or b.tag() == .nil) return .false_val; try expectArray(a, "java.util.Arrays/equals", loc); try expectArray(b, "java.util.Arrays/equals", loc); - const as = java_array.asArray(a).items(); - const bs = java_array.asArray(b).items(); - if (as.len != bs.len) return .false_val; - for (as, bs) |x, y| { - if (!try equal.valueEqual(rt, env, x, y)) return .false_val; + const aa = java_array.asArray(a); + const ba = java_array.asArray(b); + if (aa.kind != ba.kind or aa.len != ba.len) return .false_val; + switch (aa.kind) { + .object => for (aa.objects(), ba.objects()) |x, y| { + if (!try equal.valueEqual(rt, env, x, y)) return .false_val; + }, + inline else => |k| if (!primEqual(k, aa, ba)) return .false_val, } return .true_val; } @@ -140,7 +180,7 @@ fn fillFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) any _ = env; try error_catalog.checkArity("java.util.Arrays/fill", args, 2, loc); try expectArray(args[0], "java.util.Arrays/fill", loc); - @memset(java_array.asArray(args[0]).items(), args[1]); + try java_array.fill(java_array.asArray(args[0]), args[1], .strict, "java.util.Arrays/fill", loc); return .nil_val; } diff --git a/src/runtime/java/util/Collections.zig b/src/runtime/java/util/Collections.zig index c70d62ed6..79a56a552 100644 --- a/src/runtime/java/util/Collections.zig +++ b/src/runtime/java/util/Collections.zig @@ -98,7 +98,10 @@ fn elemsOf(rt: *Runtime, v: Value, fn_name: []const u8, loc: SourceLocation) !El while (i < n) : (i += 1) buf[i] = vector_mod.nth(v, i); return .{ .items = buf, .owned = buf }; } - if (java_array.isArray(v)) return .{ .items = java_array.asArray(v).items() }; + if (java_array.isArray(v)) { + const buf = try java_array.boxAll(rt, java_array.asArray(v), rt.gpa); + return .{ .items = buf, .owned = buf }; + } if (array_list_surface.isArrayList(v)) return .{ .items = array_list_surface.itemsOf(v) }; return error_catalog.raise(.type_arg_invalid, loc, .{ .fn_name = fn_name, .expected = "a vector, Java array, or ArrayList", .actual = @tagName(v.tag()) }); } diff --git a/src/runtime/java/util/Date.zig b/src/runtime/java/util/Date.zig index 38d8d21d9..36847c310 100644 --- a/src/runtime/java/util/Date.zig +++ b/src/runtime/java/util/Date.zig @@ -23,6 +23,7 @@ const SourceLocation = @import("../../error/info.zig").SourceLocation; const error_catalog = @import("../../error/catalog.zig"); const date_impl = @import("../../time/date.zig"); const instant = @import("../../time/instant.zig"); +const instant_value = @import("../../time/instant_value.zig"); /// `(java.util.Date.)` → a Date for the current epoch-ms (wall clock). /// `(java.util.Date. ms)` → a Date for the given epoch-ms (long). @@ -37,6 +38,17 @@ fn dateCtor(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) a return error_catalog.raise(.arity_not_expected, loc, .{ .got = args.len, .fn_name = "java.util.Date.", .expected = 1 }); } +/// `(java.util.Date/from instant)` — JVM `Date.from(Instant)`: the epoch +/// millis of the instant (sub-millisecond nanos truncated toward the past). +fn from(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("java.util.Date/from", args, 1, loc); + if (!instant_value.isInstant(args[0])) + return error_catalog.raise(.type_arg_invalid, loc, .{ .fn_name = "java.util.Date/from", .expected = "an Instant", .actual = @tagName(args[0].tag()) }); + const secs = @divFloor(instant_value.epochMsOf(args[0]), 1000); + return date_impl.make(rt, secs * 1000 + @divFloor(instant_value.nanosOf(args[0]), 1_000_000)); +} + fn initDate(td: *type_descriptor.TypeDescriptor, gpa: std.mem.Allocator) anyerror!void { // ADR-0174 merge: ONE descriptor carries the ctor AND the instance // methods (Date values point at it). Each side appends behind its own @@ -45,6 +57,7 @@ fn initDate(td: *type_descriptor.TypeDescriptor, gpa: std.mem.Allocator) anyerro if (td.lookupMethod(null, "") == null) { try type_descriptor.appendMethodEntries(td, gpa, .{ .{ "", &dateCtor }, + .{ "from", &from }, }); } td.print_tag = "inst"; // impl-first path sets it too; assign is idempotent diff --git a/src/runtime/keyword.zig b/src/runtime/keyword.zig index 931471e2a..c9b2f478c 100644 --- a/src/runtime/keyword.zig +++ b/src/runtime/keyword.zig @@ -81,6 +81,13 @@ pub const KeywordInterner = struct { /// already hold the lock or are running in a known-single-threaded /// path (tests, fixed-input bootstrap). pub fn internUnlocked(self: *KeywordInterner, ns: ?[]const u8, name_: []const u8) !Value { + // PERF: probe with a stack-built key first; the hit path (re-interning + // an existing keyword, e.g. `(keyword (str "k" i))` in a loop) then + // allocates nothing. [refs: O-076] + var kbuf: [128]u8 = undefined; + if (stackKey(&kbuf, ns, name_)) |probe| { + if (self.table.get(probe)) |existing| return Value.encodeHeapPtr(.keyword, existing); + } const key = try formatKey(self.alloc, ns, name_); if (self.table.get(key)) |existing| { @@ -103,6 +110,10 @@ pub const KeywordInterner = struct { /// Low-level lookup — does not lock. See `internUnlocked` for /// when to prefer this over the rt-aware top-level `find`. pub fn findUnlocked(self: *KeywordInterner, ns: ?[]const u8, name_: []const u8) ?Value { + var kbuf: [128]u8 = undefined; + if (stackKey(&kbuf, ns, name_)) |probe| { + return if (self.table.get(probe)) |kw| Value.encodeHeapPtr(.keyword, kw) else null; + } const key = formatKey(self.alloc, ns, name_) catch return null; defer self.alloc.free(key); @@ -136,6 +147,20 @@ pub fn asKeyword(val: Value) *const Keyword { // --- internal helpers --- +/// `formatKey` into a caller buffer; null when the key does not fit. +fn stackKey(buf: []u8, ns: ?[]const u8, name_: []const u8) ?[]const u8 { + if (ns) |n| { + const len = n.len + 1 + name_.len; + if (len > buf.len) return null; + @memcpy(buf[0..n.len], n); + buf[n.len] = '/'; + @memcpy(buf[n.len + 1 ..][0..name_.len], name_); + return buf[0..len]; + } + // No ns: the key IS the name; no copy needed. + return name_; +} + fn formatKey(alloc: std.mem.Allocator, ns: ?[]const u8, name_: []const u8) ![]u8 { if (ns) |n| { const key = try alloc.alloc(u8, n.len + 1 + name_.len); diff --git a/src/runtime/numeric/big_decimal.zig b/src/runtime/numeric/big_decimal.zig index eea747775..00dd1ff9a 100644 --- a/src/runtime/numeric/big_decimal.zig +++ b/src/runtime/numeric/big_decimal.zig @@ -125,7 +125,7 @@ pub fn allocFromManagedScale( /// Parse a plain/scientific decimal string (`"-1.50"`, `"1.0E10"`) into a /// BigDecimal, or `null` when the text is not a finite decimal numeral -/// (empty, double dot, non-digit, pathologically wide, or |scale| > i32). +/// (empty, double dot, non-digit, or |scale| > i32). /// The scale is `frac_digits - exponent`. Shared by `clojure.core/bigdec` /// (string + float arms) and `BigDecimal/valueOf` (the float arm formats /// via `printFloat` first). Lives here — with the BigDecimal representation @@ -149,7 +149,9 @@ pub fn allocFromDecimalString(rt: *Runtime, s: []const u8) !?Value { if (mant.len == 0) return null; // Collect mantissa digits (drop the dot); count fractional digits. - var digits: [512]u8 = undefined; + // Sized to the mantissa, so the significand has no width ceiling. + const digits = try rt.gc.infra.alloc(u8, mant.len); + defer rt.gc.infra.free(digits); var dlen: usize = 0; var frac: i64 = 0; var seen_dot = false; @@ -161,7 +163,6 @@ pub fn allocFromDecimalString(rt: *Runtime, s: []const u8) !?Value { continue; } if (c < '0' or c > '9') return null; - if (dlen >= digits.len) return null; // pathologically wide — defer digits[dlen] = c; dlen += 1; any_digit = true; diff --git a/src/runtime/numeric/big_int.zig b/src/runtime/numeric/big_int.zig index 44b5a189f..f8773c401 100644 --- a/src/runtime/numeric/big_int.zig +++ b/src/runtime/numeric/big_int.zig @@ -127,10 +127,8 @@ pub fn allocFromManaged(rt: *Runtime, src: *const std.math.big.int.Managed, orig const m_ptr = try rt.gc.infra.create(std.math.big.int.Managed); errdefer rt.gc.infra.destroy(m_ptr); m_ptr.* = try src.cloneWithDifferentAllocator(rt.gc.infra); - - const bi = try rt.gc.alloc(BigInt); - bi.* = .{ .header = HeapHeader.init(.big_int), .origin = origin, .m = m_ptr }; - return Value.encodeHeapPtr(.big_int, bi); + errdefer m_ptr.deinit(); + return wrapManaged(rt, m_ptr, origin); } /// Wrap an already-computed, `gc.infra`-owned `*Managed` as a GC BigInt Value @@ -144,6 +142,8 @@ pub fn allocFromManaged(rt: *Runtime, src: *const std.math.big.int.Managed, orig pub fn wrapManaged(rt: *Runtime, m_ptr: *std.math.big.int.Managed, origin: IntOrigin) !Value { const bi = try rt.gc.alloc(BigInt); bi.* = .{ .header = HeapHeader.init(.big_int), .origin = origin, .m = m_ptr }; + // CLJW-GC-HEAP-RETURN: the limbs live on infra, invisible to the trigger. + rt.gc.noteOwnedAlloc(ownedBytes(&bi.header)); return Value.encodeHeapPtr(.big_int, bi); } diff --git a/src/runtime/numeric/decimal_text.zig b/src/runtime/numeric/decimal_text.zig new file mode 100644 index 000000000..33aa28bac --- /dev/null +++ b/src/runtime/numeric/decimal_text.zig @@ -0,0 +1,82 @@ +// SPDX-License-Identifier: EPL-2.0 +//! Base-10 text of an arbitrary-precision integer, with no size ceiling. +//! +//! std's `Const.format` (the `{f}` spec on a `Managed` / `Const`) renders +//! through a fixed stack buffer of 64 limbs (Zig 0.16) and, past it, writes the +//! literal "(BigInt)" instead of the digits. 64 limbs is about 1233 decimal +//! digits, so `(str (reduce *' (range 1N 1000N)))` printed "(BigInt)" +//! (CLJW-BIGINT-STR-PLACEHOLDER). Every BigInt / Ratio / BigDecimal print path +//! renders through `writeDecimal` instead of `{f}`. + +const std = @import("std"); +const Managed = std.math.big.int.Managed; + +/// Largest limb count std's `Const.format` renders without the placeholder. +const std_format_max_limbs = 64; + +/// Write the base-10 digits of `m` (leading `-` when negative) to `w`. A value +/// within std's stack bound renders there; a wider one through a scratch +/// string on the Managed's own allocator. +pub fn writeDecimal(w: *std.Io.Writer, m: *const Managed) !void { + const c = m.toConst(); + if (c.limbs.len <= std_format_max_limbs) return c.format(w); + const s = try c.toStringAlloc(m.allocator, 10, .lower); + defer m.allocator.free(s); + try w.writeAll(s); +} + +/// Base-10 digits of `m` as a fresh string on `alloc` (the allocating twin of +/// `writeDecimal`). Caller owns the result. +pub fn decimalAlloc(alloc: std.mem.Allocator, m: *const Managed) ![]u8 { + return m.toConst().toStringAlloc(alloc, 10, .lower); +} + +/// Write `n/d` (a big Ratio's reader text) to `w`, no size ceiling. +pub fn writeFraction(w: *std.Io.Writer, n: *const Managed, d: *const Managed) !void { + try writeDecimal(w, n); + try w.writeByte('/'); + try writeDecimal(w, d); +} + +/// `n/d` as a fresh string on `alloc`. Caller owns the result. +pub fn fractionAlloc(alloc: std.mem.Allocator, n: *const Managed, d: *const Managed) ![]u8 { + var aw: std.Io.Writer.Allocating = .init(alloc); + errdefer aw.deinit(); + writeFraction(&aw.writer, n, d) catch return error.OutOfMemory; + return aw.toOwnedSlice(); +} + +const testing = std.testing; + +test "writeDecimal renders a value past std's 64-limb format bound" { + // 10^3000: 3001 digits, far past the 64-limb (about 1233 digit) bound. + var m = try Managed.initSet(testing.allocator, 1); + defer m.deinit(); + var ten = try Managed.initSet(testing.allocator, 10); + defer ten.deinit(); + for (0..3000) |_| try m.mul(&m, &ten); + try testing.expect(m.toConst().limbs.len > std_format_max_limbs); + + var aw: std.Io.Writer.Allocating = .init(testing.allocator); + defer aw.deinit(); + try writeDecimal(&aw.writer, &m); + const out = aw.written(); + try testing.expectEqual(@as(usize, 3001), out.len); + try testing.expectEqual(@as(u8, '1'), out[0]); + for (out[1..]) |ch| try testing.expectEqual(@as(u8, '0'), ch); + + m.negate(); + aw.clearRetainingCapacity(); + try writeDecimal(&aw.writer, &m); + try testing.expectEqual(@as(usize, 3002), aw.written().len); + try testing.expectEqual(@as(u8, '-'), aw.written()[0]); +} + +test "writeDecimal small values match std format" { + var m = try Managed.initSet(testing.allocator, -12345); + defer m.deinit(); + var aw: std.Io.Writer.Allocating = .init(testing.allocator); + defer aw.deinit(); + try writeDecimal(&aw.writer, &m); + try testing.expectEqualStrings("-12345", aw.written()); +} diff --git a/src/runtime/numeric/promote.zig b/src/runtime/numeric/promote.zig index 0de834f80..d9611a304 100644 --- a/src/runtime/numeric/promote.zig +++ b/src/runtime/numeric/promote.zig @@ -32,7 +32,7 @@ const host_instance = @import("../host_instance.zig"); const Managed = std.math.big.int.Managed; -fn inI48(x: i64) bool { +pub fn inI48(x: i64) bool { return x >= nb.NB_I48_MIN and x <= nb.NB_I48_MAX; } diff --git a/src/runtime/print.zig b/src/runtime/print.zig index e1794082d..058863060 100644 --- a/src/runtime/print.zig +++ b/src/runtime/print.zig @@ -55,6 +55,7 @@ const persistent_queue = @import("collection/persistent_queue.zig"); const sorted_collection = @import("collection/sorted.zig"); const ex_info_collection = @import("collection/ex_info.zig"); const big_int_mod = @import("numeric/big_int.zig"); +const decimal_text = @import("numeric/decimal_text.zig"); const ratio_mod = @import("numeric/ratio.zig"); const big_decimal_mod = @import("numeric/big_decimal.zig"); const regex_mod = @import("regex/value.zig"); @@ -178,7 +179,7 @@ pub fn writeStrValue(rt: *Runtime, env: *env_mod.Env, w: *Writer, v: Value) anye // `str`/`.toString` of a BigInt/BigDecimal drops the `N`/`M` reader // suffix that `pr`/`prn` keep — JVM `BigInteger`/`BigDecimal.toString` // emit plain digits. Ratio's `1/2` form is already suffix-free. - .big_int => try w.print("{f}", .{big_int_mod.asManaged(v)}), + .big_int => try decimal_text.writeDecimal(w, big_int_mod.asManaged(v)), .big_decimal => try writeBigDecimalDigits(w, v), // A deftype/reify `Object (toString [this] …)` impl supplies the // `str`/`.toString` representation — consulted like the `Seqable/-seq` @@ -1164,19 +1165,16 @@ fn printBigInt(w: *Writer, v: Value) anyerror!void { // WITHOUT `N` (it is a primitive Long that merely overflowed cljw's i48 // inline range), matching clj `(parse-long "999999999999999")`. const m = big_int_mod.asManaged(v); - if (big_int_mod.originOf(v) == .long) { - try w.print("{f}", .{m}); - } else { - try w.print("{f}N", .{m}); - } + try decimal_text.writeDecimal(w, m); + if (big_int_mod.originOf(v) != .long) try w.writeByte('N'); } fn printRatio(w: *Writer, v: Value) anyerror!void { // Render numerator `/` denominator without a trailing `N` (small = - // inline i64s; big = the stored BigInt Manageds via `{f}`). + // inline i64s; big = the stored BigInt Manageds, no size ceiling). switch (ratio_mod.parts(v)) { .small => |s| try w.print("{d}/{d}", .{ s.n, s.d }), - .big => |b| try w.print("{f}/{f}", .{ b.n.m, b.d.m }), + .big => |b| try decimal_text.writeFraction(w, b.n.m, b.d.m), } } @@ -1220,14 +1218,11 @@ fn writePlainDigits(w: *Writer, digits: []const u8, scale: i32) anyerror!void { /// plain arm. pub fn writeBigDecimalPlain(w: *Writer, v: Value) anyerror!void { const bd = v.decodePtr(*const big_decimal_mod.BigDecimal); - var buf: [1024]u8 = undefined; - var sw: std.Io.Writer = .fixed(&buf); - sw.print("{f}", .{bd.unscaled.m}) catch { - // Unscaled wider than the buffer — lossy digits-only fallback, - // mirroring writeBigDecimalDigits' stance. - return w.print("{f}", .{bd.unscaled.m}); - }; - const written = buf[0..sw.end]; + // Stack scratch for the common width, heap past it: no size ceiling. + var sfa = std.heap.stackFallback(2048, bd.unscaled.m.allocator); + const alloc = sfa.get(); + const written = try decimal_text.decimalAlloc(alloc, bd.unscaled.m); + defer alloc.free(written); const neg = written.len > 0 and written[0] == '-'; const digits = if (neg) written[1..] else written; if (neg) try w.writeByte('-'); @@ -1240,15 +1235,12 @@ pub fn writeBigDecimalDigits(w: *Writer, v: Value) anyerror!void { // `(precision-1) - scale >= -6`, otherwise scientific `d.dddE±exp`. const bd = v.decodePtr(*const big_decimal_mod.BigDecimal); - // Render unscaled digits into a scratch buffer; split off the sign. - var buf: [1024]u8 = undefined; - var sw: std.Io.Writer = .fixed(&buf); - sw.print("{f}", .{bd.unscaled.m}) catch { - // Unscaled wider than the buffer — fall back to the lossy form - // (digits only; the caller appends `M` for the reader form). - return w.print("{f}", .{bd.unscaled.m}); - }; - const written = buf[0..sw.end]; + // Render unscaled digits (no size ceiling); split off the sign. + // Stack scratch for the common width, heap past it: no size ceiling. + var sfa = std.heap.stackFallback(2048, bd.unscaled.m.allocator); + const alloc = sfa.get(); + const written = try decimal_text.decimalAlloc(alloc, bd.unscaled.m); + defer alloc.free(written); const neg = written.len > 0 and written[0] == '-'; const digits = if (neg) written[1..] else written; const n: i64 = @intCast(digits.len); @@ -1600,7 +1592,14 @@ pub fn printArraySeq(ports: ?Ports, w: *Writer, v: Value) anyerror!void { while (i < n) : (i += 1) { if (try lengthTruncated(w, @intCast(i), " ")) break; if (i > 0) try w.writeByte(' '); - try printValue(ports, w, array_seq_collection.nth(v, i)); + if (array_seq_collection.peekNth(v, i)) |e| { + try printValue(ports, w, e); + } else { + // A primitive long beyond the immediate range: its digits, which + // is how its heap box prints. + const a = array_seq_collection.primitiveBacking(v).?; + try w.print("{d}", .{a.slice(.long)[array_seq_collection.indexOf(v) + i]}); + } } try w.writeByte(')'); } diff --git a/src/runtime/protocol.zig b/src/runtime/protocol.zig index abf97b044..61790e2ca 100644 --- a/src/runtime/protocol.zig +++ b/src/runtime/protocol.zig @@ -17,6 +17,7 @@ const HeapTag = value.HeapTag; const Runtime = @import("runtime.zig").Runtime; const td_mod = @import("type_descriptor.zig"); const TypeDescriptor = td_mod.TypeDescriptor; +const DispatchCache = @import("dispatch/method_table.zig").DispatchCache; /// One method on a protocol — name + arity (Clojure protocols are /// arity-overloaded). The implementation pointer lives on the @@ -134,9 +135,8 @@ pub fn asProtocol(val: Value) *const ProtocolDescriptor { /// pair that cycle 4's `(m receiver args)` callable Value uses to /// route through the row 7.1 dispatch ABI. Sits on F-004 Group B /// slot 19 (`.protocol_fn`, declared in heap_tag.zig); CallSite is -/// NOT carried here — analyzer-time per-call-site CallSite slots -/// remain the cache location, this struct is just the per-method -/// handle. +/// NOT carried here; the per-method `dispatch_cache` (ADR-0206) is the +/// protocol-fn call cache, keyed on the receiver's descriptor. pub const ProtocolFn = extern struct { header: HeapHeader, // Pad to align Value-sized fields to 16 — header is 8 bytes, @@ -150,11 +150,20 @@ pub const ProtocolFn = extern struct { /// Interned method name (bytes owned by the symbol pool). method_name_ptr: [*]const u8, method_name_len: usize, + /// Receiver-type to resolved-method cache (ADR-0206). Interior-mutable: + /// written through `cache()` on a `*const ProtocolFn`. + dispatch_cache: DispatchCache = .{}, /// Return the method name as a `[]const u8` slice. pub fn methodName(self: *const ProtocolFn) []const u8 { return self.method_name_ptr[0..self.method_name_len]; } + + /// The dispatch cache, mutable through a const handle: the cache is a + /// memo of a pure function, invisible to every reader of the fn. + pub fn cache(self: *const ProtocolFn) *DispatchCache { + return @constCast(&self.dispatch_cache); + } }; /// Allocate a `ProtocolFn` Value on `rt.gc.infra`. Caller supplies @@ -307,6 +316,11 @@ pub fn addProtocolImpl(rt: *Runtime, td: *TypeDescriptor, proto_name: []const u8 combined[old.len] = proto_name; td.protocol_impls = combined; if (old.len > 0) rt.gc.infra.free(old); + // Membership feeds the ADR-0187 protocol-target fallback that the + // ProtocolFn dispatch cache memoises (ADR-0206). The caller's earlier + // method-table bump happened BEFORE this write, so a fill in between + // would cache pre-membership state: bump again, after the write. + rt.protocol_generation +%= 1; } // --- tests --- diff --git a/src/runtime/regex/match.zig b/src/runtime/regex/match.zig index 6eead13c1..d099e9953 100644 --- a/src/runtime/regex/match.zig +++ b/src/runtime/regex/match.zig @@ -172,6 +172,27 @@ fn scanFrom( return null; } +/// A reusable leftmost-match scanner: ONE VM state for a whole multi-match +/// walk (`split`), where `findFrom` would allocate and free the thread lists +/// on every call. Same results as `findFrom(alloc, program, input, start)`. +/// [refs: O-072] +pub const Scanner = struct { + vm: Vm, + program: *const compile.Program, + + pub fn init(alloc: std.mem.Allocator, program: *const compile.Program) MatchError!Scanner { + return .{ .vm = try Vm.init(alloc, program.insts.len, slotCount(program)), .program = program }; + } + + pub fn deinit(self: *Scanner, alloc: std.mem.Allocator) void { + self.vm.deinit(alloc); + } + + pub fn findFrom(self: *Scanner, alloc: std.mem.Allocator, input: []const u8, start: u32) MatchError!?MatchResult { + return scanFrom(&self.vm, alloc, self.program, input, start); + } +}; + /// Append every non-overlapping match (scanning from offset 0) to `out`, reusing /// ONE ThreadList pair across the whole scan — the one-pass backing for `re-seq` /// (O-035, ADR-0147 Stage 1b). A zero-width match advances the scan by 1 so the diff --git a/src/runtime/regex/replace.zig b/src/runtime/regex/replace.zig index 840da4ec6..f1791d9f5 100644 --- a/src/runtime/regex/replace.zig +++ b/src/runtime/regex/replace.zig @@ -107,6 +107,14 @@ pub fn splitToVector( ) anyerror!Value { var parts: std.ArrayList(Value) = .empty; defer parts.deinit(rt.gpa); + // The pieces sit in an off-heap list until `fromSlice` publishes them; + // no collect may run in between (ADR-0150 fabrication region). + rt.gc.enterFabrication(); + defer rt.gc.exitFabrication(); + // PERF: one scanner for the whole split, not a VM alloc per match, and a + // bulk trie build instead of a persistent conj per piece. [refs: O-072] + var scanner = try regex_match.Scanner.init(rt.gpa, program); + defer scanner.deinit(rt.gpa); if (haystack.len == 0) { try parts.append(rt.gpa, try string_collection.alloc(rt, "")); @@ -124,7 +132,7 @@ pub fn splitToVector( var matched = false; while (true) { if (limit > 0 and nsplits >= limit - 1) break; - const match = (try regex_match.findFrom(rt.gpa, program, haystack, cursor)) orelse break; + const match = (try scanner.findFrom(rt.gpa, haystack, cursor)) orelse break; defer match.deinit(rt.gpa); if (index == 0 and match.start == 0 and match.end == 0) { if (match.end >= haystack.len) break; @@ -154,9 +162,7 @@ pub fn splitToVector( } } - var result = vector_collection.empty(); - for (parts.items) |p| result = try vector_collection.conj(rt, result, p); - return result; + return vector_collection.fromSlice(rt, parts.items); } // --- tests --- diff --git a/src/runtime/runtime.zig b/src/runtime/runtime.zig index a508c7c0e..2d863b53c 100644 --- a/src/runtime/runtime.zig +++ b/src/runtime/runtime.zig @@ -292,6 +292,11 @@ pub const Runtime = struct { /// UNLIMITED), gc.infra-allocated host_instances lazily filled by /// `math_context.zig::singleton`; freed in `deinit`. math_contexts: [4]@import("value/value.zig").Value = @splat(.nil_val), + /// The interned `java.time.format.DateTimeFormatter/ISO_*` constants, + /// gc.infra-allocated host_instances over the engine's static programs, + /// lazily filled by `date_time_formatter_value.zig::isoConstant`; freed in + /// `deinit`. + iso_formatters: [@import("time/date_time_format.zig").ISO_CONSTANTS.len]@import("value/value.zig").Value = @splat(.nil_val), /// `clojure.lang.Compiler/specials` — a map `{ nil…}` /// whose keys are cljw's SPECIAL_FORMS (the SSOT in analyzer.zig). Unlike @@ -731,6 +736,7 @@ pub const Runtime = struct { @import("locale.zig").deinitSingletons(self); @import("host_enum.zig").deinitConsts(self); @import("math_context.zig").deinitSingletons(self); + @import("time/date_time_formatter_value.zig").deinitIsoConstants(self); // User-set system properties (gpa-owned key+value dupes). { diff --git a/src/runtime/seqable.zig b/src/runtime/seqable.zig index 156b4ae7a..9aaaae673 100644 --- a/src/runtime/seqable.zig +++ b/src/runtime/seqable.zig @@ -112,7 +112,7 @@ pub fn first(rt: *Runtime, env: *Env, v: Value, loc: SourceLocation) anyerror!Va // PERF: O(1) head (start), no chunk materialised for just first [refs: O-001] .range => try range.first(rt, v), // PERF: O(1) reads through the views [refs: O-058, D-179] - .array_seq => array_seq.first(v), + .array_seq => try array_seq.first(rt, v), .string_seq => string_seq.first(v), // PERF: indexed collections answer without a view. .vector => if (vector.count(v) > 0) vector.nth(v, 0) else .nil_val, diff --git a/src/runtime/time/date_time_format.zig b/src/runtime/time/date_time_format.zig new file mode 100644 index 000000000..42f35b71f --- /dev/null +++ b/src/runtime/time/date_time_format.zig @@ -0,0 +1,2060 @@ +// SPDX-License-Identifier: EPL-2.0 +//! The `java.time.format` engine: a DateTimeFormatter is a tree of printer- +//! parser ops (`Op`), built by `Builder` (directly, or from a pattern string), +//! that formats a `Temporal` view into text and parses text into a field set +//! that `resolve` turns into a date / time / instant. +//! +//! Backend: impl-only +//! Impl deps: instant, zone_offset, host_enum +//! Clojure peer: none +//! +//! Never touches a Runtime (it borrows only the pure `Zone` arithmetic of +//! `zone_offset.zig` and the ChronoField names of the neutral `host_enum` +//! registry): the Value layer (`date_time_formatter_value.zig`) wraps a +//! `Program` / `Builder` in a host instance and maps a `Diag` onto the error +//! catalog. Behaviour follows the JDK 25 `DateTimeFormatterBuilder` printer- +//! parsers one to one (op set, adjacent value parsing, optional sections, the +//! SMART / STRICT resolver, the toString forms), with English text (cljw's only +//! locale). What is left out is refused loudly, never approximated: week-based +//! and quarter fields, localized zone names, padding, the LENIENT resolver. + +const std = @import("std"); +const Allocator = std.mem.Allocator; +const instant = @import("instant.zig"); +const zone_offset = @import("zone_offset.zig"); +const Zone = zone_offset.Zone; +const host_enum = @import("../host_enum.zig"); + +// --- fields --- + +/// `java.time.temporal.ChronoField`, in JVM ordinal order (pinned against the +/// host-enum registry below). +pub const Field = enum(u8) { + nano_of_second, + nano_of_day, + micro_of_second, + micro_of_day, + milli_of_second, + milli_of_day, + second_of_minute, + second_of_day, + minute_of_hour, + minute_of_day, + hour_of_ampm, + clock_hour_of_ampm, + hour_of_day, + clock_hour_of_day, + ampm_of_day, + day_of_week, + aligned_day_of_week_in_month, + aligned_day_of_week_in_year, + day_of_month, + day_of_year, + epoch_day, + aligned_week_of_month, + aligned_week_of_year, + month_of_year, + proleptic_month, + year_of_era, + year, + era, + instant_seconds, + offset_seconds, + + /// The display name ("HourOfDay") every message and toString uses. + pub fn display(self: Field) []const u8 { + return host_enum.toStringOf(.chrono_field, @intFromEnum(self)); + } + + pub fn isTimeBased(self: Field) bool { + return @intFromEnum(self) <= @intFromEnum(Field.ampm_of_day); + } + + pub fn isDateBased(self: Field) bool { + return @intFromEnum(self) >= @intFromEnum(Field.day_of_week) and @intFromEnum(self) <= @intFromEnum(Field.era); + } +}; + +pub const FIELD_COUNT = @typeInfo(Field).@"enum".fields.len; + +comptime { + @setEvalBranchQuota(20_000); + std.debug.assert(FIELD_COUNT == host_enum.count(.chrono_field)); + for (@typeInfo(Field).@"enum".fields, 0..) |f, i| { + var upper: [f.name.len]u8 = undefined; + for (f.name, 0..) |c, j| upper[j] = std.ascii.toUpper(c); + if (!std.mem.eql(u8, &upper, host_enum.name(.chrono_field, i))) + @compileError("Field order drifted from the ChronoField registry at " ++ f.name); + } +} + +/// A `ValueRange`: `min - max`, with the smallest maximum for the month- and +/// year-length dependent fields ("1 - 28/31"). +const Range = struct { + min: i64, + max: i64, + smallest_max: ?i64 = null, + + fn isValid(self: Range, v: i64) bool { + return v >= self.min and v <= self.max; + } +}; + +const YEAR_MAX: i64 = 999_999_999; + +fn rangeOf(f: Field) Range { + return switch (f) { + .nano_of_second => .{ .min = 0, .max = 999_999_999 }, + .nano_of_day => .{ .min = 0, .max = 86_400 * 1_000_000_000 - 1 }, + .micro_of_second => .{ .min = 0, .max = 999_999 }, + .micro_of_day => .{ .min = 0, .max = 86_400 * 1_000_000 - 1 }, + .milli_of_second => .{ .min = 0, .max = 999 }, + .milli_of_day => .{ .min = 0, .max = 86_400 * 1000 - 1 }, + .second_of_minute => .{ .min = 0, .max = 59 }, + .second_of_day => .{ .min = 0, .max = 86_400 - 1 }, + .minute_of_hour => .{ .min = 0, .max = 59 }, + .minute_of_day => .{ .min = 0, .max = 24 * 60 - 1 }, + .hour_of_ampm => .{ .min = 0, .max = 11 }, + .clock_hour_of_ampm => .{ .min = 1, .max = 12 }, + .hour_of_day => .{ .min = 0, .max = 23 }, + .clock_hour_of_day => .{ .min = 1, .max = 24 }, + .ampm_of_day => .{ .min = 0, .max = 1 }, + .day_of_week, .aligned_day_of_week_in_month, .aligned_day_of_week_in_year => .{ .min = 1, .max = 7 }, + .day_of_month => .{ .min = 1, .max = 31, .smallest_max = 28 }, + .day_of_year => .{ .min = 1, .max = 366, .smallest_max = 365 }, + .epoch_day => .{ .min = -365_243_219_162, .max = 365_241_780_471 }, + .aligned_week_of_month => .{ .min = 1, .max = 5, .smallest_max = 4 }, + .aligned_week_of_year => .{ .min = 1, .max = 53 }, + .month_of_year => .{ .min = 1, .max = 12 }, + .proleptic_month => .{ .min = -YEAR_MAX * 12, .max = YEAR_MAX * 12 + 11 }, + .year_of_era => .{ .min = 1, .max = YEAR_MAX + 1, .smallest_max = YEAR_MAX }, + .year => .{ .min = -YEAR_MAX, .max = YEAR_MAX }, + .era => .{ .min = 0, .max = 1 }, + .instant_seconds => .{ .min = std.math.minInt(i64), .max = std.math.maxInt(i64) }, + .offset_seconds => .{ .min = -18 * 3600, .max = 18 * 3600 }, + }; +} + +// --- diagnostics --- + +/// Why an engine call failed. The Value layer maps each kind to its catalog +/// code (and so its JVM exception class). +pub const Diag = struct { + kind: Kind = .value, + field: Field = .year, + buf: [256]u8 = undefined, + len: usize = 0, + + pub const Kind = enum { + /// DateTimeException with `msg()`. + value, + /// UnsupportedTemporalTypeException for `field`. + unsupported_field, + /// IllegalArgumentException with `msg()` (a malformed pattern / builder call). + pattern, + /// A JVM pattern letter cljw does not implement; `msg()` is the letter. + pattern_letter, + /// IllegalStateException with `msg()` (builder misuse). + state, + }; + + pub fn msg(self: *const Diag) []const u8 { + return self.buf[0..self.len]; + } + + pub fn fail(self: *Diag, kind: Kind, comptime fmt: []const u8, args: anytype) error{DateTime} { + self.kind = kind; + self.len = if (std.fmt.bufPrint(&self.buf, fmt, args)) |s| s.len else |_| self.buf.len; + return error.DateTime; + } + + fn unsupported(self: *Diag, f: Field) error{DateTime} { + self.field = f; + return self.fail(.unsupported_field, "Unsupported field: {s}", .{f.display()}); + } + + /// `ChronoField.checkValidValue`'s "Invalid value for X (valid values …): v". + fn invalidValue(self: *Diag, f: Field, v: i64) error{DateTime} { + const r = rangeOf(f); + var rb: [64]u8 = undefined; + const rs = if (r.smallest_max) |sm| + std.fmt.bufPrint(&rb, "{d} - {d}/{d}", .{ r.min, sm, r.max }) catch unreachable + else + std.fmt.bufPrint(&rb, "{d} - {d}", .{ r.min, r.max }) catch unreachable; + return self.fail(.value, "Invalid value for {s} (valid values {s}): {d}", .{ f.display(), rs, v }); + } +}; + +pub const Error = error{ DateTime, OutOfMemory }; + +fn checkValid(f: Field, v: i64, diag: *Diag) error{DateTime}!i64 { + if (!rangeOf(f).isValid(v)) return diag.invalidValue(f, v); + return v; +} + +// --- the field set a parse fills --- + +pub const FieldSet = struct { + vals: [FIELD_COUNT]i64 = @splat(0), + mask: u32 = 0, + + pub fn has(self: *const FieldSet, f: Field) bool { + return self.mask & bit(f) != 0; + } + + pub fn get(self: *const FieldSet, f: Field) ?i64 { + return if (self.has(f)) self.vals[@intFromEnum(f)] else null; + } + + /// Store, returning the previous value. + pub fn put(self: *FieldSet, f: Field, v: i64) ?i64 { + const old = self.get(f); + self.vals[@intFromEnum(f)] = v; + self.mask |= bit(f); + return old; + } + + pub fn remove(self: *FieldSet, f: Field) ?i64 { + const old = self.get(f); + self.mask &= ~bit(f); + return old; + } + + pub fn isEmpty(self: *const FieldSet) bool { + return self.mask == 0; + } + + fn bit(f: Field) u32 { + return @as(u32, 1) << @intCast(@intFromEnum(f)); + } +}; + +// --- the temporal view a formatter reads --- + +/// What a TemporalAccessor exposes to the printer: a date (epoch day), a time +/// (nano of day), an offset, a zone, an instant, and (for a parse result) the +/// leftover field set, consulted first like `Parsed.getLong`. +pub const Temporal = struct { + date: ?i64 = null, + time: ?i64 = null, + offset: ?i32 = null, + zone: ?Zone = null, + instant_secs: ?i64 = null, + instant_nanos: i32 = 0, + fields: ?*const FieldSet = null, + + /// The ISO view of an instant at a fixed offset (a ZonedDateTime). + pub fn atZone(secs: i64, nanos: i32, zone: Zone) Temporal { + const local = secs + zone.seconds; + return .{ + .date = @divFloor(local, 86_400), + .time = @mod(local, 86_400) * 1_000_000_000 + nanos, + .offset = zone.seconds, + .zone = zone, + .instant_secs = secs, + .instant_nanos = nanos, + }; + } + + pub fn getField(self: *const Temporal, f: Field) ?i64 { + if (self.fields) |fs| { + if (fs.get(f)) |v| return v; + } + if (f.isDateBased() or f == .epoch_day) { + const ed = self.date orelse return null; + return dateField(ed, f); + } + if (f.isTimeBased()) { + if (self.time) |nod| return timeField(nod, f); + if (self.fields == null and self.instant_secs != null) { + const n: i64 = self.instant_nanos; + return switch (f) { + .nano_of_second => n, + .micro_of_second => @divFloor(n, 1000), + .milli_of_second => @divFloor(n, 1_000_000), + else => null, + }; + } + return null; + } + if (self.fields != null) return null; // a Parsed derives nothing else + return switch (f) { + .offset_seconds => if (self.offset) |o| @as(i64, o) else null, + .instant_seconds => self.instant_secs, + else => unreachable, + }; + } +}; + +fn dateField(ed: i64, f: Field) i64 { + const c = instant.civilFromDays(ed); + const doy = ed - instant.daysFromCivil(c.y, 1, 1) + 1; + return switch (f) { + .day_of_week => @mod(ed + 3, 7) + 1, + .aligned_day_of_week_in_month => @mod(c.d - 1, 7) + 1, + .aligned_day_of_week_in_year => @mod(doy - 1, 7) + 1, + .day_of_month => c.d, + .day_of_year => doy, + .epoch_day => ed, + .aligned_week_of_month => @divFloor(c.d - 1, 7) + 1, + .aligned_week_of_year => @divFloor(doy - 1, 7) + 1, + .month_of_year => c.m, + .proleptic_month => c.y * 12 + c.m - 1, + .year_of_era => if (c.y >= 1) c.y else 1 - c.y, + .year => c.y, + .era => if (c.y >= 1) 1 else 0, + else => unreachable, + }; +} + +fn timeField(nod: i64, f: Field) i64 { + const hod = @divFloor(nod, 3_600_000_000_000); + return switch (f) { + .nano_of_second => @mod(nod, 1_000_000_000), + .nano_of_day => nod, + .micro_of_second => @divFloor(@mod(nod, 1_000_000_000), 1000), + .micro_of_day => @divFloor(nod, 1000), + .milli_of_second => @divFloor(@mod(nod, 1_000_000_000), 1_000_000), + .milli_of_day => @divFloor(nod, 1_000_000), + .second_of_minute => @mod(@divFloor(nod, 1_000_000_000), 60), + .second_of_day => @divFloor(nod, 1_000_000_000), + .minute_of_hour => @mod(@divFloor(nod, 60_000_000_000), 60), + .minute_of_day => @divFloor(nod, 60_000_000_000), + .hour_of_ampm => @mod(hod, 12), + .clock_hour_of_ampm => if (@mod(hod, 12) == 0) 12 else @mod(hod, 12), + .hour_of_day => hod, + .clock_hour_of_day => if (hod == 0) 24 else hod, + .ampm_of_day => @divFloor(hod, 12), + else => unreachable, + }; +} + +// --- the op tree --- + +/// `SignStyle`, JVM declaration order. +pub const SignStyle = enum(u8) { + normal, + always, + never, + not_negative, + exceeds_pad, + + fn upper(self: SignStyle) []const u8 { + return switch (self) { + .normal => "NORMAL", + .always => "ALWAYS", + .never => "NEVER", + .not_negative => "NOT_NEGATIVE", + .exceeds_pad => "EXCEEDS_PAD", + }; + } + + /// `SignStyle.parse(positive, strict, fixedWidth)`: may this sign appear? + fn allows(self: SignStyle, positive: bool, strict: bool, fixed_width: bool) bool { + return switch (self) { + .normal => !positive or !strict, + .always, .exceeds_pad => true, + .never, .not_negative => !strict and !fixed_width, + }; + } +}; + +pub const TextStyle = enum(u8) { + full, + full_standalone, + short, + short_standalone, + narrow, + narrow_standalone, + + fn upper(self: TextStyle) []const u8 { + return switch (self) { + .full => "FULL", + .full_standalone => "FULL_STANDALONE", + .short => "SHORT", + .short_standalone => "SHORT_STANDALONE", + .narrow => "NARROW", + .narrow_standalone => "NARROW_STANDALONE", + }; + } + + fn base(self: TextStyle) TextStyle { + return switch (self) { + .full, .full_standalone => .full, + .short, .short_standalone => .short, + .narrow, .narrow_standalone => .narrow, + }; + } +}; + +pub const ResolverStyle = enum(u8) { strict, smart }; + +/// The offset patterns `appendOffset` accepts, indexed like the JVM +/// `OffsetIdPrinterParser.PATTERNS` (the padded-hour half). +pub const OFFSET_PATTERNS = [_][]const u8{ "+HH", "+HHmm", "+HH:mm", "+HHMM", "+HH:MM", "+HHMMss", "+HH:MM:ss", "+HHMMSS", "+HH:MM:SS", "+HHmmss", "+HH:mm:ss" }; + +pub const ValueOp = struct { + field: Field, + min: u8, + max: u8, + sign: SignStyle, + /// -1 = fixed width (an adjacent follower); >0 = digits reserved for the + /// fixed-width values that follow; 0 = neither. + subsequent: i16 = 0, + /// A ReducedValue (`yy`): the two-digit window starting at `base`. + reduced_base: ?i64 = null, +}; + +pub const FractionOp = struct { + field: Field, + min: u8, + max: u8, + decimal_point: bool, +}; + +pub const Op = union(enum) { + value: ValueOp, + fraction: FractionOp, + text: struct { field: Field, style: TextStyle }, + literal: []const u8, + offset: struct { pattern: u8, no_offset_text: []const u8 }, + localized_offset: TextStyle, + zone_id, + zone_region_id, + instant, + case_sensitive: bool, + strict: bool, + default_value: struct { field: Field, value: i64 }, + composite: struct { ops: []const Op, optional: bool }, +}; + +/// A finished formatter: the root op list plus its resolver style and zone. +pub const Program = struct { + ops: []const Op, + resolver: ResolverStyle = .smart, + zone: ?Zone = null, + + /// Deep copy onto `gpa` (literal text and nested op lists included). + pub fn clone(self: *const Program, gpa: Allocator) Allocator.Error!*Program { + const p = try gpa.create(Program); + errdefer gpa.destroy(p); + p.* = .{ .ops = try cloneOps(gpa, self.ops), .resolver = self.resolver, .zone = self.zone }; + return p; + } + + pub fn destroy(self: *Program, gpa: Allocator) void { + freeOps(gpa, self.ops); + gpa.destroy(self); + } +}; + +fn cloneOp(gpa: Allocator, op: Op) Allocator.Error!Op { + return switch (op) { + .literal => |s| .{ .literal = try gpa.dupe(u8, s) }, + .offset => |o| .{ .offset = .{ .pattern = o.pattern, .no_offset_text = try gpa.dupe(u8, o.no_offset_text) } }, + .composite => |c| .{ .composite = .{ .ops = try cloneOps(gpa, c.ops), .optional = c.optional } }, + else => op, + }; +} + +fn cloneOps(gpa: Allocator, ops: []const Op) Allocator.Error![]const Op { + const out = try gpa.alloc(Op, ops.len); + var done: usize = 0; + errdefer { + freeOps(gpa, out[0..done]); + gpa.free(out[done..]); + } + for (ops, 0..) |op, i| { + out[i] = try cloneOp(gpa, op); + done = i + 1; + } + return out; +} + +fn freeOp(gpa: Allocator, op: Op) void { + switch (op) { + .literal => |s| gpa.free(s), + .offset => |o| gpa.free(o.no_offset_text), + .composite => |c| freeOps(gpa, c.ops), + else => {}, + } +} + +fn freeOps(gpa: Allocator, ops: []const Op) void { + for (ops) |op| freeOp(gpa, op); + gpa.free(ops); +} + +// --- the builder --- + +/// `DateTimeFormatterBuilder`: a stack of levels (the root, then one per open +/// `optionalStart`), each tracking the active adjacent value parser. +pub const Builder = struct { + levels: std.ArrayList(Level) = .empty, + + const Level = struct { + ops: std.ArrayList(Op) = .empty, + optional: bool, + value_parser: ?usize = null, + }; + + pub fn create(gpa: Allocator) Allocator.Error!*Builder { + const b = try gpa.create(Builder); + b.* = .{}; + errdefer gpa.destroy(b); + try b.levels.append(gpa, .{ .optional = false }); + return b; + } + + pub fn destroy(self: *Builder, gpa: Allocator) void { + for (self.levels.items) |*lv| { + for (lv.ops.items) |op| freeOp(gpa, op); + lv.ops.deinit(gpa); + } + self.levels.deinit(gpa); + gpa.destroy(self); + } + + fn active(self: *Builder) *Level { + return &self.levels.items[self.levels.items.len - 1]; + } + + /// `appendInternal`: append an owned op, ending adjacent value parsing. + fn appendOwned(self: *Builder, gpa: Allocator, op: Op) Allocator.Error!usize { + const lv = self.active(); + lv.ops.append(gpa, op) catch |e| { + freeOp(gpa, op); + return e; + }; + lv.value_parser = null; + return lv.ops.items.len - 1; + } + + /// Append a copy of `op` (its text and nested ops are duplicated). + pub fn append(self: *Builder, gpa: Allocator, op: Op) Allocator.Error!void { + _ = try self.appendOwned(gpa, try cloneOp(gpa, op)); + } + + /// `appendValue(NumberPrinterParser)` with adjacent value parsing: a + /// fixed-width NOT_NEGATIVE value after an active one reserves its width + /// in that parser; anything else fixes the active one and takes over. + fn appendValueOp(self: *Builder, gpa: Allocator, v: ValueOp) Allocator.Error!void { + const lv = self.active(); + if (lv.value_parser) |base_idx| { + const base = &lv.ops.items[base_idx]; + const follower_fixed = v.min == v.max and v.sign == .not_negative; + if (follower_fixed) { + switch (base.*) { + .value => |*bv| bv.subsequent = if (bv.subsequent == -1) -1 else bv.subsequent + v.max, + else => {}, + } + var fixed = v; + fixed.subsequent = -1; + _ = try self.appendOwned(gpa, .{ .value = fixed }); + self.active().value_parser = base_idx; + } else { + switch (base.*) { + .value => |*bv| bv.subsequent = -1, + else => {}, + } + const idx = try self.appendOwned(gpa, .{ .value = v }); + self.active().value_parser = idx; + } + } else { + const idx = try self.appendOwned(gpa, .{ .value = v }); + self.active().value_parser = idx; + } + } + + /// `appendValue(field)`: 1 to 19 digits, NORMAL sign. + pub fn appendValue(self: *Builder, gpa: Allocator, field: Field) Allocator.Error!void { + try self.appendValueOp(gpa, .{ .field = field, .min = 1, .max = 19, .sign = .normal }); + } + + /// `appendValue(field, width)`: exactly `width` digits, NOT_NEGATIVE. + pub fn appendValueWidth(self: *Builder, gpa: Allocator, field: Field, width: i64, diag: *Diag) Error!void { + if (width < 1 or width > 19) + return diag.fail(.pattern, "The width must be from 1 to 19 inclusive but was {d}", .{width}); + try self.appendValueOp(gpa, .{ .field = field, .min = @intCast(width), .max = @intCast(width), .sign = .not_negative }); + } + + /// `appendValue(field, min, max, signStyle)`. + pub fn appendValueRange(self: *Builder, gpa: Allocator, field: Field, min: i64, max: i64, sign: SignStyle, diag: *Diag) Error!void { + if (min == max and sign == .not_negative) return self.appendValueWidth(gpa, field, max, diag); + if (min < 1 or min > 19) + return diag.fail(.pattern, "The minimum width must be from 1 to 19 inclusive but was {d}", .{min}); + if (max < 1 or max > 19) + return diag.fail(.pattern, "The maximum width must be from 1 to 19 inclusive but was {d}", .{max}); + if (max < min) + return diag.fail(.pattern, "The maximum width must exceed or equal the minimum width but {d} < {d}", .{ max, min }); + try self.appendValueOp(gpa, .{ .field = field, .min = @intCast(min), .max = @intCast(max), .sign = sign }); + } + + /// `appendValueReduced(field, 2, 2, LocalDate.of(2000, 1, 1))` (`yy` / `uu`). + fn appendValueReduced(self: *Builder, gpa: Allocator, field: Field) Allocator.Error!void { + try self.appendValueOp(gpa, .{ .field = field, .min = 2, .max = 2, .sign = .not_negative, .reduced_base = 2000 }); + } + + /// `appendFraction(field, min, max, decimalPoint)`. + pub fn appendFraction(self: *Builder, gpa: Allocator, field: Field, min: i64, max: i64, decimal_point: bool, diag: *Diag) Error!void { + const r = rangeOf(field); + if (r.min != 0 or !isPowerOfTen(r.max + 1)) + return diag.fail(.pattern, "Field must have a fixed set of values: {s}", .{field.display()}); + if (min < 0 or min > 9) + return diag.fail(.pattern, "Minimum width must be from 0 to 9 inclusive but was {d}", .{min}); + if (max < 1 or max > 9) + return diag.fail(.pattern, "Maximum width must be from 1 to 9 inclusive but was {d}", .{max}); + if (max < min) + return diag.fail(.pattern, "Maximum width must exceed or equal the minimum width but {d} < {d}", .{ max, min }); + const op: Op = .{ .fraction = .{ .field = field, .min = @intCast(min), .max = @intCast(max), .decimal_point = decimal_point } }; + // A fixed-width fraction without a decimal point joins adjacent value + // parsing as a follower (`yyyyMMddHHmmssSSS`): the active value parser + // reserves its width. It never becomes the active parser itself. + const lv = self.active(); + if (min == max and !decimal_point) { + if (lv.value_parser) |base_idx| { + switch (lv.ops.items[base_idx]) { + .value => |*bv| if (bv.subsequent != -1) { + bv.subsequent += @intCast(max); + }, + else => {}, + } + _ = try self.appendOwned(gpa, op); + self.active().value_parser = base_idx; + return; + } + } + _ = try self.appendOwned(gpa, op); + } + + pub fn appendText(self: *Builder, gpa: Allocator, field: Field, style: TextStyle) Allocator.Error!void { + _ = try self.appendOwned(gpa, .{ .text = .{ .field = field, .style = style } }); + } + + /// `appendLiteral`: an empty string appends nothing. + pub fn appendLiteral(self: *Builder, gpa: Allocator, s: []const u8) Allocator.Error!void { + if (s.len == 0) return; + _ = try self.appendOwned(gpa, .{ .literal = try gpa.dupe(u8, s) }); + } + + pub fn appendOffset(self: *Builder, gpa: Allocator, pattern: []const u8, no_offset_text: []const u8, diag: *Diag) Error!void { + for (OFFSET_PATTERNS, 0..) |p, i| { + if (std.mem.eql(u8, p, pattern)) { + _ = try self.appendOwned(gpa, .{ .offset = .{ .pattern = @intCast(i), .no_offset_text = try gpa.dupe(u8, no_offset_text) } }); + return; + } + } + if (pattern.len >= 2 and pattern[0] == '+' and pattern[1] == 'H' and (pattern.len == 2 or pattern[2] != 'H')) + return diag.fail(.pattern_letter, "{s}", .{pattern}); + return diag.fail(.pattern, "Invalid zone offset pattern: {s}", .{pattern}); + } + + pub fn appendLocalizedOffset(self: *Builder, gpa: Allocator, style: TextStyle, diag: *Diag) Error!void { + if (style != .full and style != .short) + return diag.fail(.pattern, "Style must be either full or short", .{}); + _ = try self.appendOwned(gpa, .{ .localized_offset = style }); + } + + /// `optionalStart`: open a nested optional level. + pub fn optionalStart(self: *Builder, gpa: Allocator) Allocator.Error!void { + self.active().value_parser = null; + try self.levels.append(gpa, .{ .optional = true }); + } + + /// `optionalEnd`: close the innermost optional level into its parent. + pub fn optionalEnd(self: *Builder, gpa: Allocator, diag: *Diag) Error!void { + if (self.levels.items.len <= 1) + return diag.fail(.state, "Cannot call optionalEnd() as there was no previous call to optionalStart()", .{}); + var lv = self.levels.pop().?; + if (lv.ops.items.len == 0) { + lv.ops.deinit(gpa); + return; + } + const ops = lv.ops.toOwnedSlice(gpa) catch |e| { + for (lv.ops.items) |op| freeOp(gpa, op); + lv.ops.deinit(gpa); + return e; + }; + _ = try self.appendOwned(gpa, .{ .composite = .{ .ops = ops, .optional = lv.optional } }); + } + + /// `append(formatter)` / `appendOptional(formatter)`. + pub fn appendProgram(self: *Builder, gpa: Allocator, p: *const Program, optional: bool) Allocator.Error!void { + _ = try self.appendOwned(gpa, .{ .composite = .{ .ops = try cloneOps(gpa, p.ops), .optional = optional } }); + } + + /// `toFormatter`: close every open optional level, copy the root. + pub fn toProgram(self: *Builder, gpa: Allocator, resolver: ResolverStyle) Error!*Program { + var diag: Diag = .{}; + while (self.levels.items.len > 1) self.optionalEnd(gpa, &diag) catch |e| switch (e) { + error.DateTime => unreachable, + error.OutOfMemory => return error.OutOfMemory, + }; + const root: Program = .{ .ops = self.levels.items[0].ops.items, .resolver = resolver }; + return root.clone(gpa); + } + + /// `appendPattern`: compile a pattern string into this builder. + pub fn appendPattern(self: *Builder, gpa: Allocator, pattern: []const u8, diag: *Diag) Error!void { + var pos: usize = 0; + while (pos < pattern.len) : (pos += 1) { + const cur = pattern[pos]; + if (std.ascii.isAlphabetic(cur)) { + const start = pos; + while (pos + 1 < pattern.len and pattern[pos + 1] == cur) pos += 1; + const count = pos - start + 1; + try self.patternLetter(gpa, cur, count, diag); + } else if (cur == '\'') { + const start = pos; + pos += 1; + while (pos < pattern.len) : (pos += 1) { + if (pattern[pos] == '\'') { + if (pos + 1 < pattern.len and pattern[pos + 1] == '\'') { + pos += 1; + } else break; + } + } + if (pos >= pattern.len) + return diag.fail(.pattern, "Pattern ends with an incomplete string literal: {s}", .{pattern}); + const raw = pattern[start + 1 .. pos]; + if (raw.len == 0) { + try self.appendLiteral(gpa, "'"); + } else { + const unq = try std.mem.replaceOwned(u8, gpa, raw, "''", "'"); + defer gpa.free(unq); + try self.appendLiteral(gpa, unq); + } + } else if (cur == '[') { + try self.optionalStart(gpa); + } else if (cur == ']') { + if (self.levels.items.len <= 1) + return diag.fail(.pattern, "Pattern invalid as it contains ] without previous [", .{}); + try self.optionalEnd(gpa, diag); + } else if (cur == '{' or cur == '}' or cur == '#') { + return diag.fail(.pattern, "Pattern includes reserved character: '{c}'", .{cur}); + } else { + // One literal per code point, like the JVM's per-char appendLiteral. + const n = std.unicode.utf8ByteSequenceLength(cur) catch 1; + const end = @min(pos + n, pattern.len); + try self.appendLiteral(gpa, pattern[pos..end]); + pos = end - 1; + } + } + } + + fn patternLetter(self: *Builder, gpa: Allocator, cur: u8, count: usize, diag: *Diag) Error!void { + const field: ?Field = switch (cur) { + 'G' => .era, + 'y' => .year_of_era, + 'u' => .year, + 'M', 'L' => .month_of_year, + 'D' => .day_of_year, + 'd' => .day_of_month, + 'F' => .aligned_week_of_month, + 'E' => .day_of_week, + 'a' => .ampm_of_day, + 'H' => .hour_of_day, + 'k' => .clock_hour_of_day, + 'K' => .hour_of_ampm, + 'h' => .clock_hour_of_ampm, + 'm' => .minute_of_hour, + 's' => .second_of_minute, + 'S', 'n' => .nano_of_second, + 'A' => .milli_of_day, + 'N' => .nano_of_day, + else => null, + }; + if (field) |f| return self.patternField(gpa, cur, count, f, diag); + switch (cur) { + 'V' => { + if (count != 2) return diag.fail(.pattern, "Pattern letter count must be 2: {c}", .{cur}); + _ = try self.appendOwned(gpa, .zone_id); + }, + 'Z' => { + if (count < 4) return self.appendOffset(gpa, "+HHMM", "+0000", diag); + if (count == 4) return self.appendLocalizedOffset(gpa, .full, diag); + if (count == 5) return self.appendOffset(gpa, "+HH:MM:ss", "Z", diag); + return diag.fail(.pattern, "Too many pattern letters: {c}", .{cur}); + }, + 'O' => { + if (count == 1) return self.appendLocalizedOffset(gpa, .short, diag); + if (count == 4) return self.appendLocalizedOffset(gpa, .full, diag); + return diag.fail(.pattern, "Pattern letter count must be 1 or 4: {c}", .{cur}); + }, + 'X', 'x' => { + if (count > 5) return diag.fail(.pattern, "Too many pattern letters: {c}", .{cur}); + const pat = OFFSET_PATTERNS[count + @as(usize, if (count == 1) 0 else 1)]; + const zero: []const u8 = if (cur == 'X') "Z" else if (count == 1) "+00" else if (count % 2 == 0) "+0000" else "+00:00"; + return self.appendOffset(gpa, pat, zero, diag); + }, + // Valid JVM letters cljw does not implement: quarter (IsoFields), + // week-based fields, localized zone and day-period names, Julian + // day, padding. + 'Q', 'q', 'c', 'e', 'Y', 'w', 'W', 'z', 'v', 'B', 'g', 'p' => return diag.fail(.pattern_letter, "{c}", .{cur}), + else => return diag.fail(.pattern, "Unknown pattern letter: {c}", .{cur}), + } + } + + fn patternField(self: *Builder, gpa: Allocator, cur: u8, count: usize, f: Field, diag: *Diag) Error!void { + const n: i64 = @intCast(count); + switch (cur) { + 'u', 'y' => { + if (count == 2) return self.appendValueReduced(gpa, f); + if (count < 4) return self.appendValueRange(gpa, f, n, 19, .normal, diag); + return self.appendValueRange(gpa, f, n, 19, .exceeds_pad, diag); + }, + 'M', 'L', 'E' => { + const standalone = cur == 'L'; + switch (count) { + 1, 2 => { + if (cur == 'E') return self.appendText(gpa, f, .short); + if (count == 1) return self.appendValue(gpa, f); + return self.appendValueWidth(gpa, f, 2, diag); + }, + 3 => return self.appendText(gpa, f, if (standalone) .short_standalone else .short), + 4 => return self.appendText(gpa, f, if (standalone) .full_standalone else .full), + 5 => return self.appendText(gpa, f, if (standalone) .narrow_standalone else .narrow), + else => return diag.fail(.pattern, "Too many pattern letters: {c}", .{cur}), + } + }, + 'a' => { + if (count != 1) return diag.fail(.pattern, "Too many pattern letters: {c}", .{cur}); + return self.appendText(gpa, f, .short); + }, + 'G' => switch (count) { + 1, 2, 3 => return self.appendText(gpa, f, .short), + 4 => return self.appendText(gpa, f, .full), + 5 => return self.appendText(gpa, f, .narrow), + else => return diag.fail(.pattern, "Too many pattern letters: {c}", .{cur}), + }, + 'S' => return self.appendFraction(gpa, .nano_of_second, n, n, false, diag), + 'F' => { + if (count != 1) return diag.fail(.pattern, "Too many pattern letters: {c}", .{cur}); + return self.appendValue(gpa, f); + }, + 'd', 'h', 'H', 'k', 'K', 'm', 's' => { + if (count == 1) return self.appendValue(gpa, f); + if (count == 2) return self.appendValueWidth(gpa, f, 2, diag); + return diag.fail(.pattern, "Too many pattern letters: {c}", .{cur}); + }, + 'D' => { + if (count == 1) return self.appendValue(gpa, f); + if (count == 2 or count == 3) return self.appendValueRange(gpa, f, n, 3, .not_negative, diag); + return diag.fail(.pattern, "Too many pattern letters: {c}", .{cur}); + }, + 'A', 'n', 'N' => return self.appendValueRange(gpa, f, n, 19, .not_negative, diag), + else => unreachable, + } + } +}; + +fn isPowerOfTen(v: i64) bool { + var x = v; + if (x < 10) return false; + while (@rem(x, 10) == 0) x = @divExact(x, 10); + return x == 1; +} + +/// `DateTimeFormatter.ofPattern(pattern)`: SMART resolver, no zone. +pub fn compilePattern(gpa: Allocator, pattern: []const u8, diag: *Diag) Error!*Program { + const b = try Builder.create(gpa); + defer b.destroy(gpa); + try b.appendPattern(gpa, pattern, diag); + return b.toProgram(gpa, .smart); +} + +// --- toString --- + +/// `DateTimeFormatter.toString`: the root composite without its parentheses. +pub fn writeProgram(out: *std.ArrayList(u8), gpa: Allocator, p: *const Program) Allocator.Error!void { + for (p.ops) |op| try writeOp(out, gpa, op); +} + +fn writeOp(out: *std.ArrayList(u8), gpa: Allocator, op: Op) Allocator.Error!void { + var b: [128]u8 = undefined; + const s: []const u8 = switch (op) { + .value => |v| blk: { + if (v.reduced_base) |base| break :blk std.fmt.bufPrint(&b, "ReducedValue({s},{d},{d},{d}-01-01)", .{ v.field.display(), v.min, v.max, base }) catch unreachable; + if (v.min == 1 and v.max == 19 and v.sign == .normal) break :blk std.fmt.bufPrint(&b, "Value({s})", .{v.field.display()}) catch unreachable; + if (v.min == v.max and v.sign == .not_negative) break :blk std.fmt.bufPrint(&b, "Value({s},{d})", .{ v.field.display(), v.min }) catch unreachable; + break :blk std.fmt.bufPrint(&b, "Value({s},{d},{d},{s})", .{ v.field.display(), v.min, v.max, v.sign.upper() }) catch unreachable; + }, + .fraction => |fr| std.fmt.bufPrint(&b, "Fraction({s},{d},{d}{s})", .{ fr.field.display(), fr.min, fr.max, if (fr.decimal_point) ",DecimalPoint" else "" }) catch unreachable, + .text => |t| if (t.style == .full) + std.fmt.bufPrint(&b, "Text({s})", .{t.field.display()}) catch unreachable + else + std.fmt.bufPrint(&b, "Text({s},{s})", .{ t.field.display(), t.style.upper() }) catch unreachable, + .literal => |lit| { + if (std.mem.eql(u8, lit, "'")) return out.appendSlice(gpa, "''"); + try out.append(gpa, '\''); + for (lit) |c| { + if (c == '\'') try out.append(gpa, '\''); + try out.append(gpa, c); + } + return out.append(gpa, '\''); + }, + .offset => |o| { + try out.appendSlice(gpa, "Offset("); + try out.appendSlice(gpa, OFFSET_PATTERNS[o.pattern]); + try out.appendSlice(gpa, ",'"); + for (o.no_offset_text) |c| { + if (c == '\'') try out.append(gpa, '\''); + try out.append(gpa, c); + } + return out.appendSlice(gpa, "')"); + }, + .localized_offset => |st| std.fmt.bufPrint(&b, "LocalizedOffset({s})", .{st.upper()}) catch unreachable, + .zone_id => "ZoneId()", + .zone_region_id => "ZoneRegionId()", + .instant => "Instant()", + .case_sensitive => |on| if (on) "ParseCaseSensitive(true)" else "ParseCaseSensitive(false)", + .strict => |on| if (on) "ParseStrict(true)" else "ParseStrict(false)", + .default_value => |d| std.fmt.bufPrint(&b, "Default({s},{d})", .{ d.field.display(), d.value }) catch unreachable, + .composite => |c| { + try out.append(gpa, if (c.optional) '[' else '('); + for (c.ops) |sub| try writeOp(out, gpa, sub); + return out.append(gpa, if (c.optional) ']' else ')'); + }, + }; + try out.appendSlice(gpa, s); +} + +// --- English text --- + +const MONTH_FULL = [_][]const u8{ "January", "February", "March", "April", "May", "June", "July", "August", "September", "October", "November", "December" }; +const MONTH_SHORT = [_][]const u8{ "Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec" }; +const MONTH_NARROW = [_][]const u8{ "J", "F", "M", "A", "M", "J", "J", "A", "S", "O", "N", "D" }; +const DOW_FULL = [_][]const u8{ "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "Sunday" }; +const DOW_SHORT = [_][]const u8{ "Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun" }; +const DOW_NARROW = [_][]const u8{ "M", "T", "W", "T", "F", "S", "S" }; +const AMPM = [_][]const u8{ "AM", "PM" }; +const AMPM_NARROW = [_][]const u8{ "a", "p" }; +const ERA_FULL = [_][]const u8{ "Before Christ", "Anno Domini" }; +const ERA_SHORT = [_][]const u8{ "BC", "AD" }; +const ERA_NARROW = [_][]const u8{ "B", "A" }; + +/// The text table for `field` in `style`, and the value of its first entry. +fn textTable(field: Field, style: TextStyle) ?struct { names: []const []const u8, first: i64 } { + return switch (field) { + .month_of_year => .{ .names = switch (style.base()) { + .full => &MONTH_FULL, + .short => &MONTH_SHORT, + else => &MONTH_NARROW, + }, .first = 1 }, + .day_of_week => .{ .names = switch (style.base()) { + .full => &DOW_FULL, + .short => &DOW_SHORT, + else => &DOW_NARROW, + }, .first = 1 }, + .ampm_of_day => .{ .names = if (style.base() == .narrow) &M_NARROW else &M, .first = 0 }, + .era => .{ .names = switch (style.base()) { + .full => &ERA_FULL, + .short => &ERA_SHORT, + else => &ERA_NARROW, + }, .first = 0 }, + else => null, + }; +} + +// --- formatting --- + +const PrintCtx = struct { + t: *const Temporal, + out: *std.ArrayList(u8), + gpa: Allocator, + diag: *Diag, + optional: u32 = 0, + + /// `DateTimePrintContext.getValue`: absent inside an optional section, + /// an UnsupportedTemporalTypeException outside one. + fn value(self: *PrintCtx, f: Field) error{DateTime}!?i64 { + if (self.t.getField(f)) |v| return v; + if (self.optional > 0) return null; + return self.diag.unsupported(f); + } + + fn put(self: *PrintCtx, s: []const u8) Allocator.Error!void { + try self.out.appendSlice(self.gpa, s); + } +}; + +/// `DateTimeFormatter.format`: print `t` with `p` into `out`. +pub fn format(gpa: Allocator, p: *const Program, t: *const Temporal, out: *std.ArrayList(u8), diag: *Diag) Error!void { + var ctx: PrintCtx = .{ .t = t, .out = out, .gpa = gpa, .diag = diag }; + _ = try printOps(&ctx, p.ops, false); +} + +fn printOps(ctx: *PrintCtx, ops: []const Op, optional: bool) Error!bool { + const mark = ctx.out.items.len; + if (optional) ctx.optional += 1; + defer { + if (optional) ctx.optional -= 1; + } + for (ops) |op| { + if (!try printOp(ctx, op)) { + ctx.out.shrinkRetainingCapacity(mark); + return true; + } + } + return true; +} + +fn printOp(ctx: *PrintCtx, op: Op) Error!bool { + switch (op) { + .value => |v| return printValue(ctx, v), + .fraction => |fr| return printFraction(ctx, fr), + .text => |t| { + const v = try ctx.value(t.field) orelse return false; + if (textTable(t.field, t.style)) |tbl| { + const idx = v - tbl.first; + if (idx >= 0 and idx < tbl.names.len) { + try ctx.put(tbl.names[@intCast(idx)]); + return true; + } + } + return printValue(ctx, .{ .field = t.field, .min = 1, .max = 19, .sign = .normal }); + }, + .literal => |s| { + try ctx.put(s); + return true; + }, + .offset => |o| return printOffset(ctx, o.pattern, o.no_offset_text), + .localized_offset => |st| return printLocalizedOffset(ctx, st), + .zone_id, .zone_region_id => { + const zone = ctx.t.zone orelse { + if (ctx.optional > 0) return false; + var tb: [96]u8 = undefined; + return ctx.diag.fail(.value, "Unable to extract ZoneId from temporal {s}", .{temporalText(&tb, ctx.t)}); + }; + if (op == .zone_region_id and !zone.isRegion()) return false; + var zb: [24]u8 = undefined; + try ctx.put(zone_offset.formatId(&zb, zone)); + return true; + }, + .instant => { + const secs = try ctx.value(.instant_seconds) orelse return false; + const nanos = ctx.t.getField(.nano_of_second) orelse 0; + var ib: [64]u8 = undefined; + try ctx.put(instant.formatIsoInstant(&ib, secs * 1000, @intCast(nanos))); + return true; + }, + .case_sensitive, .strict, .default_value => return true, + .composite => |c| return printOps(ctx, c.ops, c.optional), + } +} + +const EXCEED_POINTS = [_]i64{ 0, 10, 100, 1000, 10_000, 100_000, 1_000_000, 10_000_000, 100_000_000, 1_000_000_000, 10_000_000_000 }; + +fn printValue(ctx: *PrintCtx, v: ValueOp) Error!bool { + var val = try ctx.value(v.field) orelse return false; + if (v.reduced_base) |base| { + const abs: i64 = @intCast(@abs(val)); + const range = EXCEED_POINTS[v.min]; + val = if (val >= base and val < base + range) @mod(abs, range) else @mod(abs, EXCEED_POINTS[v.max]); + } + var db: [24]u8 = undefined; + const digits = std.fmt.bufPrint(&db, "{d}", .{@abs(val)}) catch unreachable; + if (digits.len > v.max) + return ctx.diag.fail(.value, "Field {s} cannot be printed as the value {d} exceeds the maximum print width of {d}", .{ v.field.display(), val, v.max }); + if (val >= 0) { + switch (v.sign) { + .exceeds_pad => if (v.min < 19 and v.min < EXCEED_POINTS.len and val >= EXCEED_POINTS[v.min]) try ctx.put("+"), + .always => try ctx.put("+"), + else => {}, + } + } else switch (v.sign) { + .normal, .exceeds_pad, .always => try ctx.put("-"), + .not_negative => return ctx.diag.fail(.value, "Field {s} cannot be printed as the value {d} cannot be negative according to the SignStyle", .{ v.field.display(), val }), + .never => {}, + } + var pad = @as(usize, v.min) -| digits.len; + while (pad > 0) : (pad -= 1) try ctx.put("0"); + try ctx.put(digits); + return true; +} + +/// Digits of the field's range size (10^k): the fraction scale. +fn fractionScale(f: Field) u8 { + var n = rangeOf(f).max + 1; + var k: u8 = 0; + while (n > 1) : (n = @divExact(n, 10)) k += 1; + return k; +} + +fn printFraction(ctx: *PrintCtx, fr: FractionOp) Error!bool { + const val = try ctx.value(fr.field) orelse return false; + if (!rangeOf(fr.field).isValid(val)) return ctx.diag.invalidValue(fr.field, val); + const scale = fractionScale(fr.field); + var db: [16]u8 = undefined; + const all = std.fmt.bufPrint(&db, "{d:0>[1]}", .{ @as(u64, @intCast(val)), scale }) catch unreachable; + var sig = all.len; + while (sig > 0 and all[sig - 1] == '0') sig -= 1; // stripTrailingZeros + if (sig == 0) { + if (fr.min > 0) { + if (fr.decimal_point) try ctx.put("."); + var i: usize = 0; + while (i < fr.min) : (i += 1) try ctx.put("0"); + } + return true; + } + const width = @min(@max(sig, fr.min), fr.max); + if (fr.decimal_point) try ctx.put("."); + var i: usize = 0; + while (i < width) : (i += 1) try ctx.put(if (i < all.len) all[i .. i + 1] else "0"); + return true; +} + +fn isColonStyle(style: u8) bool { + return style > 0 and style % 2 == 0; +} + +fn printOffset(ctx: *PrintCtx, style: u8, no_offset_text: []const u8) Error!bool { + const total = try ctx.value(.offset_seconds) orelse return false; + if (total == 0) { + try ctx.put(no_offset_text); + return true; + } + const abs_h: u64 = @intCast(@mod(@as(i64, @intCast(@abs(@divTrunc(total, 3600)))), 100)); + const abs_m: u64 = @intCast(@abs(@rem(@divTrunc(total, 60), 60))); + const abs_s: u64 = @intCast(@abs(@rem(total, 60))); + const mark = ctx.out.items.len; + var b: [16]u8 = undefined; + try ctx.put(if (total < 0) "-" else "+"); + try ctx.put(std.fmt.bufPrint(&b, "{d:0>2}", .{abs_h}) catch unreachable); + var output = abs_h; + const colon = isColonStyle(style); + if ((style >= 3 and style <= 8) or (style >= 9 and abs_s > 0) or (style >= 1 and abs_m > 0)) { + if (colon) try ctx.put(":"); + try ctx.put(std.fmt.bufPrint(&b, "{d:0>2}", .{abs_m}) catch unreachable); + output += abs_m; + if (style == 7 or style == 8 or (style >= 5 and abs_s > 0)) { + if (colon) try ctx.put(":"); + try ctx.put(std.fmt.bufPrint(&b, "{d:0>2}", .{abs_s}) catch unreachable); + output += abs_s; + } + } + if (output == 0) { + ctx.out.shrinkRetainingCapacity(mark); + try ctx.put(no_offset_text); + } + return true; +} + +fn printLocalizedOffset(ctx: *PrintCtx, style: TextStyle) Error!bool { + const total = try ctx.value(.offset_seconds) orelse return false; + try ctx.put("GMT"); + if (total == 0) return true; + const abs_h: u64 = @intCast(@mod(@as(i64, @intCast(@abs(@divTrunc(total, 3600)))), 100)); + const abs_m: u64 = @intCast(@abs(@rem(@divTrunc(total, 60), 60))); + const abs_s: u64 = @intCast(@abs(@rem(total, 60))); + var b: [16]u8 = undefined; + try ctx.put(if (total < 0) "-" else "+"); + if (style == .full) { + try ctx.put(std.fmt.bufPrint(&b, "{d:0>2}:{d:0>2}", .{ abs_h, abs_m }) catch unreachable); + if (abs_s != 0) try ctx.put(std.fmt.bufPrint(&b, ":{d:0>2}", .{abs_s}) catch unreachable); + } else { + try ctx.put(std.fmt.bufPrint(&b, "{d}", .{abs_h}) catch unreachable); + if (abs_m != 0 or abs_s != 0) { + try ctx.put(std.fmt.bufPrint(&b, ":{d:0>2}", .{abs_m}) catch unreachable); + if (abs_s != 0) try ctx.put(std.fmt.bufPrint(&b, ":{d:0>2}", .{abs_s}) catch unreachable); + } + } + return true; +} + +/// A short rendering of a temporal for an error message (the JVM prints the +/// value's toString). +pub fn temporalText(buf: []u8, t: *const Temporal) []const u8 { + if (t.date) |ed| { + var db: [24]u8 = undefined; + const ds = instant.formatLocalDate(&db, ed); + if (t.time) |nod| { + var tb: [24]u8 = undefined; + return std.fmt.bufPrint(buf, "{s}T{s}", .{ ds, instant.formatLocalTime(&tb, nod) }) catch buf[0..0]; + } + return std.fmt.bufPrint(buf, "{s}", .{ds}) catch buf[0..0]; // ds lives in db, not buf + } + if (t.time) |nod| return instant.formatLocalTime(buf, nod); + if (t.instant_secs) |s| return instant.formatIsoInstant(buf, s * 1000, t.instant_nanos); + return buf[0..0]; +} + +// --- parsing --- + +const ParseState = struct { + fields: FieldSet = .{}, + zone: ?Zone = null, +}; + +const ParseCtx = struct { + text: []const u8, + case_sensitive: bool = true, + strict: bool = true, + state: ParseState = .{}, + diag: *Diag, + + /// `setParsedField`: a second, different value for one field fails. + fn setField(self: *ParseCtx, f: Field, v: i64, err_pos: usize, ok_pos: usize) i64 { + const old = self.state.fields.put(f, v); + if (old != null and old.? != v) return fail(err_pos); + return @intCast(ok_pos); + } + + fn regionMatches(self: *const ParseCtx, pos: usize, s: []const u8) bool { + if (pos + s.len > self.text.len) return false; + const t = self.text[pos .. pos + s.len]; + return if (self.case_sensitive) std.mem.eql(u8, t, s) else std.ascii.eqlIgnoreCase(t, s); + } +}; + +/// The JVM's `~position` error encoding. +fn fail(pos: usize) i64 { + return ~@as(i64, @intCast(pos)); +} + +/// How a parse failed, before resolution: `index` is where the text stopped +/// matching; `unparsed` means the whole program matched but text was left. +pub const ParseFailure = union(enum) { index: usize, unparsed: usize }; + +/// `DateTimeFormatter.parse(text)` to the resolved field set. On a parse +/// failure returns `error.Parse` with `failure` set; on a resolve failure +/// returns `error.DateTime` with `diag` set (the caller words it as +/// "could not be parsed: "). +pub fn parse(p: *const Program, text: []const u8, diag: *Diag, failure: *ParseFailure) (Error || error{Parse})!Parsed { + var ctx: ParseCtx = .{ .text = text, .diag = diag }; + const r = try parseOps(&ctx, p.ops, 0, false); + if (r < 0) { + failure.* = .{ .index = @intCast(~r) }; + return error.Parse; + } + if (r != text.len) { + failure.* = .{ .unparsed = @intCast(r) }; + return error.Parse; + } + var parsed: Parsed = .{ .fields = ctx.state.fields, .zone = ctx.state.zone orelse p.zone }; + try parsed.resolve(p.resolver, diag); + return parsed; +} + +fn parseOps(ctx: *ParseCtx, ops: []const Op, position: usize, optional: bool) error{DateTime}!i64 { + if (optional) { + const saved = ctx.state; + var pos: i64 = @intCast(position); + for (ops, 0..) |op, i| { + pos = try parseOp(ctx, ops, i, op, @intCast(pos)); + if (pos < 0) { + ctx.state = saved; + return @intCast(position); + } + } + return pos; + } + var pos: i64 = @intCast(position); + for (ops, 0..) |op, i| { + pos = try parseOp(ctx, ops, i, op, @intCast(pos)); + if (pos < 0) break; + } + return pos; +} + +fn parseOp(ctx: *ParseCtx, ops: []const Op, idx: usize, op: Op, pos: usize) error{DateTime}!i64 { + _ = ops; + _ = idx; + switch (op) { + .value => |v| return parseValue(ctx, v, pos), + .fraction => |fr| return parseFraction(ctx, fr, pos), + .text => |t| return parseText(ctx, t.field, t.style, pos), + .literal => |s| { + if (pos > ctx.text.len) return fail(pos); + if (!ctx.regionMatches(pos, s)) return fail(pos); + return @intCast(pos + s.len); + }, + .offset => |o| return parseOffset(ctx, o.pattern, o.no_offset_text, pos), + .localized_offset => |st| return parseLocalizedOffset(ctx, st, pos), + .zone_id, .zone_region_id => return parseZone(ctx, pos), + .instant => return parseInstant(ctx, pos), + .case_sensitive => |on| { + ctx.case_sensitive = on; + return @intCast(pos); + }, + .strict => |on| { + ctx.strict = on; + return @intCast(pos); + }, + .default_value => |d| { + if (!ctx.state.fields.has(d.field)) _ = ctx.state.fields.put(d.field, d.value); + return @intCast(pos); + }, + .composite => |c| return parseOps(ctx, c.ops, pos, c.optional), + } +} + +fn digitAt(text: []const u8, pos: usize) ?u8 { + if (pos >= text.len) return null; + const c = text[pos]; + return if (c >= '0' and c <= '9') c - '0' else null; +} + +fn isFixedWidth(v: ValueOp, strict: bool) bool { + if (v.reduced_base != null and !strict) return false; + return v.subsequent == -1 or (v.subsequent > 0 and v.min == v.max and v.sign == .not_negative); +} + +/// `NumberPrinterParser.parse` (and the ReducedPrinterParser window). +fn parseValue(ctx: *ParseCtx, v: ValueOp, position_in: usize) error{DateTime}!i64 { + var position = position_in; + const text = ctx.text; + if (position >= text.len) return fail(position); + const fixed = isFixedWidth(v, ctx.strict); + const eff_min: usize = if (ctx.strict or fixed) v.min else 1; + var negative = false; + var positive = false; + const sign = text[position]; + if (sign == '+') { + if (!v.sign.allows(true, ctx.strict, v.min == v.max)) return fail(position); + positive = true; + position += 1; + } else if (sign == '-') { + if (!v.sign.allows(false, ctx.strict, v.min == v.max)) return fail(position); + negative = true; + position += 1; + } else if (v.sign == .always and ctx.strict) { + return fail(position); + } + var eff_max: usize = (if (ctx.strict or fixed) @as(usize, v.max) else 9) + @as(usize, @intCast(@max(v.subsequent, 0))); + const min_end = position + eff_min; + if (min_end > text.len) return fail(position); + var total: i128 = 0; + var pos = position; + var pass: u8 = 0; + while (pass < 2) : (pass += 1) { + const max_end = @min(pos + eff_max, text.len); + while (pos < max_end) { + const d = digitAt(text, pos) orelse { + if (pos < min_end) return fail(position); + break; + }; + total = total * 10 + d; + pos += 1; + } + if (v.subsequent > 0 and pass == 0) { + const parse_len = pos - position; + eff_max = @max(eff_min, parse_len -| @as(usize, @intCast(v.subsequent))); + pos = position; + total = 0; + } else break; + } + if (total > std.math.maxInt(i64)) return fail(position); + if (negative) { + if (total == 0 and ctx.strict) return fail(position - 1); + total = -total; + } else if (v.sign == .exceeds_pad and ctx.strict) { + const parse_len = pos - position; + if (positive) { + if (parse_len <= v.min) return fail(position - 1); + } else if (parse_len > v.min) { + return fail(position); + } + } + var value: i64 = @intCast(total); + if (v.reduced_base) |base| { + const parse_len = pos - position; + if (parse_len == v.min and value >= 0) { + const range = EXCEED_POINTS[parse_len]; + const last = @mod(base, range); + const base_part = base - last; + value = if (base > 0) base_part + value else base_part - value; + if (value < base) value += range; + } + } + return ctx.setField(v.field, value, position, pos); +} + +/// `FractionPrinterParser.parse`. +fn parseFraction(ctx: *ParseCtx, fr: FractionOp, position_in: usize) error{DateTime}!i64 { + var position = position_in; + const text = ctx.text; + const eff_min: usize = if (ctx.strict) fr.min else 0; + const eff_max: usize = if (ctx.strict) fr.max else 9; + if (position == text.len) return if (eff_min > 0) fail(position) else @intCast(position); + if (fr.decimal_point) { + if (text[position] != '.') return if (eff_min > 0) fail(position) else @intCast(position); + position += 1; + } + const min_end = position + eff_min; + if (min_end > text.len) return fail(position); + const max_end = @min(position + eff_max, text.len); + var total: i64 = 0; + var pos = position; + while (pos < max_end) { + const d = digitAt(text, pos) orelse { + if (pos < min_end) return fail(position); + break; + }; + total = total * 10 + d; + pos += 1; + } + // value = floor(total / 10^digits × 10^scale) + const digits = pos - position; + const scale = fractionScale(fr.field); + var value: i64 = total; + if (digits > scale) { + var k = digits - scale; + while (k > 0) : (k -= 1) value = @divFloor(value, 10); + } else { + var k = scale - digits; + while (k > 0) : (k -= 1) value *= 10; + } + return ctx.setField(fr.field, value, position, pos); +} + +fn parseText(ctx: *ParseCtx, field: Field, style: TextStyle, position: usize) error{DateTime}!i64 { + if (textTable(field, style)) |_| { + // Strict: the one style; lenient: every style. Longest match wins. + const styles: []const TextStyle = if (ctx.strict) &.{style} else &.{ .full, .short, .narrow }; + var best_len: usize = 0; + var best_val: i64 = 0; + for (styles) |st| { + const tbl = textTable(field, st).?; + for (tbl.names, 0..) |nm, i| { + if (nm.len > best_len and ctx.regionMatches(position, nm)) { + best_len = nm.len; + best_val = tbl.first + @as(i64, @intCast(i)); + } + } + } + if (best_len > 0) return ctx.setField(field, best_val, position, position + best_len); + if (ctx.strict) return fail(position); + } + return parseValue(ctx, .{ .field = field, .min = 1, .max = 19, .sign = .normal }, position); +} + +/// Two digits at `array[0]`, after a colon when `colon` and not the hour. +fn offsetDigits(text: []const u8, colon: bool, index: usize, array: *[4]i64) bool { + var pos = array[0]; + if (pos < 0) return true; + if (colon and index != 1) { + if (pos + 1 > text.len or text[@intCast(pos)] != ':') return false; + pos += 1; + } + if (pos + 2 > text.len) return false; + const a = digitAt(text, @intCast(pos)) orelse return false; + const b = digitAt(text, @intCast(pos + 1)) orelse return false; + const v: i64 = @as(i64, a) * 10 + b; + if (v > 59) return false; + array[index] = v; + array[0] = pos + 2; + return true; +} + +/// `OffsetIdPrinterParser.parse` (padded-hour patterns). +fn parseOffset(ctx: *ParseCtx, style: u8, no_offset_text: []const u8, position: usize) error{DateTime}!i64 { + const text = ctx.text; + if (no_offset_text.len == 0) { + if (position == text.len) return ctx.setField(.offset_seconds, 0, position, position); + } else { + if (position == text.len) return fail(position); + if (ctx.regionMatches(position, no_offset_text)) + return ctx.setField(.offset_seconds, 0, position, position + no_offset_text.len); + } + const sign = text[position]; + if (sign == '+' or sign == '-') { + const negative: i64 = if (sign == '-') -1 else 1; + var colon = isColonStyle(style); + var array = [4]i64{ @intCast(position + 1), 0, 0, 0 }; + var parse_type = style; + if (!ctx.strict) { + if (colon or (parse_type == 0 and text.len > position + 3 and text[position + 3] == ':')) { + colon = true; + parse_type = 10; + } else parse_type = 9; + } + if (!offsetDigits(text, false, 1, &array)) array[0] = ~array[0]; + switch (parse_type) { + 0 => {}, + 1, 2 => _ = offsetDigits(text, colon, 2, &array), + 3, 4 => if (!offsetDigits(text, colon, 2, &array)) { + array[0] = ~array[0]; + }, + 5, 6 => { + if (!offsetDigits(text, colon, 2, &array)) array[0] = ~array[0]; + _ = offsetDigits(text, colon, 3, &array); + }, + 7, 8 => { + if (!offsetDigits(text, colon, 2, &array)) array[0] = ~array[0]; + if (!offsetDigits(text, colon, 3, &array)) array[0] = ~array[0]; + }, + 9, 10 => if (offsetDigits(text, colon, 2, &array)) { + _ = offsetDigits(text, colon, 3, &array); + }, + else => unreachable, + } + if (array[0] > 0) { + if (array[1] > 23 or array[2] > 59 or array[3] > 59) + return ctx.diag.fail(.value, "Value out of range: Hour[0-23], Minute[0-59], Second[0-59]", .{}); + const secs = negative * (array[1] * 3600 + array[2] * 60 + array[3]); + return ctx.setField(.offset_seconds, secs, position, @intCast(array[0])); + } + } + if (no_offset_text.len == 0) return ctx.setField(.offset_seconds, 0, position, position); + return fail(position); +} + +/// `LocalizedOffsetIdPrinterParser.parse`: `GMT`, then an optional signed +/// `h[h][:mm[:ss]]` (SHORT) or `hh:mm[:ss]` (FULL). +fn parseLocalizedOffset(ctx: *ParseCtx, style: TextStyle, position: usize) error{DateTime}!i64 { + const text = ctx.text; + if (!ctx.regionMatches(position, "GMT")) return fail(position); + var pos = position + 3; + if (pos == text.len) return ctx.setField(.offset_seconds, 0, position, pos); + const negative: i64 = switch (text[pos]) { + '+' => 1, + '-' => -1, + else => return ctx.setField(.offset_seconds, 0, position, pos), + }; + pos += 1; + var h: i64 = 0; + var m: i64 = 0; + var s: i64 = 0; + if (style == .full) { + const h1 = digitAt(text, pos) orelse return fail(position); + const h2 = digitAt(text, pos + 1) orelse return fail(position); + if (pos + 2 >= text.len or text[pos + 2] != ':') return fail(position); + h = @as(i64, h1) * 10 + h2; + const m1 = digitAt(text, pos + 3) orelse return fail(position); + const m2 = digitAt(text, pos + 4) orelse return fail(position); + m = @as(i64, m1) * 10 + m2; + pos += 5; + if (pos + 2 < text.len and text[pos] == ':') { + if (digitAt(text, pos + 1)) |s1| if (digitAt(text, pos + 2)) |s2| { + s = @as(i64, s1) * 10 + s2; + pos += 3; + }; + } + } else { + h = digitAt(text, pos) orelse return fail(position); + pos += 1; + if (pos < text.len) { + if (digitAt(text, pos)) |h2| { + h = h * 10 + h2; + pos += 1; + } + if (pos + 2 < text.len and text[pos] == ':') { + if (digitAt(text, pos + 1)) |m1| if (digitAt(text, pos + 2)) |m2| { + m = @as(i64, m1) * 10 + m2; + pos += 3; + if (pos + 2 < text.len and text[pos] == ':') { + if (digitAt(text, pos + 1)) |s1| if (digitAt(text, pos + 2)) |s2| { + s = @as(i64, s1) * 10 + s2; + pos += 3; + }; + } + }; + } + } + } + return ctx.setField(.offset_seconds, negative * (h * 3600 + m * 60 + s), position, pos); +} + +/// A zone id: the longest prefix that `ZoneId.of` accepts. Region ids beyond +/// UTC / GMT / UT need the tz database cljw does not ship, so they fail to +/// parse (which an optional section absorbs). +fn parseZone(ctx: *ParseCtx, position: usize) error{DateTime}!i64 { + const text = ctx.text; + var end = position; + while (end < text.len and (std.ascii.isAlphanumeric(text[end]) or std.mem.indexOfScalar(u8, "~/._+-:", text[end]) != null)) end += 1; + var len = end - position; + while (len > 0) : (len -= 1) { + var d: zone_offset.Diag = .{}; + const zone = zone_offset.parseZoneId(text[position .. position + len], &d) catch continue; + ctx.state.zone = zone; + return @intCast(position + len); + } + return fail(position); +} + +/// The ISO_INSTANT parse: ISO local date 'T' HH:mm:ss[.fraction] offset. +const INSTANT_OPS = [_]Op{ + .{ .composite = .{ .ops = &ISO_LOCAL_DATE_OPS, .optional = false } }, + .{ .literal = "T" }, + .{ .value = .{ .field = .hour_of_day, .min = 2, .max = 2, .sign = .not_negative } }, + .{ .literal = ":" }, + .{ .value = .{ .field = .minute_of_hour, .min = 2, .max = 2, .sign = .not_negative } }, + .{ .literal = ":" }, + .{ .value = .{ .field = .second_of_minute, .min = 2, .max = 2, .sign = .not_negative } }, + .{ .fraction = .{ .field = .nano_of_second, .min = 0, .max = 9, .decimal_point = true } }, + .{ .offset = .{ .pattern = 6, .no_offset_text = "Z" } }, +}; + +fn parseInstant(ctx: *ParseCtx, position: usize) error{DateTime}!i64 { + var sub: ParseCtx = .{ .text = ctx.text, .case_sensitive = ctx.case_sensitive, .strict = ctx.strict, .diag = ctx.diag }; + const pos = try parseOps(&sub, &INSTANT_OPS, position, false); + if (pos < 0) return pos; + const f = &sub.state.fields; + const year = f.get(.year).?; + const month = f.get(.month_of_year).?; + const day = f.get(.day_of_month).?; + var hour = f.get(.hour_of_day).?; + const min = f.get(.minute_of_hour).?; + var sec = f.get(.second_of_minute) orelse 0; + const nano = f.get(.nano_of_second) orelse 0; + const offset = f.get(.offset_seconds).?; + var days: i64 = 0; + if (hour == 24 and min == 0 and sec == 0 and nano == 0) { + hour = 0; + days = 1; + } else if (hour == 23 and min == 59 and sec == 60) { + sec = 59; + } + _ = try checkValid(.year, year, ctx.diag); + _ = try checkValid(.month_of_year, month, ctx.diag); + _ = try checkValid(.day_of_month, day, ctx.diag); + _ = try checkValid(.hour_of_day, hour, ctx.diag); + _ = try checkValid(.minute_of_hour, min, ctx.diag); + _ = try checkValid(.second_of_minute, sec, ctx.diag); + const ed = (try dateOf(year, month, day, ctx.diag)) + days; + const secs = ed * 86_400 + hour * 3600 + min * 60 + sec - offset; + const r = ctx.setField(.instant_seconds, secs, position, @intCast(pos)); + if (r < 0) return r; + return ctx.setField(.nano_of_second, nano, position, @intCast(r)); +} + +/// `LocalDate.of` validation: the epoch day, or "Invalid date …". +fn dateOf(y: i64, m: i64, d: i64, diag: *Diag) error{DateTime}!i64 { + if (d > 28 and d > instant.lengthOfMonth(y, m)) { + if (d == 29) return diag.fail(.value, "Invalid date 'February 29' as '{d}' is not a leap year", .{y}); + return diag.fail(.value, "Invalid date '{s} {d}'", .{ host_enum.name(.month, @intCast(m - 1)), d }); + } + return instant.daysFromCivil(y, m, d); +} + +// --- resolution --- + +/// `java.time.format.Parsed` after `resolve`: the leftover fields plus the +/// resolved date / time and the zone. +pub const Parsed = struct { + fields: FieldSet, + zone: ?Zone = null, + date: ?i64 = null, + time: ?i64 = null, + excess_days: i64 = 0, + resolver: ResolverStyle = .smart, + + pub fn temporal(self: *const Parsed) Temporal { + return .{ .date = self.date, .time = self.time, .zone = self.zone, .fields = &self.fields }; + } + + fn addFieldValue(self: *Parsed, f: Field, v: i64, diag: *Diag) error{DateTime}!void { + if (self.fields.get(f)) |old| { + if (old != v) return diag.fail(.value, "Conflict found: {s} {d} differs from {s} {d}", .{ f.display(), old, f.display(), v }); + } + _ = self.fields.put(f, v); + } + + fn updateCheckConflict(self: *Parsed, target: Field, change: Field, v: i64, diag: *Diag) error{DateTime}!void { + const old = self.fields.put(change, v); + if (old != null and old.? != v) + return diag.fail(.value, "Conflict found: {s} {d} differs from {s} {d} while resolving {s}", .{ change.display(), old.?, change.display(), v, target.display() }); + } + + fn setDate(self: *Parsed, ed: i64, diag: *Diag) error{DateTime}!void { + if (self.date) |cur| { + if (cur != ed) { + var a: [24]u8 = undefined; + var b: [24]u8 = undefined; + return diag.fail(.value, "Conflict found: Fields resolved to two different dates: {s} {s}", .{ instant.formatLocalDate(&a, cur), instant.formatLocalDate(&b, ed) }); + } + } else self.date = ed; + } + + fn setTime(self: *Parsed, nod: i64, excess: i64, diag: *Diag) error{DateTime}!void { + if (self.time) |cur| { + if (cur != nod) { + var a: [24]u8 = undefined; + var b: [24]u8 = undefined; + return diag.fail(.value, "Conflict found: Fields resolved to different times: {s} {s}", .{ instant.formatLocalTime(&a, cur), instant.formatLocalTime(&b, nod) }); + } + if (self.excess_days != 0 and excess != 0 and self.excess_days != excess) + return diag.fail(.value, "Conflict found: Fields resolved to different excess periods: P{d}D P{d}D", .{ self.excess_days, excess }); + self.excess_days = excess; + } else { + self.time = nod; + self.excess_days = excess; + } + } + + /// `Parsed.resolve` for the ISO chronology, SMART or STRICT. + pub fn resolve(self: *Parsed, style: ResolverStyle, diag: *Diag) error{DateTime}!void { + self.resolver = style; + try self.resolveDate(diag); + try self.resolveTimeFields(diag); + try self.resolveTimeLenient(diag); + try self.crossCheck(diag); + if (self.date != null and self.time != null and self.excess_days != 0) { + self.date = self.date.? + self.excess_days; + self.excess_days = 0; + } + self.resolveFractional(); + self.resolveInstant(); + } + + fn resolveDate(self: *Parsed, diag: *Diag) error{DateTime}!void { + const f = &self.fields; + if (f.remove(.epoch_day)) |ed| return self.setDate(try checkValid(.epoch_day, ed, diag), diag); + if (f.remove(.proleptic_month)) |pm| { + _ = try checkValid(.proleptic_month, pm, diag); + try self.addFieldValue(.month_of_year, @mod(pm, 12) + 1, diag); + try self.addFieldValue(.year, @divFloor(pm, 12), diag); + } + if (f.remove(.year_of_era)) |yoe| { + _ = try checkValid(.year_of_era, yoe, diag); + if (f.remove(.era)) |era| { + if (era == 1) { + try self.addFieldValue(.year, yoe, diag); + } else if (era == 0) { + try self.addFieldValue(.year, 1 - yoe, diag); + } else return diag.fail(.value, "Invalid value for era: {d}", .{era}); + } else if (self.resolver == .strict) { + if (f.get(.year)) |y| { + try self.addFieldValue(.year, if (y > 0) yoe else 1 - yoe, diag); + } else _ = f.put(.year_of_era, yoe); + } else { + const y = f.get(.year); + try self.addFieldValue(.year, if (y == null or y.? > 0) yoe else 1 - yoe, diag); + } + } else if (f.get(.era)) |era| { + _ = try checkValid(.era, era, diag); + } + if (!f.has(.year)) return; + if (f.has(.month_of_year) and f.has(.day_of_month)) { + const y = try checkValid(.year, f.remove(.year).?, diag); + const m = try checkValid(.month_of_year, f.remove(.month_of_year).?, diag); + var d = try checkValid(.day_of_month, f.remove(.day_of_month).?, diag); + if (self.resolver == .smart) { + if (m == 4 or m == 6 or m == 9 or m == 11) { + d = @min(d, 30); + } else if (m == 2) d = @min(d, instant.lengthOfMonth(y, 2)); + } + return self.setDate(try dateOf(y, m, d, diag), diag); + } + if (f.has(.day_of_year)) { + const y = try checkValid(.year, f.remove(.year).?, diag); + const doy = try checkValid(.day_of_year, f.remove(.day_of_year).?, diag); + if (doy == 366 and !instant.isLeap(y)) + return diag.fail(.value, "Invalid date 'DayOfYear 366' as '{d}' is not a leap year", .{y}); + return self.setDate(instant.daysFromCivil(y, 1, 1) + doy - 1, diag); + } + } + + fn checkUnlessZero(self: *Parsed, f: Field, v: i64, diag: *Diag) error{DateTime}!void { + if (self.resolver == .strict or v != 0) _ = try checkValid(f, v, diag); + } + + fn resolveTimeFields(self: *Parsed, diag: *Diag) error{DateTime}!void { + const f = &self.fields; + if (f.remove(.clock_hour_of_day)) |ch| { + try self.checkUnlessZero(.clock_hour_of_day, ch, diag); + try self.updateCheckConflict(.clock_hour_of_day, .hour_of_day, if (ch == 24) 0 else ch, diag); + } + if (f.remove(.clock_hour_of_ampm)) |ch| { + try self.checkUnlessZero(.clock_hour_of_ampm, ch, diag); + try self.updateCheckConflict(.clock_hour_of_ampm, .hour_of_ampm, if (ch == 12) 0 else ch, diag); + } + if (f.has(.ampm_of_day) and f.has(.hour_of_ampm)) { + const ap = try checkValid(.ampm_of_day, f.remove(.ampm_of_day).?, diag); + const hap = try checkValid(.hour_of_ampm, f.remove(.hour_of_ampm).?, diag); + try self.updateCheckConflict(.ampm_of_day, .hour_of_day, ap * 12 + hap, diag); + } + if (f.remove(.nano_of_day)) |nod| { + _ = try checkValid(.nano_of_day, nod, diag); + try self.updateCheckConflict(.nano_of_day, .hour_of_day, @divFloor(nod, 3_600_000_000_000), diag); + try self.updateCheckConflict(.nano_of_day, .minute_of_hour, @mod(@divFloor(nod, 60_000_000_000), 60), diag); + try self.updateCheckConflict(.nano_of_day, .second_of_minute, @mod(@divFloor(nod, 1_000_000_000), 60), diag); + try self.updateCheckConflict(.nano_of_day, .nano_of_second, @mod(nod, 1_000_000_000), diag); + } + if (f.remove(.micro_of_day)) |cod| { + _ = try checkValid(.micro_of_day, cod, diag); + try self.updateCheckConflict(.micro_of_day, .second_of_day, @divFloor(cod, 1_000_000), diag); + try self.updateCheckConflict(.micro_of_day, .micro_of_second, @mod(cod, 1_000_000), diag); + } + if (f.remove(.milli_of_day)) |lod| { + _ = try checkValid(.milli_of_day, lod, diag); + try self.updateCheckConflict(.milli_of_day, .second_of_day, @divFloor(lod, 1000), diag); + try self.updateCheckConflict(.milli_of_day, .milli_of_second, @mod(lod, 1000), diag); + } + if (f.remove(.second_of_day)) |sod| { + _ = try checkValid(.second_of_day, sod, diag); + try self.updateCheckConflict(.second_of_day, .hour_of_day, @divFloor(sod, 3600), diag); + try self.updateCheckConflict(.second_of_day, .minute_of_hour, @mod(@divFloor(sod, 60), 60), diag); + try self.updateCheckConflict(.second_of_day, .second_of_minute, @mod(sod, 60), diag); + } + if (f.remove(.minute_of_day)) |mod| { + _ = try checkValid(.minute_of_day, mod, diag); + try self.updateCheckConflict(.minute_of_day, .hour_of_day, @divFloor(mod, 60), diag); + try self.updateCheckConflict(.minute_of_day, .minute_of_hour, @mod(mod, 60), diag); + } + if (f.get(.nano_of_second)) |nos_in| { + var nos = try checkValid(.nano_of_second, nos_in, diag); + if (f.remove(.micro_of_second)) |cos| { + _ = try checkValid(.micro_of_second, cos, diag); + nos = cos * 1000 + @mod(nos, 1000); + try self.updateCheckConflict(.micro_of_second, .nano_of_second, nos, diag); + } + if (f.remove(.milli_of_second)) |los| { + _ = try checkValid(.milli_of_second, los, diag); + try self.updateCheckConflict(.milli_of_second, .nano_of_second, los * 1_000_000 + @mod(nos, 1_000_000), diag); + } + } + if (f.has(.hour_of_day) and f.has(.minute_of_hour) and f.has(.second_of_minute) and f.has(.nano_of_second)) { + const hod = f.remove(.hour_of_day).?; + const moh = f.remove(.minute_of_hour).?; + const som = f.remove(.second_of_minute).?; + const nos = f.remove(.nano_of_second).?; + try self.resolveTime(hod, moh, som, nos, diag); + } + } + + fn resolveTimeLenient(self: *Parsed, diag: *Diag) error{DateTime}!void { + const f = &self.fields; + if (self.time == null) { + if (f.remove(.milli_of_second)) |los| { + if (f.get(.micro_of_second)) |cos_in| { + const cos = los * 1000 + @mod(cos_in, 1000); + try self.updateCheckConflict(.milli_of_second, .micro_of_second, cos, diag); + _ = f.remove(.micro_of_second); + _ = f.put(.nano_of_second, cos * 1000); + } else _ = f.put(.nano_of_second, los * 1_000_000); + } else if (f.remove(.micro_of_second)) |cos| { + _ = f.put(.nano_of_second, cos * 1000); + } + if (f.get(.hour_of_day)) |hod| { + const moh = f.get(.minute_of_hour); + const som = f.get(.second_of_minute); + const nos = f.get(.nano_of_second); + const undefaultable = (moh == null and (som != null or nos != null)) or (moh != null and som == null and nos != null); + if (!undefaultable) { + try self.resolveTime(hod, moh orelse 0, som orelse 0, nos orelse 0, diag); + _ = f.remove(.hour_of_day); + _ = f.remove(.minute_of_hour); + _ = f.remove(.second_of_minute); + _ = f.remove(.nano_of_second); + } + } + } + // Validate the time-based fields left over. + for (0..FIELD_COUNT) |i| { + const fld: Field = @enumFromInt(i); + if (!fld.isTimeBased()) continue; + if (f.get(fld)) |v| _ = try checkValid(fld, v, diag); + } + } + + fn resolveTime(self: *Parsed, hod: i64, moh: i64, som: i64, nos: i64, diag: *Diag) error{DateTime}!void { + const m = try checkValid(.minute_of_hour, moh, diag); + const n = try checkValid(.nano_of_second, nos, diag); + if (self.resolver == .smart and hod == 24 and m == 0 and som == 0 and n == 0) + return self.setTime(0, 1, diag); + const h = try checkValid(.hour_of_day, hod, diag); + const s = try checkValid(.second_of_minute, som, diag); + try self.setTime(((h * 60 + m) * 60 + s) * 1_000_000_000 + n, 0, diag); + } + + /// Every leftover field the resolved date / time also carries must agree + /// with it, and is then dropped. + fn crossCheck(self: *Parsed, diag: *Diag) error{DateTime}!void { + if (self.date == null and self.time == null) return; + const view: Temporal = .{ .date = self.date, .time = self.time }; + for (0..FIELD_COUNT) |i| { + const fld: Field = @enumFromInt(i); + const v2 = self.fields.get(fld) orelse continue; + const v1 = view.getField(fld) orelse continue; + if (v1 != v2) { + var tb: [64]u8 = undefined; + return diag.fail(.value, "Conflict found: Field {s} {d} differs from {s} {d} derived from {s}", .{ fld.display(), v1, fld.display(), v2, temporalText(&tb, &view) }); + } + _ = self.fields.remove(fld); + } + } + + fn resolveFractional(self: *Parsed) void { + const f = &self.fields; + if (self.time == null and (f.has(.instant_seconds) or f.has(.second_of_day) or f.has(.second_of_minute))) { + if (f.get(.nano_of_second)) |nos| { + _ = f.put(.micro_of_second, @divFloor(nos, 1000)); + _ = f.put(.milli_of_second, @divFloor(nos, 1_000_000)); + } else { + _ = f.put(.nano_of_second, 0); + _ = f.put(.micro_of_second, 0); + _ = f.put(.milli_of_second, 0); + } + } + } + + fn resolveInstant(self: *Parsed) void { + const ed = self.date orelse return; + const nod = self.time orelse return; + const offset: i64 = self.fields.get(.offset_seconds) orelse if (self.zone) |z| z.seconds else return; + _ = self.fields.put(.instant_seconds, ed * 86_400 + @divFloor(nod, 1_000_000_000) - offset); + } + + /// `Parsed.toString`: `{F=v, …},ISO[,zone][ resolved to date[Ttime]]`. + pub fn write(self: *const Parsed, out: *std.ArrayList(u8), gpa: Allocator) Allocator.Error!void { + var b: [64]u8 = undefined; + try out.append(gpa, '{'); + var first = true; + for (0..FIELD_COUNT) |i| { + const fld: Field = @enumFromInt(i); + const v = self.fields.get(fld) orelse continue; + if (!first) try out.appendSlice(gpa, ", "); + first = false; + try out.appendSlice(gpa, std.fmt.bufPrint(&b, "{s}={d}", .{ fld.display(), v }) catch unreachable); + } + try out.appendSlice(gpa, "},ISO"); + if (self.zone) |z| { + try out.append(gpa, ','); + try out.appendSlice(gpa, zone_offset.formatId(&b, z)); + } + if (self.date != null or self.time != null) { + try out.appendSlice(gpa, " resolved to "); + const view: Temporal = .{ .date = self.date, .time = self.time }; + try out.appendSlice(gpa, temporalText(&b, &view)); + } + } +}; + +// --- the ISO constants --- + +fn isoValue(f: Field, min: u8, max: u8, sign: SignStyle) Op { + return .{ .value = .{ .field = f, .min = min, .max = max, .sign = sign } }; +} + +fn isoFixed(f: Field, w: u8) Op { + return isoValue(f, w, w, .not_negative); +} + +fn comp(ops: []const Op, optional: bool) Op { + return .{ .composite = .{ .ops = ops, .optional = optional } }; +} + +const OFFSET_ID: Op = .{ .offset = .{ .pattern = 6, .no_offset_text = "Z" } }; + +const ISO_LOCAL_DATE_OPS = [_]Op{ isoValue(.year, 4, 10, .exceeds_pad), .{ .literal = "-" }, isoFixed(.month_of_year, 2), .{ .literal = "-" }, isoFixed(.day_of_month, 2) }; +const FRACTION_TAIL = [_]Op{.{ .fraction = .{ .field = .nano_of_second, .min = 0, .max = 9, .decimal_point = true } }}; +const SECONDS_TAIL = [_]Op{ .{ .literal = ":" }, isoFixed(.second_of_minute, 2), comp(&FRACTION_TAIL, true) }; +const ISO_LOCAL_TIME_OPS = [_]Op{ isoFixed(.hour_of_day, 2), .{ .literal = ":" }, isoFixed(.minute_of_hour, 2), comp(&SECONDS_TAIL, true) }; +const ISO_LOCAL_DATE_TIME_OPS = [_]Op{ .{ .case_sensitive = false }, comp(&ISO_LOCAL_DATE_OPS, false), .{ .literal = "T" }, comp(&ISO_LOCAL_TIME_OPS, false) }; +const ISO_OFFSET_DATE_TIME_OPS = [_]Op{ .{ .case_sensitive = false }, comp(&ISO_LOCAL_DATE_TIME_OPS, false), .{ .strict = false }, OFFSET_ID, .{ .strict = true } }; +const ISO_OFFSET_DATE_OPS = [_]Op{ .{ .case_sensitive = false }, comp(&ISO_LOCAL_DATE_OPS, false), OFFSET_ID }; +const ISO_OFFSET_TIME_OPS = [_]Op{ .{ .case_sensitive = false }, comp(&ISO_LOCAL_TIME_OPS, false), OFFSET_ID }; +const OPT_OFFSET = [_]Op{OFFSET_ID}; +const ISO_DATE_OPS = [_]Op{ .{ .case_sensitive = false }, comp(&ISO_LOCAL_DATE_OPS, false), comp(&OPT_OFFSET, true) }; +const ISO_TIME_OPS = [_]Op{ .{ .case_sensitive = false }, comp(&ISO_LOCAL_TIME_OPS, false), comp(&OPT_OFFSET, true) }; +const REGION_TAIL = [_]Op{ .{ .literal = "[" }, .{ .case_sensitive = true }, .zone_region_id, .{ .literal = "]" } }; +const ZONED_TAIL = [_]Op{ OFFSET_ID, comp(®ION_TAIL, true) }; +const ISO_DATE_TIME_OPS = [_]Op{ comp(&ISO_LOCAL_DATE_TIME_OPS, false), comp(&ZONED_TAIL, true) }; +const ISO_ORDINAL_DATE_OPS = [_]Op{ .{ .case_sensitive = false }, isoValue(.year, 4, 10, .exceeds_pad), .{ .literal = "-" }, isoFixed(.day_of_year, 3), comp(&OPT_OFFSET, true) }; +const BASIC_OFFSET = [_]Op{ .{ .strict = false }, .{ .offset = .{ .pattern = 5, .no_offset_text = "Z" } }, .{ .strict = true } }; +const BASIC_ISO_DATE_OPS = [_]Op{ + .{ .case_sensitive = false }, + .{ .value = .{ .field = .year, .min = 4, .max = 4, .sign = .not_negative, .subsequent = 4 } }, + .{ .value = .{ .field = .month_of_year, .min = 2, .max = 2, .sign = .not_negative, .subsequent = -1 } }, + .{ .value = .{ .field = .day_of_month, .min = 2, .max = 2, .sign = .not_negative, .subsequent = -1 } }, + comp(&BASIC_OFFSET, true), +}; +const ISO_INSTANT_OPS = [_]Op{ .{ .case_sensitive = false }, .instant }; + +/// The `DateTimeFormatter/ISO_*` constants (all STRICT, no zone), in the +/// static-field index order the surface declares. +pub const ISO_CONSTANTS = [_]struct { name: []const u8, ops: []const Op }{ + .{ .name = "ISO_LOCAL_DATE", .ops = &ISO_LOCAL_DATE_OPS }, + .{ .name = "ISO_LOCAL_TIME", .ops = &ISO_LOCAL_TIME_OPS }, + .{ .name = "ISO_LOCAL_DATE_TIME", .ops = &ISO_LOCAL_DATE_TIME_OPS }, + .{ .name = "ISO_OFFSET_DATE_TIME", .ops = &ISO_OFFSET_DATE_TIME_OPS }, + .{ .name = "ISO_OFFSET_DATE", .ops = &ISO_OFFSET_DATE_OPS }, + .{ .name = "ISO_OFFSET_TIME", .ops = &ISO_OFFSET_TIME_OPS }, + .{ .name = "ISO_DATE", .ops = &ISO_DATE_OPS }, + .{ .name = "ISO_TIME", .ops = &ISO_TIME_OPS }, + .{ .name = "ISO_DATE_TIME", .ops = &ISO_DATE_TIME_OPS }, + .{ .name = "ISO_ORDINAL_DATE", .ops = &ISO_ORDINAL_DATE_OPS }, + .{ .name = "BASIC_ISO_DATE", .ops = &BASIC_ISO_DATE_OPS }, + .{ .name = "ISO_INSTANT", .ops = &ISO_INSTANT_OPS }, +}; + +/// The `ISO_CONSTANTS` index of `name` (comptime: a typo is a build error). +pub fn isoIndex(comptime name: []const u8) u8 { + inline for (ISO_CONSTANTS, 0..) |c, i| { + if (comptime std.mem.eql(u8, c.name, name)) return i; + } + @compileError("no ISO constant " ++ name); +} + +/// ISO constant `idx` as a borrowed (static, never freed) program. +pub fn isoStatic(idx: u8) Program { + return ISO_PROGRAMS[idx]; +} + +/// The ISO constants as static programs (STRICT, no zone): the interned +/// `DateTimeFormatter/ISO_*` instances point here and never free them. +pub const ISO_PROGRAMS = build: { + var arr: [ISO_CONSTANTS.len]Program = undefined; + for (&arr, ISO_CONSTANTS) |*p, c| p.* = .{ .ops = c.ops, .resolver = .strict }; + break :build arr; +}; + +// --- tests --- + +const testing = std.testing; + +fn formatToString(gpa: Allocator, p: *const Program, t: Temporal) ![]u8 { + var out: std.ArrayList(u8) = .empty; + errdefer out.deinit(gpa); + var diag: Diag = .{}; + try format(gpa, p, &t, &out, &diag); + return out.toOwnedSlice(gpa); +} + +test "ofPattern compiles to the JVM toString form" { + const gpa = testing.allocator; + var diag: Diag = .{}; + const p = try compilePattern(gpa, "yyyy-MM-dd'T'HH:mm:ss.SSSXXX", &diag); + defer p.destroy(gpa); + var out: std.ArrayList(u8) = .empty; + defer out.deinit(gpa); + try writeProgram(&out, gpa, p); + try testing.expectEqualStrings("Value(YearOfEra,4,19,EXCEEDS_PAD)'-'Value(MonthOfYear,2)'-'Value(DayOfMonth,2)'T'Value(HourOfDay,2)':'Value(MinuteOfHour,2)':'Value(SecondOfMinute,2)'.'Fraction(NanoOfSecond,3,3)Offset(+HH:MM,'Z')", out.items); + try testing.expectError(error.DateTime, compilePattern(gpa, "bogus{", &diag)); + try testing.expectEqualStrings("Unknown pattern letter: b", diag.msg()); +} + +test "format: local date patterns and the optional section" { + const gpa = testing.allocator; + var diag: Diag = .{}; + const ed = instant.daysFromCivil(2024, 3, 9); + const p = try compilePattern(gpa, "EEEE, MMMM dd yy D['T'HH]", &diag); + defer p.destroy(gpa); + const s = try formatToString(gpa, p, .{ .date = ed }); + defer gpa.free(s); + try testing.expectEqualStrings("Saturday, March 09 24 69", s); +} + +test "parse + resolve: ISO_OFFSET_DATE_TIME to an instant" { + const gpa = testing.allocator; + _ = gpa; + const p = isoStatic(isoIndex("ISO_OFFSET_DATE_TIME")); + var diag: Diag = .{}; + var failure: ParseFailure = undefined; + const parsed = try parse(&p, "2024-01-01T10:00:00+02:00", &diag, &failure); + try testing.expectEqual(@as(?i64, 1_704_096_000), parsed.fields.get(.instant_seconds)); + try testing.expectError(error.Parse, parse(&p, "2024-01-01T10:00:00+02:00x", &diag, &failure)); + try testing.expectEqual(ParseFailure{ .unparsed = 25 }, failure); +} + +test "parse: adjacent value parsing splits yyyyMMdd" { + const gpa = testing.allocator; + var diag: Diag = .{}; + const p = try compilePattern(gpa, "yyyyMMdd", &diag); + defer p.destroy(gpa); + var failure: ParseFailure = undefined; + const parsed = try parse(p, "20240309", &diag, &failure); + try testing.expectEqual(@as(?i64, instant.daysFromCivil(2024, 3, 9)), parsed.date); +} diff --git a/src/runtime/time/date_time_formatter_value.zig b/src/runtime/time/date_time_formatter_value.zig new file mode 100644 index 000000000..3254ef847 --- /dev/null +++ b/src/runtime/time/date_time_formatter_value.zig @@ -0,0 +1,732 @@ +// SPDX-License-Identifier: EPL-2.0 +//! The Value layer of `java.time.format`: `DateTimeFormatter`, +//! `DateTimeFormatterBuilder` and the parse result `java.time.format.Parsed` +//! as `.host_instance` values, plus the `TemporalAccessor` bridge every +//! java.time `from` / `parse(text, formatter)` / `format` goes through. +//! +//! Each instance holds one heap object of the rt-free engine +//! (`date_time_format.zig`) in `state[0]`, allocated on `rt.gc.infra` and +//! freed by the descriptor's `host_finalise` hook: a formatter owns an +//! immutable `Program` (`withZone` copies it), a builder owns its mutable +//! `Builder` (each `append…` mutates it and returns the same instance, like the +//! JVM's fluent builder), a Parsed owns its resolved field set. +//! +//! An engine `Diag` maps onto the error catalog here, so the JVM exception +//! class survives (`DateTimeParseException`, `UnsupportedTemporalTypeException`, +//! `DateTimeException`, `IllegalArgumentException`). + +const std = @import("std"); +const Value = @import("../value/value.zig").Value; +const HeapHeader = @import("../value/value.zig").HeapHeader; +const Runtime = @import("../runtime.zig").Runtime; +const Env = @import("../env.zig").Env; +const td_mod = @import("../type_descriptor.zig"); +const TypeDescriptor = td_mod.TypeDescriptor; +const TypedInstance = td_mod.TypedInstance; +const error_catalog = @import("../error/catalog.zig"); +const SourceLocation = @import("../error/info.zig").SourceLocation; +const host_instance = @import("../host_instance.zig"); +const host_enum = @import("../host_enum.zig"); +const string_collection = @import("../collection/string.zig"); +const engine = @import("date_time_format.zig"); +const zone_offset = @import("zone_offset.zig"); +const instant_value = @import("instant_value.zig"); +const local_date_value = @import("local_date_value.zig"); +const local_time_value = @import("local_time_value.zig"); +const local_date_time_value = @import("local_date_time_value.zig"); + +pub const FORMATTER_FQCN = "java.time.format.DateTimeFormatter"; +pub const BUILDER_FQCN = "java.time.format.DateTimeFormatterBuilder"; +pub const PARSED_FQCN = "java.time.format.Parsed"; + +const BuiltinFn = *const fn (*Runtime, *Env, []const Value, SourceLocation) anyerror!Value; + +// --- host-instance plumbing --- + +fn isHostOf(v: Value, fqcn: []const u8) bool { + if (v.tag() != .host_instance) return false; + const fq = host_instance.asHostInstance(v).descriptor.fqcn orelse return false; + return std.mem.eql(u8, fq, fqcn); +} + +pub fn isFormatter(v: Value) bool { + return isHostOf(v, FORMATTER_FQCN); +} + +pub fn isParsed(v: Value) bool { + return isHostOf(v, PARSED_FQCN); +} + +fn payload(comptime T: type, v: Value) *T { + return @ptrFromInt(@as(usize, @intCast(host_instance.asHostInstance(v).state[0]))); +} + +pub fn programOf(v: Value) *const engine.Program { + return payload(engine.Program, v); +} + +/// The receiver of an instance method, type-checked. `Class/.member` hands out +/// the raw method fn, so args[0] can be any value; reading `state[0]` of a +/// non-instance would dereference garbage. +fn receiver(comptime T: type, v: Value, fqcn: []const u8, method: []const u8, loc: SourceLocation) !*T { + if (!isHostOf(v, fqcn)) + return error_catalog.raise(.type_arg_invalid, loc, .{ .fn_name = method, .expected = fqcn, .actual = classNameOf(v) }); + return payload(T, v); +} + +fn fmtRecv(v: Value, method: []const u8, loc: SourceLocation) !*const engine.Program { + return receiver(engine.Program, v, FORMATTER_FQCN, method, loc); +} + +fn builderRecv(v: Value, method: []const u8, loc: SourceLocation) !*engine.Builder { + return receiver(engine.Builder, v, BUILDER_FQCN, method, loc); +} + +fn parsedRecv(v: Value, method: []const u8, loc: SourceLocation) !*const engine.Parsed { + return receiver(engine.Parsed, v, PARSED_FQCN, method, loc); +} + +fn parsedOf(v: Value) *const engine.Parsed { + return payload(engine.Parsed, v); +} + +fn finaliseProgram(infra: std.mem.Allocator, state: *[host_instance.STATE_WORDS]u64) void { + if (state[1] == BORROWED) return; // an interned ISO constant's static program + const p: *engine.Program = @ptrFromInt(@as(usize, @intCast(state[0]))); + p.destroy(infra); +} + +fn finaliseBuilder(infra: std.mem.Allocator, state: *[host_instance.STATE_WORDS]u64) void { + const b: *engine.Builder = @ptrFromInt(@as(usize, @intCast(state[0]))); + b.destroy(infra); +} + +fn finaliseParsed(infra: std.mem.Allocator, state: *[host_instance.STATE_WORDS]u64) void { + const p: *engine.Parsed = @ptrFromInt(@as(usize, @intCast(state[0]))); + infra.destroy(p); +} + +/// Wrap an owned heap `Program` in a formatter instance (ownership moves). +pub fn wrapProgram(rt: *Runtime, p: *engine.Program) !Value { + errdefer p.destroy(rt.gc.infra); + const td = try formatterDescriptor(rt); + return host_instance.alloc(rt, td, .{ @intFromPtr(p), 0, 0, 0 }); +} + +fn wrapParsed(rt: *Runtime, parsed: engine.Parsed) !Value { + const p = try rt.gc.infra.create(engine.Parsed); + p.* = parsed; + errdefer rt.gc.infra.destroy(p); + const td = try td_mod.ensureRegistered(rt, PARSED_FQCN, &configureParsed); + return host_instance.alloc(rt, td, .{ @intFromPtr(p), 0, 0, 0 }); +} + +/// A fresh builder instance (the `(DateTimeFormatterBuilder.)` constructor). +pub fn newBuilder(rt: *Runtime) !Value { + const b = try engine.Builder.create(rt.gc.infra); + errdefer b.destroy(rt.gc.infra); + const td = try builderDescriptor(rt); + return host_instance.alloc(rt, td, .{ @intFromPtr(b), 0, 0, 0 }); +} + +/// `state[1]` of a formatter whose program is static (never freed). +const BORROWED: u64 = 1; + +/// `DateTimeFormatter/ISO_*` constant `idx`: one interned instance per +/// Runtime over the engine's static program (JVM `identical?` parity), on +/// `gc.infra` like the host-enum singletons; freed by `deinitIsoConstants`. +pub fn isoConstant(rt: *Runtime, idx: u8) !Value { + const slot = &rt.iso_formatters[idx]; + if (!slot.isNil()) return slot.*; + const td = try formatterDescriptor(rt); + const inst = try rt.gc.infra.create(host_instance.HostInstance); + inst.* = .{ + .header = HeapHeader.init(.host_instance), + .descriptor = td, + .state = .{ @intFromPtr(&engine.ISO_PROGRAMS[idx]), BORROWED, 0, 0 }, + }; + slot.* = Value.encodeHeapPtr(.host_instance, inst); + return slot.*; +} + +/// Release the interned ISO formatters. Called from `Runtime.deinit`. +pub fn deinitIsoConstants(rt: *Runtime) void { + for (&rt.iso_formatters) |*slot| { + if (slot.isNil()) continue; + rt.gc.infra.destroy(@constCast(host_instance.asHostInstance(slot.*))); + slot.* = .nil_val; + } +} + +// --- errors --- + +/// Raise the catalog error an engine `Diag` describes. +pub fn raiseDiag(diag: *const engine.Diag, loc: SourceLocation) anyerror { + return switch (diag.kind) { + .value => error_catalog.raise(.date_time_value_invalid, loc, .{ .detail = diag.msg() }), + .unsupported_field => error_catalog.raise(.temporal_field_unsupported, loc, .{ .field = diag.field.display() }), + .pattern => error_catalog.raise(.date_time_pattern_invalid, loc, .{ .detail = diag.msg() }), + .pattern_letter => error_catalog.raise(.date_time_pattern_letter_unsupported, loc, .{ .letter = diag.msg() }), + .state => error_catalog.raise(.date_time_builder_state_invalid, loc, .{ .detail = diag.msg() }), + }; +} + +/// Map an engine error: OOM passes through, `DateTime` raises its Diag. +fn engineErr(e: engine.Error, diag: *const engine.Diag, loc: SourceLocation) anyerror { + return switch (e) { + error.OutOfMemory => error.OutOfMemory, + error.DateTime => raiseDiag(diag, loc), + }; +} + +/// The JVM shortens a long input in a parse message to 64 chars + "...". +fn abbreviate(text: []const u8, buf: *[80]u8) []const u8 { + if (text.len <= 64) return text; + @memcpy(buf[0..64], text[0..64]); + @memcpy(buf[64..67], "..."); + return buf[0..67]; +} + +fn raiseUnparseable(text: []const u8, detail: []const u8, loc: SourceLocation) anyerror { + var ab: [80]u8 = undefined; + return error_catalog.raise(.date_time_text_unparseable, loc, .{ .text = abbreviate(text, &ab), .detail = detail }); +} + +// --- the TemporalAccessor bridge --- + +/// The engine view of a TemporalAccessor value, or null when `v` is not one. +pub fn temporalOf(v: Value) ?engine.Temporal { + if (isParsed(v)) return parsedOf(v).temporal(); + if (v.tag() != .typed_instance) return null; + if (local_date_value.isLocalDate(v)) return .{ .date = local_date_value.epochDayOf(v) }; + if (local_time_value.isLocalTime(v)) return .{ .time = local_time_value.nanoOfDayOf(v) }; + if (local_date_time_value.isLocalDateTime(v)) return .{ .date = local_date_time_value.epochDayOf(v), .time = local_date_time_value.nanoOfDayOf(v) }; + if (instant_value.isInstant(v)) return .{ .instant_secs = @divFloor(instant_value.epochMsOf(v), 1000), .instant_nanos = instant_value.nanosOf(v) }; + return null; +} + +/// The JVM class name of a value, for the "of type …" / ClassCastException text. +pub fn classNameOf(v: Value) []const u8 { + return switch (v.tag()) { + .host_instance => host_instance.asHostInstance(v).descriptor.fqcn orelse "java.lang.Object", + .typed_instance => v.decodePtr(*const TypedInstance).descriptor.fqcn orelse "java.lang.Object", + .string => "java.lang.String", + .integer => "java.lang.Long", + .float => "java.lang.Double", + .keyword => "clojure.lang.Keyword", + .nil => "nil", + else => @tagName(v.tag()), + }; +} + +/// The value's toString, for the "Unable to obtain" message. +fn temporalTextOf(rt: *Runtime, v: Value, t: *const engine.Temporal, buf: []u8) ![]const u8 { + if (isParsed(v)) { + var out: std.ArrayList(u8) = .empty; + defer out.deinit(rt.gpa); + try parsedOf(v).write(&out, rt.gpa); + const n = @min(out.items.len, buf.len); + @memcpy(buf[0..n], out.items[0..n]); + return buf[0..n]; + } + return engine.temporalText(buf, t); +} + +fn requireTemporal(v: Value, loc: SourceLocation) !engine.Temporal { + return temporalOf(v) orelse error_catalog.raise(.temporal_accessor_expected, loc, .{ .class = classNameOf(v) }); +} + +fn raiseUnconvertible(rt: *Runtime, target: []const u8, v: Value, t: *const engine.Temporal, loc: SourceLocation) anyerror { + var tb: [256]u8 = undefined; + const text = try temporalTextOf(rt, v, t, &tb); + return error_catalog.raise(.temporal_accessor_unconvertible, loc, .{ .target = target, .temporal = text, .class = classNameOf(v) }); +} + +/// The four `from` conversions (`LocalDate/from` …). +pub const Target = enum { + local_date, + local_time, + local_date_time, + instant, + + fn simpleName(self: Target) []const u8 { + return switch (self) { + .local_date => "LocalDate", + .local_time => "LocalTime", + .local_date_time => "LocalDateTime", + .instant => "Instant", + }; + } +}; + +/// `Target.from(temporal)`: the value, or the JVM DateTimeException. +pub fn convert(rt: *Runtime, target: Target, v: Value, loc: SourceLocation) !Value { + const t = try requireTemporal(v, loc); + return convertTemporal(rt, target, v, &t, loc); +} + +fn convertTemporal(rt: *Runtime, target: Target, v: Value, t: *const engine.Temporal, loc: SourceLocation) !Value { + switch (target) { + .local_date => { + const ed = t.date orelse return raiseUnconvertible(rt, "LocalDate", v, t, loc); + return local_date_value.make(rt, ed); + }, + .local_time => { + const nod = t.time orelse return raiseUnconvertible(rt, "LocalTime", v, t, loc); + return local_time_value.make(rt, nod); + }, + .local_date_time => { + const ed = t.date orelse return raiseUnconvertible(rt, "LocalDateTime", v, t, loc); + const nod = t.time orelse return raiseUnconvertible(rt, "LocalDateTime", v, t, loc); + return local_date_time_value.make(rt, ed, nod); + }, + .instant => { + if (instant_value.isInstant(v)) return v; + const secs = t.getField(.instant_seconds) orelse return raiseUnconvertible(rt, "Instant", v, t, loc); + const nanos = t.getField(.nano_of_second) orelse return raiseUnconvertible(rt, "Instant", v, t, loc); + return instant_value.make(rt, secs * 1000, @intCast(nanos)); + }, + } +} + +/// `Target.parse(text, formatter)`: parse, then convert; a conversion failure +/// is reworded as the DateTimeParseException the JVM's `parse(text, query)` +/// throws. +pub fn parseTo(rt: *Runtime, target: Target, text_v: Value, fmt_v: Value, fn_name: []const u8, loc: SourceLocation) !Value { + if (!isFormatter(fmt_v)) + return error_catalog.raise(.type_arg_invalid, loc, .{ .fn_name = fn_name, .expected = "a DateTimeFormatter", .actual = classNameOf(fmt_v) }); + return parseWith(rt, target, text_v, programOf(fmt_v), fn_name, loc); +} + +/// `Target.parse(text)`: the 1-arity form, which the JVM defines as +/// `parse(text, ISO_)`. +pub fn parseIso(rt: *Runtime, target: Target, text_v: Value, comptime iso_name: []const u8, fn_name: []const u8, loc: SourceLocation) !Value { + const p = engine.isoStatic(engine.isoIndex(iso_name)); + return parseWith(rt, target, text_v, &p, fn_name, loc); +} + +fn parseWith(rt: *Runtime, target: Target, text_v: Value, p: *const engine.Program, fn_name: []const u8, loc: SourceLocation) !Value { + if (text_v.tag() != .string) + return error_catalog.raise(.type_arg_not_string, loc, .{ .fn_name = fn_name, .actual = @tagName(text_v.tag()) }); + const text = string_collection.asString(text_v); + const parsed = try parseText(p, text, loc); + const t = parsed.temporal(); + const ok = switch (target) { + .local_date => t.date != null, + .local_time => t.time != null, + .local_date_time => t.date != null and t.time != null, + .instant => t.getField(.instant_seconds) != null and t.getField(.nano_of_second) != null, + }; + if (!ok) { + var out: std.ArrayList(u8) = .empty; + defer out.deinit(rt.gpa); + try out.appendSlice(rt.gpa, ": Unable to obtain "); + try out.appendSlice(rt.gpa, target.simpleName()); + try out.appendSlice(rt.gpa, " from TemporalAccessor: "); + try parsed.write(&out, rt.gpa); + try out.appendSlice(rt.gpa, " of type " ++ PARSED_FQCN); + return raiseUnparseable(text, out.items, loc); + } + // `ok` guarantees the conversion succeeds, so no receiver is needed for + // an error message. + return convertTemporal(rt, target, .nil_val, &t, loc); +} + +/// Run the engine parse, raising the DateTimeParseException on any failure. +fn parseText(p: *const engine.Program, text: []const u8, loc: SourceLocation) !engine.Parsed { + var diag: engine.Diag = .{}; + var failure: engine.ParseFailure = undefined; + return engine.parse(p, text, &diag, &failure) catch |e| { + var db: [300]u8 = undefined; + const detail = switch (e) { + error.OutOfMemory => return error.OutOfMemory, + error.Parse => switch (failure) { + .index => |i| std.fmt.bufPrint(&db, " at index {d}", .{i}) catch unreachable, + .unparsed => |i| std.fmt.bufPrint(&db, ", unparsed text found at index {d}", .{i}) catch unreachable, + }, + error.DateTime => std.fmt.bufPrint(&db, ": {s}", .{diag.msg()}) catch db[0..], + }; + return raiseUnparseable(text, detail, loc); + }; +} + +/// Format `v` with `fmt_v` (the shared body of `fmt.format(t)` and +/// `t.format(fmt)`). +pub fn formatValue(rt: *Runtime, fmt_v: Value, v: Value, loc: SourceLocation) !Value { + const p = programOf(fmt_v); + var t = try requireTemporal(v, loc); + if (p.zone) |zone| { + // The override zone re-projects anything that is an instant; other + // temporals only gain it for zone-id printing (DateTimePrintContext). + if (t.instant_secs != null and t.fields == null) { + t = engine.Temporal.atZone(t.instant_secs.?, t.instant_nanos, zone); + } else if (t.getField(.instant_seconds)) |secs| { + t = engine.Temporal.atZone(secs, @intCast(t.getField(.nano_of_second) orelse 0), zone); + } else { + t.zone = zone; + } + } + var out: std.ArrayList(u8) = .empty; + defer out.deinit(rt.gpa); + var diag: engine.Diag = .{}; + engine.format(rt.gpa, p, &t, &out, &diag) catch |e| return engineErr(e, &diag, loc); + return string_collection.alloc(rt, out.items); +} + +// --- DateTimeFormatter instance methods --- + +fn fmtFormat(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("format", args, 2, loc); + _ = try fmtRecv(args[0], "format", loc); + return formatValue(rt, args[0], args[1], loc); +} + +/// `(.parse fmt text)` — the resolved `Parsed` TemporalAccessor. +fn fmtParse(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("parse", args, 2, loc); + if (args[1].tag() != .string) + return error_catalog.raise(.type_arg_not_string, loc, .{ .fn_name = "parse", .actual = @tagName(args[1].tag()) }); + const parsed = try parseText(try fmtRecv(args[0], "parse", loc), string_collection.asString(args[1]), loc); + return wrapParsed(rt, parsed); +} + +/// `(.withZone fmt zone)` — a copy carrying the override zone. +fn fmtWithZone(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("withZone", args, 2, loc); + if (!args[1].isNil() and !zone_offset.isZone(args[1])) + return error_catalog.raise(.type_arg_invalid, loc, .{ .fn_name = "withZone", .expected = "a ZoneId", .actual = classNameOf(args[1]) }); + const copy = try (try fmtRecv(args[0], "withZone", loc)).clone(rt.gc.infra); + copy.zone = if (args[1].isNil()) null else zone_offset.zoneOf(args[1]); + return wrapProgram(rt, copy); +} + +fn fmtGetZone(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("getZone", args, 1, loc); + const zone = (try fmtRecv(args[0], "getZone", loc)).zone orelse return .nil_val; + return zone_offset.make(rt, zone); +} + +fn fmtToString(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("toString", args, 1, loc); + var out: std.ArrayList(u8) = .empty; + defer out.deinit(rt.gpa); + try engine.writeProgram(&out, rt.gpa, try fmtRecv(args[0], "toString", loc)); + return string_collection.alloc(rt, out.items); +} + +/// `(.withLocale fmt locale)`: cljw has one locale (English), so every +/// formatter already is in it; the JVM returns the receiver when the locale +/// is unchanged. +fn fmtWithLocale(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = rt; + _ = env; + try error_catalog.checkArity("withLocale", args, 2, loc); + _ = try fmtRecv(args[0], "withLocale", loc); + if (!isHostOf(args[1], "java.util.Locale")) + return error_catalog.raise(.type_arg_invalid, loc, .{ .fn_name = "withLocale", .expected = "a java.util.Locale", .actual = classNameOf(args[1]) }); + return args[0]; +} + +pub const FORMATTER_METHODS = [_]struct { []const u8, BuiltinFn }{ + .{ "format", &fmtFormat }, + .{ "withLocale", &fmtWithLocale }, + .{ "parse", &fmtParse }, + .{ "withZone", &fmtWithZone }, + .{ "getZone", &fmtGetZone }, + .{ "toString", &fmtToString }, +}; + +pub fn ensureFormatterMethods(td: *TypeDescriptor, gpa: std.mem.Allocator) anyerror!void { + td.host_finalise = &finaliseProgram; + if (td.lookupMethod(null, "format") != null) return; + try td_mod.appendMethodEntries(td, gpa, FORMATTER_METHODS); +} + +fn formatterDescriptor(rt: *Runtime) !*const TypeDescriptor { + return td_mod.ensureRegistered(rt, FORMATTER_FQCN, &ensureFormatterMethods); +} + +// --- DateTimeFormatterBuilder instance methods --- + +fn fieldArg(v: Value, fn_name: []const u8, loc: SourceLocation) !engine.Field { + if (v.tag() == .host_instance) { + const hi = host_instance.asHostInstance(v); + if (hi.descriptor.host_enum_idx) |idx| { + if (idx == @intFromEnum(host_enum.Idx.chrono_field)) return @enumFromInt(@as(u8, @intCast(hi.state[0]))); + } + } + return error_catalog.raise(.type_arg_invalid, loc, .{ .fn_name = fn_name, .expected = "a ChronoField", .actual = classNameOf(v) }); +} + +fn intArg(v: Value, fn_name: []const u8, loc: SourceLocation) !i64 { + if (v.tag() != .integer) + return error_catalog.raise(.type_arg_invalid, loc, .{ .fn_name = fn_name, .expected = "integer", .actual = @tagName(v.tag()) }); + return v.asInteger(); +} + +fn stringArg(v: Value, fn_name: []const u8, loc: SourceLocation) ![]const u8 { + if (v.tag() != .string) + return error_catalog.raise(.type_arg_not_string, loc, .{ .fn_name = fn_name, .actual = @tagName(v.tag()) }); + return string_collection.asString(v); +} + +fn formatterArg(v: Value, fn_name: []const u8, loc: SourceLocation) !*const engine.Program { + if (!isFormatter(v)) + return error_catalog.raise(.type_arg_invalid, loc, .{ .fn_name = fn_name, .expected = "a DateTimeFormatter", .actual = classNameOf(v) }); + return programOf(v); +} + +/// Run one builder mutation, mapping its Diag; the builder instance is the +/// result (the JVM builder returns `this`). +fn mutated(args: []const Value, r: engine.Error!void, diag: *const engine.Diag, loc: SourceLocation) !Value { + r catch |e| return engineErr(e, diag, loc); + return args[0]; +} + +fn bAppendPattern(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("appendPattern", args, 2, loc); + var diag: engine.Diag = .{}; + return mutated(args, (try builderRecv(args[0], "appendPattern", loc)).appendPattern(rt.gc.infra, try stringArg(args[1], "appendPattern", loc), &diag), &diag, loc); +} + +/// `appendValue(field)` / `appendValue(field, width)`. The 4-arity form +/// takes a `SignStyle`, which cljw does not surface. +fn bAppendValue(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArityRange("appendValue", args, 2, 3, loc); + const f = try fieldArg(args[1], "appendValue", loc); + var diag: engine.Diag = .{}; + const b = (try builderRecv(args[0], "appendValue", loc)); + if (args.len == 2) return mutated(args, b.appendValue(rt.gc.infra, f), &diag, loc); + return mutated(args, b.appendValueWidth(rt.gc.infra, f, try intArg(args[2], "appendValue", loc), &diag), &diag, loc); +} + +fn bAppendFraction(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("appendFraction", args, 5, loc); + const f = try fieldArg(args[1], "appendFraction", loc); + const min = try intArg(args[2], "appendFraction", loc); + const max = try intArg(args[3], "appendFraction", loc); + if (args[4].tag() != .boolean) + return error_catalog.raise(.type_arg_invalid, loc, .{ .fn_name = "appendFraction", .expected = "boolean", .actual = @tagName(args[4].tag()) }); + var diag: engine.Diag = .{}; + return mutated(args, (try builderRecv(args[0], "appendFraction", loc)).appendFraction(rt.gc.infra, f, min, max, args[4].asBoolean(), &diag), &diag, loc); +} + +/// `appendText(field)` — the FULL text style. +fn bAppendText(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("appendText", args, 2, loc); + var diag: engine.Diag = .{}; + return mutated(args, (try builderRecv(args[0], "appendText", loc)).appendText(rt.gc.infra, try fieldArg(args[1], "appendText", loc), .full), &diag, loc); +} + +/// `appendLiteral(char)` / `appendLiteral(String)`. +fn bAppendLiteral(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("appendLiteral", args, 2, loc); + var cb: [4]u8 = undefined; + const s: []const u8 = switch (args[1].tag()) { + .char => cb[0 .. std.unicode.utf8Encode(args[1].asChar(), &cb) catch 0], + .string => string_collection.asString(args[1]), + else => return error_catalog.raise(.type_arg_invalid, loc, .{ .fn_name = "appendLiteral", .expected = "a char or string", .actual = @tagName(args[1].tag()) }), + }; + var diag: engine.Diag = .{}; + return mutated(args, (try builderRecv(args[0], "appendLiteral", loc)).appendLiteral(rt.gc.infra, s), &diag, loc); +} + +fn bAppendOffset(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("appendOffset", args, 3, loc); + const pattern = try stringArg(args[1], "appendOffset", loc); + const no_offset = try stringArg(args[2], "appendOffset", loc); + var diag: engine.Diag = .{}; + return mutated(args, (try builderRecv(args[0], "appendOffset", loc)).appendOffset(rt.gc.infra, pattern, no_offset, &diag), &diag, loc); +} + +/// `appendOffsetId` = `appendOffset("+HH:MM:ss", "Z")`. +fn bAppendOffsetId(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("appendOffsetId", args, 1, loc); + var diag: engine.Diag = .{}; + return mutated(args, (try builderRecv(args[0], "appendOffsetId", loc)).appendOffset(rt.gc.infra, "+HH:MM:ss", "Z", &diag), &diag, loc); +} + +fn bAppendZoneId(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("appendZoneId", args, 1, loc); + var diag: engine.Diag = .{}; + return mutated(args, (try builderRecv(args[0], "appendZoneId", loc)).append(rt.gc.infra, .zone_id), &diag, loc); +} + +/// `appendInstant()` — ISO_INSTANT's printer-parser. +fn bAppendInstant(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("appendInstant", args, 1, loc); + var diag: engine.Diag = .{}; + return mutated(args, (try builderRecv(args[0], "appendInstant", loc)).append(rt.gc.infra, .instant), &diag, loc); +} + +fn bOptionalStart(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("optionalStart", args, 1, loc); + var diag: engine.Diag = .{}; + return mutated(args, (try builderRecv(args[0], "optionalStart", loc)).optionalStart(rt.gc.infra), &diag, loc); +} + +fn bOptionalEnd(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("optionalEnd", args, 1, loc); + var diag: engine.Diag = .{}; + return mutated(args, (try builderRecv(args[0], "optionalEnd", loc)).optionalEnd(rt.gc.infra, &diag), &diag, loc); +} + +fn setting(rt: *Runtime, args: []const Value, name: []const u8, op: engine.Op, loc: SourceLocation) !Value { + try error_catalog.checkArity(name, args, 1, loc); + var diag: engine.Diag = .{}; + return mutated(args, (try builderRecv(args[0], "optionalEnd", loc)).append(rt.gc.infra, op), &diag, loc); +} + +fn bParseCaseInsensitive(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + return setting(rt, args, "parseCaseInsensitive", .{ .case_sensitive = false }, loc); +} + +fn bParseCaseSensitive(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + return setting(rt, args, "parseCaseSensitive", .{ .case_sensitive = true }, loc); +} + +fn bParseStrict(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + return setting(rt, args, "parseStrict", .{ .strict = true }, loc); +} + +fn bParseLenient(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + return setting(rt, args, "parseLenient", .{ .strict = false }, loc); +} + +fn bParseDefaulting(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("parseDefaulting", args, 3, loc); + const f = try fieldArg(args[1], "parseDefaulting", loc); + const v = try intArg(args[2], "parseDefaulting", loc); + var diag: engine.Diag = .{}; + return mutated(args, (try builderRecv(args[0], "parseDefaulting", loc)).append(rt.gc.infra, .{ .default_value = .{ .field = f, .value = v } }), &diag, loc); +} + +fn bAppend(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("append", args, 2, loc); + var diag: engine.Diag = .{}; + return mutated(args, (try builderRecv(args[0], "append", loc)).appendProgram(rt.gc.infra, try formatterArg(args[1], "append", loc), false), &diag, loc); +} + +fn bAppendOptional(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("appendOptional", args, 2, loc); + var diag: engine.Diag = .{}; + return mutated(args, (try builderRecv(args[0], "appendOptional", loc)).appendProgram(rt.gc.infra, try formatterArg(args[1], "appendOptional", loc), true), &diag, loc); +} + +/// `toFormatter()` / `toFormatter(locale)` (cljw has one locale): SMART. +fn bToFormatter(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArityRange("toFormatter", args, 1, 2, loc); + var diag: engine.Diag = .{}; + const p = (try builderRecv(args[0], "toFormatter", loc)).toProgram(rt.gc.infra, .smart) catch |e| return engineErr(e, &diag, loc); + return wrapProgram(rt, p); +} + +pub const BUILDER_METHODS = [_]struct { []const u8, BuiltinFn }{ + .{ "appendPattern", &bAppendPattern }, + .{ "appendValue", &bAppendValue }, + .{ "appendFraction", &bAppendFraction }, + .{ "appendText", &bAppendText }, + .{ "appendLiteral", &bAppendLiteral }, + .{ "appendOffset", &bAppendOffset }, + .{ "appendOffsetId", &bAppendOffsetId }, + .{ "appendZoneId", &bAppendZoneId }, + .{ "appendInstant", &bAppendInstant }, + .{ "optionalStart", &bOptionalStart }, + .{ "optionalEnd", &bOptionalEnd }, + .{ "parseCaseInsensitive", &bParseCaseInsensitive }, + .{ "parseCaseSensitive", &bParseCaseSensitive }, + .{ "parseStrict", &bParseStrict }, + .{ "parseLenient", &bParseLenient }, + .{ "parseDefaulting", &bParseDefaulting }, + .{ "append", &bAppend }, + .{ "appendOptional", &bAppendOptional }, + .{ "toFormatter", &bToFormatter }, +}; + +pub fn ensureBuilderMethods(td: *TypeDescriptor, gpa: std.mem.Allocator) anyerror!void { + td.host_finalise = &finaliseBuilder; + if (td.lookupMethod(null, "appendPattern") != null) return; + try td_mod.appendMethodEntries(td, gpa, BUILDER_METHODS); +} + +fn builderDescriptor(rt: *Runtime) !*const TypeDescriptor { + return td_mod.ensureRegistered(rt, BUILDER_FQCN, &ensureBuilderMethods); +} + +// --- Parsed instance methods --- + +fn parsedGetLong(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = rt; + _ = env; + try error_catalog.checkArity("getLong", args, 2, loc); + const f = try fieldArg(args[1], "getLong", loc); + const t = (try parsedRecv(args[0], "getLong", loc)).temporal(); + const v = t.getField(f) orelse return error_catalog.raise(.temporal_field_unsupported, loc, .{ .field = f.display() }); + return Value.initInteger(v); +} + +fn parsedIsSupported(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = rt; + _ = env; + try error_catalog.checkArity("isSupported", args, 2, loc); + const f = try fieldArg(args[1], "isSupported", loc); + return Value.initBoolean((try parsedRecv(args[0], "isSupported", loc)).temporal().getField(f) != null); +} + +fn parsedToString(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("toString", args, 1, loc); + var out: std.ArrayList(u8) = .empty; + defer out.deinit(rt.gpa); + try (try parsedRecv(args[0], "toString", loc)).write(&out, rt.gpa); + return string_collection.alloc(rt, out.items); +} + +fn configureParsed(td: *TypeDescriptor, gpa: std.mem.Allocator) anyerror!void { + td.host_finalise = &finaliseParsed; + try td_mod.appendMethodEntries(td, gpa, .{ + .{ "getLong", &parsedGetLong }, + .{ "isSupported", &parsedIsSupported }, + .{ "toString", &parsedToString }, + }); +} + +// --- tests --- + +const testing = std.testing; + +test "temporalOf: the local types and Instant; non-temporals are null" { + var th = std.Io.Threaded.init(testing.allocator, .{}); + defer th.deinit(); + var rt = Runtime.init(th.io(), testing.allocator); + defer rt.deinit(); + const d = try local_date_value.make(&rt, 19_791); + try testing.expectEqual(@as(?i64, 19_791), temporalOf(d).?.date); + const i = try instant_value.make(&rt, 5000, 7); + try testing.expectEqual(@as(?i64, 5), temporalOf(i).?.instant_secs); + try testing.expect(temporalOf(Value.initInteger(1)) == null); +} diff --git a/src/runtime/time/instant.zig b/src/runtime/time/instant.zig index 57841915a..3d1a76dfc 100644 --- a/src/runtime/time/instant.zig +++ b/src/runtime/time/instant.zig @@ -388,54 +388,6 @@ pub fn formatLocalTime(buf: []u8, nano_of_day: i64) []const u8 { return buf[0..len]; } -/// Parse the ISO local date `yyyy-MM-dd` → `epoch_day` (signed). The year is -/// read as a positive 4-digit field. -pub fn parseLocalDate(s: []const u8) ParseError!i64 { - var i: usize = 0; - const y = try readN(s, &i, 4); - if (i >= s.len or s[i] != '-') return error.InvalidInstant; - i += 1; - const m = try readN(s, &i, 2); - if (i >= s.len or s[i] != '-') return error.InvalidInstant; - i += 1; - const d = try readN(s, &i, 2); - if (i != s.len) return error.InvalidInstant; - if (m < 1 or m > 12 or d < 1 or d > 31) return error.InvalidInstant; - return daysFromCivil(y, m, d); -} - -/// Parse the ISO local time `HH:mm[:ss[.fraction]]` → `nano_of_day` (the -/// fraction is 1-9 digits scaled to nanos). No offset / `Z`. -pub fn parseLocalTime(s: []const u8) ParseError!i64 { - var i: usize = 0; - const h = try readN(s, &i, 2); - if (i >= s.len or s[i] != ':') return error.InvalidInstant; - i += 1; - const mi = try readN(s, &i, 2); - var sec: i64 = 0; - var nanos: i64 = 0; - if (i < s.len and s[i] == ':') { - i += 1; - sec = try readN(s, &i, 2); - if (i < s.len and s[i] == '.') { - i += 1; - // Fractional seconds: 1-9 digits scaled to nanos (pad / truncate to 9). - var digits: usize = 0; - var acc: i64 = 0; - while (i < s.len and s[i] >= '0' and s[i] <= '9') : (i += 1) { - if (digits < 9) acc = acc * 10 + (s[i] - '0'); - digits += 1; - } - if (digits == 0) return error.InvalidInstant; - while (digits < 9) : (digits += 1) acc *= 10; - nanos = acc; - } - } - if (i != s.len) return error.InvalidInstant; - if (h > 23 or mi > 59 or sec > 59) return error.InvalidInstant; - return ((h * 60 + mi) * 60 + sec) * NANO_PER_SECOND + nanos; -} - // --- tests --- const testing = std.testing; @@ -531,20 +483,6 @@ test "formatLocalTime: conditional seconds + variable fraction (clj str form)" { try testing.expectEqualStrings("12:30:00.250", formatLocalTime(&buf, (12 * 60 + 30) * 60_000_000_000 + 250_000_000)); } -test "parseLocalDate / parseLocalTime round-trip + reject malformed" { - try testing.expectEqual(daysFromCivil(2024, 2, 29), try parseLocalDate("2024-02-29")); - try testing.expectEqual(@as(i64, 0), try parseLocalDate("1970-01-01")); - try testing.expectError(error.InvalidInstant, parseLocalDate("2024-13-01")); - try testing.expectError(error.InvalidInstant, parseLocalDate("not-a-date")); - try testing.expectError(error.InvalidInstant, parseLocalDate("2024-01-01T00:00")); - - try testing.expectEqual(((6 * 60 + 7) * 60 + 8) * 1_000_000_000, try parseLocalTime("06:07:08")); - try testing.expectEqual((12 * 60 + 30) * 60_000_000_000, try parseLocalTime("12:30")); - try testing.expectEqual(((12 * 60 + 30) * 60 + 45) * 1_000_000_000 + 500_000_000, try parseLocalTime("12:30:45.5")); - try testing.expectError(error.InvalidInstant, parseLocalTime("25:00")); - try testing.expectError(error.InvalidInstant, parseLocalTime("12")); -} - test "nowEpochMillis returns a sensible 2026-era epoch ms" { var th = std.Io.Threaded.init(testing.allocator, .{}); defer th.deinit(); diff --git a/src/runtime/time/zone_offset.zig b/src/runtime/time/zone_offset.zig new file mode 100644 index 000000000..cf37250ff --- /dev/null +++ b/src/runtime/time/zone_offset.zig @@ -0,0 +1,334 @@ +// SPDX-License-Identifier: EPL-2.0 +//! Fixed-offset time zones: `java.time.ZoneOffset` and the offset-backed +//! `java.time.ZoneRegion` ids `UTC` / `GMT` / `UT` (optionally suffixed +//! `UTC+01:00`). The zone a `DateTimeFormatter.withZone` carries and the +//! `ZoneId/of` result. +//! +//! Backend: impl-only +//! Impl deps: none +//! Clojure peer: none +//! +//! cljw ships no time-zone database, so a region id such as `Europe/Paris` +//! is refused (`zone_id_unknown`) instead of resolving to a guessed offset. +//! Every zone that IS representable has one fixed offset, so a zone is two +//! integers: the total offset seconds and the id prefix. A value is a +//! `.typed_instance` carrying exactly those two fields; the prefix selects the +//! class (`.none` = ZoneOffset, otherwise ZoneRegion), so `=` / `hash` by +//! fields + descriptor match the JVM's `ZoneOffset.equals` / `ZoneRegion.equals` +//! (both by id). +//! +//! The parse rules and the DateTimeException texts follow +//! `ZoneOffset.of` / `ZoneOffset.ofHoursMinutesSeconds` / `ZoneId.of`. + +const std = @import("std"); +const Value = @import("../value/value.zig").Value; +const Runtime = @import("../runtime.zig").Runtime; +const Env = @import("../env.zig").Env; +const td_mod = @import("../type_descriptor.zig"); +const TypeDescriptor = td_mod.TypeDescriptor; +const TypedInstance = td_mod.TypedInstance; +const error_catalog = @import("../error/catalog.zig"); +const SourceLocation = @import("../error/info.zig").SourceLocation; +const string_collection = @import("../collection/string.zig"); + +/// The id prefix of a region zone. `.none` is a bare ZoneOffset. +pub const Prefix = enum(u8) { none = 0, utc = 1, gmt = 2, ut = 3 }; + +pub const Zone = struct { + /// Total offset from UTC in seconds, within ±64800 (±18:00). + seconds: i32, + prefix: Prefix = .none, + + pub const utc_offset: Zone = .{ .seconds = 0 }; + + pub fn isRegion(self: Zone) bool { + return self.prefix != .none; + } +}; + +pub const MAX_SECONDS: i32 = 18 * 3600; + +/// A failed parse / validation: the JVM DateTimeException text, or `region` +/// when the id is a well-formed region id the missing tz database would serve. +pub const Diag = struct { + buf: [160]u8 = undefined, + len: usize = 0, + region: bool = false, + + pub fn msg(self: *const Diag) []const u8 { + return self.buf[0..self.len]; + } + + fn set(self: *Diag, comptime fmt: []const u8, args: anytype) error{Invalid} { + self.len = if (std.fmt.bufPrint(&self.buf, fmt, args)) |s| s.len else |_| self.buf.len; + return error.Invalid; + } +}; + +/// `ZoneOffset.ofHoursMinutesSeconds` validation + fold to total seconds. +pub fn ofHoursMinutesSeconds(h: i64, m: i64, s: i64, diag: *Diag) error{Invalid}!i32 { + if (h < -18 or h > 18) + return diag.set("Zone offset hours not in valid range: value {d} is not in the range -18 to 18", .{h}); + if (h > 0) { + if (m < 0 or s < 0) return diag.set("Zone offset minutes and seconds must be positive because hours is positive", .{}); + } else if (h < 0) { + if (m > 0 or s > 0) return diag.set("Zone offset minutes and seconds must be negative because hours is negative", .{}); + } else if ((m > 0 and s < 0) or (m < 0 and s > 0)) { + return diag.set("Zone offset minutes and seconds must have the same sign", .{}); + } + if (m < -59 or m > 59) + return diag.set("Zone offset minutes not in valid range: value {d} is not in the range -59 to 59", .{m}); + if (s < -59 or s > 59) + return diag.set("Zone offset seconds not in valid range: value {d} is not in the range -59 to 59", .{s}); + if ((h == 18 or h == -18) and (m != 0 or s != 0)) + return diag.set("Zone offset not in valid range: -18:00 to +18:00", .{}); + return @intCast(h * 3600 + m * 60 + s); +} + +/// `ZoneOffset.ofTotalSeconds` validation. +pub fn ofTotalSeconds(total: i64, diag: *Diag) error{Invalid}!i32 { + if (total < -MAX_SECONDS or total > MAX_SECONDS) + return diag.set("Zone offset not in valid range: -18:00 to +18:00", .{}); + return @intCast(total); +} + +fn twoDigits(id: []const u8, pos: usize, preceded_by_colon: bool, diag: *Diag) error{Invalid}!i64 { + if (preceded_by_colon and id[pos - 1] != ':') + return diag.set("Invalid ID for ZoneOffset, colon not found when expected: {s}", .{id}); + const a = id[pos]; + const b = id[pos + 1]; + if (a < '0' or a > '9' or b < '0' or b > '9') + return diag.set("Invalid ID for ZoneOffset, non numeric characters found: {s}", .{id}); + return (a - '0') * 10 + (b - '0'); +} + +/// `ZoneOffset.of(id)`: `Z`, `+h`, `+hh`, `+hh:mm`, `+hhmm`, `+hh:mm:ss`, +/// `+hhmmss` (and the `-` forms). +pub fn parseOffsetId(id: []const u8, diag: *Diag) error{Invalid}!i32 { + if (std.mem.eql(u8, id, "Z")) return 0; + var padded: [3]u8 = undefined; + var src = id; + if (id.len == 2) { + padded = .{ id[0], '0', id[1] }; + src = &padded; + } + var h: i64 = 0; + var m: i64 = 0; + var s: i64 = 0; + switch (src.len) { + 3 => h = try twoDigits(src, 1, false, diag), + 5 => { + h = try twoDigits(src, 1, false, diag); + m = try twoDigits(src, 3, false, diag); + }, + 6 => { + h = try twoDigits(src, 1, false, diag); + m = try twoDigits(src, 4, true, diag); + }, + 7 => { + h = try twoDigits(src, 1, false, diag); + m = try twoDigits(src, 3, false, diag); + s = try twoDigits(src, 5, false, diag); + }, + 9 => { + h = try twoDigits(src, 1, false, diag); + m = try twoDigits(src, 4, true, diag); + s = try twoDigits(src, 7, true, diag); + }, + else => return diag.set("Invalid ID for ZoneOffset, invalid format: {s}", .{id}), + } + const first = src[0]; + if (first != '+' and first != '-') + return diag.set("Invalid ID for ZoneOffset, plus/minus not found when expected: {s}", .{id}); + return if (first == '-') ofHoursMinutesSeconds(-h, -m, -s, diag) else ofHoursMinutesSeconds(h, m, s, diag); +} + +fn isRegionIdChar(c: u8, first: bool) bool { + if (std.ascii.isAlphabetic(c)) return true; + if (first) return false; + return std.ascii.isDigit(c) or c == '~' or c == '/' or c == '.' or c == '_' or c == '+' or c == '-'; +} + +/// `ZoneId.of(id)`: an offset id, `UTC` / `GMT` / `UT` with an optional +/// `+offset` suffix, or a region id (refused: `diag.region` is set). +pub fn parseZoneId(id: []const u8, diag: *Diag) error{Invalid}!Zone { + if (id.len <= 1 or id[0] == '+' or id[0] == '-') return .{ .seconds = try parseOffsetId(id, diag) }; + const prefixes = [_]struct { text: []const u8, p: Prefix }{ + .{ .text = "UTC", .p = .utc }, .{ .text = "GMT", .p = .gmt }, .{ .text = "UT", .p = .ut }, + }; + for (prefixes) |pre| { + if (!std.mem.startsWith(u8, id, pre.text)) continue; + const rest = id[pre.text.len..]; + if (rest.len == 0) return .{ .seconds = 0, .prefix = pre.p }; + if (rest[0] == '+' or rest[0] == '-') return .{ .seconds = try parseOffsetId(rest, diag), .prefix = pre.p }; + } + for (id, 0..) |c, i| { + if (!isRegionIdChar(c, i == 0)) + return diag.set("Invalid ID for region-based ZoneId, invalid format: {s}", .{id}); + } + diag.region = true; + return error.Invalid; +} + +/// The offset id: `Z`, or `±hh:mm` with `:ss` only when non-zero. +pub fn formatOffsetId(buf: []u8, seconds: i32) []const u8 { + if (seconds == 0) return std.fmt.bufPrint(buf, "Z", .{}) catch unreachable; + const abs: u32 = @abs(seconds); + const sign: u8 = if (seconds < 0) '-' else '+'; + const h = abs / 3600; + const m = (abs / 60) % 60; + const s = abs % 60; + if (s != 0) return std.fmt.bufPrint(buf, "{c}{d:0>2}:{d:0>2}:{d:0>2}", .{ sign, h, m, s }) catch unreachable; + return std.fmt.bufPrint(buf, "{c}{d:0>2}:{d:0>2}", .{ sign, h, m }) catch unreachable; +} + +/// The zone id: the offset id for a ZoneOffset; the prefix, then the offset id +/// when non-zero, for a region (`UTC`, `UTC+01:00`). +pub fn formatId(buf: []u8, zone: Zone) []const u8 { + const prefix: []const u8 = switch (zone.prefix) { + .none => return formatOffsetId(buf, zone.seconds), + .utc => "UTC", + .gmt => "GMT", + .ut => "UT", + }; + @memcpy(buf[0..prefix.len], prefix); + if (zone.seconds == 0) return buf[0..prefix.len]; + const tail = formatOffsetId(buf[prefix.len..], zone.seconds); + return buf[0 .. prefix.len + tail.len]; +} + +// --- the Value layer --- + +pub const OFFSET_FQCN = "java.time.ZoneOffset"; +pub const REGION_FQCN = "java.time.ZoneRegion"; +const ZONE_SUPERTYPES = [_][]const u8{"java.time.ZoneId"}; + +fn getIdFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("getId", args, 1, loc); + var buf: [24]u8 = undefined; + return string_collection.alloc(rt, formatId(&buf, zoneOf(args[0]))); +} + +fn getTotalSecondsFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = rt; + _ = env; + try error_catalog.checkArity("getTotalSeconds", args, 1, loc); + return Value.initInteger(zoneOf(args[0]).seconds); +} + +/// `(.normalized z)` — the ZoneOffset a fixed-offset zone always reduces to. +fn normalizedFn(rt: *Runtime, env: *Env, args: []const Value, loc: SourceLocation) anyerror!Value { + _ = env; + try error_catalog.checkArity("normalized", args, 1, loc); + return make(rt, .{ .seconds = zoneOf(args[0]).seconds }); +} + +/// Register the `Object/toString` the `str` path dispatches plus the shared +/// readers. Sentinel-guarded on `getId` (both registration orders). +fn ensureCommonMethods(td: *TypeDescriptor, gpa: std.mem.Allocator) !void { + if (td.lookupMethod(null, "getId") != null) return; + try td_mod.appendMethodEntries(td, gpa, .{ + .{ "getId", &getIdFn }, + .{ "normalized", &normalizedFn }, + }); + try appendObjectToString(td, gpa, &getIdFn); + td.host_supertypes = &ZONE_SUPERTYPES; +} + +/// Append a `toString` entry under the `Object` protocol, the one the +/// `str` path consults for a native `.typed_instance` (`dispatchOrNull`). +pub fn appendObjectToString(td: *TypeDescriptor, gpa: std.mem.Allocator, f: anytype) !void { + const old = td.method_table; + const entries = try gpa.alloc(TypeDescriptor.MethodEntry, old.len + 1); + @memcpy(entries[0..old.len], old); + entries[old.len] = .{ + .protocol_name = "Object", // borrowed: Runtime.deinit frees method_name only + .method_name = try gpa.dupe(u8, "toString"), + .method_val = Value.initBuiltinFn(f), + }; + if (old.len > 0) gpa.free(old); + td.method_table = entries; +} + +/// ZoneOffset instance methods (also run by the surface `init`). +pub fn ensureOffsetMethods(td: *TypeDescriptor, gpa: std.mem.Allocator) anyerror!void { + try ensureCommonMethods(td, gpa); + if (td.lookupMethod(null, "getTotalSeconds") == null) + try td_mod.appendMethodEntries(td, gpa, .{.{ "getTotalSeconds", &getTotalSecondsFn }}); +} + +fn configureRegion(td: *TypeDescriptor, gpa: std.mem.Allocator) anyerror!void { + try ensureCommonMethods(td, gpa); +} + +/// Mint the zone Value: a ZoneOffset for `.none`, else a ZoneRegion. +pub fn make(rt: *Runtime, zone: Zone) !Value { + const td = if (zone.isRegion()) + try td_mod.ensureRegistered(rt, REGION_FQCN, &configureRegion) + else + try td_mod.ensureRegistered(rt, OFFSET_FQCN, &ensureOffsetMethods); + return td_mod.allocInstance(rt, td, &.{ Value.initInteger(zone.seconds), Value.initInteger(@intFromEnum(zone.prefix)) }); +} + +/// True for a ZoneOffset or a ZoneRegion value. +pub fn isZone(v: Value) bool { + if (v.tag() != .typed_instance) return false; + const fq = v.decodePtr(*const TypedInstance).descriptor.fqcn orelse return false; + return std.mem.eql(u8, fq, OFFSET_FQCN) or std.mem.eql(u8, fq, REGION_FQCN); +} + +/// True for a ZoneOffset value only. +pub fn isZoneOffset(v: Value) bool { + return isZone(v) and zoneOf(v).prefix == .none; +} + +/// The zone a value carries. Caller must have checked `isZone`. +pub fn zoneOf(v: Value) Zone { + const f = v.decodePtr(*const TypedInstance).fields(); + return .{ .seconds = @intCast(f[0].asInteger()), .prefix = @enumFromInt(@as(u8, @intCast(f[1].asInteger()))) }; +} + +/// Raise the DateTimeException a failed `parseZoneId` / `parseOffsetId` +/// described. +pub fn raiseDiag(diag: *const Diag, id: []const u8, loc: SourceLocation) anyerror { + if (diag.region) return error_catalog.raise(.zone_id_unknown, loc, .{ .id = id }); + return error_catalog.raise(.date_time_value_invalid, loc, .{ .detail = diag.msg() }); +} + +// --- tests --- + +const testing = std.testing; + +test "parseOffsetId: the JVM ZoneOffset.of id forms" { + var d: Diag = .{}; + try testing.expectEqual(@as(i32, 0), try parseOffsetId("Z", &d)); + try testing.expectEqual(@as(i32, 7200), try parseOffsetId("+2", &d)); + try testing.expectEqual(@as(i32, 7200), try parseOffsetId("+02", &d)); + try testing.expectEqual(@as(i32, -19800), try parseOffsetId("-05:30", &d)); + try testing.expectEqual(@as(i32, -19800), try parseOffsetId("-0530", &d)); + try testing.expectEqual(@as(i32, 3661), try parseOffsetId("+01:01:01", &d)); + try testing.expectError(error.Invalid, parseOffsetId("+25:00", &d)); + try testing.expectEqualStrings("Zone offset hours not in valid range: value 25 is not in the range -18 to 18", d.msg()); + try testing.expectError(error.Invalid, parseOffsetId("+0x:00", &d)); + try testing.expectEqualStrings("Invalid ID for ZoneOffset, non numeric characters found: +0x:00", d.msg()); +} + +test "parseZoneId: prefixes, offsets, and a refused region" { + var d: Diag = .{}; + try testing.expectEqual(Zone{ .seconds = 0, .prefix = .utc }, try parseZoneId("UTC", &d)); + try testing.expectEqual(Zone{ .seconds = 3600, .prefix = .utc }, try parseZoneId("UTC+01:00", &d)); + try testing.expectEqual(Zone{ .seconds = 0, .prefix = .ut }, try parseZoneId("UT", &d)); + try testing.expectEqual(Zone{ .seconds = 0 }, try parseZoneId("Z", &d)); + try testing.expectError(error.Invalid, parseZoneId("Europe/Paris", &d)); + try testing.expect(d.region); +} + +test "formatId: offset and region ids" { + var buf: [24]u8 = undefined; + try testing.expectEqualStrings("Z", formatId(&buf, .{ .seconds = 0 })); + try testing.expectEqualStrings("+05:45", formatId(&buf, .{ .seconds = 20700 })); + try testing.expectEqualStrings("-00:00:30", formatId(&buf, .{ .seconds = -30 })); + try testing.expectEqualStrings("UTC", formatId(&buf, .{ .seconds = 0, .prefix = .utc })); + try testing.expectEqualStrings("GMT-03:00", formatId(&buf, .{ .seconds = -10800, .prefix = .gmt })); +} diff --git a/src/runtime/type_descriptor.zig b/src/runtime/type_descriptor.zig index de2e8a663..97524def3 100644 --- a/src/runtime/type_descriptor.zig +++ b/src/runtime/type_descriptor.zig @@ -232,6 +232,9 @@ pub const TypeDescriptor = struct { time_local_date_epoch, time_local_date_time_min, time_local_date_time_max, + time_zone_offset_utc, + time_zone_offset_min, // -18:00 + time_zone_offset_max, // +18:00 }; /// The shapes a static field can carry. `int` lifts via @@ -256,6 +259,9 @@ pub const TypeDescriptor = struct { /// A `java.math.MathContext` standard constant (0-3 = DECIMAL32/64/128/ /// UNLIMITED) → `math_context.zig::singleton` (D-511). math_context: u8, + /// A `java.time.format.DateTimeFormatter` ISO constant: the index into + /// `time/date_time_format.zig::ISO_CONSTANTS` → a fresh formatter. + date_time_formatter: u8, /// A small-integer `java.math.BigDecimal` constant (ZERO/ONE/TWO/TEN) — /// lifted via the BigDecimal constructor at analyze time (ADR-0174 D7), /// scale 0. diff --git a/test/clj/suites/arrays_test.clj b/test/clj/suites/arrays_test.clj index b0e57bb58..fe987da8e 100644 --- a/test/clj/suites/arrays_test.clj +++ b/test/clj/suites/arrays_test.clj @@ -1,16 +1,15 @@ ;; test/e2e/phase14_arrays.sh ;; -;; ADR-0105 / D-287 — Java arrays. Type-erased uniform []Value over -;; cljw.internal/__array-make + aget/aset/alength/aclone; the clojure.core surface -;; (object-array / int-array / byte-array / make-array / to-array / aset-* / -;; amap / areduce) composed in core.clj. Per-constructor init defaults + -;; byte/short/char wrap give clj-faithful VALUES (F-011); element type erased -;; (AD-019); identity equality; simple class name (AD-003). +;; ADR-0105 / D-287, ADR-0204 — Java arrays with an element kind: object +;; arrays hold Values, the typed constructors native elements (byte i8 ... +;; double f64). Constructors convert like clj (wrap, truncate), `aset` is +;; strict; identity equality; simple class name (AD-003); class arguments +;; are advisory (AD-019). ;; Migrated from test/e2e/phase14_arrays.sh. Original named assertions retain ;; their exact printed expectations. (ns suites.arrays-test - (:require [clojure.test :refer [deftest is]])) + (:require [clojure.test :refer [deftest is testing]])) (deftest arrays-cases (is (= ":x" (pr-str (let [a (object-array 3)] (aset a 1 :x) (aget a 1)))) "aset_aget") @@ -21,7 +20,7 @@ (is (= "[0.0 0.0]" (pr-str (vec (double-array 2)))) "double_array_zero") (is (= "[false false]" (pr-str (vec (boolean-array 2)))) "boolean_array_false") (is (= "[1 2 44]" (pr-str (vec (byte-array [1 2 300])))) "byte_array_wrap") - (is (= "-56" (pr-str (let [a (byte-array 1)] (aset-byte a 0 200) (aget a 0)))) "aset_byte_wrap") + (is (= "-56" (pr-str (let [a (byte-array 1)] (aset-byte a 0 -56) (aget a 0)))) "aset_byte") (is (= "[:a :b :c]" (pr-str (vec (object-array [:a :b :c])))) "object_array_from_seq") (is (= "[1 2 3]" (pr-str (vec (seq (int-array [1 2 3]))))) "seq_over_array") (is (= "3" (pr-str (count (int-array [1 2 3])))) "count_array") @@ -41,3 +40,63 @@ (is (= "false" (pr-str (array? [1 2]))) "array_pred_neg") (is (= "array" (pr-str (class (object-array 0)))) "array_class_simple") (is (= "65" (pr-str (let [^"[B" b (byte-array 1)] (aset b 0 65) (aget b 0)))) "type_hint_advisory")) + +(defn- thrown-msg [thunk] + (try (thunk) nil (catch Exception e (ex-message e)))) + +(deftest typed-array-values + (testing "constructors convert each element to the kind (clj RT.xArray)" + (is (= [1 2 44] (vec (byte-array [1 2 300])))) + (is (= [1 -2] (vec (long-array [1.9 -2.7])))) + (is (= [1.0 2.0] (vec (double-array [1 2])))) + (is (= [1 -32768] (vec (short-array [1 32768])))) + (is (= [true false true] (vec (boolean-array [1 nil :x])))) + (is (= [\A \b] (vec (char-array [65 \b]))))) + (testing "a long keeps all 64 bits" + (is (= [Long/MAX_VALUE Long/MIN_VALUE] (vec (long-array [Long/MAX_VALUE Long/MIN_VALUE])))) + (is (= Long/MAX_VALUE (aget (long-array [Long/MAX_VALUE]) 0))) + (is (= (list Long/MAX_VALUE 1) (seq (long-array [Long/MAX_VALUE 1])))) + (is (= "(9223372036854775807 1)" (pr-str (seq (long-array [Long/MAX_VALUE 1])))))) + (testing "a float array stores single precision" + (is (= [1.1 1.6777216E7] (vec (float-array [1.1 16777217]))))) + (testing "fresh slots hold the kind's default" + (is (= [0 0] (vec (long-array 2)))) + (is (= [nil] (vec (object-array 1)))) + (is (= 0 (int (aget (char-array 1) 0))))) + (testing "the 2-arg forms fill or copy" + (is (= [7 7 7] (vec (long-array 3 7)))) + (is (= [1 2 0] (vec (int-array 3 [1 2])))) + (is (= [1] (vec (int-array 1 [1 2 3]))))) + (testing "aclone keeps the kind" + (is (bytes? (aclone (byte-array 2)))))) + +(deftest aset-is-strict + (is (re-find #"in range for a byte array" (thrown-msg #(aset (byte-array 1) 0 200)))) + (is (re-find #"integer" (thrown-msg #(aset (long-array 1) 0 1.5)))) + (is (re-find #"integer" (thrown-msg #(aset (int-array 1) 0 nil)))) + (is (re-find #"boolean" (thrown-msg #(aset (boolean-array 1) 0 1)))) + (is (re-find #"char" (thrown-msg #(aset (char-array 1) 0 65)))) + (is (some? (thrown-msg #(aset-byte (byte-array 1) 0 200))) "aset-byte range-checks as clj") + (is (= 2.0 (let [a (double-array 1)] (aset a 0 2) (aget a 0))) "an integer widens into a double") + (is (= :k (let [a (object-array 1)] (aset a 0 :k) (aget a 0))))) + +(deftest kind-predicates-and-host-statics + (is (bytes? (byte-array 1))) + (is (not (bytes? (int-array 1)))) + (is (not (bytes? (object-array 1)))) + (is (not (bytes? [1]))) + (is (java.util.Arrays/equals (double-array [1]) (double-array [1.0]))) + (is (not (java.util.Arrays/equals (int-array [1]) (long-array [1])))) + (is (= [-0.0 0.0 1.0 3.0 :nan] + (map #(if (Double/isNaN %) :nan %) + (let [a (double-array [3 ##NaN 0.0 -0.0 1])] (java.util.Arrays/sort a) (vec a))))) + (is (= [1 2 3] (let [a (int-array [3 1 2])] (java.util.Arrays/sort a) (vec a)))) + (is (= [5 0 0] (vec (java.util.Arrays/copyOf (long-array [5]) 3)))) + (is (= [9 9] (let [a (long-array 2)] (java.util.Arrays/fill a 9) (vec a)))) + (is (= [1 1 2] (let [a (long-array [1 2 3])] (System/arraycopy a 0 a 1 2) (vec a)))) + (is (some? (thrown-msg #(System/arraycopy (long-array 1) 0 (int-array 1) 0 1)))) + (is (= "hi" (String. (byte-array [104 105])))) + (is (= "hi" (String. (char-array [\h \i])))) + (is (some? (thrown-msg #(String. (int-array [104 105]))))) + (is (bytes? (.getBytes "abc")))) + diff --git a/test/clj/suites/bignum_print_width_test.clj b/test/clj/suites/bignum_print_width_test.clj new file mode 100644 index 000000000..5d5f6bd6a --- /dev/null +++ b/test/clj/suites/bignum_print_width_test.clj @@ -0,0 +1,71 @@ +;; CLJW-BIGINT-STR-PLACEHOLDER: printing a BigInt / Ratio / BigDecimal has no +;; width ceiling. Zig's std big-int formatter renders through a 64-limb stack +;; buffer (about 1233 digits) and writes the literal "(BigInt)" past it, so +;; `(str (reduce *' (range 1N 1000N)))` used to answer "(BigInt)". Every print +;; path (str, pr-str, .toString, print, format, json) now renders the digits, +;; and the decimal parser behind `bigdec` reads a significand of any width. +;; +;; Expected values were taken from clj (JVM) for the same expressions: digit +;; count, digit sum, leading digits and trailing-zero count of 3500!, which is +;; 10887 digits wide. +(ns suites.bignum-print-width-test + (:require [clojure.test :refer [deftest is testing]])) + +(defn- fact [n] (reduce *' (range 1N (inc n)))) + +(defn- digit-sum [s] + (reduce + (map #(- (int %) 48) (filter #(Character/isDigit ^char %) s)))) + +;; Built per test, never at load: run_suites loads every suite before running +;; any, and a load-time 3500! would sit in the heap under gc_heap_return_test's +;; committed-memory baseline. +(defn- big3500 [] (fact 3500)) + +(deftest bigint-past-the-std-format-bound + (let [big (big3500)] + (testing "str of 3500! is its 10887 digits, never the placeholder" + (let [s (str big)] + (is (= 10887 (count s))) + (is (= "23911281994776495250" (subs s 0 20))) + (is (= 45387 (digit-sum s))) + (is (= 874 (count (re-find #"0+$" s)))) + (is (not (re-find #"BigInt" s))))) + (testing "pr-str keeps the N suffix after the full digits" + (let [s (pr-str big)] + (is (= 10888 (count s))) + (is (= \N (last s))))) + (testing ".toString agrees with str; print keeps the N like pr-str (clj)" + (is (= (str big) (.toString big))) + (is (= (pr-str big) (with-out-str (print big))))) + (testing "format %s is .toString: no N/M reader suffix (clj)" + (is (= (str big) (format "%s" big))) + (is (= "1 1.5" (format "%s %s" 1N 1.5M)))) + (testing "1000!, the reported case (999! = (reduce *' (range 1N 1000N)))" + (is (= 2565 (count (str (fact 999))))) + (is (= 2566 (count (pr-str (fact 999)))))))) + +(deftest ratio-past-the-std-format-bound + (let [big (big3500) + r (/ 1 big)] + (is (= 10889 (count (pr-str r)))) + (is (= (str "1/" (str big)) (str r))) + (is (= 45388 (digit-sum (pr-str r)))))) + +(deftest bigdecimal-past-the-old-buffer + (let [big (big3500) + s (str big) + d (bigdec s)] + (is (= s (str d))) + (is (= (str s "M") (pr-str d))) + (is (= 45387 (digit-sum (str d))))) + (testing "scientific and plain forms of a 10887-digit unscaled value" + (let [d (bigdec (str (str (big3500)) "E-5000"))] + (is (= "239112819947" (subs (str d) 0 12))) + (is (= 10888 (count (.toPlainString d))))))) + +(deftest rationalize-a-wide-bigdecimal + (testing "no digit or scale ceiling" + (is (= 1203 (count (str (rationalize (bigdec "1E-1200")))))) + (is (= 1501 (count (str (rationalize (bigdec "7E+1500")))))) + (let [big (big3500)] + (is (= big (rationalize (bigdec (str big)))))))) diff --git a/test/clj/suites/chunked_seq_test.clj b/test/clj/suites/chunked_seq_test.clj index 6b2f8d2c0..b1a0774a6 100644 --- a/test/clj/suites/chunked_seq_test.clj +++ b/test/clj/suites/chunked_seq_test.clj @@ -48,3 +48,45 @@ (is (= "(0 1 2)" (str (seq (range 3))))) ; print_chunked_cons (is (= "(1 2 3 4)" (str (rest (range 5))))) ; print_cc_rest (is (= "[(0 1 2)]" (pr-str [(seq (range 3))])))) ; print_cc_nested + +;; The seq of a vector is IChunkedSeq, as clj's PersistentVector$ChunkedSeq: +;; its chunk is the 32-element leaf holding the cursor, so map/filter over a +;; vector realize a leaf at a time. Values measured on clj/bb (ADR-0205): the +;; first element of (map f v) runs f over the whole first leaf; a cursor at 5 +;; exposes the rest of that leaf (27). subvec and empty seqs are not chunked. +(deftest a-vector-seq-is-chunked-by-leaf + (let [v (vec (range 70)) + log (atom 0) + m (map (fn* [x] (swap! log inc) x) v)] + (is (= 0 (first m))) + (is (= 32 @log)) ; vec_map_first_realizes_leaf + (reset! log 0) + (is (= 5 (first (map (fn* [x] (swap! log inc) x) (drop 5 v))))) + (is (= 27 @log)) ; vec_map_mid_leaf + (is (true? (chunked-seq? (seq v)))) ; vec_seq_chunked + (is (false? (chunked-seq? (seq (subvec v 1 5))))) ; subvec_not_chunked + (is (= 38 (count (chunk-rest (seq v))))) ; vec_chunk_rest + (is (= 38 (count (chunk-rest (next (seq v)))))) ; vec_chunk_rest_mid + (is (= 2485 (reduce + (map inc v)))) ; vec_reduce_map + (is (= (range 1 71) (map inc v))) ; vec_map_eq + (is (= (filter odd? (range 70)) (filter odd? v))))) ; vec_filter_eq + +;; Multi-coll map is UNCHUNKED in clj: one f call per realized node, even over +;; chunked sources, stopping at the shortest source (ADR-0205, O-062). Values +;; measured on bb. +(deftest multi-coll-map-is-unchunked + (let [log (atom 0) + m (map (fn* [a b] (swap! log inc) (+ a b)) (vec (range 70)) (range 100 170))] + (is (= 100 (first m))) + (is (= 1 @log)) ; map2_first_one_call + (is (= '(100 102 104) (take 3 m))) + (is (= 11830 (reduce + m))) + (is (= 70 @log))) ; map2_each_once + (let [log (atom 0) + m (map (fn* [a b c] (swap! log inc) [a b c]) [1 2 3] [4 5] (range))] + (is (= '([1 4 0] [2 5 1]) (doall m))) + (is (= 2 @log))) ; map3_shortest + (is (= '() (map + [1 2] nil))) ; map2_nil_source + (is (= '(4 10 18) (map * [1 2 3] [4 5 6 7]))) + (is (= '(10 12 14) (take 3 (map + (range) (iterate inc 10))))) + (is (thrown? Exception (doall (map (fn* [a b] (throw (ex-info "boom" {}))) [1] [2]))))) diff --git a/test/clj/suites/concurrency_host_test.clj b/test/clj/suites/concurrency_host_test.clj new file mode 100644 index 000000000..e2973f5ca --- /dev/null +++ b/test/clj/suites/concurrency_host_test.clj @@ -0,0 +1,179 @@ +;; STM, Thread lifecycle, watch/validator re-entry overflow, System properties, +;; filesystem require and Var source meta (Phase B #5 / AD-013, ADR-0174 D6, +;; ADR-0157 / D-485 / D-486, D-158 / ADR-0084 / ADR-0189, D-563b / D-316). +;; +;; Migrated from the phase15 e2e scripts stm_dosync, thread_lifecycle, +;; watch_recursion, system_property, require_fs and var_source_meta. What +;; still needs a process stays in those scripts, each marked +;; e2e-process-boundary: join-at-exit and daemon-no-wait (process EXIT), +;; the no-SIGSEGV exit codes, env vars / -cp / CLJW_PATH argv, and the AOT +;; `cljw build` artifact. +(ns suites.concurrency-host-test + (:require [clojure.test :refer [deftest is testing]] + [clojure.java.io :as io] + [clojure.string :as str])) + +(deftest p15-stm-dosync + (testing "phase15_stm_dosync single-thread" + (is (= 42 (let [r (ref 0)] (dosync (ref-set r 42)) @r)) "dosync_ref_set") + (is (= 5 (let [r (ref 0)] (dosync (alter r + 5)) @r)) "dosync_alter") + (is (= 22 (let [r (ref 10)] (dosync (alter r + 1) (alter r * 2)) @r)) "dosync_sees_own_writes") + (is (= 6 (let [r (ref 0)] (dosync (ref-set r 5) (alter r inc)) @r)) "dosync_in_txn_read") + (is (= 2 (let [r (ref 0)] (dosync (alter r inc) (dosync (alter r inc))) @r)) "dosync_nested") + (is (= [10 20] (let [a (ref 1) b (ref 2)] (dosync (ref-set a 10) (ref-set b 20)) [@a @b])) "dosync_multi_ref") + (is (= 5 (let [r (ref 0)] (dosync (commute r + 5)) @r)) "dosync_commute") + (is (= [2 15] (let [a (ref 1) b (ref 10)] (dosync (alter a inc) (commute b + 5)) [@a @b])) "dosync_alter_plus_commute") + (is (= [1 3] (let [a (ref 1) b (ref 2)] (dosync (ensure a) (alter b + (deref a))) [@a @b])) "dosync_ensure")) + (testing "concurrency (AD-013: retry, no barge)" + (is (= [0 200] (let [a (ref 0) b (ref 0)] + (run! deref (mapv (fn [_] (future (dotimes [_ 50] (dosync (ensure a) (alter b inc))))) (range 4))) + [@a @b])) "dosync_concurrent_ensure") + (is (= 400 (let [c (ref 0)] + (run! deref (mapv (fn [_] (future (dotimes [_ 100] (dosync (alter c inc))))) (range 4))) + @c)) "dosync_concurrent_serializable") + (is (= [-100 200] (let [a (ref 100) b (ref 0)] + (run! deref (mapv (fn [_] (future (dotimes [_ 50] (dosync (alter a dec) (alter b inc))))) (range 4))) + [@a @b])) "dosync_concurrent_multi_ref_transfer") + (is (= 400 (let [c (ref 0)] + (run! deref (mapv (fn [_] (future (dotimes [_ 100] (dosync (commute c inc))))) (range 4))) + @c)) "dosync_concurrent_commute")) + (testing "errors / abort" + (is (thrown-with-msg? Throwable #"must be called inside a \(dosync" (alter (ref 0) inc)) "alter_out_of_txn_errors") + (is (= 1 (let [r (ref 1)] (try (dosync (ref-set r 2) (throw (ex-info "x" {}))) (catch Throwable e :c)) @r)) "dosync_abort_on_throw")) + (testing "ref watches" + (is (= [[0 2]] (let [log (atom []) r (ref 0)] + (add-watch r :k (fn [k rf o n] (swap! log conj [o n]))) + (dosync (alter r inc) (alter r inc)) @log)) "ref_add_watch_net") + (is (= [] (let [log (atom []) r (ref 0)] + (add-watch r :k (fn [k rf o n] (swap! log conj n))) + (dosync @r) @log)) "ref_watch_readonly_silent") + (is (= [1] (let [log (atom []) r (ref 0)] + (add-watch r :k (fn [k rf o n] (swap! log conj n))) + (dosync (alter r inc)) (remove-watch r :k) (dosync (alter r inc)) @log)) "ref_remove_watch") + (is (= [[:a 1] [:b 11]] (let [log (atom []) a (ref 0) b (ref 10)] + (add-watch a :a (fn [k r o n] (swap! log conj [:a n]))) + (add-watch b :b (fn [k r o n] (swap! log conj [:b n]))) + (dosync (alter a inc) (alter b inc)) + (vec (sort-by second @log)))) "ref_watch_multi"))) + +(deftest p15-thread-lifecycle + (testing "phase15_thread_lifecycle" + (is (= "Hello from thread\n" + (with-out-str (let [w *out* t (Thread. (fn [] (binding [*out* w] (println "Hello from thread"))))] (.start t) (.join t)))) + "ctor_start_join") + (is (boolean (re-find #"^Thread-\d+$" (.getName (Thread. (fn []))))) "auto_name") + (is (= "worker-1" (.getName (Thread. (fn []) "worker-1"))) "ctor_name") + (is (= "renamed" (let [t (Thread. (fn []))] (.setName t "renamed") (.getName t))) "set_name") + (is (= "inner" (let [seen (promise) + t (Thread. (fn [] (deliver seen (.getName (Thread/currentThread)))) "inner")] + (.start t) (.join t) @seen)) "current_thread_inside") + (is (= "main" (.getName (Thread/currentThread))) "main_thread_name") + (is (= [false true false] (let [t (Thread. (fn [] (Thread/sleep 100))) + a (.isAlive t) _ (.start t) b (.isAlive t) _ (.join t)] + [a b (.isAlive t)])) "is_alive") + (is (thrown-with-msg? Throwable #"already started" + (let [t (Thread. (fn []))] (.start t) (.join t) (.start t))) "double_start") + (is (= [true false] (let [t (Thread. (fn [] (Thread/sleep 300)) "slow")] + (.start t) (.join t 30) + (let [a (.isAlive t)] (.join t) [a (.isAlive t)]))) "timed_join") + (is (= [false true] (let [t (Thread. (fn []))] + (let [a (.isDaemon t)] (.setDaemon t true) [a (.isDaemon t)]))) "daemon_flag") + (let [t (Thread. (fn [] (Thread/sleep 50)))] + (.start t) + (is (thrown-with-msg? Throwable #"already started" (.setDaemon t true)) "setdaemon_after_start") + (.join t)) + (is (= [nil nil] [(Thread/yield) (Thread/onSpinWait)]) "yield_spin") + (is (= [1 5 10] [Thread/MIN_PRIORITY Thread/NORM_PRIORITY Thread/MAX_PRIORITY]) "priorities") + (is (= "java.lang.Thread" (str (class (Thread. (fn []))))) "thread_class") + (is (instance? Thread (Thread. (fn []))) "thread_instance"))) + +(deftest p15-watch-recursion + (let [wrec #(let [a (atom 0)] (add-watch a :w (fn [k r o n] (swap! r inc))) (swap! a inc))] + (testing "phase15_watch_recursion: runaway re-entry is a catchable StackOverflowError (ADR-0157)" + (is (thrown-with-msg? StackOverflowError #"Stack overflow" (wrec)) "watch_recursion_graceful / catch_as_stackoverflow") + (is (= :caught (try (wrec) (catch Throwable e :caught))) "catch_as_throwable") + (is (= :as-thr (try (wrec) (catch Exception e :as-exc) (catch Throwable e :as-thr))) "catch_not_as_exception") + (is (= :caught (try (let [a (atom 0)] (set-validator! a (fn [n] (swap! a inc) true)) (reset! a 5)) + (catch StackOverflowError e :caught))) "validator_reentry_caught") + (is (= :caught (try (reduce (fn f [acc x] (reduce f acc [x])) 0 [1 2 3]) (catch Throwable e :caught))) "reducer_reentry_caught") + (is (= :caught (try ((fn f [n] (f (inc n))) 0) (catch StackOverflowError e :caught))) "direct_recursion_caught") + (is (= :caught (try ((fn f [n] (f (inc n))) 0) (catch Throwable e :caught))) "direct_recursion_thr")) + (testing "legit chains / bounded recursion are untouched" + (is (= [1 10] (let [a (atom 0) b (atom 0)] + (add-watch a :w (fn [k r o n] (reset! b (* 10 n)))) + (swap! a inc) [@a @b])) "legit_watch_chain") + (is (= [2] (let [a (atom 0) b (atom 0) log (atom [])] + (add-watch a :wa (fn [k r o n] (reset! b (inc n)))) + (add-watch b :wb (fn [k r o n] (swap! log conj n))) + (swap! a inc) @log)) "two_level_watch") + ;; depths are the script's tree_walk-safe values (D-555): the suite runs + ;; on whichever backend the binary was built with. + (is (= :done ((fn f [n] (if (zero? n) :done (first (map (fn [_] (f (dec n))) [1])))) 100)) "legit_hof_recursion") + (is (= :done ((fn f [n] (if (zero? n) :done (f (dec n)))) 300)) "legit_direct_recursion")))) + +(deftest p15-system-property + (testing "phase15_system_property (process-independent cases)" + (is (= "\n" (System/getProperty "line.separator")) "line-sep") + (is (= ":" (System/getProperty "path.separator")) "path-sep") + (is (= "/" (System/getProperty "file.separator")) "file-sep") + (is (= "UTF-8" (System/getProperty "file.encoding")) "encoding") + (is (string? (System/getProperty "os.name")) "os-name") + (is (string? (System/getProperty "os.arch")) "os-arch") + (is (string? (System/getProperty "user.dir")) "user-dir") + (is (nil? (System/getProperty "no.such.prop.xyz")) "unknown") + (is (= "fallback" (System/getProperty "no.such.prop.xyz" "fallback")) "default") + (is (= "/" (java.lang.System/getProperty "file.separator")) "fqcn") + (is (nil? (System/getenv "CLJW_DEFINITELY_UNSET_XYZ")) "getenv-missing"))) + +(defn- tmp-root [] + (let [d (io/file (System/getProperty "java.io.tmpdir") (str "cljw-p15-" (System/nanoTime) "-" (rand-int 1000000)))] + (.mkdirs d) + d)) + +(defn- spit-in [root rel text] + (let [f (io/file root rel)] + (.mkdirs (.getParentFile f)) + (spit f text))) + +(defn- delete-tree [f] + (when (.isDirectory f) (run! delete-tree (.listFiles f))) + (.delete f)) + +(deftest p15-require-fs + ;; -cp / CLJW_PATH roots are argv / environment (they stay in + ;; test/e2e/phase15_require_fs.sh); these cases load through + ;; clojure.core/load-file, which shares the on-disk loader. + (testing "phase15_require_fs: a disk test-ns requiring clojure.test + a disk lib" + (load-file "test/e2e/fixtures/cljwlib/demo/math.clj") + (load-file "test/e2e/fixtures/cljwlib/demo/math_test.clj") + (is (= 49 ((ns-resolve 'demo.math 'square) 7)) "cp-load") + (is (= 64 ((ns-resolve 'demo.math 'cube) 4)) "env-path") + (let [s (binding [clojure.test/*test-out* (java.io.StringWriter.) + clojure.test/*report-counters* nil + clojure.test/*testing-vars* (list) + clojure.test/*testing-contexts* (list)] + (clojure.test/run-tests 'demo.math-test))] + (is (= [4 0] [(:pass s) (:fail s)]) "run-suite"))) + (testing "missing lib" + (is (thrown-with-msg? Throwable #"Could not locate" (require 'p15.no.such.fs.lib)) "missing"))) + +(deftest p15-var-source-meta + (let [ev (fn [src] (binding [*ns* (create-ns (gensym "p15.vsm"))] + (refer 'clojure.core) + (load-string src)))] + (testing "phase15_var_source_meta" + (is (= [1 true 2] (ev "(def a 1)\n(defn f [x] x)\n[(:line (meta (var a))) (pos? (:column (meta (var a)))) (:line (meta (var f)))]")) "line_col") + (is (true? (ev "(def a 1)\n(string? (:file (meta (var a))))")) "file_is_label") + (is (= ["gdoc" "me"] (ev "(defn g \"gdoc\" {:author \"me\"} [x] x)\n[(:doc (meta (var g))) (:author (meta (var g)))]")) "doc_and_user_meta") + (is (= [1 "me"] (ev "(def ^{:line 999 :author \"me\"} h 1)\n[(:line (meta (var h))) (:author (meta (var h)))]")) "compiler_line_wins") + (is (= [3 '(a b) "s"] (ev "(def ^{:computed (+ 1 2) :quoted (quote (a b)) :lit \"s\"} mv 1)\n[(:computed (meta (var mv))) (:quoted (meta (var mv))) (:lit (meta (var mv)))]")) "computed_meta") + (is (= [6 '([x])] (ev "(defn mf \"d\" {:extra (* 2 3)} [x] x)\n[(:extra (meta (var mf))) (:arglists (meta (var mf)))]")) "defn_attr_computed")) + (testing "fixture file: :file is the real path, :line the real line" + (let [root (tmp-root)] + (try + (spit-in root "src_meta_fix.clj" "(ns p15.src-meta-fix)\n(defn fixture-fn [x] x)\n") + (load-file (str (io/file root "src_meta_fix.clj"))) + (let [m (meta (ns-resolve 'p15.src-meta-fix 'fixture-fn))] + (is (= 2 (:line m)) "fixture_line_file: line") + (is (str/includes? (str (:file m)) "src_meta_fix") "fixture_line_file: file")) + (finally (delete-tree root))))))) diff --git a/test/clj/suites/ffi_test.clj b/test/clj/suites/ffi_test.clj index 622588b66..0b97e7e17 100644 --- a/test/clj/suites/ffi_test.clj +++ b/test/clj/suites/ffi_test.clj @@ -90,6 +90,40 @@ (testing "sym answers the address function uses" (is (pos? (ffi/sym @lib "add_i"))))) +(deftest primitive-arrays-cross-as-contiguous-storage + (testing ":pointer passes a primitive array's storage, no copy" + (let [a (double-array [1 2 3]) + b (double-array [4 5 6])] + (is (= 32.0 ((f "dot" [:pointer :pointer :long] :double) a b 3)))) + (is (= -6 ((f "sum_i32" [:pointer :long] :long) (int-array [1 -2 3 -8]) 4))) + (is (= 3.75 ((f "sum_f32" [:pointer :long] :double) (float-array [1.5 2.25]) 2)))) + (testing "C writes a :pointer array in place" + (let [l (long-array 3)] + ((f "fill_i64" [:pointer :long :long] :void) l 3 9000000000) + (is (= [9000000000 9000000001 9000000002] (vec l))))) + (testing ":bytes passes a copy, so C writes do not reach the array" + (let [b (byte-array [1 2 3])] + ((f "bump_bytes" [:bytes :long] :void) b 3) + (is (= [1 2 3] (vec b))) + ((f "bump_bytes" [:pointer :long] :void) b 3) + (is (= [2 3 4] (vec b))))) + (testing "a large byte array crosses whole" + (let [b (byte-array 65536 1)] + (is (= 65536 ((f "sum_bytes" [:bytes :long] :long) b 65536))) + (is (= 65536 ((f "sum_bytes" [:pointer :long] :long) b 65536))))) + (testing "bytes copies out into a byte array" + (let [b (byte-array [7 8 9]) + p ((f "dup_upper" [:string] :pointer) "xyz")] + (is (= [88 89 90] (vec (ffi/bytes p 3)))) + (is (bytes? (ffi/bytes p 3))) + ((f "fixture_free" [:pointer] :void) p) + (is (= [7 8 9] (vec b))))) + (testing "only a byte array is :bytes, only a primitive array is a :pointer" + (is (= {:ffi/error :arg-type :index 0 :type :bytes} + (ffi-error #((f "sum_bytes" [:bytes :long] :long) (int-array 2) 2)))) + (is (= {:ffi/error :arg-type :index 0 :type :pointer} + (ffi-error #((f "sum_bytes" [:pointer :long] :long) (object-array 2) 2)))))) + (deftest ordinary-clojure-fns (let [add (f "add_i" [:int :int] :int)] (is (fn? add)) diff --git a/test/clj/suites/forms_and_literals_test.clj b/test/clj/suites/forms_and_literals_test.clj new file mode 100644 index 000000000..106d598b6 --- /dev/null +++ b/test/clj/suites/forms_and_literals_test.clj @@ -0,0 +1,227 @@ +;; Macros, reader literals, special forms and definition forms (D-158/D-165/ +;; D-232/D-273/D-382, ADR-0113/0188, CLJW-UNCHECKED-MATH). +;; +;; Migrated from the phase15 e2e scripts macro_literals, macro_heaplong, +;; literal_roundtrip, reader_conditional, unchecked_math, campaign_keywords +;; (regex-macro + thrown-with-msg?), clojure_template, clojure_uuid, +;; clojure_instant, new_ctor, defmulti_defmethod, defmethod_destructure, +;; marker_protocol, defprotocol_docstring, threaddeath_multifn, with_open, +;; pr_newline and deferred_host_ref (all but its call-time raise, which is an +;; uncatchable not_implemented and stays bash). +;; +;; A macro is only registered once its top-level form completes, so every +;; defmacro-then-call case goes through `ev` (load-string in a fresh scratch +;; ns, the same per-form evaluation `cljw -e` did). *unchecked-math* is a +;; COMPILE-time flag read by the analyzer, so those cases bind it around `ev`. +;; Type / protocol / multimethod names carry a p15 prefix: the suite image is +;; shared. +(ns suites.forms-and-literals-test + (:require [clojure.test :as t :refer [deftest is testing]] + [clojure.template] + [clojure.uuid] + [clojure.instant])) + +(def ^:private scratch-seq (atom 0)) + +(defn- ev [src] + (let [n (symbol (str "p15.forms" (swap! scratch-seq inc)))] + (binding [*ns* (create-ns n)] + (refer 'clojure.core) + (load-string src)))) + +(defn- rd + "Read `s` the way a .cljc file / `cljw -e` reads it: reader conditionals on." + [s] + (read-string {:read-cond :allow} s)) + +(deftest p15-macro-literals + (testing "phase15_macro_literals" + (is (= "1/2" (pr-str (ev "(defmacro m [] `~(/ 1 2)) (m)"))) "ratio") + (is (= "9999999999800000000001N" (pr-str (ev "(defmacro m [] `~(* 99999999999 99999999999)) (m)"))) "bigint") + (is (= "3.75M" (pr-str (ev "(defmacro m [] `~(+ 1.50M 2.25M)) (m)"))) "bigdec") + (is (= "\\b" (pr-str (ev "(defmacro m [] `~(char 98)) (m)"))) "char") + (is (= "[3/4 123N \\c]" (pr-str (ev "(defmacro m [] `[~(/ 3 4) ~(bigint 123) ~(char 99)]) (m)"))) "mixed") + (is (= "abc" (ev "(defmacro m [] `(re-find #\"a.c\" \"xabcy\")) (m)")) "regex / campaign regex-macro"))) + +(def ^:private longify-src + "(defmacro longify [num] (if (> num Long/MAX_VALUE) (-> num (- 18446744073709551616N) (long) (bit-or -9223372036854775808)) num)) ") + +(deftest p15-macro-heaplong + (testing "phase15_macro_heaplong" + (is (= "Long" (str (ev (str longify-src "(class (longify 0xbf58476d1ce4e5b9))")))) "longify_returns_long") + (is (= -4658895280553007687 (ev (str longify-src "(longify 0xbf58476d1ce4e5b9)"))) "longify_value") + (is (= "Long" (str (binding [*unchecked-math* true] + (ev (str longify-src "(class (unsigned-bit-shift-right (* 9223372036854775807 (longify 0xbf58476d1ce4e5b9)) 11))"))))) + "longify_splitmix_compose_no_bigint") + (is (= "Long" (str (ev "(do (defmacro biglong [] 4658895280553007687) (class (biglong)))"))) "biglong_positive_stays_long") + (is (= "BigInt" (str (ev "(do (defmacro reallybig [] (* 99999999999999999N 99999999999999999N)) (class (reallybig)))"))) "genuine_bigint_preserved"))) + +(deftest p15-unchecked-math + (let [um (fn [src] (binding [*unchecked-math* true] (ev src)))] + (testing "phase15_unchecked_math: flag off promotes" + (is (= 18446744073709551614N (ev "(* 9223372036854775807 2)")) "checked_mul_promotes") + (is (= "BigInt" (str (ev "(class (* 9223372036854775807 2))"))) "checked_mul_class_bigint")) + (testing "flag on wraps" + (is (= -2 (um "(* 9223372036854775807 2)")) "unchecked_mul_wraps") + (is (= "Long" (str (um "(class (* 9223372036854775807 2))"))) "unchecked_mul_class_long") + (is (= -9223372036854775808 (um "(+ 9223372036854775807 1)")) "unchecked_add_wraps") + (is (= -9223372036854775808 (um "(inc 9223372036854775807)")) "unchecked_inc_wraps") + (is (= 9223372036854775807 (um "(dec -9223372036854775808)")) "unchecked_dec_wraps")) + (testing "arity parity" + (is (= [10 24 5 -5 0 1] (um "[(+ 1 2 3 4) (* 2 3 4) (- 10 3 2) (- 5) (+) (*)]")) + "unchecked nary/negate/identity")) + (testing "n-ary overflow + splitmix shape + value position" + (is (= "Long" (str (um "(class (* 1000000000000 1000000000 1000000000))"))) "unchecked_nary_wraps") + (is (= 4503599627370495 (um "(unsigned-bit-shift-right (* 9223372036854775807 7) 11)")) "unchecked_splitmix_shape") + (is (= "BigInt" (str (um "(class (reduce + [9223372036854775807 2]))"))) "value_position_still_checked")))) + +(deftest p15-literal-roundtrip + (let [mac "(defmacro m [& body] `(do ~@body)) "] + (testing "phase15_literal_roundtrip: reader literals through a macro" + (is (= "#uuid \"550e8400-e29b-41d4-a716-446655440000\"" + (pr-str (ev (str mac "(m #uuid \"550e8400-e29b-41d4-a716-446655440000\")")))) "uuid-through-macro") + (is (true? (ev (str mac "(uuid? (m #uuid \"550e8400-e29b-41d4-a716-446655440000\"))"))) "uuid-through-macro-type") + (is (true? (ev (str mac "(inst? (m #inst \"2010-11-12T13:14:15.666-05:00\"))"))) "inst-through-macro") + (is (true? (ev (str mac "(= #inst \"2010-11-12T18:14:15.666-00:00\" (m #inst \"2010-11-12T13:14:15.666-05:00\"))"))) + "inst-round-trip-equal"))) + (testing "resolve follows a require alias" + (is (= #'clojure.string/blank? (ev "(do (require '[clojure.string :as str]) (resolve 'str/blank?))")) "resolve-through-alias") + (is (= #'clojure.string/blank? (ev "(do (require '[clojure.string :as str]) (resolve 'clojure.string/blank?))")) "resolve-real-ns-still-works") + (is (nil? (ev "(do (require '[clojure.string :as str]) (resolve 'str/no-such-var))")) "resolve-alias-miss") + (is (= #'clojure.string/upper-case + (ev "(do (require '[clojure.string :as str]) (ns-resolve *ns* 'str/upper-case))")) "ns-resolve-through-alias")) + (testing "BigDecimal exponents" + (is (= "1E+5M" (pr-str 1e5M)) "bigdec-positive-exponent") + (is (= "1E-10M" (pr-str 1E-10M)) "bigdec-negative-exponent") + (is (= "-1.5E-10M" (pr-str -1.5E-10M)) "bigdec-signed-mantissa-exponent") + (is (= 150000M 1.5e5M) "bigdec-exponent-value") + (is (= "123.456M" (pr-str 123.456M)) "bigdec-plain-unchanged")) + (testing "when-let / when-first with no body" + (is (nil? (when-let [x 1])) "when-let-empty-body-truthy") + (is (nil? (when-let [x nil])) "when-let-empty-body-falsey") + (is (= 10 (when-let [x 5] (* x 2))) "when-let-body-still-runs") + (is (nil? (when-first [x [1 2]])) "when-first-empty-body") + (is (= 8 (when-first [x [7 8]] (inc x))) "when-first-body-still-runs"))) + +(deftest p15-reader-conditional + (testing "phase15_reader_conditional: feature set {:cljw :clj :default}, left to right" + (is (= 1 (rd "#?(:clj 1 :cljs 2)")) "clj-branch") + (is (= 8 (rd "#?(:cljs 9 :clj 8)")) "clj-branch-reordered") + (is (= 1 (rd "#?(:cljw 1 :clj 2 :cljs 3)")) "cljw-branch") + (is (= 2 (rd "#?(:clj 2 :cljw 1)")) "cljw-after-clj-loses") + (is (= 4 (rd "#?(:cljs 9 :cljw 4)")) "cljw-alone") + (is (= 7 (rd "#?(:cljs 1 :default 7)")) "default-fallback") + (is (= [1 3] (rd "[1 #?(:cljs 2) 3]")) "no-match-skips") + (is (= [1] (rd "[1 #?(:cljs 2)]")) "no-match-trailing") + (is (= '(quote (ns x)) (rd "(quote (ns x #?(:cljs :m)))")) "no-match-ns-tail")) + (testing "#?@ splicing" + (is (= [1 2 3 4] (rd "[1 #?@(:clj [2 3]) 4]")) "splice-clj") + (is (= [1 2 3 4] (rd "[1 #?@(:cljw [2 3] :clj [8 9]) 4]")) "splice-cljw") + (is (= [0 5] (rd "[0 #?@(:cljs [9]) 5]")) "splice-no-match") + (is (= '(list 1 2 3) (rd "(list 1 #?@(:clj [2 3]))")) "splice-list") + (is (thrown-with-msg? Throwable #"only allowed inside" (rd "#?@(:clj [1 2])")) "splice-top-level"))) + +(deftest p15-thrown-with-msg + (testing "phase15_campaign_keywords twm-pass / twm-fail" + (let [run (fn [re] + (binding [t/*test-out* (java.io.StringWriter.) + t/*report-counters* nil + t/*testing-vars* (list) + t/*testing-contexts* (list)] + (let [s (ev (str "(ns p15.twm" (swap! scratch-seq inc) " (:use clojure.test)) " + "(deftest x (is (thrown-with-msg? Throwable #\"" re "\" (throw (ex-info \"boom!\" {}))))) " + "(let [s (run-tests)] [(:pass s) (:fail s)])"))] + s)))] + (is (= [1 0] (run "boom")) "twm-pass") + (is (= [0 1] (run "nope")) "twm-fail")))) + +(deftest p15-bundled-small-nses + (testing "phase15_clojure_template" + (is (= '(+ 5 5) (clojure.template/apply-template '[x] '(+ x x) '[5])) "apply-template") + (is (= 30 (clojure.template/do-template [x y] (+ x y) 1 2 10 20)) "do-template")) + (testing "phase15_clojure_uuid" + (is (= #uuid "12345678-1234-1234-1234-123456789abc" + (clojure.uuid/default-uuid-reader "12345678-1234-1234-1234-123456789abc"))) + (is (= "#uuid \"12345678-1234-1234-1234-123456789abc\"" + (pr-str #uuid "12345678-1234-1234-1234-123456789abc")) "uuid_require_and_reader")) + (testing "phase15_clojure_instant" + (is (= #inst "2026-06-10T12:00:00.000-00:00" (clojure.instant/read-instant-date "2026-06-10T12:00:00Z")) "read_instant_date") + (is (= "java.util.Date" (str (class (clojure.instant/read-instant-date "2026-06-10")))) "read_instant_date class") + (let [ts (clojure.instant/read-instant-timestamp "2024-01-01T00:00:00.123456789Z")] + (is (= "[#inst \"2024-01-01T00:00:00.123456789-00:00\" java.sql.Timestamp true 1704067200123 true false]" + (pr-str [ts (class ts) (inst? ts) (inst-ms ts) + (= ts (clojure.instant/read-instant-timestamp "2024-01-01T00:00:00.123456789Z")) + (= ts (clojure.instant/read-instant-date "2024-01-01T00:00:00.123Z"))])) + "read_instant_timestamp")) + (is (thrown? Exception (clojure.instant/read-instant-date "not-an-instant")) "malformed_throws") + (is (thrown? Exception (clojure.instant/read-instant-date "2024-01-01T00:00:00\"x")) "quote_chars_rejected") + (is (thrown? Exception (clojure.instant/read-instant-date "a\\b")) "quote_chars_rejected backslash"))) + +(deftest p15-new-ctor + (testing "phase15_new_ctor" + (is (= "boom" (.getMessage (new Exception "boom"))) "new-exception") + (is (= ["x" "x"] [(.getMessage (new Exception "x")) (.getMessage (Exception. "x"))]) "new-eq-sugar") + (is (instance? Exception (new RuntimeException "r")) "new-instance") + (is (thrown-with-msg? Throwable #"requires a class" (ev "(new)")) "new-no-class"))) + +(deftest p15-multimethods + (testing "phase15_defmulti_defmethod" + (is (= 9 (ev "(do (defmulti area \"the area\" {:arglists (quote ([shape]))} :kind) (defmethod area :square [s] (* (:side s) (:side s))) (area {:kind :square :side 3}))")) + "defmulti-docstring-dispatch") + (is (= "the doc" (ev "(do (defmulti m \"the doc\" {:x 1} identity) (:doc (meta (var m))))")) "defmulti-doc-meta") + (is (= 1 (ev "(do (defmulti m \"d\" {:x 1} identity) (:x (meta (var m))))")) "defmulti-attr-meta") + (is (nil? (ev "(do (defmulti m identity) (defmethod m :default [_]) (m :anything))")) "defmethod-empty-body")) + (testing "phase15_defmethod_destructure" + (is (= 7 (ev "(defmulti foo (fn [x] (:t x))) (defmethod foo :pair [{:keys [a b]}] (+ a b)) (foo {:t :pair :a 3 :b 4})")) "assoc-param") + (is (= [:x 1 2 '(3 4) [:tag 1 2 3 4]] + (ev "(defmulti bar (fn [m form] m)) (defmethod bar :x [msg [_ a b & rest :as form]] [msg a b rest form]) (bar :x [:tag 1 2 3 4])")) + "nested-seq-param") + (is (= 42 (ev "(defmulti baz class) (defmethod baz Long [n] (* n 2)) (baz 21)")) "symbol-param-regress")) + (testing "phase15_threaddeath_multifn" + (is (= 1 (try 1 (catch java.lang.ThreadDeath t (throw t)) (catch Throwable e 3))) "threaddeath_guard_analyses") + (is (= :ok (try :ok (catch ThreadDeath t :td))) "threaddeath_simple_name") + (is (true? (ev "(do (defmulti mm identity) (instance? clojure.lang.MultiFn mm))")) "multifn_instance_true") + (is (false? (instance? clojure.lang.MultiFn +)) "multifn_instance_fn_false") + (is (false? (instance? clojure.lang.MultiFn 5)) "multifn_instance_scalar_false") + (is (true? (ev "(do (defmulti mm2 identity) (= (class mm2) clojure.lang.MultiFn))")) "multifn_class_identity") + (is (true? (ev "(do (defmulti mm3 identity) (isa? (class mm3) clojure.lang.IFn))")) "multifn_isa_ifn"))) + +(deftest p15-protocols + (testing "phase15_marker_protocol" + (is (true? (ev "(defprotocol P15M1 \"marker\") (defrecord P15RM1 [] P15M1) (satisfies? P15M1 (->P15RM1))")) "marker-defrecord") + (is (true? (ev "(defprotocol P15M2) (defrecord P15RM2 []) (extend-type P15RM2 P15M2) (satisfies? P15M2 (->P15RM2))")) "marker-extend-type") + (is (false? (ev "(defprotocol P15M3 \"marker\") (defrecord P15Other3 []) (satisfies? P15M3 (->P15Other3))")) "non-member-false") + (is (= [true 7] (ev "(defprotocol P15G (g [x])) (defrecord P15R4 [n] P15G (g [x] n)) [(satisfies? P15G (->P15R4 1)) (g (->P15R4 7))]")) "method-proto-regress")) + (testing "phase15_defprotocol_docstring" + (is (= ["hi ann" "HI bob" true] + (ev "(defprotocol P15Greet \"a greeting protocol\" (greet [this] \"say hi\") (loud [this] \"SHOUT\")) (defrecord P15Person [nm] P15Greet (greet [this] (str \"hi \" nm)) (loud [this] (str \"HI \" nm))) [(greet (->P15Person \"ann\")) (loud (->P15Person \"bob\")) (satisfies? P15Greet (->P15Person \"x\"))]")) + "dispatch-with-docstrings") + (is (false? (ev "(defprotocol P15Hint (^String baz [a] [a b] \"multi-arity, hinted\")) (satisfies? P15Hint 1)")) "hinted-multi-arity") + (is (false? (ev "(defprotocol P15Doc \"marker only\") (satisfies? P15Doc 1)")) "marker-docstring-only") + (is (= ["<7>" true] + (ev "(defprotocol P15Sz :extend-via-metadata true (sz [this] \"render\")) (defrecord P15Lit [v] P15Sz (sz [this] (str \"<\" v \">\"))) [(sz (->P15Lit 7)) (satisfies? P15Sz (->P15Lit 1))]")) + "option-pair-extend-via-metadata"))) + +(deftest p15-with-open + (let [res "(def log (atom [])) (defprotocol P15PC (close [this])) (deftype P15Res [id] P15PC (close [this] (swap! log conj id))) "] + (testing "phase15_with_open" + (is (= [:body 2 1] (ev (str res "(with-open [a (->P15Res 1) b (->P15Res 2)] (swap! log conj :body)) @log"))) "reverse-order") + (is (= [1] (ev (str res "(try (with-open [a (->P15Res 1)] (throw (ex-info \"boom\" {}))) (catch Throwable e :caught)) @log"))) "close-on-throw") + (is (= 3 (with-open [] (+ 1 2))) "empty-bindings") + (is (= :result (ev (str res "(with-open [a (->P15Res 1)] :result)"))) "returns-body")))) + +(deftest p15-pr-newline + (testing "phase15_pr_newline" + (is (= "\"hi\"" (with-out-str (pr "hi"))) "pr-string-quoted") + (is (= "hi" (with-out-str (print "hi"))) "print-string-unquoted") + (is (= ":a" (with-out-str (pr :a))) "pr-keyword") + (is (= "x\ny" (with-out-str (print "x") (newline) (print "y"))) "newline-line-break"))) + +(deftest p15-deferred-host-ref + ;; ADR-0113 / AD-022: an unresolved clojure.lang.* ref defers to call time. + ;; The call-time raise itself is an uncatchable not_implemented and stays in + ;; test/e2e/phase15_deferred_host_ref.sh. + (testing "phase15_deferred_host_ref" + (is (= :ok (ev "(do (defn f [] (clojure.lang.RT/baseLoader)) :ok)")) "defines_and_loads") + (is (= :defined (ev "(do (def g (fn [] clojure.lang.Compiler/CHAR_MAP)) :defined)")) "value_position_defines") + (is (thrown-with-msg? Throwable #"No namespace: 'myalias'" (ev "(myalias/foo 1)")) "typo_stays_loud"))) diff --git a/test/clj/suites/gc_heap_return_test.clj b/test/clj/suites/gc_heap_return_test.clj new file mode 100644 index 000000000..3a8cfad4a --- /dev/null +++ b/test/clj/suites/gc_heap_return_test.clj @@ -0,0 +1,50 @@ +;; CLJW-GC-HEAP-RETURN: the collector reuses freed cell space and hands empty +;; pages back to the OS. `(.totalMemory r)` reports the committed cell heap +;; (mapped GC pages plus live large cells), so both halves are observable from +;; Clojure without reading RSS, which is host noise. The RSS probe lives in +;; bench/gc_heap_return.clj. +(ns suites.gc-heap-return-test + (:require [clojure.test :refer [deftest is testing]])) + +(def ^:private mb (* 1024 1024)) + +(defn- committed [] (.totalMemory (Runtime/getRuntime))) + +(defn- burst + "Allocate a ~34 MB vector trie (4M elements) and drop it. It must be much + larger than the free space earlier suites leave in partly used pages of the + shared run_suites image, or it lands in already-committed pages and the + growth is invisible (#80's macOS gate: base 208 MB, burst +0.8 MB)." + [] + (count (vec (range 4000000)))) + +(deftest requested-collect-returns-empty-pages + (System/gc) + (let [base (committed) + _ (burst) + grown (committed) + _ (System/gc) + after (committed)] + (testing "the burst grew the committed heap" + (is (> grown (+ base (* 16 mb))))) + (testing "System/gc gave the dropped burst's pages back" + (is (< after (- grown (* 16 mb)))) + (is (<= after (+ base (* 2 mb))))))) + +(deftest freed-pages-are-reused-by-the-next-burst + (System/gc) + (burst) + (let [first-peak (committed)] + (dotimes [_ 4] + (burst)) + (testing "bursts of one size after a collect refill the same pages" + (System/gc) + (burst) + (is (<= (committed) (+ first-peak (* 2 mb))))))) + +(deftest memory-readings-stay-coherent + (let [r (Runtime/getRuntime) + t (.totalMemory r) + f (.freeMemory r)] + (is (pos? t)) + (is (<= 0 f t)))) diff --git a/test/clj/suites/java_time_format_test.clj b/test/clj/suites/java_time_format_test.clj new file mode 100644 index 000000000..acf1f7351 --- /dev/null +++ b/test/clj/suites/java_time_format_test.clj @@ -0,0 +1,207 @@ +;; java.time.format: DateTimeFormatter, DateTimeFormatterBuilder and the +;; Parsed TemporalAccessor; java.time.temporal.ChronoField; the fixed-offset +;; java.time.ZoneId / ZoneOffset; the `from` conversions. +;; +;; Every expected string is what JVM Clojure 1.12 on JDK 25 printed for the +;; same form: the `str` of the value, or `: ` of the +;; exception. A ClassCastException message names JVM classes cljw does not +;; have, so those rows compare the class only (`outcome-class`). +;; +;; The malli 0.20.2 `malli.transform` string<->date formatters are the +;; driving consumer, so they come first, built exactly as malli builds them. +(ns suites.java-time-format-test + (:require [clojure.test :refer [deftest is testing]]) + (:import (java.time Instant LocalDate LocalDateTime LocalTime ZoneId ZoneOffset) + (java.time.format DateTimeFormatter DateTimeFormatterBuilder) + (java.time.temporal ChronoField) + (java.util Date))) + +(defmacro outcome + "The str of `form`'s value, or `: ` of what it threw." + [form] + `(try (str ~form) + (catch Exception ex# (str (.getSimpleName (class ex#)) ": " (ex-message ex#))))) + +(defmacro outcome-class + "The str of `form`'s value, or the simple class name of what it threw." + [form] + `(try (str ~form) + (catch Exception ex# (.getSimpleName (class ex#))))) + +(def malli-in + (-> (DateTimeFormatterBuilder.) + (.appendPattern "yyyy-MM-dd['T'HH:mm:ss]") + (.optionalStart) + (.appendFraction ChronoField/MICRO_OF_SECOND, 0, 9, true) + (.optionalEnd) + (.optionalStart) + (.appendOffset "+HHMMss", "Z") + (.optionalEnd) + (.optionalStart) + (.appendOffset "+HH:MM:ss", "Z") + (.optionalEnd) + (.parseDefaulting ChronoField/HOUR_OF_DAY 0) + (.parseDefaulting ChronoField/OFFSET_SECONDS 0) + (.toFormatter))) + +(def malli-out + (-> (DateTimeFormatter/ofPattern "yyyy-MM-dd'T'HH:mm:ss.SSSXXX") + (.withZone (ZoneId/of "UTC")))) + +(deftest malli-string->date + (is (= ["2024-01-02T00:00:00Z" + "2024-01-02T03:04:05Z" + "2024-01-02T03:04:05.123456Z" + "2024-01-02T03:04:05Z" + "2024-01-02T01:34:05Z" + "2024-01-02T01:34:05.500Z" + "2024-01-02T08:04:05Z"] + (mapv #(outcome (Instant/from (.parse malli-in %))) + ["2024-01-02" "2024-01-02T03:04:05" "2024-01-02T03:04:05.123456789" + "2024-01-02T03:04:05Z" "2024-01-02T03:04:05+0130" "2024-01-02T03:04:05.5+01:30" + "2024-01-02T03:04:05-05:00"]))) + (is (= "DateTimeParseException: Text '2024-01-02T' could not be parsed, unparsed text found at index 10" + (outcome (Instant/from (.parse malli-in "2024-01-02T"))))) + (is (= "DateTimeParseException: Text 'nope' could not be parsed at index 0" + (outcome (Instant/from (.parse malli-in "nope"))))) + (is (= 1704164645500 (inst-ms (Date/from (Instant/from (.parse malli-in "2024-01-02T03:04:05.5Z"))))))) + +(deftest malli-date->string + (is (= "2024-01-02T03:04:05.123Z" (outcome (.format malli-out (Instant/ofEpochMilli 1704164645123))))) + (is (= "1970-01-01T00:00:00.000Z" (outcome (.format malli-out (Instant/ofEpochMilli 0))))) + (testing "a Date is not a TemporalAccessor" + (is (= "ClassCastException" (outcome-class (.format malli-out (Date/from (Instant/ofEpochSecond 1)))))))) + +(deftest of-pattern-format + (is (= "09/03/2024" (outcome (.format (DateTimeFormatter/ofPattern "dd/MM/yyyy") (LocalDate/of 2024 3 9))))) + (is (= "Mar 9, 2024" (outcome (.format (DateTimeFormatter/ofPattern "MMM d, yyyy") (LocalDate/of 2024 3 9))))) + (is (= "Saturday, March 09" (outcome (.format (DateTimeFormatter/ofPattern "EEEE, MMMM dd") (LocalDate/of 2024 3 9))))) + (is (= "Sat 24 69" (outcome (.format (DateTimeFormatter/ofPattern "EEE yy D") (LocalDate/of 2024 3 9))))) + (is (= "2024-03-09 07:05" (outcome (.format (DateTimeFormatter/ofPattern "yyyy-MM-dd HH:mm") (LocalDateTime/of 2024 3 9 7 5))))) + (is (= "7:05 PM" (outcome (.format (DateTimeFormatter/ofPattern "h:mm a") (LocalTime/of 19 5))))) + (is (= "01:02:03.004 4500000" (outcome (.format (DateTimeFormatter/ofPattern "HH:mm:ss.SSS n") (LocalTime/of 1 2 3 4500000))))) + (testing "a field the temporal does not carry" + (is (= "UnsupportedTemporalTypeException: Unsupported field: YearOfEra" + (outcome (.format (DateTimeFormatter/ofPattern "yyyy-MM-dd") (LocalTime/of 1 2))))) + (is (= "UnsupportedTemporalTypeException: Unsupported field: YearOfEra" + (outcome (.format (DateTimeFormatter/ofPattern "yyyy") (Instant/ofEpochSecond 0)))))) + (testing "an optional section drops when its fields are absent" + (is (= "2024-01-02" (outcome (.format (DateTimeFormatter/ofPattern "yyyy-MM-dd['T'HH:mm]") (LocalDate/of 2024 1 2))))) + (is (= "2024-01-02T03:04" (outcome (.format (DateTimeFormatter/ofPattern "yyyy-MM-dd['T'HH:mm]") (LocalDateTime/of 2024 1 2 3 4)))))) + (testing "withZone re-projects an Instant" + (is (= "ClassCastException" + (outcome-class (.format (DateTimeFormatter/ofPattern "yyyyMMdd'T'HHmmssX") + (.withZone (DateTimeFormatter/ofPattern "yyyy") (ZoneOffset/ofHours 2)))))) + (is (= "19700101T020000+0200" + (outcome (.format (.withZone (DateTimeFormatter/ofPattern "yyyyMMdd'T'HHmmssXX") (ZoneOffset/ofHours 2)) (Instant/ofEpochSecond 0))))) + (is (= "1969-12-31T18:30-05:30 -05:30" + (outcome (.format (.withZone (DateTimeFormatter/ofPattern "yyyy-MM-dd'T'HH:mmxxx VV") (ZoneOffset/of "-05:30")) (Instant/ofEpochSecond 0)))))) + (testing "toString and pattern errors" + (is (= "Value(YearOfEra,4,19,EXCEEDS_PAD)'-'Value(MonthOfYear,2)'-'Value(DayOfMonth,2)'T'Value(HourOfDay,2)':'Value(MinuteOfHour,2)':'Value(SecondOfMinute,2)'.'Fraction(NanoOfSecond,3,3)Offset(+HH:MM,'Z')" + (outcome (DateTimeFormatter/ofPattern "yyyy-MM-dd'T'HH:mm:ss.SSSXXX")))) + (is (= "IllegalArgumentException: Unknown pattern letter: b" (outcome (DateTimeFormatter/ofPattern "bogus{")))))) + +(deftest iso-constants-format + (is (= "2024-03-09" (outcome (.format DateTimeFormatter/ISO_LOCAL_DATE (LocalDate/of 2024 3 9))))) + (is (= "10:15:30.5" (outcome (.format DateTimeFormatter/ISO_LOCAL_TIME (LocalTime/of 10 15 30 500000000))))) + (is (= "10:15:00" (outcome (.format DateTimeFormatter/ISO_LOCAL_TIME (LocalTime/of 10 15))))) + (is (= "2024-03-09T10:15:00" (outcome (.format DateTimeFormatter/ISO_LOCAL_DATE_TIME (LocalDateTime/of 2024 3 9 10 15))))) + (is (= "2024-01-01T00:00:00.500Z" (outcome (.format DateTimeFormatter/ISO_INSTANT (Instant/ofEpochSecond 1704067200 500000000))))) + (is (= "20240309" (outcome (.format DateTimeFormatter/BASIC_ISO_DATE (LocalDate/of 2024 3 9))))) + (is (= "2024-03-09" (outcome (.format DateTimeFormatter/ISO_DATE (LocalDate/of 2024 3 9))))) + (is (= "UnsupportedTemporalTypeException: Unsupported field: OffsetSeconds" + (outcome (.format DateTimeFormatter/ISO_OFFSET_DATE_TIME (LocalDateTime/of 2024 3 9 10 15))))) + (is (= "1969-12-31T21:00:00-03:00" + (outcome (.format (.withZone DateTimeFormatter/ISO_OFFSET_DATE_TIME (ZoneOffset/ofHours -3)) (Instant/ofEpochSecond 0))))) + (is (= "Value(Year,4,10,EXCEEDS_PAD)'-'Value(MonthOfYear,2)'-'Value(DayOfMonth,2)" (outcome DateTimeFormatter/ISO_LOCAL_DATE))) + (is (= "ParseCaseSensitive(false)Instant()" (outcome DateTimeFormatter/ISO_INSTANT)))) + +(deftest parse-and-convert + (is (= "2024-01-01T00:00:00.250Z" (outcome (Instant/from (.parse DateTimeFormatter/ISO_INSTANT "2024-01-01T00:00:00.25Z"))))) + (is (= "2024-01-01T08:00:00Z" (outcome (Instant/from (.parse DateTimeFormatter/ISO_OFFSET_DATE_TIME "2024-01-01T10:00:00+02:00"))))) + (is (= "2024-03-09" (outcome (LocalDate/parse "20240309" DateTimeFormatter/BASIC_ISO_DATE)))) + (is (= "2024-03-09" (outcome (LocalDate/parse "09/03/2024" (DateTimeFormatter/ofPattern "dd/MM/yyyy"))))) + (is (= "2024-03-09T07:05" (outcome (LocalDateTime/parse "2024-03-09 07:05" (DateTimeFormatter/ofPattern "yyyy-MM-dd HH:mm"))))) + (is (= "19:05" (outcome (LocalTime/parse "7:05 PM" (DateTimeFormatter/ofPattern "h:mm a"))))) + (is (= "2024-03-09" (outcome (LocalDate/parse "Mar 9, 2024" (DateTimeFormatter/ofPattern "MMM d, yyyy"))))) + (is (= "2024-03-09" (outcome (LocalDate/from (.parse DateTimeFormatter/ISO_LOCAL_DATE_TIME "2024-03-09T01:02"))))) + (is (= "01:02" (outcome (LocalTime/from (.parse DateTimeFormatter/ISO_LOCAL_DATE_TIME "2024-03-09T01:02"))))) + (is (= "2024-03-09T01:02:03.400" (outcome (LocalDateTime/from (.parse DateTimeFormatter/ISO_LOCAL_DATE_TIME "2024-03-09T01:02:03.4"))))) + (testing "failures" + (is (= "DateTimeParseException: Text '2024-02-30' could not be parsed: Invalid date 'FEBRUARY 30'" + (outcome (LocalDate/parse "2024-02-30" DateTimeFormatter/ISO_LOCAL_DATE)))) + (is (= "DateTimeParseException: Text '2024-03-09x' could not be parsed, unparsed text found at index 10" + (outcome (LocalDate/parse "2024-03-09x" DateTimeFormatter/ISO_LOCAL_DATE)))) + (is (= "DateTimeException: Unable to obtain LocalDate from TemporalAccessor: {},ISO resolved to 10:15 of type java.time.format.Parsed" + (outcome (LocalDate/from (.parse (DateTimeFormatter/ofPattern "HH:mm") "10:15")))))) + (testing "the Parsed accessor" + (is (= 2024 (.getLong (.parse DateTimeFormatter/ISO_LOCAL_DATE "2024-03-09") ChronoField/YEAR))) + (is (false? (.isSupported (.parse DateTimeFormatter/ISO_LOCAL_DATE "2024-03-09") ChronoField/HOUR_OF_DAY)))) + (testing "the exception classes catch by their JVM names" + (is (= :caught (try (LocalDate/parse "x" DateTimeFormatter/ISO_LOCAL_DATE) + (catch java.time.format.DateTimeParseException _ :caught)))) + (is (= :caught (try (.format DateTimeFormatter/ISO_LOCAL_DATE (LocalTime/of 1 2)) + (catch java.time.DateTimeException _ :caught)))))) + +(deftest adjacent-value-parsing + (testing "a fixed-width fraction reserves its digits in the variable-width year" + (is (= "2024-01-01T12:00:00.123" + (outcome (LocalDateTime/parse "20240101120000123" (DateTimeFormatter/ofPattern "yyyyMMddHHmmssSSS")))))) + (is (= "Value(YearOfEra,4,19,EXCEEDS_PAD)Value(MonthOfYear,2)Value(DayOfMonth,2)Value(HourOfDay,2)Value(MinuteOfHour,2)Value(SecondOfMinute,2)Fraction(NanoOfSecond,3,3)" + (outcome (DateTimeFormatter/ofPattern "yyyyMMddHHmmssSSS"))))) + +(deftest formatter-identity-and-locale + (is (identical? DateTimeFormatter/ISO_DATE DateTimeFormatter/ISO_DATE)) + (is (= "Mar 9" (outcome (.format (.withLocale (DateTimeFormatter/ofPattern "MMM d") java.util.Locale/US) + (LocalDate/of 2024 3 9)))))) + +(deftest instant-parse-is-iso-instant + (is (= "DateTimeParseException: Text 'x' could not be parsed at index 0" (outcome (Instant/parse "x"))))) + +(deftest a-method-fn-checks-its-receiver + (testing "Class/.member hands out the raw method; a foreign receiver raises instead of being read as a formatter" + (is (thrown? Exception (DateTimeFormatter/.parse "x" "y"))) + (is (thrown? Exception (DateTimeFormatterBuilder/.appendLiteral "s" "t"))))) + +(deftest builder + (is (= "2024/3 at 07" + (outcome (.format (-> (DateTimeFormatterBuilder.) (.appendValue ChronoField/YEAR 4) (.appendLiteral \/) + (.appendValue ChronoField/MONTH_OF_YEAR) (.appendLiteral " at ") + (.appendValue ChronoField/HOUR_OF_DAY 2) (.toFormatter)) + (LocalDateTime/of 2024 3 9 7 5))))) + (is (= "March" (outcome (.format (-> (DateTimeFormatterBuilder.) (.appendText ChronoField/MONTH_OF_YEAR) (.toFormatter)) + (LocalDate/of 2024 3 9))))) + (is (= "2024-03-09" (outcome (LocalDate/parse "MARCH 2024 9" + (-> (DateTimeFormatterBuilder.) (.parseCaseInsensitive) + (.appendPattern "MMMM yyyy d") (.toFormatter)))))) + (is (= "2024-03-09!" (outcome (.format (-> (DateTimeFormatterBuilder.) (.append DateTimeFormatter/ISO_LOCAL_DATE) + (.appendLiteral "!") (.toFormatter)) + (LocalDate/of 2024 3 9)))))) + +(deftest chrono-field + (is (= "HourOfDay" (outcome ChronoField/HOUR_OF_DAY))) + (is (= "MICRO_OF_SECOND" (.name ChronoField/MICRO_OF_SECOND))) + (is (= 30 (count (ChronoField/values)))) + (is (= ChronoField/YEAR (ChronoField/valueOf "YEAR")))) + +(deftest zones + (is (= ["UTC" "Z" "+02:00" "UTC+01:00"] + (mapv #(outcome (ZoneId/of %)) ["UTC" "Z" "+02:00" "UTC+01:00"]))) + (is (= "+02:00" (outcome (ZoneOffset/ofHours 2)))) + (is (= "Z" (outcome ZoneOffset/UTC))) + (is (= -19800 (.getTotalSeconds (ZoneOffset/of "-05:30")))) + (is (= "+05:45" (outcome (ZoneOffset/ofHoursMinutes 5 45)))) + (is (= "GMT" (.getId (ZoneId/of "GMT")))) + (testing "= and hash by value, with the JDK hashCode" + (is (= (ZoneOffset/ofHours 2) (ZoneOffset/of "+02:00"))) + (is (not= (ZoneOffset/of "Z") (ZoneId/of "UTC"))) + (is (= 7200 (hash (ZoneOffset/ofHours 2)))) + (is (= 84356 (hash (ZoneId/of "UTC"))))) + (is (= "DateTimeException: Zone offset hours not in valid range: value 25 is not in the range -18 to 18" + (outcome (ZoneOffset/of "+25:00")))) + (testing "a region zone needs the tz database cljw does not ship: it raises, never guesses" + (is (re-find #"Unknown time-zone ID: Europe/Paris" (outcome (ZoneId/of "Europe/Paris")))))) + +(deftest from-conversions + (is (= 5000 (inst-ms (Date/from (Instant/ofEpochSecond 5))))) + (is (= "1970-01-01T00:00:07Z" (outcome (Instant/from (Instant/ofEpochSecond 7)))))) diff --git a/test/clj/suites/java_time_values_test.clj b/test/clj/suites/java_time_values_test.clj new file mode 100644 index 000000000..e37e432d6 --- /dev/null +++ b/test/clj/suites/java_time_values_test.clj @@ -0,0 +1,313 @@ +;; java.time value types (D-462 / ADR-0161 / D-510..D-512): Instant, Duration, +;; LocalDate, LocalTime, LocalDateTime, DayOfWeek / Month / ChronoUnit enums. +;; `(str v)` is the ISO toString (clj-grounded); pr-str's `#object[…]` wrapper is +;; the locked divergence AD-042, so every case reads values through `str` or the +;; instance methods. +;; +;; Migrated from the phase15 java_time e2e scripts (one cljw process each): +;; instant, duration, local_date_time, local_date, compare, sort, +;; local_date_arith, ldt_arith, arith3, duration_between, dow_month, +;; ldt_calendar. `testing` strings carry the old block names. +(ns suites.java-time-values-test + (:require [clojure.test :refer [deftest is testing]])) + +(deftest java-time-instant + (testing "phase15_java_time_instant epoch0" + (let [i (java.time.Instant/ofEpochSecond 0)] + (is (= "1970-01-01T00:00:00Z" (str i))) + (is (= [0 0 0] [(.getEpochSecond i) (.getNano i) (.toEpochMilli i)])))) + (testing "sec-nanos" + (let [i (java.time.Instant/ofEpochSecond 1704067200 500000000)] + (is (= "2024-01-01T00:00:00.500Z" (str i))) + (is (= 500000000 (.getNano i))) + (is (= 1704067200500 (.toEpochMilli i))))) + (testing "parse / ofEpochMilli" + (is (= "2024-01-01T00:00:00Z" (str (java.time.Instant/parse "2024-01-01T00:00:00Z")))) + (is (= "2024-01-01T00:00:00Z" (str (java.time.Instant/ofEpochMilli 1704067200000))))) + (testing "value-eq" + (is (= (java.time.Instant/ofEpochSecond 5) (java.time.Instant/ofEpochSecond 5))) + (is (not= (java.time.Instant/ofEpochSecond 5) (java.time.Instant/ofEpochSecond 6)))) + (testing "instant-until (D-512)" + (let [a (java.time.Instant/ofEpochSecond 0) b (java.time.Instant/ofEpochSecond 3661)] + (is (= [3661 61 1 2] + [(.until a b java.time.temporal.ChronoUnit/SECONDS) + (.until a b java.time.temporal.ChronoUnit/MINUTES) + (.until a b java.time.temporal.ChronoUnit/HOURS) + (.until (java.time.Instant/ofEpochSecond 0) (java.time.Instant/ofEpochSecond 200000) + java.time.temporal.ChronoUnit/DAYS)]))) + (is (thrown-with-msg? Throwable #"time-based ChronoUnit" + (.until (java.time.Instant/ofEpochSecond 0) (java.time.Instant/ofEpochSecond 1) + java.time.temporal.ChronoUnit/WEEKS)) + "instant-until-unsupported"))) + +(deftest java-time-duration + (testing "phase15_java_time_duration str-forms" + (is (= ["PT1M30S" "PT2H" "PT1H30M" "PT1.5S" "PT5.5S" "PT0S" "PT-30S" "PT1H1M1S" + "PT-1H-1M-1S" "PT-1.5S" "PT-0.5S" "PT0.123456789S" "PT24H"] + (mapv str [(java.time.Duration/ofSeconds 90) + (java.time.Duration/ofHours 2) + (java.time.Duration/ofMinutes 90) + (java.time.Duration/ofMillis 1500) + (java.time.Duration/ofSeconds 5 500000000) + (java.time.Duration/ofSeconds 0) + (java.time.Duration/ofSeconds -30) + (java.time.Duration/ofSeconds 3661) + (java.time.Duration/ofSeconds -3661) + (java.time.Duration/ofMillis -1500) + (java.time.Duration/ofSeconds -1 500000000) + (java.time.Duration/ofNanos 123456789) + (java.time.Duration/ofDays 1)])))) + (testing "readers" + (let [d (java.time.Duration/ofMillis -1500)] + (is (= [-2 500000000 -1500] [(.getSeconds d) (.getNano d) (.toMillis d)]))) + (let [d (java.time.Duration/ofSeconds 90)] + (is (= [90 0 90000 1] [(.getSeconds d) (.getNano d) (.toMillis d) (.toMinutes d)])))) + (testing "value-eq" + (is (= (java.time.Duration/ofSeconds 90) (java.time.Duration/ofMillis 90000))) + (is (not= (java.time.Duration/ofSeconds 90) (java.time.Duration/ofSeconds 91))))) + +(deftest java-time-local-date-time + (testing "phase15_java_time_local_date_time str-forms" + (is (= ["2024-01-01T12:30" "2024-01-01T12:30:45" "2024-01-01T12:30:45.500" + "2024-06-15T00:00" "2024-12-31T00:00"] + (mapv str [(java.time.LocalDateTime/of 2024 1 1 12 30) + (java.time.LocalDateTime/of 2024 1 1 12 30 45) + (java.time.LocalDateTime/of 2024 1 1 12 30 45 500000000) + (java.time.LocalDateTime/of 2024 6 15 0 0 0) + (java.time.LocalDateTime/of 2024 12 31 0 0)])))) + (testing "readers" + (let [d (java.time.LocalDateTime/of 2024 3 9 14 5 45 123456789)] + (is (= [2024 3 9 14 5 45 123456789] + [(.getYear d) (.getMonthValue d) (.getDayOfMonth d) + (.getHour d) (.getMinute d) (.getSecond d) (.getNano d)])))) + (testing "parse" + (is (= "2024-01-01T12:30:45" (str (java.time.LocalDateTime/parse "2024-01-01T12:30:45")))) + (is (= "2024-01-01T12:30" (str (java.time.LocalDateTime/parse "2024-01-01T12:30"))))) + (testing "value-eq" + (is (= (java.time.LocalDateTime/of 2024 1 1 12 30 0) (java.time.LocalDateTime/of 2024 1 1 12 30))) + (is (not= (java.time.LocalDateTime/of 2024 1 1 12 30) (java.time.LocalDateTime/of 2024 1 1 12 31))))) + +(deftest java-time-local-date-and-time + (testing "phase15_java_time_local_date localdate" + (is (= ["2024-01-01" "2024-12-31" "0001-01-01"] + (mapv str [(java.time.LocalDate/of 2024 1 1) (java.time.LocalDate/of 2024 12 31) + (java.time.LocalDate/of 1 1 1)]))) + (let [d (java.time.LocalDate/of 2024 3 9)] + (is (= [2024 3 9] [(.getYear d) (.getMonthValue d) (.getDayOfMonth d)]))) + (is (= "2024-02-29" (str (java.time.LocalDate/parse "2024-02-29")))) + (is (= "2024-02-10" (str (.plusDays (java.time.LocalDate/of 2024 1 1) 40)))) + (is (= (java.time.LocalDate/of 2024 1 1) (java.time.LocalDate/of 2024 1 1))) + (is (not= (java.time.LocalDate/of 2024 1 1) (java.time.LocalDate/of 2024 1 2)))) + (testing "localtime" + (is (= ["12:30" "12:30:45" "12:30:45.500" "00:00"] + (mapv str [(java.time.LocalTime/of 12 30) (java.time.LocalTime/of 12 30 45) + (java.time.LocalTime/of 12 30 45 500000000) (java.time.LocalTime/of 0 0)]))) + (let [t (java.time.LocalTime/of 14 5 45 123456789)] + (is (= [14 5 45 123456789] [(.getHour t) (.getMinute t) (.getSecond t) (.getNano t)]))) + (is (= "06:07:08" (str (java.time.LocalTime/parse "06:07:08")))) + (is (= (java.time.LocalTime/of 12 30) (java.time.LocalTime/of 12 30 0))) + (is (not= (java.time.LocalTime/of 12 30) (java.time.LocalTime/of 12 31)))) + (testing "to-local" + (let [dt (java.time.LocalDateTime/of 2024 3 9 14 5 45 500000000)] + (is (= "2024-03-09" (str (.toLocalDate dt)))) + (is (= "14:05:45.500" (str (.toLocalTime dt))))))) + +(deftest java-time-compare-predicates + (testing "phase15_java_time_compare instant" + (let [a (java.time.Instant/ofEpochSecond 5) b (java.time.Instant/ofEpochSecond 9)] + (is (= [true false false true] [(.isBefore a b) (.isAfter a b) (.isBefore a a) (.isAfter b a)])))) + (testing "localdate" + (let [a (java.time.LocalDate/of 2024 1 1) b (java.time.LocalDate/of 2024 6 1)] + (is (= [true false false true] + [(.isBefore a b) (.isAfter a b) (.isEqual a b) (.isEqual a (java.time.LocalDate/of 2024 1 1))])))) + (testing "localtime-ldt" + (let [a (java.time.LocalTime/of 6 0) b (java.time.LocalTime/of 18 0)] + (is (= [true false] [(.isBefore a b) (.isAfter a b)]))) + (let [a (java.time.LocalDateTime/of 2024 1 1 6 0) b (java.time.LocalDateTime/of 2024 1 1 18 0)] + (is (= [true false false] [(.isBefore a b) (.isAfter a b) (.isEqual a b)])))) + (testing "duration-unary" + (is (= [true false true false] + [(.isZero (java.time.Duration/ofSeconds 0)) (.isZero (java.time.Duration/ofSeconds 5)) + (.isNegative (java.time.Duration/ofSeconds -1)) (.isNegative (java.time.Duration/ofSeconds 1))])) + (is (= ["PT-1.5S" "PT30S" "PT30S" "PT0S"] + [(str (.negated (java.time.Duration/ofMillis 1500))) + (str (.abs (java.time.Duration/ofSeconds -30))) + (str (.abs (java.time.Duration/ofSeconds 30))) + (str (.negated (java.time.Duration/ofSeconds 0)))])))) + +(deftest java-time-compare-and-sort + ;; AD-043 pin: TEMPORAL compare is SIGN-based (clj answers a magnitude for + ;; LocalDate/LocalDateTime); the string family is clj's compareTo magnitude. + (testing "phase15_java_time_sort compare-sign" + (is (= [-1 1 0] + [(compare (java.time.Instant/ofEpochSecond 5) (java.time.Instant/ofEpochSecond 9)) + (compare (java.time.Instant/ofEpochSecond 9) (java.time.Instant/ofEpochSecond 5)) + (compare (java.time.Instant/ofEpochSecond 5) (java.time.Instant/ofEpochSecond 5))])) + (is (= [1 -1 -1 -1] + [(compare (java.time.LocalDate/of 2024 6 1) (java.time.LocalDate/of 2024 1 1)) + (compare (java.time.LocalTime/of 6 0) (java.time.LocalTime/of 18 0)) + (compare (java.time.Duration/ofSeconds 1) (java.time.Duration/ofSeconds 9)) + (compare (java.time.LocalDateTime/of 2024 1 1 6 0) (java.time.LocalDateTime/of 2024 1 1 18 0))]))) + (testing "sort" + (is (= ["2024-01-01" "2024-03-01" "2024-06-01"] + (mapv str (sort [(java.time.LocalDate/of 2024 6 1) (java.time.LocalDate/of 2024 1 1) + (java.time.LocalDate/of 2024 3 1)]))))) + (testing "string-family-magnitude" + (is (= [-2 1 -2 -2 -2] + [(compare "a" "c") (compare "abc" "ab") (compare :a :c) (compare 'aa 'ac) (compare \a \c)])))) + +(deftest java-time-local-date-arith + (testing "phase15_java_time_local_date_arith day-week" + (is (= ["2024-02-10" "2023-12-31" "2024-01-15" "2024-01-01"] + [(str (.plusDays (java.time.LocalDate/of 2024 1 1) 40)) + (str (.minusDays (java.time.LocalDate/of 2024 1 1) 1)) + (str (.plusWeeks (java.time.LocalDate/of 2024 1 1) 2)) + (str (.minusWeeks (java.time.LocalDate/of 2024 1 15) 2))]))) + (testing "month-year (civil clamp)" + (is (= ["2024-02-29" "2023-02-28" "2025-01-15" "2024-02-29" "2025-02-28" "2019-06-15"] + [(str (.plusMonths (java.time.LocalDate/of 2024 1 31) 1)) + (str (.plusMonths (java.time.LocalDate/of 2023 1 31) 1)) + (str (.plusMonths (java.time.LocalDate/of 2024 12 15) 1)) + (str (.minusMonths (java.time.LocalDate/of 2024 3 31) 1)) + (str (.plusYears (java.time.LocalDate/of 2024 2 29) 1)) + (str (.minusYears (java.time.LocalDate/of 2024 6 15) 5))]))) + (testing "leap-length" + (is (= [true false false true] + (mapv #(.isLeapYear (java.time.LocalDate/of % 1 1)) [2024 2023 1900 2000]))) + (is (= [29 28 30] + [(.lengthOfMonth (java.time.LocalDate/of 2024 2 1)) + (.lengthOfMonth (java.time.LocalDate/of 2023 2 1)) + (.lengthOfMonth (java.time.LocalDate/of 2024 4 1))]))) + (testing "at-time" + (let [d (java.time.LocalDate/of 2026 6 20)] + (is (= ["2026-06-20T00:00" "2026-06-20T13:30" "2026-06-20T13:30:45" + "2026-06-20T00:00:00.500" "2026-06-20"] + [(str (.atStartOfDay d)) (str (.atTime d 13 30)) (str (.atTime d 13 30 45)) + (str (.atTime d 0 0 0 500000000)) (str (.toLocalDate (.atTime d 13 30)))]))))) + +(deftest java-time-ldt-arith + (testing "phase15_java_time_ldt_arith day-week" + (is (= ["2024-02-10T12:30" "2024-02-29T12:00" "2024-01-15T00:00"] + [(str (.plusDays (java.time.LocalDateTime/of 2024 1 1 12 30) 40)) + (str (.minusDays (java.time.LocalDateTime/of 2024 3 1 12 0) 1)) + (str (.plusWeeks (java.time.LocalDateTime/of 2024 1 1 0 0) 2))]))) + (testing "time-carry (midnight carry/borrow)" + (is (= ["2024-01-02T01:30" "2024-01-02T00:01" "2023-12-31T23:00" + "2024-01-01T00:01:15" "2024-01-01T12:00:01.500"] + [(str (.plusHours (java.time.LocalDateTime/of 2024 1 1 23 30) 2)) + (str (.plusMinutes (java.time.LocalDateTime/of 2024 1 1 23 59) 2)) + (str (.minusHours (java.time.LocalDateTime/of 2024 1 1 1 0) 2)) + (str (.plusSeconds (java.time.LocalDateTime/of 2024 1 1 0 0 30) 45)) + (str (.plusNanos (java.time.LocalDateTime/of 2024 1 1 12 0 0 0) 1500000000))]))) + (testing "phase15_java_time_ldt_calendar months" + (is (= ["2024-02-29T14:30" "2023-02-28T09:00" "2025-02-15T06:00" "2024-02-29T00:00"] + [(str (.plusMonths (java.time.LocalDateTime/of 2024 1 31 14 30) 1)) + (str (.plusMonths (java.time.LocalDateTime/of 2023 1 31 9 0) 1)) + (str (.plusMonths (java.time.LocalDateTime/of 2024 12 15 6 0) 2)) + (str (.minusMonths (java.time.LocalDateTime/of 2024 3 31 0 0) 1))]))) + (testing "years" + (is (= ["2025-02-28T23:59:59" "2019-06-15T12:00"] + [(str (.plusYears (java.time.LocalDateTime/of 2024 2 29 23 59 59) 1)) + (str (.minusYears (java.time.LocalDateTime/of 2024 6 15 12 0) 5))])))) + +(deftest java-time-arith3 + (testing "phase15_java_time_arith3 instant" + (let [i (java.time.Instant/ofEpochSecond 100 0)] + (is (= [105 101500 1 50] + [(.getEpochSecond (.plusSeconds i 5)) (.toEpochMilli (.plusMillis i 1500)) + (.getNano (.plusNanos i 1)) (.getEpochSecond (.minusSeconds i 50))])))) + (testing "duration" + (is (= ["PT15S" "PT2M" "PT5.5S" "PT12S" "PT2.5S" "PT7S"] + [(str (.plusSeconds (java.time.Duration/ofSeconds 5) 10)) + (str (.plusMinutes (java.time.Duration/ofSeconds 0) 2)) + (str (.plus (java.time.Duration/ofSeconds 5) (java.time.Duration/ofMillis 500))) + (str (.multipliedBy (java.time.Duration/ofSeconds 3) 4)) + (str (.dividedBy (java.time.Duration/ofSeconds 10) 4)) + (str (.minusSeconds (java.time.Duration/ofSeconds 10) 3))]))) + (testing "localtime (wraps mod 24h)" + (is (= ["01:00" "23:00" "13:15" "12:00:01.500"] + [(str (.plusHours (java.time.LocalTime/of 23 0) 2)) + (str (.minusHours (java.time.LocalTime/of 1 0) 2)) + (str (.plusMinutes (java.time.LocalTime/of 12 30) 45)) + (str (.plusNanos (java.time.LocalTime/of 12 0 0 0) 1500000000))])))) + +(deftest java-time-duration-between + (testing "phase15_java_time_duration_between between" + (is (= ["PT1M30S" "PT-1M-30S" "PT1H30M" "PT2H30M"] + [(str (java.time.Duration/between (java.time.Instant/ofEpochSecond 100) (java.time.Instant/ofEpochSecond 190))) + (str (java.time.Duration/between (java.time.Instant/ofEpochSecond 190) (java.time.Instant/ofEpochSecond 100))) + (str (java.time.Duration/between (java.time.LocalDateTime/of 2024 1 1 12 0) (java.time.LocalDateTime/of 2024 1 1 13 30))) + (str (java.time.Duration/between (java.time.LocalTime/of 6 0) (java.time.LocalTime/of 8 30)))]))) + (testing "instant-plus-dur" + (is (= ["1970-01-01T00:01:45.500Z" "1970-01-01T00:00:50Z"] + [(str (.plus (java.time.Instant/ofEpochSecond 100) (java.time.Duration/ofSeconds 5 500000000))) + (str (.minus (java.time.Instant/ofEpochSecond 100) (java.time.Duration/ofSeconds 50)))]))) + (testing "ldt-plus-dur" + (is (= ["2024-01-02T01:00" "2024-01-01T23:00"] + [(str (.plus (java.time.LocalDateTime/of 2024 1 1 23 0) (java.time.Duration/ofHours 2))) + (str (.minus (java.time.LocalDateTime/of 2024 1 2 1 0) (java.time.Duration/ofHours 2)))])))) + +(deftest java-time-dow-month-enums + (testing "phase15_java_time_dow_month localdate" + (let [d (java.time.LocalDate/of 2024 3 9)] + (is (= ["SATURDAY" 6 "MARCH" 3 69] + [(str (.getDayOfWeek d)) (.getValue (.getDayOfWeek d)) + (str (.getMonth d)) (.getValue (.getMonth d)) (.getDayOfYear d)])))) + (testing "localdatetime" + (let [dt (java.time.LocalDateTime/of 2024 12 25 10 0)] + (is (= ["WEDNESDAY" "DECEMBER"] [(str (.getDayOfWeek dt)) (str (.getMonth dt))])))) + (testing "enum-eq" + (is (= (.getDayOfWeek (java.time.LocalDate/of 2024 3 9)) (.getDayOfWeek (java.time.LocalDate/of 2024 3 16)))) + (is (not= (.getMonth (java.time.LocalDate/of 2024 3 9)) (.getMonth (java.time.LocalDate/of 2024 4 9))))) + (testing "dow-boundary" + (is (= ["THURSDAY" "SUNDAY"] + [(str (.getDayOfWeek (java.time.LocalDate/of 1970 1 1))) + (str (.getDayOfWeek (java.time.LocalDate/of 2024 3 10)))]))) + (testing "chrono-unit" + (is (= ["Days" "DAYS" "HalfDays" "MILLENNIA" true false] + [(str java.time.temporal.ChronoUnit/DAYS) + (.name java.time.temporal.ChronoUnit/DAYS) + (str java.time.temporal.ChronoUnit/HALF_DAYS) + (.name java.time.temporal.ChronoUnit/MILLENNIA) + (= java.time.temporal.ChronoUnit/DAYS java.time.temporal.ChronoUnit/DAYS) + (= java.time.temporal.ChronoUnit/DAYS java.time.temporal.ChronoUnit/WEEKS)]))) + (testing "until" + (let [a (java.time.LocalDate/of 2020 1 1) b (java.time.LocalDate/of 2021 3 15)] + (is (= [439 62 14 1 1 -60] + [(.until a b java.time.temporal.ChronoUnit/DAYS) + (.until a b java.time.temporal.ChronoUnit/WEEKS) + (.until a b java.time.temporal.ChronoUnit/MONTHS) + (.until a b java.time.temporal.ChronoUnit/YEARS) + (.until (java.time.LocalDate/of 2020 1 31) (java.time.LocalDate/of 2020 3 1) + java.time.temporal.ChronoUnit/MONTHS) + (.until (java.time.LocalDate/of 2020 3 1) (java.time.LocalDate/of 2020 1 1) + java.time.temporal.ChronoUnit/DAYS)]))) + (is (thrown-with-msg? Throwable #"date-based ChronoUnit" + (.until (java.time.LocalDate/of 2020 1 1) (java.time.LocalDate/of 2020 1 2) + java.time.temporal.ChronoUnit/HOURS)) + "until-unsupported")) + (testing "ldt-until" + (let [a (java.time.LocalDateTime/of 2020 1 1 0 0 0) b (java.time.LocalDateTime/of 2021 3 2 6 30 15)] + (is (= [426 10230 613830 36829815 14 1] + (mapv #(.until a b %) + [java.time.temporal.ChronoUnit/DAYS java.time.temporal.ChronoUnit/HOURS + java.time.temporal.ChronoUnit/MINUTES java.time.temporal.ChronoUnit/SECONDS + java.time.temporal.ChronoUnit/MONTHS java.time.temporal.ChronoUnit/YEARS]))))) + (testing "enum-const" + (is (= ["MONDAY" 7 "MONDAY" "JANUARY" 12 "MARCH"] + [(str java.time.DayOfWeek/MONDAY) (.getValue java.time.DayOfWeek/SUNDAY) + (.name java.time.DayOfWeek/MONDAY) (str java.time.Month/JANUARY) + (.getValue java.time.Month/DECEMBER) (.name java.time.Month/MARCH)]))) + (testing "enum-identity" + (let [d (java.time.LocalDate/of 2024 3 9)] + (is (= (.getDayOfWeek d) java.time.DayOfWeek/SATURDAY)) + (is (identical? (.getDayOfWeek d) java.time.DayOfWeek/SATURDAY)) + (is (= (.getMonth d) java.time.Month/MARCH)) + (is (identical? (.getMonth d) java.time.Month/MARCH)))) + (testing "enum-compare (AD-043 sign pin)" + (is (= [-1 1 -1 ["MONDAY" "WEDNESDAY" "FRIDAY"]] + [(compare java.time.DayOfWeek/MONDAY java.time.DayOfWeek/SUNDAY) + (compare java.time.Month/MARCH java.time.Month/JANUARY) + (compare java.math.RoundingMode/UP java.math.RoundingMode/DOWN) + (mapv str (sort [java.time.DayOfWeek/WEDNESDAY java.time.DayOfWeek/MONDAY + java.time.DayOfWeek/FRIDAY]))])))) diff --git a/test/clj/suites/protocol_dispatch_cache_test.clj b/test/clj/suites/protocol_dispatch_cache_test.clj new file mode 100644 index 000000000..554978d54 --- /dev/null +++ b/test/clj/suites/protocol_dispatch_cache_test.clj @@ -0,0 +1,103 @@ +;; ADR-0206: every ProtocolFn memoises (receiver descriptor, protocol +;; generation) -> resolved method. These pins call a protocol fn FIRST, so the +;; cache is filled, then change what the call must resolve to, then call +;; again. A stale memo answers the old value. Every expected value is the +;; JVM's (clojure -M); the cache must be invisible. +(ns suites.protocol-dispatch-cache-test + (:require [clojure.test :refer [deftest is]])) + +(defprotocol PCache (pc [x])) + +(defrecord CacheA [v]) +(defrecord CacheB [v]) + +;; --- extend after the first call: a miss that raised must not stick --- +(deftest extend-type-after-a-failed-first-call + (let [a (->CacheA 1)] + (is (thrown? Exception (pc a))) + (extend-type CacheA PCache (pc [x] [:a (:v x)])) + (is (= [:a 1] (pc a))))) + +;; --- a re-extend replaces the impl the cache already holds --- +(deftest re-extend-replaces-a-cached-impl + (extend-type CacheB PCache (pc [_] :first)) + (let [b (->CacheB 1)] + (is (= :first (pc b))) + (is (= :first (pc b))) + (extend-type CacheB PCache (pc [_] :second)) + (is (= :second (pc b))))) + +;; --- an Object default cached for a type is displaced by a later +;; type-specific extension --- +(defprotocol PFallback (pf [x])) +(defrecord FallbackR []) +(extend-protocol PFallback Object (pf [_] :object)) + +(deftest specific-extend-displaces-a-cached-object-default + (let [r (->FallbackR)] + (is (= :object (pf r))) + (is (= :object (pf r))) + (extend-type FallbackR PFallback (pf [_] :specific)) + (is (= :specific (pf r))) + (is (= :object (pf "a string still falls back"))))) + +;; --- one call site alternating receiver types (the slot is monomorphic, +;; so this is the refill path every iteration) --- +(defprotocol PPoly (pp [x])) +(defrecord PolyA []) +(defrecord PolyB []) +(extend-protocol PPoly + PolyA (pp [_] 1) + PolyB (pp [_] 10)) + +(deftest alternating-receivers-at-one-site + (let [xs (vec (take 100 (cycle [(->PolyA) (->PolyB)])))] + (is (= 550 (reduce + (map pp xs)))))) + +;; --- redefining the record: the old vintage keeps its impl, the new one +;; must not be answered from the old vintage's cache entry --- +(defprotocol PRedef (pr-v [x])) +(defrecord RedefR [a]) +(extend-protocol PRedef RedefR (pr-v [x] [:old (:a x)])) +(def ^:private old-r (->RedefR 1)) +(def ^:private old-answer (pr-v old-r)) +(defrecord RedefR [a]) +(extend-protocol PRedef RedefR (pr-v [x] [:new (:a x)])) +(def ^:private new-r (->RedefR 2)) + +(deftest redefined-record-dispatches-per-vintage + (is (= [:old 1] old-answer)) + (is (= [:new 2] (pr-v new-r))) + (is (= [:old 1] (pr-v old-r))) + (is (= [:new 2] (pr-v new-r)))) + +;; --- re-evaluating defprotocol, then re-extending: clj and bb both keep the +;; extend-type impl across a same-shape defprotocol re-eval (measured: :v1 +;; twice), and the re-extend after it must win over the cached :v1 --- +(defprotocol PReproto (rp [x])) +(defrecord ReprotoR []) +(extend-type ReprotoR PReproto (rp [_] :v1)) +(def ^:private before-redefine (rp (->ReprotoR))) +(defprotocol PReproto (rp [x])) +(def ^:private after-redefine (rp (->ReprotoR))) + +(deftest redefined-protocol-then-re-extend + (is (= :v1 before-redefine)) + (is (= :v1 after-redefine)) + (extend-type ReprotoR PReproto (rp [_] :v2)) + (is (= :v2 (rp (->ReprotoR))))) + +;; --- native receivers share one descriptor per tag --- +(defprotocol PNative (pn [x])) + +(deftest extend-a-native-type-after-first-call + (is (thrown? Exception (pn 1))) + (extend-protocol PNative Long (pn [x] (inc x))) + (is (= 2 (pn 1))) + (extend-protocol PNative Long (pn [x] (dec x))) + (is (= 0 (pn 1)))) + +;; --- reify: each reify value carries its own anonymous descriptor --- +(deftest reify-receivers-do-not-alias + (let [rs (mapv (fn [i] (reify PPoly (pp [_] i))) (range 5))] + (is (= [0 1 2 3 4] (mapv pp rs))))) diff --git a/test/clj/suites/test_reporters_test.clj b/test/clj/suites/test_reporters_test.clj new file mode 100644 index 000000000..2cc948757 --- /dev/null +++ b/test/clj/suites/test_reporters_test.clj @@ -0,0 +1,215 @@ +;; clojure.test itself and its reporters, plus the other bundled REPL/debug +;; nses (D-227/D-273/D-463/D-475/D-502/D-513/D-563b, ADR-0140/0156). +;; +;; Migrated from the phase15 e2e scripts clojure_test, clojure_test_output, +;; clojure_test_tap, test_junit, spec, repl_discovery, tap and +;; clojure_stacktrace. A case that defines deftests and calls run-tests is a +;; NESTED run inside this suite's own run, so `nested-run` isolates the +;; clojure.test dynamic state (report counters, testing vars/contexts, +;; *test-out*) and evaluates the program in a fresh p15 namespace. The program +;; text in a case is what the script fed to `cljw -`; only the namespace names +;; differ (unique across the shared suite image). +(ns suites.test-reporters-test + (:require [clojure.test :as t :refer [deftest is testing]] + [clojure.string :as str] + [clojure.spec.alpha :as s] + [clojure.repl] + [clojure.set] + [clojure.stacktrace])) + +(def ^:private scratch-seq (atom 0)) + +(defn- fresh-ns-name [] (symbol (str "p15.rep" (swap! scratch-seq inc)))) + +(defn- nested-run + "Evaluate `src` (which may contain {{ns}}, replaced by a fresh ns name) as a + nested clojure.test program. Returns {:value v :out captured-stdout}." + [src] + (let [n (fresh-ns-name) + src (str/replace src "{{ns}}" (str n)) + w (java.io.StringWriter.)] + (binding [t/*report-counters* nil + t/*testing-vars* (list) + t/*testing-contexts* (list) + t/*test-out* w + *ns* (create-ns n)] + (refer 'clojure.core) + (let [out (with-out-str (binding [t/*test-out* *out*] + (def ^:private last-value (load-string src))))] + {:value last-value :out out})))) + +(defn- counts [src] + (let [{s :value} (nested-run src)] [(:test s) (:pass s) (:fail s) (:error s)])) + +(defn- out-lines [src] (set (str/split-lines (:out (nested-run src))))) + +(deftest p15-clojure-test-counts + (let [hdr "(ns {{ns}} (:require [clojure.test :refer [deftest is are testing run-tests]])) "] + (testing "phase15_clojure_test" + (is (= [1 3 0 0] (counts (str hdr "(deftest a (is (= 1 1)) (is (pos? 3)) (is (not (nil? [])))) (run-tests '{{ns}})"))) "all-pass") + (is (= [1 1 1 0] (counts (str hdr "(deftest a (is (= 1 1)) (is (= 1 2))) (run-tests '{{ns}})"))) "one-fail") + (is (= [1 0 0 1] (counts (str hdr "(deftest a (is (throw (ex-info \"x\" {})))) (run-tests '{{ns}})"))) "error") + (is (= [1 1 0 0] (counts (str hdr "(deftest a (is (thrown? Throwable (throw (ex-info \"x\" {}))))) (run-tests '{{ns}})"))) "thrown") + (is (= [1 3 0 0] (counts (str hdr "(deftest a (are [x y] (= x y) 1 1 2 2 3 3)) (run-tests '{{ns}})"))) "are") + (is (= [2 3 0 0] (counts (str hdr "(deftest a (is (= 1 1))) (deftest b (is (= 2 2)) (is (= 3 3))) (run-tests)"))) "multi-deftest") + (is (= [1 2 0 0] (counts (str hdr "(deftest a (testing \"ctx\" (is (= 1 1)) (is (= 2 2)))) (run-tests '{{ns}})"))) "testing")))) + +(deftest p15-clojure-test-output + (let [hdr "(ns {{ns}} (:require [clojure.test :refer [deftest is testing run-tests]])) "] + (testing "phase15_clojure_test_output A (= fail)" + (let [{:keys [out]} (nested-run (str hdr "(deftest eqfail (is (= 1 2))) (run-tests '{{ns}})")) + ls (set (str/split-lines out))] + (is (some #(re-matches #"Testing p15\.rep\d+" %) ls) "A-testing") + (is (str/includes? out "FAIL in (eqfail) (") "A-fail-in (D-563b suffix)") + (is (ls "expected: (= 1 2)") "A-expected") + (is (ls " actual: (not (= 1 2))") "A-actual") + (is (ls "Ran 1 tests containing 1 assertions.") "A-ran") + (is (ls "1 failures, 0 errors.") "A-summary"))) + (testing "B predicate fail" + (is ((out-lines (str hdr "(deftest pf (is (pos? -1))) (run-tests '{{ns}})")) " actual: (not (pos? -1))") "B-actual")) + (testing "C testing context" + (is ((out-lines (str hdr "(deftest cf (testing \"outer\" (testing \"inner\" (is (= :a :b))))) (run-tests '{{ns}})")) "outer inner") "C-context")) + (testing "E/F *test-out* rebinding" + (is (some? (resolve 'clojure.test/*test-out*)) "E-resolvable") + (let [{:keys [value]} (nested-run (str hdr "(deftest f (is (= 1 2))) (with-out-str (binding [clojure.test/*test-out* *out*] (run-tests '{{ns}})))"))] + (is (str/includes? value "FAIL in (f) (") "F-captured-fail"))))) + +(deftest p15-clojure-test-tap + (testing "phase15_clojure_test_tap" + (let [{:keys [out]} (nested-run "(require '[clojure.test :as t] '[clojure.test.tap :as tap]) +(t/deftest pass-t (t/is (= 1 1))) +(t/deftest fail-t (t/is (= 1 2))) +(tap/with-tap-output (t/run-tests '{{ns}}))") + ;; normalise the fresh ns name and the source label (`` in the + ;; script, NO_SOURCE_PATH under load-string; the ` (file:line)` suffix + ;; itself is the D-563b behaviour under test) + norm (-> out + (str/replace #"p15\.rep\d+" "NS") + (str/replace #"\((?:NO_SOURCE_PATH|):\d+\)" "(SRC)"))] + (is (= (str/join "\n" ["# {:type :begin-test-ns, :ns NS}" + "# {:type :begin-test-var, :var #'NS/pass-t}" + "ok (pass-t) (SRC)" + "# expected:(= 1 1)" + "# actual:(= 1 1)" + "# {:type :end-test-var, :var #'NS/pass-t}" + "# {:type :begin-test-var, :var #'NS/fail-t}" + "not ok (fail-t) (SRC)" + "# expected:(= 1 2)" + "# actual:(not (= 1 2))" + "# {:type :end-test-var, :var #'NS/fail-t}" + "# {:type :end-test-ns, :ns NS}" + "1..2"]) + (str/trim norm)) + "tap_output (also clojure_test_output D-pass-actual)")))) + +(deftest p15-test-junit + (testing "phase15_test_junit" + (let [{:keys [out]} (nested-run "(require '[clojure.test :as t] '[clojure.test.junit :as ju]) +(t/deftest a-pass (t/is (= 1 1))) +(t/deftest a-fail (t/is (= 1 2))) +(ju/with-junit-output (t/run-tests '{{ns}}))") + out (str/replace out #"p15\.rep\d+" "NS")] + (doseq [needle ["" + "" + "" + "" + "expected: (= 1 2)" + "" + ""]] + (is (str/includes? out needle) needle)) + (is (re-find #"" out) + "testsuite element names the ns"))) + (testing "begin/end-test-var fire for a custom reporter" + ;; report is ^:dynamic (D-273), so a custom reporter is a binding, which + ;; leaves the shared multimethod untouched. + (let [evs (atom [])] + (binding [t/report (fn [m] (when (#{:begin-test-var :end-test-var} (:type m)) + (swap! evs conj ({:begin-test-var :begin :end-test-var :end} (:type m)))))] + (nested-run "(clojure.test/deftest z (clojure.test/is true)) (clojure.test/run-tests '{{ns}})")) + (is (= [:begin :end] @evs))))) + +(s/def ::p15-age (s/and int? pos?)) +(s/def ::p15-r int?) +(defmulti p15-shp :t) +(defmethod p15-shp :c [_] (s/keys :req-un [::p15-r])) +(s/def ::p15-shp (s/multi-spec p15-shp :t)) + +(deftest p15-spec + (testing "phase15_spec regex ops" + (is (true? (s/valid? (s/and int? pos?) 5)) "load_valid") + (is (= {:a 1 :b "x"} (s/conform (s/cat :a int? :b string?) [1 "x"])) "cat") + (is (= [1 2 3] (s/conform (s/* int?) [1 2 3])) "star") + (is (= [1 2] (s/conform (s/+ int?) [1 2])) "plus") + (is (nil? (s/conform (s/? int?) [])) "opt") + (is (= [:s "x"] (s/conform (s/alt :i int? :s string?) ["x"])) "alt") + (is (= [1 2] (s/conform (s/& (s/* int?) (fn [v] (even? (count v)))) [1 2])) "amp") + (is (= {:a [:x 1]} (s/conform (s/cat :a (s/alt :x int? :y (s/cat :p int? :q int?))) [1])) "nestreg")) + (testing "collection / scalar specs" + (is (= [1 2 3] (s/conform (s/coll-of int?) [1 2 3])) "coll") + (is (= {:a 1} (s/conform (s/map-of keyword? int?) {:a 1})) "mapof") + (is (= [1 "x"] (s/conform (s/tuple int? string?) [1 "x"])) "tuple") + (is (nil? (s/conform (s/nilable int?) nil)) "nilable") + (is (true? (s/valid? (s/keys :req [::p15-age]) {::p15-age 3})) "keys") + (is (true? (s/valid? ::p15-shp {:t :c :p15-r 2})) "multi")) + (testing "explain / describe / form" + (is (= "\"x\" - failed: int?\n" (s/explain-str int? "x")) "explain_named") + (is (= "-1 - failed: pos?\n" (s/explain-str (s/and int? pos?) -1)) "explain_and") + (is (= '(and int? pos?) (s/describe (s/and int? pos?))) "describe") + (is (= "-1 - failed: :clojure.spec.alpha/unknown\n" (s/explain-str (fn [n] (pos? n)) -1)) "explain_anon") + (is (= '(clojure.spec.alpha/cat :a clojure.core/int?) (s/form (s/cat :a int?))) "form_parity"))) + +(deftest p15-repl-discovery + (testing "phase15_repl_discovery" + (is (= '(difference index intersection join map-invert project rename rename-keys select subset? superset? union) + (clojure.repl/dir-fn 'clojure.set)) "dir_fn_set") + (let [out (with-out-str (clojure.repl/dir clojure.set))] + (is (str/includes? out "difference") "dir_first") + (is (str/includes? out "union") "dir_last")) + (is (some #{'clojure.core/reduce-kv} (clojure.repl/apropos "reduce-kv")) "apropos_kv") + (is (some #{'clojure.core/reductions} (clojure.repl/apropos #"^reductions$")) "apropos_re") + (is (seq? (clojure.repl/apropos "definitely-no-such-var-xyzzy")) "apropos_empty"))) + +(deftest p15-tap + ;; tap> delivers ASYNCHRONOUSLY (an agent queue, D-502), so a value tapped + ;; earlier may still reach a tap added later: each tap filters on its own + ;; value instead of taking whatever arrives first. + (testing "phase15_tap" + (is (every? some? [(resolve 'tap>) (resolve 'add-tap) (resolve 'remove-tap)]) "resolve") + (is (true? (tap> 42)) "tap_returns_true") + (let [p (promise) f (fn [x] (when (= x 99) (deliver p x)))] + (add-tap f) + (try (tap> 99) + (is (= 99 (deref p 2000 :timeout)) "delivers") + (finally (remove-tap f)))) + (let [p (promise) f (fn [x] (when (= x 7) (deliver p x)))] + (add-tap f) + (remove-tap f) + (tap> 7) + (is (= :timeout (deref p 300 :timeout)) "remove")) + (let [p (promise) bad (fn [_] (throw (ex-info "boom" {}))) good (fn [x] (when (= x 5) (deliver p x)))] + (add-tap bad) + (add-tap good) + (try (tap> 5) + (is (= 5 (deref p 2000 :timeout)) "fault_isolation") + (finally (remove-tap bad) (remove-tap good)))))) + +(defn- p15-boom [] (/ 1 0)) + +(deftest p15-clojure-stacktrace + (testing "phase15_clojure_stacktrace" + (is (= "inner" (ex-message (clojure.stacktrace/root-cause (ex-info "outer" {} (ex-info "mid" {} (ex-info "inner" {})))))) "root_cause") + (is (= "solo" (ex-message (clojure.stacktrace/root-cause (ex-info "solo" {})))) "root_cause_solo") + (is (= "ExceptionInfo: boom\n{:k 1}" (with-out-str (clojure.stacktrace/print-throwable (ex-info "boom" {:k 1})))) + "print_throwable") + ;; AD-029 pin: a never-thrown ex-info keeps the frame-less marker. + (is (= (str "ExceptionInfo: outer\n{}\n at [no stack trace available]\n" + "Caused by: ExceptionInfo: inner\n{}\n at [no stack trace available]") + (str/trim-newline (with-out-str (clojure.stacktrace/print-cause-trace (ex-info "outer" {} (ex-info "inner" {})))))) + "print_cause_trace") + (let [e (try (p15-boom) (catch Throwable e e)) + out (with-out-str (clojure.stacktrace/print-stack-trace e))] + (is (str/includes? out "suites.test-reporters-test/p15-boom") "caught_prints_user_frame") + (is (not (str/includes? out "[no stack trace available]")) "caught_no_marker") + (is (some #{"p15-boom"} (map :fn (stack-trace e))) "stack_trace_fn_key") + (is (some #{"suites.test-reporters-test"} (map :ns (stack-trace e))) "stack_trace_ns_key")) + (is (= 0 (count (stack-trace (ex-info "x" {})))) "unthrown_no_frames"))) diff --git a/test/clj/suites/vars_namespaces_test.clj b/test/clj/suites/vars_namespaces_test.clj new file mode 100644 index 000000000..1ea8af8bf --- /dev/null +++ b/test/clj/suites/vars_namespaces_test.clj @@ -0,0 +1,207 @@ +;; Vars, dynamic binding and namespaces as runtime values (D-230/D-232/D-235/ +;; D-237/D-239/D-554, ADR-0083/0085/0096/0097/0171, AD-010/AD-015/AD-053). +;; +;; Migrated from the phase15 e2e scripts alias, var_get_set, dynamic_var, +;; with_local_vars, set_bang, ns_binding_roundtrip, ns_runtime_fns, namespace, +;; ns_docstring, campaign_keywords, use_refer_all, use_libspec_filter, +;; require_runtime_fn, ns_import, import_macro, aliased_macro. Each script ran +;; one `cljw -e` program per case; here a case that needs top-level forms +;; (ns / def / in-ns) evaluates them with `ev`, a load-string inside a FRESH +;; scratch namespace, so a case never leaks its ns switch or defs into the +;; suite (memory 20260911151735: in-ns under a test runner must go through a +;; *ns* binding). Every namespace and var a case creates is prefixed p15-. +(ns suites.vars-namespaces-test + (:require [clojure.test :refer [deftest is testing]])) + +(def ^:private scratch-seq (atom 0)) + +(defn- ev + "load-string `src` in a fresh namespace that refers clojure.core, the way + `cljw -e` evaluated it in `user`. *ns* is thread-bound around it, so the + program's own ns / in-ns switches pop when it returns." + [src] + (let [n (symbol (str "p15.scratch" (swap! scratch-seq inc)))] + (binding [*ns* (create-ns n)] + (refer 'clojure.core) + (load-string src)))) + +(deftest p15-alias + (testing "phase15_alias" + (is (= "HI" (ev "(alias (quote str) (quote clojure.string)) (str/upper-case \"hi\")")) "alias-use") + (is (true? (ev "(do (alias (quote s) (quote clojure.set)) (contains? (ns-aliases *ns*) (quote s)))")) "ns-aliases") + (is (= 'clojure.walk (ev "(do (alias (quote w) (quote clojure.walk)) (ns-name ((ns-aliases *ns*) (quote w))))")) "alias-target") + (is (thrown-with-msg? Throwable #"Could not locate" (ev "(alias (quote x) (quote no.such.ns))")) "alias-unknown"))) + +(def ^:dynamic *p15-x* 0) + +(deftest p15-var-get-set + (testing "phase15_var_get_set" + (is (= 5 (ev "(def y 5) (var-get (var y))")) "var-get-root") + (is (= 7 (ev "(def y 7) (var-get #'y)")) "var-get-hash") + (is (= 9 (binding [*p15-x* 1] (var-set (var *p15-x*) 9) *p15-x*)) "var-set-bind") + (is (= 9 (binding [*p15-x* 1] (var-set (var *p15-x*) 9))) "var-set-ret") + (is (= 0 (do (binding [*p15-x* 1] (var-set (var *p15-x*) 9)) *p15-x*)) "var-set-frame") + (is (thrown-with-msg? Throwable #"not thread-bound" (var-set (var *p15-x*) 9)) "var-set-unbd"))) + +(deftest p15-dynamic-var + (testing "phase15_dynamic_var" + (is (= 2 (ev "(do (def ^:dynamic *x* 1) (binding [*x* 2] *x*))")) "bind") + (is (= [2 1] (ev "(do (def ^:dynamic *x* 1) [(binding [*x* 2] *x*) *x*])")) "restore") + (is (= 30 (ev "(do (def ^:dynamic *y* 10) (binding [*y* 20] (binding [*y* 30] *y*)))")) "nested") + (is (= [:inner :root] (ev "(do (def ^:dynamic *a* :root) (defn pa [] *a*) [(binding [*a* :inner] (pa)) (pa)])")) "fn-sees") + (is (thrown-with-msg? Throwable #"non-dynamic" (ev "(do (def plain 1) (binding [plain 2] plain))")) "guard")) + (testing "core version / flag vars" + (is (= 12 (:minor *clojure-version*)) "cv_minor") + (is (= "1.12.0" (clojure-version)) "cv_string") + (is (false? *unchecked-math*) "unchecked") + (is (true? (binding [*unchecked-math* true] *unchecked-math*)) "um_bind"))) + +(deftest p15-with-local-vars + (testing "phase15_with_local_vars" + (is (= 12 (with-local-vars [x 1 y 2] (var-set x 10) (+ (var-get x) (var-get y)))) "basic") + (is (= 20 (with-local-vars [a 5] (var-set a (* @a 4)) @a)) "deref") + (is (= 99 (with-local-vars [x 1] (var-set x 99))) "set-ret") + (is (= 3 (with-local-vars [x 1] (with-local-vars [y 2] (+ (var-get x) (var-get y))))) "nested") + ;; AD-015 pin: an escaped local var deref'd after its extent is nil in cljw. + (is (nil? (var-get (with-local-vars [x 5] x))) "escape-nil") + (is (= :done (do (dotimes [_ 50] (with-local-vars [x 1] (var-set x 2))) :done)) "loop-clean"))) + +(deftest p15-set-bang + (testing "phase15_set_bang" + (is (= 99 (ev "(def ^:dynamic *x* 1) (binding [*x* 10] (set! *x* 99) *x*)")) "bind-set") + (is (= 5 (ev "(def ^:dynamic *x* 1) (binding [*x* 0] (set! *x* 5))")) "set-ret") + (is (= 1 (ev "(def ^:dynamic *x* 1) (do (binding [*x* 0] (set! *x* 5)) *x*)")) "frame-only") + (is (thrown-with-msg? Throwable #"Can't set! var that is not thread-bound: p15\.scratch\d+/\*x\*" + (ev "(def ^:dynamic *x* 1) (set! *x* 8)")) "unbound-errs") + (let [old *warn-on-reflection*] + (try (is (true? (ev "(set! *warn-on-reflection* true)")) "warn-refl") + (finally (set! *warn-on-reflection* old)))) + (is (= 9 (ev "(do (def ^:dynamic zz 0) (binding [zz 1] (set! zz 9)))")) "same-unit") + (is (thrown-with-msg? Throwable #"Can't set! var that is not thread-bound: p15\.scratch\d+/y" + (ev "(def y 1) (set! y 2)")) "not-dyn") + (is (thrown-with-msg? Throwable #"set! expects 2 args, got 1" + (ev "(set! *warn-on-reflection*)")) "arity"))) + +(deftest p15-ns-binding-roundtrip + (testing "phase15_ns_binding_roundtrip" + (let [[inside after before] + (ev "(def b (ns-name *ns*)) (def inside (binding [*ns* *ns*] (in-ns (quote p15.tmp.x)) (ns-name *ns*))) [inside (ns-name *ns*) b]")] + (is (= 'p15.tmp.x inside) "inside-and-restore") + (is (= before after) "inside-and-restore: restored")) + (is (= 42 (ev "(binding [*ns* *ns*] (in-ns (quote p15.tmp.y))) (inc 41)")) "core-resolves-after") + (is (re-find #"^p15\.scratch" (str (ev "(binding [*ns* *ns*] (in-ns (quote p15.a)) (binding [*ns* *ns*] (in-ns (quote p15.b)))) (ns-name *ns*)"))) + "nested-roundtrip"))) + +(deftest p15-ns-runtime-fns + (testing "phase15_ns_runtime_fns" + (is (= [42 true] + (ev "(def outer (ns-name *ns*)) (def n (binding [*ns* *ns*] (in-ns (gensym)) (apply clojure.core/use [(quote clojure.core)]) (inc 41))) [n (= outer (ns-name *ns*))]")) + "temp-ns-shape") + (is (= 'p15.some.tmp (binding [*ns* *ns*] (ns-name (in-ns 'p15.some.tmp)))) "in-ns-returns-ns") + (is (= 'p15.t2 (binding [*ns* *ns*] (ns-name (apply in-ns ['p15.t2])))) "apply-in-ns"))) + +(deftest p15-namespace-values + ;; AD-010 pin: a Namespace prints as #object[Namespace "name"]. + (testing "phase15_namespace" + (let [u (the-ns 'user)] + (is (= 'user (ns-name u)) "ns-name") + (is (= "user" (str u)) "str-ns") + (is (= "#object[Namespace \"user\"]" (pr-str u)) "pr-ns") + (is (= u (the-ns 'user)) "identity")) + (is (nil? (find-ns 'p15-nope)) "find-miss") + (is (= 'p15.my.app (ns-name (create-ns 'p15.my.app))) "create") + (is (= "#object[Namespace \"p15.foo.bar\"]" + (pr-str (ev "(do (in-ns (quote p15.foo.bar)) clojure.core/*ns*)"))) "in-ns") + (is (= #'clojure.core/map (ns-resolve 'clojure.core 'map)) "resolve") + (is (true? (ev "(do (def zzz 7) (contains? (ns-interns *ns*) (quote zzz)))")) "interns") + (is (contains? (ns-publics (the-ns 'clojure.core)) 'reduce) "core-pub-builtin") + (is (contains? (ns-map (the-ns 'clojure.core)) 'reduce) "core-map-builtin")) + (testing "AD-053 kernel-helper namespace" + (is (some? (find-ns 'cljw.internal)) "ad053-internal-ns") + (is (nil? (find-ns 'rt)) "ad053-rt-gone") + (is (not (contains? (ns-publics (the-ns 'clojure.core)) '__class)) "ad053-no-leak")) + (testing "intern" + (is (= 42 (deref (intern (create-ns 'p15.foo.iv) 'x 42))) "intern-3ary") + (is (= "#'p15.foo.iv5/q" (pr-str (intern (create-ns 'p15.foo.iv5) 'q 1))) "intern-ret") + (is (= 5 (do (intern (create-ns 'p15.foo.iv6) 'k 5) (deref (intern (the-ns 'p15.foo.iv6) 'k)))) "intern-2ary-keeps") + (is (= "#'p15.foo.iv4/w" (pr-str (do (intern (create-ns 'p15.foo.iv4) 'w) (find-var 'p15.foo.iv4/w)))) "intern-2ary-var"))) + +(deftest p15-ns-docstring + (testing "phase15_ns_docstring" + (is (= ["p15.foo.bar2" 42] (ev "(ns p15.foo.bar2 \"my docstring\" (:refer-clojure)) [(name (ns-name *ns*)) (inc 41)]")) "ns-docstring") + (is (= ["p15.baz" 2] (ev "(ns p15.baz \"doc\" {:author \"x\"} (:refer-clojure)) [(name (ns-name *ns*)) (inc 1)]")) "ns-docstring-attrmap") + (is (= "p15.plain" (ev "(ns p15.plain (:refer-clojure)) (name (ns-name *ns*))")) "ns-no-docstring") + (ev "(ns p15.dm.doc \"the doc\")") + (is (= "the doc" (:doc (meta (find-ns 'p15.dm.doc)))) "ns-doc-meta") + (let [n (create-ns 'p15.dm.alter)] + (alter-meta! n assoc :k 1) + (is (= [1 nil] [(:k (meta n)) (do (reset-meta! n nil) (meta n))]) "ns-alter-meta")) + (ev "(ns p15.dm.attr \"d\" {:author \"a\" :added \"1.0\"})") + (is (= ["d" "a" "1.0"] ((juxt :doc :author :added) (meta (find-ns 'p15.dm.attr)))) "ns-attr-meta") + (ev "(ns ^{:k 1} p15.dm.sym)") + (is (= 1 (:k (meta (find-ns 'p15.dm.sym)))) "ns-name-sym-meta"))) + +(deftest p15-campaign-keywords + (testing "phase15_campaign_keywords: Namespace interop" + (is (symbol? (.name *ns*)) "ns-dot-name") + (is (= "user" (str (.getName (the-ns 'user)))) "ns-dot-getname") + (is (= "user" (.toString (the-ns 'user))) "ns-dot-str"))) + +(deftest p15-use-refer-all + (testing "phase15_use_refer_all" + (is (= "HI" (ev "(require (quote [clojure.string :refer :all])) (upper-case \"hi\")")) "refer-all") + (is (= "HI" (ev "(ns p15.use1 (:use clojure.string)) (upper-case \"hi\")")) "use-string") + (is (= [2 0] (binding [clojure.test/*test-out* (java.io.StringWriter.) + clojure.test/*report-counters* nil + clojure.test/*testing-vars* (list) + clojure.test/*testing-contexts* (list)] + (ev "(ns p15.use2 (:use clojure.test)) (deftest t (is (= 1 1)) (is (= 2 2))) (let [s (run-tests)] [(:pass s) (:fail s)])"))) + "use-test") + (load-file "test/e2e/fixtures/cljwlib/demo/math.clj") + (is (= 36 (ev "(ns p15.use3 (:use demo.math)) (square 6)")) "use-disk") + (is (= ["X" #{1 2}] (ev "(ns p15.use4 (:use clojure.string clojure.set)) [(upper-case \"x\") (union #{1} #{2})]")) "use-multi"))) + +(deftest p15-use-libspec-filter + (testing "phase15_use_libspec_filter" + (is (= #{1 2} (ev "(ns p15.lf1 (:use [clojure.set :only (union)])) (union #{1} #{2})")) "only-refers") + (is (nil? (ev "(ns p15.lf2 (:use [clojure.set :only (union)])) (resolve (quote intersection))")) "only-excludes") + (is (= #{1 3} (ev "(ns p15.lf3 (:use [clojure.set :only [union]])) (union #{1} #{3})")) "only-vector") + (is (= #{1 2} (ev "(ns p15.lf4 (:use [clojure.set :exclude (intersection difference)])) (union #{1} #{2})")) "excl-refers") + (is (nil? (ev "(ns p15.lf5 (:use [clojure.set :exclude (intersection)])) (resolve (quote intersection))")) "excl-blocks") + (is (= "HI" (ev "(require (quote [clojure.string :only (upper-case)])) (upper-case \"hi\")")) "req-only"))) + +(deftest p15-require-runtime-fn + (testing "phase15_require_runtime_fn" + (is (= "HI" (ev "(let [n (quote clojure.string)] (require n)) (clojure.string/upper-case \"hi\")")) "require-computed") + (is (= #{1 2} (ev "(apply require [(quote clojure.set)]) (clojure.set/union #{1} #{2})")) "apply-require") + (is (= "ab" (ev "(let [s (quote [clojure.string :as cs])] (require s)) (cs/lower-case \"AB\")")) "require-as-alias") + (is (= "OK" (ev "(require '[clojure.string :as s]) (s/upper-case \"ok\")")) "literal-still") + (is (thrown-with-msg? Throwable #"Could not locate" (require 'p15.no.such.lib)) "missing-lib (from phase15_require_fs)"))) + +(deftest p15-ns-import + (testing "phase15_ns_import" + (is (true? (ev "(ns p15.im1 (:import java.util.UUID)) (uuid? (UUID/randomUUID))")) "sym-form") + (is (true? (ev "(ns p15.im2 (:import [java.util UUID])) (uuid? (UUID/randomUUID))")) "prefix-form") + (is (true? (ev "(ns p15.im3 (:import java.util.UUID)) (uuid? (java.util.UUID/randomUUID))")) "fqcn-still") + (is (= 3 (ev "(ns p15.im4) (+ 1 2)")) "no-import") + (is (= :loaded (ev "(ns p15.im5 (:import java.util.ServiceLoader)) :loaded")) "unsupp-load") + (is (thrown-with-msg? Throwable #"Unable to resolve symbol: 'ServiceLoader'" + (ev "(ns p15.im6 (:import java.util.ServiceLoader)) (ServiceLoader.)")) "unsupp-use"))) + +(deftest p15-import-macro + (testing "phase15_import_macro" + (is (= 42 (ev "(import (quote (java.lang Integer))) (Integer/parseInt \"42\")")) "prefix-list") + (is (= 7 (ev "(import (quote java.lang.Long)) (Long/parseLong \"7\")")) "bare-symbol") + (is (= [3 4] (ev "(import (quote (java.lang Integer Long))) [(Integer/parseInt \"3\") (Long/parseLong \"4\")]")) "multi-class") + (is (= '(do (import* "java.lang.Boolean") (import* "java.lang.Integer")) + (macroexpand-1 '(import (quote (java.lang Boolean Integer))))) "expands-to-import-star") + (is (= [5 6] (ev "(import (quote [java.lang Integer Long])) [(Integer/parseInt \"5\") (Long/parseLong \"6\")]")) "prefix-vector"))) + +(deftest p15-aliased-macro + ;; The script built a disk lib off deps.edn :paths; the analyzer path under + ;; test (alias-translate a qualified macro head) is the same for an ns that + ;; already exists in the image. + (testing "phase15_aliased_macro" + (ev "(ns p15.mlib.core) (defmacro pick [sym] (list (quote quote) sym))") + (is (= 'wildname (ev "(require '[p15.mlib.core :as m]) (m/pick wildname)")) "aliased_macro_expands") + (is (= 'wildname (ev "(require 'p15.mlib.core) (p15.mlib.core/pick wildname)")) "qualified_macro_expands"))) diff --git a/test/clj/suites/wasm_i64_range_test.clj b/test/clj/suites/wasm_i64_range_test.clj new file mode 100644 index 000000000..c716f5489 --- /dev/null +++ b/test/clj/suites/wasm_i64_range_test.clj @@ -0,0 +1,85 @@ +;; Full-range integers across the wasm boundary (CLJW-WASM-I64-FULL-RANGE). +;; A cljw integer is an immediate only inside the i48 NaN-box window (about +;; +/-2^47); past it the value is a heap Long. `wasm/call` (and the dot form, +;; which shares its path) refused a heap Long as an i64 arg ("argument is not a +;; number") and lifted an i64 result past the window to a lossy Double. Every +;; case is a round trip through an identity export, so a pass proves lower and +;; lift both. Needs the -Dwasm binary (the gate configuration). +(ns suites.wasm-i64-range-test + (:require [clojure.test :refer [deftest is testing]])) + +(def ^:private fixture "test/e2e/fixtures/wasm/i64_range.wasm") +(def ^:private echo "test/e2e/fixtures/wasm/echo.component.wasm") + +(def ^:private i64-edges + [0 1 -1 + 70368744177664 -70368744177664 ; +/-2^46, inside the window + 140737488355328 -140737488355328 ; +/-2^47, the first heap Long is +2^47 + 9007199254740993 -9007199254740993 ; +/-(2^53+1), not representable as f64 + Long/MIN_VALUE Long/MAX_VALUE]) + +(defn- exact-long? [want got] + (and (instance? Long got) (= want got))) + +(defn- msg [f] + (try (f) :not-raised (catch Throwable e (.getMessage e)))) + +(def ^:private out-of-range + "wasm/call: an integer argument is out of range for the parameter type") + +(deftest i64-args-and-results-round-trip + (let [m (wasm/load fixture)] + (doseq [v i64-edges] + (testing (str "wasm/call " v) + (is (exact-long? v (wasm/call m "id64" v)))) + (testing (str "dot form " v) + (is (exact-long? v (.id64 m v))))))) + +(deftest u64-top-bit-reads-as-signed-long + ;; wasm i64 is signless: a guest u64 with the top bit set (what + ;; `@bitReverse(@as(u64, 1))` returns) is the negative Long with the same + ;; bits, JVM semantics, as cljw.ffi's :long. Was -9.223372036854776E18. + (is (exact-long? Long/MIN_VALUE (wasm/call (wasm/load fixture) "top_bit")))) + +(deftest several-i64-results + (let [[a b] (wasm/call (wasm/load fixture) "swap64" Long/MIN_VALUE 9007199254740993)] + (is (exact-long? 9007199254740993 a)) + (is (exact-long? Long/MIN_VALUE b)))) + +(deftest i32-and-f64-params + (let [m (wasm/load fixture)] + (doseq [v [-2147483648 2147483647 0 -1]] + (is (exact-long? v (wasm/call m "id32" v)))) + (testing "a heap Long widens to an f64 param, as `double` does" + (is (= 1.40737488355328E14 (wasm/call m "idf64" 140737488355328)))))) + +(deftest linear-memory + (let [m (wasm/load fixture)] + (wasm/mem-write! m :i64 0 i64-edges) + (is (every? true? (map exact-long? i64-edges (wasm/mem-read m :i64 0 (count i64-edges))))) + (wasm/mem-write! m :u32 256 [4294967295 0]) + (is (= [4294967295 0] (wasm/mem-read m :u32 256 2))) + (wasm/mem-write! m :i32 264 [-2147483648 2147483647]) + (is (= [-2147483648 2147483647] (wasm/mem-read m :i32 264 2))) + (testing "a heap Long widens into an :f64 element" + (wasm/mem-write! m :f64 512 [140737488355328]) + (is (= [1.40737488355328E14] (wasm/mem-read m :f64 512 1)))))) + +(deftest refusals-are-catchable + ;; An integral 2^63 float passed the old `> maxInt(i64)` test (maxInt as an + ;; f64 rounds to 2^63) and reached `@intFromFloat`: a ReleaseSafe host panic. + (let [m (wasm/load fixture)] + (is (= out-of-range (msg #(wasm/call m "id64" 9223372036854775808N)))) + (is (= out-of-range (msg #(wasm/call m "id64" -9223372036854775809N)))) + (is (= out-of-range (msg #(wasm/call m "id32" 140737488355328)))) + (is (= out-of-range (msg #(wasm/call m "id64" 9.223372036854775807E18)))) + (is (re-find #"^wasm/mem-write!: element 0 is outside this element type's range" + (str (msg #(wasm/mem-write! m :i64 0 [9.223372036854775807E18]))))))) + +(deftest wit-u64 + ;; The component path shares the integer coercion. A u64 above + ;; Long/MAX_VALUE is unsigned by its WIT type, so it lifts as an exact BigInt + ;; (component.zig `bigFromU64`), not a wrapped Long. + (doseq [v [140737488355328 9007199254740993 Long/MAX_VALUE]] + (is (exact-long? v (wasm/component-invoke echo "echo-u64" v)))) + (is (= 18446744073709551615N (wasm/component-invoke echo "echo-u64" 18446744073709551615N)))) diff --git a/test/clj/suites/wasm_thread_hop_test.clj b/test/clj/suites/wasm_thread_hop_test.clj new file mode 100644 index 000000000..5a3b42b1a --- /dev/null +++ b/test/clj/suites/wasm_thread_hop_test.clj @@ -0,0 +1,28 @@ +;; CLJW-WASM-INTERP-THREAD-TRAP: a handle warmed on the runtime thread must +;; still run when the next call comes from another thread (future / agent). +;; zwasm's interp cached the FIRST caller thread's native-stack low limit on +;; the instance's Runtime, so a later call from a thread whose stack sits below +;; that limit trapped CallStackExhausted. Needs the -Dwasm binary (the gate +;; configuration). +(ns suites.wasm-thread-hop-test + (:require [clojure.test :refer [deftest is testing]])) + +(def ^:private add-wasm "bench/wasm/ffi/add.wasm") + +(defn- call [h] + (try (wasm/call h "add" 2 3) + (catch Throwable e (str "TRAPPED " (ex-message e))))) + +(deftest warmed-handle-runs-from-a-future + (doseq [engine [:interp :jit :auto] + warm [1 5000]] + (testing (str engine " warmed " warm "x on the runtime thread") + (let [h (wasm/load add-wasm {:engine engine :fuel 0})] + (dotimes [_ warm] (call h)) + (is (= 5 @(future (call h))) "the call from a future") + (is (= 5 (call h)) "and back on the runtime thread"))))) + +(deftest worker-first-handle-runs-on-the-runtime-thread + (let [h (wasm/load add-wasm {:engine :interp :fuel 0})] + (is (= 5 @(future (call h)))) + (is (= 5 (call h))))) diff --git a/test/diff/clj_corpus/bignum_print_width.txt b/test/diff/clj_corpus/bignum_print_width.txt new file mode 100644 index 000000000..54ee5c460 --- /dev/null +++ b/test/diff/clj_corpus/bignum_print_width.txt @@ -0,0 +1,36 @@ +(count (str (reduce *' (range 1N 1000N)))) +;;=> 2565 +(subs (str (reduce *' (range 1N 1000N))) 0 30) +;;=> "402387260077093773543702433923" +(count (str (reduce *' (range 1N 3501N)))) +;;=> 10887 +(subs (str (reduce *' (range 1N 3501N))) 0 30) +;;=> "239112819947764952509538749369" +(reduce + (map #(- (int %) 48) (str (reduce *' (range 1N 3501N))))) +;;=> 45387 +(count (re-find #"0+$" (str (reduce *' (range 1N 3501N))))) +;;=> 874 +(count (pr-str (reduce *' (range 1N 3501N)))) +;;=> 10888 +(last (pr-str (reduce *' (range 1N 3501N)))) +;;=> \N +(= (str (reduce *' (range 1N 3501N))) (.toString (reduce *' (range 1N 3501N)))) +;;=> true +(= (str (reduce *' (range 1N 3501N))) (format "%s" (reduce *' (range 1N 3501N)))) +;;=> true +(format "%s %s" 1N 1.5M) +;;=> "1 1.5" +(count (pr-str (/ 1 (reduce *' (range 1N 3501N))))) +;;=> 10889 +(count (str (bigdec (str (reduce *' (range 1N 3501N)))))) +;;=> 10887 +(subs (str (bigdec (str (reduce *' (range 1N 3501N)) "E-5000"))) 0 12) +;;=> "239112819947" +(count (.toPlainString (bigdec (str (reduce *' (range 1N 3501N)) "E-5000")))) +;;=> 10888 +(count (str (rationalize (bigdec "1E-1200")))) +;;=> 1203 +(count (str (rationalize (bigdec "7E+1500")))) +;;=> 1501 +(= (reduce *' (range 1N 3501N)) (rationalize (bigdec (str (reduce *' (range 1N 3501N)))))) +;;=> true diff --git a/test/diff/clj_corpus/edn_strictness.txt b/test/diff/clj_corpus/edn_strictness.txt new file mode 100644 index 000000000..4feb971b0 --- /dev/null +++ b/test/diff/clj_corpus/edn_strictness.txt @@ -0,0 +1,118 @@ +(clojure.edn/read-string "`a") +;;=> RuntimeException +(clojure.edn/read-string "~a") +;;=> RuntimeException +(clojure.edn/read-string "~@a") +;;=> RuntimeException +(clojure.edn/read-string "@a") +;;=> RuntimeException +(clojure.edn/read-string "#'a") +;;=> RuntimeException +(clojure.edn/read-string "#(inc %)") +;;=> RuntimeException +(clojure.edn/read-string "#\"re\"") +;;=> RuntimeException +(clojure.edn/read-string "#=(+ 1 2)") +;;=> RuntimeException +(clojure.edn/read-string "::kw") +;;=> RuntimeException +(clojure.edn/read-string "::a/b") +;;=> RuntimeException +(clojure.edn/read-string "#::{:a 1}") +;;=> RuntimeException +(clojure.edn/read-string "#:a{:b 1}") +;;=> #:a{:b 1} +(clojure.edn/read-string "#:a{:_/b 1}") +;;=> {:b 1} +(clojure.edn/read-string "#?(:clj 1)") +;;=> RuntimeException +(clojure.edn/read-string "#?@(:clj [1])") +;;=> RuntimeException +(clojure.edn/read-string "#foo 1") +;;=> RuntimeException +(clojure.edn/read-string "#foo/bar 1") +;;=> RuntimeException +(clojure.edn/read-string "'a") +;;=> 'a +(clojure.edn/read-string "'(1)") +;;=> ' +(clojure.edn/read-string "[a'b]") +;;=> [a'b] +(clojure.edn/read-string "#user.R{:a 1}") +;;=> RuntimeException +(clojure.edn/read-string "#!x\n3") +;;=> RuntimeException +(clojure.edn/read-string "#_ 1 2") +;;=> 2 +(clojure.edn/read-string "[#_ ^:m x 1]") +;;=> [1] +(clojure.edn/read-string "^:m [1]") +;;=> [1] +(meta (clojure.edn/read-string "^:m [1]")) +;;=> {:m true} +(clojure.edn/read-string "#_ #foo 1") +;;=> RuntimeException +(clojure.edn/read-string "#_ `a 3") +;;=> RuntimeException +(clojure.edn/read-string "#_ #(x) 3") +;;=> RuntimeException +(clojure.edn/read-string "#_ ::k 3") +;;=> RuntimeException +(clojure.edn/read-string "[1 #_ #inst \"bad\" 2]") +;;=> RuntimeException +(clojure.edn/read-string "[#_ {:a 1 :a 2} 1]") +;;=> IllegalArgumentException +(clojure.edn/read-string "a@b") +;;=> RuntimeException +(clojure.edn/read-string "a~b") +;;=> RuntimeException +(clojure.edn/read-string "a`b") +;;=> RuntimeException +(clojure.edn/read-string "[a@b]") +;;=> RuntimeException +(clojure.edn/read-string "[:k@b]") +;;=> RuntimeException +(clojure.edn/read-string "x#") +;;=> x# +(clojure.edn/read-string "%1") +;;=> %1 +(clojure.edn/read-string "#_#_ 1 2 3") +;;=> 3 +(clojure.edn/read-string "#{#_ 1}") +;;=> #{} +(clojure.edn/read-string nil) +;;=> nil +(clojure.edn/read-string {} nil) +;;=> nil +(clojure.edn/read-string {:readers {'foo (fn [x] [:foo x])}} "#foo 1") +;;=> [:foo 1] +(clojure.edn/read-string {:default (fn [t v] [t v])} "#foo 1") +;;=> [foo 1] +(clojure.edn/read-string {:default (fn [t v] [t v])} "#user.R{:a 1}") +;;=> [user.R {:a 1}] +(clojure.edn/read-string {:readers {'inst (fn [x] [:i x])}} "#inst \"x\"") +;;=> [:i "x"] +(clojure.edn/read-string {:eof :E} "") +;;=> :E +(clojure.edn/read-string {} ";x\n") +;;=> RuntimeException +(clojure.edn/read-string {:readers {'my/boom (fn [_] (throw (ex-info "boom" {})))}} "[#_ #my/boom 0 42]") +;;=> ExceptionInfo +(clojure.edn/read-string {:default (fn [t v] (throw (ex-info "dflt" {})))} "[#_ #foo 1 2]") +;;=> ExceptionInfo +(clojure.edn/read-string {:readers {'my/t vector}} "[#_ #my/t 1 2]") +;;=> [2] +(clojure.edn/read-string "[#_ #foo 1 {:a 1 :a 2}]") +;;=> RuntimeException +(read-string "#_ #foo 1") +;;=> RuntimeException +(read-string "[#_ {:a 1 :a 2} 1]") +;;=> IllegalArgumentException +(read-string "'a") +;;=> (quote a) +(read-string "::kw") +;;=> :user/kw +(clojure.edn/read (java.io.PushbackReader. (java.io.StringReader. "@a"))) +;;=> RuntimeException +(clojure.edn/read (java.io.PushbackReader. (java.io.StringReader. "'a 1"))) +;;=> 'a diff --git a/test/e2e/fixtures/ffi/ffi_fixture.c b/test/e2e/fixtures/ffi/ffi_fixture.c index f4f2208e0..13ef812e7 100644 --- a/test/e2e/fixtures/ffi/ffi_fixture.c +++ b/test/e2e/fixtures/ffi/ffi_fixture.c @@ -52,3 +52,30 @@ double eight_doubles(double a, double b, double c, double d, double e, double f, } int is_null(const char *s) { return s == NULL; } + +// Primitive arrays passed as :pointer (ADR-0204): contiguous native storage. +double dot(const double *a, const double *b, long n) { + double s = 0; + for (long i = 0; i < n; i++) s += a[i] * b[i]; + return s; +} + +long sum_i32(const int *p, long n) { + long s = 0; + for (long i = 0; i < n; i++) s += p[i]; + return s; +} + +double sum_f32(const float *p, long n) { + double s = 0; + for (long i = 0; i < n; i++) s += p[i]; + return s; +} + +void fill_i64(long *p, long n, long v) { + for (long i = 0; i < n; i++) p[i] = v + i; +} + +void bump_bytes(unsigned char *p, long n) { + for (long i = 0; i < n; i++) p[i]++; +} diff --git a/test/e2e/fixtures/wasm/i64_range.wasm b/test/e2e/fixtures/wasm/i64_range.wasm new file mode 100644 index 000000000..1d31e0ed0 Binary files /dev/null and b/test/e2e/fixtures/wasm/i64_range.wasm differ diff --git a/test/e2e/fixtures/wasm/i64_range.wat b/test/e2e/fixtures/wasm/i64_range.wat new file mode 100644 index 000000000..dd5482243 --- /dev/null +++ b/test/e2e/fixtures/wasm/i64_range.wat @@ -0,0 +1,19 @@ +;; Full-range i64 marshalling fixture (CLJW-WASM-I64-FULL-RANGE). Identity +;; exports, so a round trip proves both directions: the arg was lowered to an +;; exact i64 and the result was lifted back to an exact Long. The i48 NaN-box +;; window (about +/-2^47) is where a cljw integer stops being an immediate, so +;; the interesting values all sit outside it. +;; top_bit : a guest computing a u64 with the top bit set (what +;; `@bitReverse(@as(u64, 1))` returns). wasm i64 is signless, so the +;; host sees Long/MIN_VALUE, the JVM reading of the same 64 bits. +;; swap64 : two i64 results, the vector-of-results path. +;; Regenerate: wat2wasm i64_range.wat -o i64_range.wasm +(module + (memory (export "memory") 1) + (func (export "id64") (param i64) (result i64) local.get 0) + (func (export "id32") (param i32) (result i32) local.get 0) + (func (export "idf64") (param f64) (result f64) local.get 0) + (func (export "top_bit") (result i64) i64.const 0x8000000000000000) + (func (export "swap64") (param i64 i64) (result i64 i64) + local.get 1 + local.get 0)) diff --git a/test/e2e/fixtures/wasm_memory_probe.clj b/test/e2e/fixtures/wasm_memory_probe.clj index 3f2aafa20..59a5b1d57 100644 --- a/test/e2e/fixtures/wasm_memory_probe.clj +++ b/test/e2e/fixtures/wasm_memory_probe.clj @@ -37,9 +37,35 @@ ;; Every failure is a CATCHABLE cljw exception naming the fn the user wrote, ;; never an internal_error and never a host panic on caller data. +;; Primitive arrays cross in bulk (ADR-0204): a double-array written as :f64 +;; is the guest's buffer byte for byte, and mem-read-array answers a typed +;; array. Every array kind round-trips through its dtype. +(println "arr-wrote" (wasm/mem-write! m :f64 0 (double-array [1 2 3 4]))) +(println "arr-newsum" (wasm/call m "scale_f64" 0 4 2.0)) +(let [d (wasm/mem-read-array m :f64 0 4)] + (println "arr-f64" (vec d) (= :double (cljw.internal/__array-kind d)))) +(doseq [[dt arr] [[:i8 (byte-array [-1 2 127])] + [:i16 (short-array [-300 2 32767])] + [:i32 (int-array [-70000 2 2147483647])] + [:i64 (long-array [9007199254740993 -1 2])] + [:f32 (float-array [1.5 -2.25 3])] + [:f64 (double-array [1.5 -2.25 3])]]] + (wasm/mem-write! m dt 1000 arr) + (println "arr-rt" dt (= (vec arr) (vec (wasm/mem-read-array m dt 1000 3))))) +(wasm/mem-write! m :u16 2000 [65535]) +(wasm/mem-write! m :u32 2008 [4294967295]) +(println "arr-unsigned" (vec (wasm/mem-read-array m :u16 2000 1)) (vec (wasm/mem-read-array m :u32 2008 1))) +;; A kind that is not the dtype's layout converts element by element. +(wasm/mem-write! m :f64 3000 (int-array [5 6])) +(println "arr-convert" (wasm/mem-read m :f64 3000 2)) + (defn probe [label f] (println label (try (do (f) "NOT-CAUGHT") (catch Exception e (.getMessage e))))) +(probe "arr-object" #(wasm/mem-write! m :f64 0 (object-array 1))) +(probe "arr-range" #(wasm/mem-write! m :i8 0 (int-array [1 300]))) +(probe "arr-oob" #(wasm/mem-read-array m :f64 65530 1)) + (probe "no-memory" #(wasm/mem-size (wasm/load "docs/examples/wasm/add.wasm"))) (probe "bad-handle" #(wasm/mem-size 42)) (probe "bad-dtype" #(wasm/mem-read m :f16 0 1)) diff --git a/test/e2e/phase14_arity_parity.sh b/test/e2e/phase14_arity_parity.sh index ad5e5f567..70f4a4e93 100644 --- a/test/e2e/phase14_arity_parity.sh +++ b/test/e2e/phase14_arity_parity.sh @@ -83,13 +83,11 @@ check "(char-array n ch)" '(int (aget (char-array 3 (char 120)) 1))' '120' check "AD-036 (byte-array n num)" '(vec (byte-array 3 65))' '[65 65 65]' check "AD-036 (char-array n int)" '(int (aget (char-array 3 65) 0))' '65' -# AD-051 pin: `bytes?` is true for ANY cljw array — arrays are type-erased -# (AD-019), so a byte-array is runtime-indistinguishable from an int/object -# array. The common positive guard matches clj; non-byte arrays diverge (clj -# false, cljw true). -check "AD-051 (bytes? byte-array)" '(bytes? (byte-array 3))' 'true' -check "AD-051 (bytes? int-array)" '(bytes? (int-array 3))' 'true' -check "AD-051 (bytes? object-array)" '(bytes? (object-array 0))' 'true' +# `bytes?` is true only for a byte array (clj parity; arrays carry their +# element kind since ADR-0204, which retired the over-broad AD-051). +check "(bytes? byte-array)" '(bytes? (byte-array 3))' 'true' +check "(bytes? int-array)" '(bytes? (int-array 3))' 'false' +check "(bytes? object-array)" '(bytes? (object-array 0))' 'false' # --- multi-dim aget/aset (D-446 close-out): the variadic forms walk nested @@ -98,9 +96,9 @@ check "AD-051 (bytes? object-array)" '(bytes? (object-array 0))' 'true' check "(aget a i j)" '(let [a (make-array Long 2 3)] (aset a 0 1 42) (aget a 0 1))' '42' check "(aset a i j k v)" '(let [b (make-array Long 2 2 2)] (aset b 1 0 1 9) (aget b 1 0 1))' '9' check "(aget oob catchable)" '(let [a (make-array Long 2 3)] (try (aget a 9 0) (catch Throwable t :oob)))' ':oob' -# AD-019 pin: the typed aset-* multidim arms store WITHOUT element-type -# checking (arrays are type-erased; clj throws IllegalArgumentException when -# the array is boxed, and cljw has no boxed/primitive distinction to key on). +# AD-019 pin: make-array ignores its class argument and builds an object +# array, and the typed aset-* multidim arms store into it (clj throws +# IllegalArgumentException on a boxed array; cljw has no Class to key on). check "AD-019 (aset-long a i j v)" '(let [b (make-array Long 2 2 2)] (aset-long b 0 1 1 5) (aget b 0 1 1))' '5' check "AD-019 (aset-double a i j v)" '(let [c (make-array Double 2 2)] (aset-double c 1 1 2.5) (aget c 1 1))' '2.5' check "AD-019 (aset-int a i j v)" '(let [c (make-array Long 2 2)] (aset-int c 0 0 3) (aget c 0 0))' '3' diff --git a/test/e2e/phase14_doc.sh b/test/e2e/phase14_doc.sh index 847125bf6..ec60b416c 100755 --- a/test/e2e/phase14_doc.sh +++ b/test/e2e/phase14_doc.sh @@ -6,7 +6,7 @@ # D-183 Var-metadata surface. `doc` lives in `clojure.repl` (D-513; the # in-core copy is removed), so non-REPL contexts require it explicitly — # clj parity: `clj -e '(doc f)'` is unresolved there too, and the REPL -# auto-refer is covered by phase15_repl_discovery. The trailing `nil` is +# auto-refer is covered by suites.test-reporters-test. The trailing `nil` is # `doc`'s (println) return, printed by `cljw -e`. # # Layer 2 (e2e CLI) per ADR-0021. diff --git a/test/e2e/phase15_alias.sh b/test/e2e/phase15_alias.sh deleted file mode 100755 index 01321d23f..000000000 --- a/test/e2e/phase15_alias.sh +++ /dev/null @@ -1,18 +0,0 @@ -#!/usr/bin/env bash -# test/e2e/phase15_alias.sh — `alias` / `ns-aliases` (D-232, ns-libs.clj). -# `(alias a ns)` adds a → (the-ns ns) to the current ns; `ns-aliases` reports -# the alias map. clj-equivalence is environmental (clj must `require` the target -# first; cljw bootstraps clojure.* ), so this is a cljw behavioral check. Layer 2. -set -euo pipefail -cd "$(dirname "$0")/../.." -BIN="zig-out/bin/cljw" -[ -n "${CLJW_SKIP_BUILD:-}" ] || zig build -Dwasm -Doptimize="${CLJW_OPT:-ReleaseSafe}" >/dev/null -fail() { echo "FAIL $1" >&2; exit 1; } -assert_eq() { local n="$1" g="$2" w="$3"; [[ "$g" == "$w" ]] || fail "$n: got '$g' want '$w'"; echo "PASS $n -> $w"; } - -assert_eq 'alias-use' "$("$BIN" -e '(alias (quote str) (quote clojure.string)) (str/upper-case "hi")' 2>&1 | tail -1)" '"HI"' -assert_eq 'ns-aliases' "$("$BIN" -e '(do (alias (quote s) (quote clojure.set)) (contains? (ns-aliases *ns*) (quote s)))' 2>&1 | tail -1)" 'true' -assert_eq 'alias-target' "$("$BIN" -e '(do (alias (quote w) (quote clojure.walk)) (ns-name ((ns-aliases *ns*) (quote w))))' 2>&1 | tail -1)" 'clojure.walk' -assert_eq 'alias-unknown' "$("$BIN" -e '(alias (quote x) (quote no.such.ns))' 2>&1 | grep -cm1 'Could not locate')" '1' - -echo "OK — phase15_alias (4 cases) green" diff --git a/test/e2e/phase15_aliased_macro.sh b/test/e2e/phase15_aliased_macro.sh deleted file mode 100755 index 901fee063..000000000 --- a/test/e2e/phase15_aliased_macro.sh +++ /dev/null @@ -1,40 +0,0 @@ -#!/usr/bin/env bash -# test/e2e/phase15_aliased_macro.sh — a macro called through an `:as` alias -# (`m/some-macro`) must be macroexpanded by the analyzer, exactly as the -# fully-qualified form (`mlib.core/some-macro`) already is. -# -# Regression: the analyzer's macro-detection path (`resolveMaybe`) resolved a -# qualified head with a bare `env.findNs(ns)` that does NOT translate require -# aliases, so `m/pick` fell through to a plain function call and its raw -# argument was analyzed as an ordinary symbol → "Unable to resolve symbol". -# Found via test/conformance/verified_projects/qbits.ex (`ex/try+ … (catch-data …)`). The -# fix mirrors `analyzeSymbol`: alias-translate, then own-interns-only (D-261). -# Layer 2 (e2e CLI). -set -euo pipefail -cd "$(dirname "$0")/../.." -BIN="$(pwd)/zig-out/bin/cljw" -[ -n "${CLJW_SKIP_BUILD:-}" ] || zig build -Dwasm -Doptimize="${CLJW_OPT:-ReleaseSafe}" >/dev/null -fail() { echo "FAIL $1" >&2; exit 1; } -last_line() { printf '%s' "$1" | tail -1; } - -WORK="$(mktemp -d)" -trap 'rm -rf "$WORK"' EXIT - -# A lib ns with a macro whose body returns a quoted form of its raw argument. -# If the macro does NOT expand, the analyzer sees `(m/pick undefined-sym)` as a -# call and chokes on `undefined-sym` — the exact failure the bug produced. -proj="$WORK/proj"; mkdir -p "$proj/src/mlib" -printf '{:paths ["src"]}\n' > "$proj/deps.edn" -printf '(ns mlib.core)\n(defmacro pick [sym] (list (quote quote) sym))\n' > "$proj/src/mlib/core.clj" - -# --- Case 1: aliased macro call expands (the regression target) --- -got="$(cd "$proj" && "$BIN" -e "(require '[mlib.core :as m]) (m/pick wildname)" 2>&1 || true)" -[[ "$(last_line "$got")" == 'wildname' ]] || fail "aliased macro: got '$(last_line "$got")'" -echo "PASS aliased_macro_expands -> wildname" - -# --- Case 2: fully-qualified form behaves identically (control) --- -got2="$(cd "$proj" && "$BIN" -e "(require 'mlib.core) (mlib.core/pick wildname)" 2>&1 || true)" -[[ "$(last_line "$got2")" == 'wildname' ]] || fail "qualified macro: got '$(last_line "$got2")'" -echo "PASS qualified_macro_expands -> wildname" - -echo "OK — phase15_aliased_macro (2 cases) green" diff --git a/test/e2e/phase15_campaign_keywords.sh b/test/e2e/phase15_campaign_keywords.sh deleted file mode 100755 index e1593ca56..000000000 --- a/test/e2e/phase15_campaign_keywords.sh +++ /dev/null @@ -1,33 +0,0 @@ -#!/usr/bin/env bash -# test/e2e/phase15_campaign_keywords.sh — validation-campaign batch (D-232) from -# running the upstream clojure.test-clojure.keywords suite: `.name`/`.getName`/ -# `.toString` interop on the Namespace value, regex in a macro expansion -# (valueToForm `.regex` arm), and clojure.test `thrown-with-msg?`. Layer 2. -set -euo pipefail -cd "$(dirname "$0")/../.." -BIN="zig-out/bin/cljw" -[ -n "${CLJW_SKIP_BUILD:-}" ] || zig build -Dwasm -Doptimize="${CLJW_OPT:-ReleaseSafe}" >/dev/null -fail() { echo "FAIL $1" >&2; exit 1; } -assert_eq() { local n="$1" g="$2" w="$3"; [[ "$g" == "$w" ]] || fail "$n: got '$g' want '$w'"; echo "PASS $n -> $w"; } - -# Namespace interop (.name → symbol, .toString → string) -assert_eq 'ns-dot-name' "$("$BIN" -e '(symbol? (.name *ns*))' 2>&1 | tail -1)" 'true' -assert_eq 'ns-dot-getname' "$("$BIN" -e '(str (.getName *ns*))' 2>&1 | tail -1)" '"user"' -assert_eq 'ns-dot-str' "$("$BIN" -e '(.toString *ns*)' 2>&1 | tail -1)" '"user"' -# regex surviving a macro expansion (valueToForm .regex arm) -assert_eq 'regex-macro' "$("$BIN" -e '(defmacro m [] `(re-find #"a.c" "xabcy")) (m)' 2>&1 | tail -1)" '"abc"' -# clojure.test thrown-with-msg? (msg matches → pass; msg mismatch → fail) -assert_eq 'twm-pass' "$("$BIN" -e '(ns t (:use clojure.test)) (deftest x (is (thrown-with-msg? Throwable #"boom" (throw (ex-info "boom!" {}))))) (let [s (run-tests)] [(:pass s) (:fail s)])' 2>&1 | tail -1)" '[1 0]' -assert_eq 'twm-fail' "$("$BIN" -e '(ns t (:use clojure.test)) (deftest x (is (thrown-with-msg? Throwable #"nope" (throw (ex-info "boom!" {}))))) (let [s (run-tests)] [(:pass s) (:fail s)])' 2>&1 | tail -1)" '[0 1]' -# the campaign capstone: the upstream keywords suite RUNS end-to-end off -cp -# (real gaps fixed; residual mismatches are accepted error-message divergences). -# Guarded — the upstream Clojure clone is a local reference, absent on CI/ubuntunote. -UPSTREAM="$HOME/Documents/OSS/clojure/test" -if [ -f "$UPSTREAM/clojure/test_clojure/keywords.clj" ]; then - KW="$("$BIN" -cp "$UPSTREAM" -e '(require (quote clojure.test-clojure.keywords)) (let [s (clojure.test/run-tests (quote clojure.test-clojure.keywords))] (>= (:pass s) 7))' 2>&1 | tail -1)" - assert_eq 'keywords-runs' "$KW" 'true' -else - echo "SKIP keywords-runs (upstream Clojure clone absent)" -fi - -echo "OK — phase15_campaign_keywords (7 cases) green" diff --git a/test/e2e/phase15_clojure_instant.sh b/test/e2e/phase15_clojure_instant.sh deleted file mode 100644 index 106143878..000000000 --- a/test/e2e/phase15_clojure_instant.sh +++ /dev/null @@ -1,66 +0,0 @@ -#!/usr/bin/env bash -# clojure.instant backfill (D-273) + the neutral richer-instant model (D-382). The -# `#inst` reader literal + Date print are cljw built-ins; read-instant-date returns a -# Date (clj-faithful), read-instant-timestamp returns a real nanosecond-precision -# java.sql.Timestamp (neutral runtime/time/timestamp.zig, NOT a Date collapse — print -# form + class + inst?/inst-ms are clj-exact). read-instant-calendar is still absent -# (no Calendar type yet, D-382). Malformed input throws. - -set -euo pipefail -cd "$(dirname "$0")/../.." - -BIN="zig-out/bin/cljw" -[ -n "${CLJW_SKIP_BUILD:-}" ] || zig build -Dwasm -Doptimize="${CLJW_OPT:-ReleaseSafe}" >/dev/null - -fail() { echo "FAIL $1" >&2; exit 1; } -assert_eq() { - local name="$1"; local got="$2"; local want="$3" - [[ "$got" == "$want" ]] || fail "$name: got '$got', want '$want'" - echo "PASS $name -> $want" -} - -# read-instant-date parses an instant string into a Date that equals the #inst literal. -got=$("$BIN" - <<'EOF' 2>/dev/null -(require 'clojure.instant) -(prn [(= (clojure.instant/read-instant-date "2026-06-10T12:00:00Z") - #inst "2026-06-10T12:00:00.000-00:00") - (class (clojure.instant/read-instant-date "2026-06-10"))]) -EOF -) -assert_eq 'read_instant_date' "$got" '[true java.util.Date]' - -# read-instant-timestamp → a nanosecond-precision Timestamp (clj-exact print + class + -# inst?/inst-ms); a Timestamp is NOT = an =-instant Date (distinct type). -got=$("$BIN" - <<'EOF' 2>/dev/null -(require 'clojure.instant) -(let [ts (clojure.instant/read-instant-timestamp "2024-01-01T00:00:00.123456789Z")] - (prn [ts (class ts) (inst? ts) (inst-ms ts) - (= ts (clojure.instant/read-instant-timestamp "2024-01-01T00:00:00.123456789Z")) - (= ts (clojure.instant/read-instant-date "2024-01-01T00:00:00.123Z"))])) -EOF -) -assert_eq 'read_instant_timestamp' "$got" '[#inst "2024-01-01T00:00:00.123456789-00:00" java.sql.Timestamp true 1704067200123 true false]' - -# Malformed input throws (clj parity: a bad instant string is rejected). -got=$("$BIN" - <<'EOF' 2>/dev/null -(require 'clojure.instant) -(print (try (clojure.instant/read-instant-date "not-an-instant") :no-throw - (catch Exception e :threw))) -EOF -) -assert_eq 'malformed_throws' "$got" ':threw' - -# Reader-quote chars are rejected before reaching read-string (the guard is a -# regex literal in a BUNDLED ns — also locks bootstrap-reader regex support, D-383). -got=$("$BIN" - <<'EOF' 2>/dev/null -(require 'clojure.instant) -(print [(try (clojure.instant/read-instant-date "2024-01-01T00:00:00\"x") :no-throw - (catch Exception e :threw)) - (try (clojure.instant/read-instant-date "a\\b") :no-throw - (catch Exception e :threw))]) -EOF -) -assert_eq 'quote_chars_rejected' "$got" '[:threw :threw]' - -echo -echo "clojure.instant backfill (D-273) e2e: all green." diff --git a/test/e2e/phase15_clojure_stacktrace.sh b/test/e2e/phase15_clojure_stacktrace.sh deleted file mode 100755 index f36a265a3..000000000 --- a/test/e2e/phase15_clojure_stacktrace.sh +++ /dev/null @@ -1,114 +0,0 @@ -#!/usr/bin/env bash -# clojure.stacktrace backfill (D-273). A pure-Clojure namespace reimplemented over -# cljw's ex-info exception model (no JVM Throwable). root-cause + print-throwable + -# the cause-chain printing work; per-frame stack printing DEGRADES (cljw exposes no -# Clojure-level frame accessor for a caught exception — the trace data exists on the -# ExInfo but its Clojure surface is owned by the clojure.repl/pst work, D-232). The -# frame degradation is the accepted divergence AD-029. `print-trace-element` and the -# `*e`-dependent REPL helper `e` are intentionally omitted (documented in the ns). -# Uses the stdin-program form (`cljw -`) so only explicit (print …) output is -# captured (the `-e` form prints every top-level form's value, incl. require's nil). - -set -euo pipefail -cd "$(dirname "$0")/../.." - -BIN="zig-out/bin/cljw" -[ -n "${CLJW_SKIP_BUILD:-}" ] || zig build -Dwasm -Doptimize="${CLJW_OPT:-ReleaseSafe}" >/dev/null - -fail() { echo "FAIL $1" >&2; exit 1; } -assert_eq() { - local name="$1"; local got="$2"; local want="$3" - [[ "$got" == "$want" ]] || fail "$name: got '$got', want '$want'" - echo "PASS $name -> $want" -} - -# root-cause walks the ex-cause chain to the last cause. -got=$("$BIN" - <<'EOF' 2>/dev/null -(require 'clojure.stacktrace) -(print (ex-message (clojure.stacktrace/root-cause (ex-info "outer" {} (ex-info "mid" {} (ex-info "inner" {})))))) -EOF -) -assert_eq 'root_cause' "$got" 'inner' - -# root-cause of a single (uncaused) throwable is itself. -got=$("$BIN" - <<'EOF' 2>/dev/null -(require 'clojure.stacktrace) -(print (ex-message (clojure.stacktrace/root-cause (ex-info "solo" {})))) -EOF -) -assert_eq 'root_cause_solo' "$got" 'solo' - -# print-throwable: ": " then the ex-data map on its own line. -got=$("$BIN" - <<'EOF' 2>/dev/null -(require 'clojure.stacktrace) -(clojure.stacktrace/print-throwable (ex-info "boom" {:k 1})) -EOF -) -assert_eq 'print_throwable' "$got" 'ExceptionInfo: boom -{:k 1}' - -# print-cause-trace: the throwable, the degraded no-frame marker, then the cause chain. -got=$("$BIN" - <<'EOF' 2>/dev/null -(require 'clojure.stacktrace) -(clojure.stacktrace/print-cause-trace (ex-info "outer" {} (ex-info "inner" {}))) -EOF -) -assert_eq 'print_cause_trace' "$got" 'ExceptionInfo: outer -{} - at [no stack trace available] -Caused by: ExceptionInfo: inner -{} - at [no stack trace available]' - -assert_contains() { - local name="$1"; local got="$2"; local needle="$3" - [[ "$got" == *"$needle"* ]] || fail "$name: '$got' does not contain '$needle'" - echo "PASS $name -> contains '$needle'" -} -assert_absent() { - local name="$1"; local got="$2"; local needle="$3" - [[ "$got" != *"$needle"* ]] || fail "$name: '$got' unexpectedly contains '$needle'" - echo "PASS $name -> absent '$needle'" -} - -# --- D2 (ADR-0140): a CAUGHT exception carries frames (ADR-0120), so -# print-stack-trace prints per-frame `/ (:)` lines -# instead of the AD-029 marker. A constructed (never-thrown) ex-info has -# NO frames, so it KEEPS the marker (the cases above stay green). --- -got=$("$BIN" - <<'EOF' 2>/dev/null -(require 'clojure.stacktrace) -(defn boom [] (/ 1 0)) -(try (boom) - (catch Throwable e - (print (with-out-str (clojure.stacktrace/print-stack-trace e))))) -EOF -) -assert_contains 'caught_prints_user_frame' "$got" 'user/boom' -assert_absent 'caught_no_marker' "$got" '[no stack trace available]' - -# --- (stack-trace e): a cljw-shaped frame seq (maps {:ns :fn :file :line :column}), -# innermost-first; :fn is the bare name, :ns separate. --- -got=$("$BIN" - <<'EOF' 2>/dev/null -(defn boom [] (/ 1 0)) -(try (boom) - (catch Throwable e - (print (boolean (some #{"boom"} (map :fn (stack-trace e))))))) -EOF -) -assert_eq 'stack_trace_fn_key' "$got" 'true' - -got=$("$BIN" - <<'EOF' 2>/dev/null -(defn boom [] (/ 1 0)) -(try (boom) - (catch Throwable e - (print (boolean (some #{"user"} (map :ns (stack-trace e))))))) -EOF -) -assert_eq 'stack_trace_ns_key' "$got" 'true' - -# --- a constructed ex-info (never thrown) has no frames → empty seq + marker kept. --- -got=$("$BIN" -e '(count (stack-trace (ex-info "x" {})))' 2>/dev/null) -assert_eq 'unthrown_no_frames' "$got" '0' - -echo -echo "clojure.stacktrace backfill (D-273) + frame accessor (ADR-0140) e2e: all green." diff --git a/test/e2e/phase15_clojure_template.sh b/test/e2e/phase15_clojure_template.sh deleted file mode 100755 index b1729e4b4..000000000 --- a/test/e2e/phase15_clojure_template.sh +++ /dev/null @@ -1,23 +0,0 @@ -#!/usr/bin/env bash -# test/e2e/phase15_clojure_template.sh — the bundled clojure.template namespace -# (apply-template / do-template, Stuart Sierra). A small standard ns over -# clojure.walk that cljw did not bundle; surfaced as a require gap by honeysql's -# honey.sql. Now embedded (bootstrap.zig FILES + lookupEmbeddedFile). Layer 2. -set -euo pipefail -cd "$(dirname "$0")/../.." -BIN="zig-out/bin/cljw" -[ -n "${CLJW_SKIP_BUILD:-}" ] || zig build -Dwasm -Doptimize="${CLJW_OPT:-ReleaseSafe}" >/dev/null -fail() { echo "FAIL $1" >&2; exit 1; } -assert_eq() { local n="$1" g="$2" w="$3"; [[ "$g" == "$w" ]] || fail "$n: got '$g' want '$w'"; echo "PASS $n -> $w"; } - -# apply-template substitutes argv symbols with values inside a quoted expr -assert_eq 'apply-template' \ - "$("$BIN" -e "(require '[clojure.template :as t]) (t/apply-template '[x] '(+ x x) '[5])" 2>&1 | tail -1)" \ - '(+ 5 5)' - -# do-template expands to (do expr…) over partitioned values; the do yields last -assert_eq 'do-template' \ - "$("$BIN" -e "(require 'clojure.template) (clojure.template/do-template [x y] (+ x y) 1 2 10 20)" 2>&1 | tail -1)" \ - '30' - -echo "OK — phase15_clojure_template (2 cases) green" diff --git a/test/e2e/phase15_clojure_test.sh b/test/e2e/phase15_clojure_test.sh deleted file mode 100755 index 67353456d..000000000 --- a/test/e2e/phase15_clojure_test.sh +++ /dev/null @@ -1,61 +0,0 @@ -#!/usr/bin/env bash -# test/e2e/phase15_clojure_test.sh — clojure.test (D-227): deftest/is/are/testing/ -# run-tests with a per-ns registry keyed by ns symbol. run-tests returns the -# summary map {:test :pass :fail :error}; the report prints to stdout. We assert -# the returned counts (the last prn line). Layer 2. -set -euo pipefail -cd "$(dirname "$0")/../.." -BIN="zig-out/bin/cljw" -[ -n "${CLJW_SKIP_BUILD:-}" ] || zig build -Dwasm -Doptimize="${CLJW_OPT:-ReleaseSafe}" >/dev/null -fail() { echo "FAIL $1" >&2; exit 1; } -assert_eq() { local n="$1" g="$2" w="$3"; [[ "$g" == "$w" ]] || fail "$n: got '$g' want '$w'"; echo "PASS $n -> $w"; } - -# counts(prog) — run a program ending in a (prn [test pass fail error]) and return that line -counts() { "$BIN" -e "$1" 2>&1 | tail -1; } - -C='(fn [s] [(:test s) (:pass s) (:fail s) (:error s)])' - -# all pass: 1 test, 3 assertions -P1='(ns t1 (:require [clojure.test :refer [deftest is run-tests]])) - (deftest a (is (= 1 1)) (is (pos? 3)) (is (not (nil? [])))) - ('"$C"' (run-tests (quote t1)))' -assert_eq 'all-pass' "$(counts "$P1")" '[1 3 0 0]' - -# one fail -P2='(ns t2 (:require [clojure.test :refer [deftest is run-tests]])) - (deftest a (is (= 1 1)) (is (= 1 2))) - ('"$C"' (run-tests (quote t2)))' -assert_eq 'one-fail' "$(counts "$P2")" '[1 1 1 0]' - -# error (uncaught throw inside is, not via thrown?) -P3='(ns t3 (:require [clojure.test :refer [deftest is run-tests]])) - (deftest a (is (throw (ex-info "x" {})))) - ('"$C"' (run-tests (quote t3)))' -assert_eq 'error' "$(counts "$P3")" '[1 0 0 1]' - -# thrown? passes -P4='(ns t4 (:require [clojure.test :refer [deftest is run-tests]])) - (deftest a (is (thrown? Throwable (throw (ex-info "x" {}))))) - ('"$C"' (run-tests (quote t4)))' -assert_eq 'thrown' "$(counts "$P4")" '[1 1 0 0]' - -# are expands to N assertions -P5='(ns t5 (:require [clojure.test :refer [deftest is are run-tests]])) - (deftest a (are [x y] (= x y) 1 1 2 2 3 3)) - ('"$C"' (run-tests (quote t5)))' -assert_eq 'are' "$(counts "$P5")" '[1 3 0 0]' - -# two deftests in one ns, run-tests with no arg (current ns) -P6='(ns t6 (:require [clojure.test :refer [deftest is run-tests]])) - (deftest a (is (= 1 1))) - (deftest b (is (= 2 2)) (is (= 3 3))) - ('"$C"' (run-tests))' -assert_eq 'multi-deftest' "$(counts "$P6")" '[2 3 0 0]' - -# testing context does not change counts -P7='(ns t7 (:require [clojure.test :refer [deftest is testing run-tests]])) - (deftest a (testing "ctx" (is (= 1 1)) (is (= 2 2)))) - ('"$C"' (run-tests (quote t7)))' -assert_eq 'testing' "$(counts "$P7")" '[1 2 0 0]' - -echo "OK — phase15_clojure_test (7 cases) green" diff --git a/test/e2e/phase15_clojure_test_output.sh b/test/e2e/phase15_clojure_test_output.sh deleted file mode 100755 index bec3a35f2..000000000 --- a/test/e2e/phase15_clojure_test_output.sh +++ /dev/null @@ -1,77 +0,0 @@ -#!/usr/bin/env bash -# test/e2e/phase15_clojure_test_output.sh — clojure.test OUTPUT fidelity (D-463). -# The report format is clj-grounded (oracle: clj 1.12). The FAIL line carries -# a ` (file:line)` suffix from the deftest VAR's source meta (D-563b) — clj -# points at the failing `is` via its JVM stack instead (deftest-line vs -# is-line = the narrowed AD-041); the :error cause trace stays AD-041. -# Everything else matches clj byte-for-line: `(not (= 1 2))` actual, -# `outer inner` context line, the `Testing ` begin line, and a re-enabled -# `*test-out*` (binding it redirects report output — was a no-JVM deferral). -# Uses `cljw -` (stdin program: only explicit output, no per-form echo). Layer 2. -set -euo pipefail -cd "$(dirname "$0")/../.." -BIN="zig-out/bin/cljw" -[ -n "${CLJW_SKIP_BUILD:-}" ] || zig build -Dwasm -Doptimize="${CLJW_OPT:-ReleaseSafe}" >/dev/null -fail() { echo "FAIL $1" >&2; exit 1; } -has() { local n="$1" out="$2" line="$3"; grep -Fqx "$line" <<<"$out" || fail "$n: missing line '$line'"; echo "PASS $n -> $line"; } -hasf() { local n="$1" out="$2" sub="$3"; grep -Fq "$sub" <<<"$out" || fail "$n: missing '$sub'"; echo "PASS $n -> $sub"; } - -# --- A. = fail: FAIL line via testing-vars-str + clj actual (not (= …)) + Testing line + summary -A=$("$BIN" - <<'EOF' 2>&1 -(ns demo (:require [clojure.test :refer [deftest is run-tests]])) -(deftest eqfail (is (= 1 2))) -(run-tests 'demo) -EOF -) -has 'A-testing' "$A" 'Testing demo' -hasf 'A-fail-in' "$A" 'FAIL in (eqfail) (' # suffix present (D-563b; stdin label) -has 'A-expected' "$A" 'expected: (= 1 2)' -has 'A-actual' "$A" ' actual: (not (= 1 2))' -has 'A-ran' "$A" 'Ran 1 tests containing 1 assertions.' -has 'A-summary' "$A" '1 failures, 0 errors.' - -# --- B. predicate (non-=) fail: actual wraps the evaluated form in (not …) -B=$("$BIN" - <<'EOF' 2>&1 -(ns demo (:require [clojure.test :refer [deftest is run-tests]])) -(deftest pf (is (pos? -1))) -(run-tests 'demo) -EOF -) -has 'B-actual' "$B" ' actual: (not (pos? -1))' - -# --- C. testing context: clj prints the joined context strings (outermost first) -C=$("$BIN" - <<'EOF' 2>&1 -(ns demo (:require [clojure.test :refer [deftest is testing run-tests]])) -(deftest cf (testing "outer" (testing "inner" (is (= :a :b))))) -(run-tests 'demo) -EOF -) -has 'C-context' "$C" 'outer inner' - -# --- D. = pass actual renders the bare form (= 1 1), not clojure.core/= -D=$("$BIN" - <<'EOF' 2>&1 -(require '[clojure.test :as t :refer [deftest is run-tests]] '[clojure.test.tap :as tap]) -(deftest p (is (= 1 1))) -(tap/with-tap-output (run-tests 'user)) -EOF -) -hasf 'D-pass-actual' "$D" '# actual:(= 1 1)' - -# --- E. *test-out* is re-enabled: resolvable + binding it redirects report output -E=$("$BIN" - <<'EOF' 2>&1 -(require '[clojure.test]) -(prn (some? (resolve 'clojure.test/*test-out*))) -EOF -) -has 'E-resolvable' "$E" 'true' - -F=$("$BIN" - <<'EOF' 2>&1 -(ns d2 (:require [clojure.test :as t :refer [deftest is run-tests]])) -(deftest f (is (= 1 2))) -(print "CAP[" (with-out-str (binding [t/*test-out* *out*] (run-tests 'd2))) "]CAP") -EOF -) -hasf 'F-capture' "$F" 'CAP[' -hasf 'F-captured-fail' "$F" 'FAIL in (f) (' - -echo "OK — phase15_clojure_test_output (D-463) green" diff --git a/test/e2e/phase15_clojure_test_tap.sh b/test/e2e/phase15_clojure_test_tap.sh deleted file mode 100755 index fb4b28fd3..000000000 --- a/test/e2e/phase15_clojure_test_tap.sh +++ /dev/null @@ -1,55 +0,0 @@ -#!/usr/bin/env bash -# clojure.test.tap backfill (D-273) — TAP (Test Anything Protocol) output for -# clojure.test. `with-tap-output` rebinds clojure.test/report (now ^:dynamic) to a -# TAP reporter emitting `ok` / `not ok` lines + `#` diagnostics + a `1..n` plan. Built -# on clj-compat clojure.test surface added here: with-test-out / inc-report-counter / -# testing-vars-str / testing-contexts-str / *testing-vars* / *stack-trace-depth* -# (also useful for running real clojure.test suites, D-232). Uses clojure.stacktrace -# (D-273) for the actual-trace diagnostic. Java `.split` → clojure.string/split-lines. - -set -euo pipefail -cd "$(dirname "$0")/../.." - -BIN="zig-out/bin/cljw" -[ -n "${CLJW_SKIP_BUILD:-}" ] || zig build -Dwasm -Doptimize="${CLJW_OPT:-ReleaseSafe}" >/dev/null - -fail() { echo "FAIL $1" >&2; exit 1; } -assert_eq() { - local name="$1"; local got="$2"; local want="$3" - [[ "$got" == "$want" ]] || fail "$name: got '$got', want '$want'" - echo "PASS $name -> ok" -} - -# with-tap-output emits valid TAP: ok / not ok lines (named by the test var) + -# `#` diagnostics + the trailing `1..n` plan. -got=$("$BIN" - <<'EOF' 2>/dev/null -(require '[clojure.test :as t] '[clojure.test.tap :as tap]) -(t/deftest pass-t (t/is (= 1 1))) -(t/deftest fail-t (t/is (= 1 2))) -(print (with-out-str (tap/with-tap-output (t/run-tests 'user)))) -EOF -) -# run-tests emits the full lifecycle event set — :begin-test-ns, :begin/:end-test-var -# (per test), :end-test-ns — which tap's `:default` renders as `# {…}` comment -# lines, EXACTLY as clj's tap does (clj-verified 2026-06-21). cljw omits only the -# the ns identity-hash in the begin/end-test-ns map (no JVM identity — -# AD-002); the ok/not-ok ` (file:line)` suffix is PRESENT since D-563(b) -# (deftest-var-sourced; clj's is assertion-stack-sourced — narrowed AD-041). Names render via testing-vars-str `(test-name)`; -# `=` actual is the bare (not (= …)) form (D-463). -want='# {:type :begin-test-ns, :ns user} -# {:type :begin-test-var, :var #'\''user/pass-t} -ok (pass-t) (:2) -# expected:(= 1 1) -# actual:(= 1 1) -# {:type :end-test-var, :var #'\''user/pass-t} -# {:type :begin-test-var, :var #'\''user/fail-t} -not ok (fail-t) (:3) -# expected:(= 1 2) -# actual:(not (= 1 2)) -# {:type :end-test-var, :var #'\''user/fail-t} -# {:type :end-test-ns, :ns user} -1..2' -assert_eq 'tap_output' "$got" "$want" - -echo -echo "clojure.test.tap backfill (D-273) e2e: all green." diff --git a/test/e2e/phase15_clojure_uuid.sh b/test/e2e/phase15_clojure_uuid.sh deleted file mode 100644 index eadd92e3e..000000000 --- a/test/e2e/phase15_clojure_uuid.sh +++ /dev/null @@ -1,33 +0,0 @@ -#!/usr/bin/env bash -# clojure.uuid backfill (D-273). In ClojureWasm the `#uuid` reader literal and the -# `#uuid "…"` print form are BUILT IN (the reader's default data-readers + the UUID -# print path), so this namespace is a thin require-compatibility shim: a real library -# that `(require 'clojure.uuid)` for its side-effects must load. It exposes the -# `default-uuid-reader` data-reader fn (faithful to upstream) over java.util.UUID. - -set -euo pipefail -cd "$(dirname "$0")/../.." - -BIN="zig-out/bin/cljw" -[ -n "${CLJW_SKIP_BUILD:-}" ] || zig build -Dwasm -Doptimize="${CLJW_OPT:-ReleaseSafe}" >/dev/null - -fail() { echo "FAIL $1" >&2; exit 1; } -assert_eq() { - local name="$1"; local got="$2"; local want="$3" - [[ "$got" == "$want" ]] || fail "$name: got '$got', want '$want'" - echo "PASS $name -> $want" -} - -# (require 'clojure.uuid) must succeed (currently fails — missing ns), and the -# #uuid reader + UUID print stay clj-faithful after it loads. -got=$("$BIN" - <<'EOF' 2>/dev/null -(require 'clojure.uuid) -(prn [(= #uuid "12345678-1234-1234-1234-123456789abc" - (clojure.uuid/default-uuid-reader "12345678-1234-1234-1234-123456789abc")) - (pr-str #uuid "12345678-1234-1234-1234-123456789abc")]) -EOF -) -assert_eq 'uuid_require_and_reader' "$got" '[true "#uuid \"12345678-1234-1234-1234-123456789abc\""]' - -echo -echo "clojure.uuid backfill (D-273) e2e: all green." diff --git a/test/e2e/phase15_deferred_host_ref.sh b/test/e2e/phase15_deferred_host_ref.sh index 2baef6090..a34039d15 100755 --- a/test/e2e/phase15_deferred_host_ref.sh +++ b/test/e2e/phase15_deferred_host_ref.sh @@ -1,35 +1,21 @@ #!/usr/bin/env bash -# test/e2e/phase15_deferred_host_ref.sh — ADR-0113 / AD-022. An unresolved -# `clojure.lang.*` / `clojure.asm.*` reference defers to a CALL-time -# feature_not_supported instead of failing the whole namespace at analysis, so a -# fn whose body names such a JVM-internal class DEFINES + LOADS; the ref errors -# only if evaluated. A user alias typo stays a LOUD analysis error (strict prefix -# allowlist). Surfaced by integrant's `(clojure.lang.RT/baseLoader)`. Layer 2. +# test/e2e/phase15_deferred_host_ref.sh — ADR-0113 / AD-022, the one case that +# needs a process. An unresolved `clojure.lang.*` reference defers to a +# CALL-time feature_not_supported; that raise is a .not_implemented error, +# which no (catch …) intercepts, so observing it means reading the stderr of a +# process that died of it. The deferral itself (defines + loads, value +# position, a user alias typo staying loud) is in-process and lives in +# test/clj/suites/forms_and_literals_test.clj (p15-deferred-host-ref). +# e2e-process-boundary: the call-time raise is an uncatchable not_implemented, observable only on stderr set -euo pipefail cd "$(dirname "$0")/../.." BIN="zig-out/bin/cljw" [ -n "${CLJW_SKIP_BUILD:-}" ] || zig build -Dwasm -Doptimize="${CLJW_OPT:-ReleaseSafe}" >/dev/null fail() { echo "FAIL $1" >&2; exit 1; } -last_line() { printf '%s' "$1" | tail -1; } - -# defines_and_loads: a fn body naming clojure.lang.RT defines; the do returns :ok -got="$("$BIN" -e '(do (defn f [] (clojure.lang.RT/baseLoader)) :ok)' 2>&1 || true)" -[[ "$(last_line "$got")" == ':ok' ]] || fail "defines_and_loads: got '$(last_line "$got")'" -echo "PASS defines_and_loads -> :ok" # raises_when_called: invoking the unsupported ref errors loudly at call time -got2="$("$BIN" -e '(do (defn f [] (clojure.lang.RT/baseLoader)) (f))' 2>&1 || true)" -grep -q 'clojure.lang.RT/baseLoader is not supported' <<<"$got2" || fail "raises_when_called: got '$got2'" +got="$("$BIN" -e '(do (defn f [] (clojure.lang.RT/baseLoader)) (f))' 2>&1 || true)" +grep -q 'clojure.lang.RT/baseLoader is not supported' <<<"$got" || fail "raises_when_called: got '$got'" echo "PASS raises_when_called -> feature_not_supported" -# value-position (a static field read) defers identically -got3="$("$BIN" -e '(do (def g (fn [] clojure.lang.Compiler/CHAR_MAP)) :defined)' 2>&1 || true)" -[[ "$(last_line "$got3")" == ':defined' ]] || fail "value_position_defines: got '$(last_line "$got3")'" -echo "PASS value_position_defines -> :defined" - -# typo_stays_loud: a non-clojure.lang.* unresolved ns is STILL a loud error -got4="$("$BIN" -e '(myalias/foo 1)' 2>&1 || true)" -grep -q "No namespace: 'myalias'" <<<"$got4" || fail "typo_stays_loud: got '$got4'" -echo "PASS typo_stays_loud -> No namespace: 'myalias'" - -echo "OK — phase15_deferred_host_ref (4 cases) green" +echo "OK — phase15_deferred_host_ref (1 process case) green" diff --git a/test/e2e/phase15_defmethod_destructure.sh b/test/e2e/phase15_defmethod_destructure.sh deleted file mode 100755 index a1cbf8aba..000000000 --- a/test/e2e/phase15_defmethod_destructure.sh +++ /dev/null @@ -1,30 +0,0 @@ -#!/usr/bin/env bash -# test/e2e/phase15_defmethod_destructure.sh — destructuring defmethod params -# (D-232). defmethod lowered its body to `fn*` (raw, symbol-only params), so a -# destructuring method param (`(defmethod m :k [a [b c]] …)` / -# `(defmethod m :k [{:keys [x]}] …)`) raised "fn* parameter must be a symbol". -# Fixed: defmethod now emits `fn` (the destructure-lowering macro). Surfaced by -# clojure.test-helper's assert-expr methods. clj-grounded. Layer 2. -set -euo pipefail -cd "$(dirname "$0")/../.." -BIN="zig-out/bin/cljw" -[ -n "${CLJW_SKIP_BUILD:-}" ] || zig build -Dwasm -Doptimize="${CLJW_OPT:-ReleaseSafe}" >/dev/null -fail() { echo "FAIL $1" >&2; exit 1; } -assert_eq() { local n="$1" g="$2" w="$3"; [[ "$g" == "$w" ]] || fail "$n: got '$g' want '$w'"; echo "PASS $n -> $w"; } - -# associative destructure in a method param -assert_eq 'assoc-param' \ - "$("$BIN" -e '(defmulti foo (fn [x] (:t x))) (defmethod foo :pair [{:keys [a b]}] (+ a b)) (foo {:t :pair :a 3 :b 4})' 2>&1 | tail -1)" \ - '7' - -# nested sequential destructure with _, & rest, :as (the test-helper shape) -assert_eq 'nested-seq-param' \ - "$("$BIN" -e '(defmulti bar (fn [m form] m)) (defmethod bar :x [msg [_ a b & rest :as form]] [msg a b rest form]) (bar :x [:tag 1 2 3 4])' 2>&1 | tail -1)" \ - '[:x 1 2 (3 4) [:tag 1 2 3 4]]' - -# a plain symbol-param method still dispatches (no regression) -assert_eq 'symbol-param-regress' \ - "$("$BIN" -e '(defmulti baz class) (defmethod baz Long [n] (* n 2)) (baz 21)' 2>&1 | tail -1)" \ - '42' - -echo "OK — phase15_defmethod_destructure (3 cases) green" diff --git a/test/e2e/phase15_defmulti_defmethod.sh b/test/e2e/phase15_defmulti_defmethod.sh deleted file mode 100755 index 0de5e6ef7..000000000 --- a/test/e2e/phase15_defmulti_defmethod.sh +++ /dev/null @@ -1,35 +0,0 @@ -#!/usr/bin/env bash -# test/e2e/phase15_defmulti_defmethod.sh — defmulti docstring/attr-map + empty -# -body defmethod (surfaced by integrant). `(defmulti name "doc" {:attr …} -# dispatch-fn)` used to mistake the docstring for the dispatch fn ("Cannot call -# value of type 'string'"); the docstring/attr-map are now skipped AND attached -# to the Var metadata (clj parity for :doc). `(defmethod m dv [params])` with no -# body now defines a nil-returning method (clj parity). Layer 2. -set -euo pipefail -cd "$(dirname "$0")/../.." -BIN="zig-out/bin/cljw" -[ -n "${CLJW_SKIP_BUILD:-}" ] || zig build -Dwasm -Doptimize="${CLJW_OPT:-ReleaseSafe}" >/dev/null -fail() { echo "FAIL $1" >&2; exit 1; } -assert_eq() { local n="$1" g="$2" w="$3"; [[ "$g" == "$w" ]] || fail "$n: got '$g' want '$w'"; echo "PASS $n -> $w"; } - -# defmulti with docstring + attr-map dispatches correctly (was a string-call err) -assert_eq 'defmulti-docstring-dispatch' \ - "$("$BIN" -e '(do (defmulti area "the area" {:arglists (quote ([shape]))} :kind) (defmethod area :square [s] (* (:side s) (:side s))) (area {:kind :square :side 3}))' 2>&1 | tail -1)" \ - '9' - -# docstring is attached to the Var metadata (not discarded) -assert_eq 'defmulti-doc-meta' \ - "$("$BIN" -e '(do (defmulti m "the doc" {:x 1} identity) (:doc (meta (var m))))' 2>&1 | tail -1)" \ - '"the doc"' - -# explicit attr-map keys are attached too -assert_eq 'defmulti-attr-meta' \ - "$("$BIN" -e '(do (defmulti m "d" {:x 1} identity) (:x (meta (var m))))' 2>&1 | tail -1)" \ - '1' - -# empty-body defmethod defines a nil-returning method (clj parity) -assert_eq 'defmethod-empty-body' \ - "$("$BIN" -e '(do (defmulti m identity) (defmethod m :default [_]) (m :anything))' 2>&1 | tail -1)" \ - 'nil' - -echo "OK — phase15_defmulti_defmethod (4 cases) green" diff --git a/test/e2e/phase15_defprotocol_docstring.sh b/test/e2e/phase15_defprotocol_docstring.sh deleted file mode 100755 index 871179060..000000000 --- a/test/e2e/phase15_defprotocol_docstring.sh +++ /dev/null @@ -1,45 +0,0 @@ -#!/usr/bin/env bash -# test/e2e/phase15_defprotocol_docstring.sh — defprotocol docstrings (D-232). -# clj's defprotocol allows an optional protocol-level docstring after the name -# and an optional docstring after each method's arity vectors. cljw used to -# reject the protocol-level docstring ("method signature must be a list"). -# Method docstrings + ^type return hints + multiple arity vectors are tolerated -# (only the method-name symbol is read). Surfaced by upstream protocols/ -# examples.clj (which then parks on definterface, a JVM-interface unit). Layer 2. -set -euo pipefail -cd "$(dirname "$0")/../.." -BIN="zig-out/bin/cljw" -[ -n "${CLJW_SKIP_BUILD:-}" ] || zig build -Dwasm -Doptimize="${CLJW_OPT:-ReleaseSafe}" >/dev/null -fail() { echo "FAIL $1" >&2; exit 1; } -assert_eq() { local n="$1" g="$2" w="$3"; [[ "$g" == "$w" ]] || fail "$n: got '$g' want '$w'"; echo "PASS $n -> $w"; } - -# protocol docstring + method docstrings, then dispatch through a defrecord -P='(defprotocol G "a greeting protocol" (greet [this] "say hi") (loud [this] "SHOUT")) (defrecord Person [nm] G (greet [this] (str "hi " nm)) (loud [this] (str "HI " nm)))' -assert_eq 'dispatch-with-docstrings' \ - "$("$BIN" -e "$P [(greet (->Person \"ann\")) (loud (->Person \"bob\")) (satisfies? G (->Person \"x\"))]" 2>&1 | tail -1)" \ - '["hi ann" "HI bob" true]' - -# ^type return hint + multiple arity vectors on one method (hint/arities ignored) -assert_eq 'hinted-multi-arity' \ - "$("$BIN" -e '(defprotocol P (^String baz [a] [a b] "multi-arity, hinted")) (satisfies? P 1)' 2>&1 | tail -1)" \ - 'false' - -# marker protocol with only a docstring (no methods): defines cleanly and -# satisfies? on a non-member returns false without error. (Membership for a -# type that *extends* a method-less marker is a separate pre-existing gap, -# tracked in D-232 — orthogonal to the docstring fix.) -assert_eq 'marker-docstring-only' \ - "$("$BIN" -e '(defprotocol M "marker only") (satisfies? M 1)' 2>&1 | tail -1)" \ - 'false' - -# keyword-value option pairs after the name (e.g. :extend-via-metadata true, -# clj 1.10+) are parsed + skipped before the method sigs, so the protocol -# defines and dispatches normally. (The :extend-via-metadata DISPATCH itself — -# metadata-based extension — is not yet honored, tracked D-314; explicit -# extend works, which is what this exercises.) Surfaced by honeysql. -OPT='(defprotocol Sz :extend-via-metadata true (sz [this] "render")) (defrecord Lit [v] Sz (sz [this] (str "<" v ">")))' -assert_eq 'option-pair-extend-via-metadata' \ - "$("$BIN" -e "$OPT [(sz (->Lit 7)) (satisfies? Sz (->Lit 1))]" 2>&1 | tail -1)" \ - '["<7>" true]' - -echo "OK — phase15_defprotocol_docstring (4 cases) green" diff --git a/test/e2e/phase15_dynamic_var.sh b/test/e2e/phase15_dynamic_var.sh deleted file mode 100755 index 7160bde35..000000000 --- a/test/e2e/phase15_dynamic_var.sh +++ /dev/null @@ -1,29 +0,0 @@ -#!/usr/bin/env bash -# test/e2e/phase15_dynamic_var.sh — `^:dynamic` on a .clj def now sets the Var's -# dynamic flag so `binding` can rebind it (analyzeDef reads :dynamic/:private off -# the def-target metadata; previously the meta was lifted into Var.meta but the -# flags stayed false). Layer 2. -set -euo pipefail -cd "$(dirname "$0")/../.." -BIN="zig-out/bin/cljw" -[ -n "${CLJW_SKIP_BUILD:-}" ] || zig build -Dwasm -Doptimize="${CLJW_OPT:-ReleaseSafe}" >/dev/null -fail() { echo "FAIL $1" >&2; exit 1; } -assert_eq() { local n="$1" g="$2" w="$3"; [[ "$g" == "$w" ]] || fail "$n: got '$g' want '$w'"; echo "PASS $n -> $w"; } - -assert_eq 'bind' "$("$BIN" -e '(do (def ^:dynamic *x* 1) (binding [*x* 2] *x*))' 2>&1 | tail -1)" '2' -assert_eq 'restore' "$("$BIN" -e '(do (def ^:dynamic *x* 1) [(binding [*x* 2] *x*) *x*])' 2>&1 | tail -1)" '[2 1]' -assert_eq 'nested' "$("$BIN" -e '(do (def ^:dynamic *y* 10) (binding [*y* 20] (binding [*y* 30] *y*)))' 2>&1 | tail -1)" '30' -assert_eq 'fn-sees' "$("$BIN" -e '(do (def ^:dynamic *a* :root) (defn pa [] *a*) [(binding [*a* :inner] (pa)) (pa)])' 2>&1 | tail -1)" '[:inner :root]' -# non-dynamic var still rejects binding (the guard still fires). Expect 2 -# occurrences: the message line + the source-window caret repeats it -# (print.zig's intended form; D-555 restored the vm's loc fidelity here). -assert_eq 'guard' "$("$BIN" -e '(do (def plain 1) (binding [plain 2] plain))' 2>&1 | grep -c 'non-dynamic')" '2' - -# --- standard core version / flag vars (cljw targets the 1.12 surface) --- -assert_eq 'cv_minor' "$("$BIN" -e '(:minor *clojure-version*)' 2>&1 | tail -1)" '12' -assert_eq 'cv_string' "$("$BIN" -e '(clojure-version)' 2>&1 | tail -1)" '"1.12.0"' -assert_eq 'unchecked' "$("$BIN" -e '*unchecked-math*' 2>&1 | tail -1)" 'false' -# *unchecked-math* is ^:dynamic — libs set! / binding it (a no-op flag in cljw) -assert_eq 'um_bind' "$("$BIN" -e '(binding [*unchecked-math* true] *unchecked-math*)' 2>&1 | tail -1)" 'true' - -echo "OK — phase15_dynamic_var (9 cases) green" diff --git a/test/e2e/phase15_import_macro.sh b/test/e2e/phase15_import_macro.sh deleted file mode 100755 index 598c6ed5f..000000000 --- a/test/e2e/phase15_import_macro.sh +++ /dev/null @@ -1,40 +0,0 @@ -#!/usr/bin/env bash -# test/e2e/phase15_import_macro.sh — the standalone (import …) macro (D-232). -# `(import '(pkg Class …))` / `(import 'pkg.Class)` register class simple-names -# in the current ns (reusing the D-235 :import map). Expands to clojure.core/ -# import* calls (the runtime registration fn), clj-faithful. cljw had only the -# `(:import …)` ns directive. Surfaced by clojure.test-clojure.evaluation:20. -# Behaviour parity pinned by corpus import_macro. Layer 2. -set -euo pipefail -cd "$(dirname "$0")/../.." -BIN="zig-out/bin/cljw" -[ -n "${CLJW_SKIP_BUILD:-}" ] || zig build -Dwasm -Doptimize="${CLJW_OPT:-ReleaseSafe}" >/dev/null -fail() { echo "FAIL $1" >&2; exit 1; } -assert_eq() { local n="$1" g="$2" w="$3"; [[ "$g" == "$w" ]] || fail "$n: got '$g' want '$w'"; echo "PASS $n -> $w"; } - -# prefix-list import, then use the class via static interop -assert_eq 'prefix-list' \ - "$("$BIN" -e '(import (quote (java.lang Integer))) (Integer/parseInt "42")' 2>&1 | tail -1)" \ - '42' - -# bare qualified-symbol import -assert_eq 'bare-symbol' \ - "$("$BIN" -e '(import (quote java.lang.Long)) (Long/parseLong "7")' 2>&1 | tail -1)" \ - '7' - -# multiple classes in one prefix list -assert_eq 'multi-class' \ - "$("$BIN" -e '(import (quote (java.lang Integer Long))) [(Integer/parseInt "3") (Long/parseLong "4")]' 2>&1 | tail -1)" \ - '[3 4]' - -# import* expansion (one registration call per class) -assert_eq 'expands-to-import-star' \ - "$("$BIN" -e '(macroexpand-1 (quote (import (quote (java.lang Boolean Integer)))))' 2>&1 | tail -1)" \ - '(do (import* "java.lang.Boolean") (import* "java.lang.Integer"))' - -# vector prefix spelling, which clj accepts and the ns :import clause already did -assert_eq 'prefix-vector' \ - "$("$BIN" -e '(import (quote [java.lang Integer Long])) [(Integer/parseInt "5") (Long/parseLong "6")]' 2>&1 | tail -1)" \ - '[5 6]' - -echo "OK — phase15_import_macro (5 cases) green" diff --git a/test/e2e/phase15_java_time_arith3.sh b/test/e2e/phase15_java_time_arith3.sh deleted file mode 100644 index ee1f4df70..000000000 --- a/test/e2e/phase15_java_time_arith3.sh +++ /dev/null @@ -1,46 +0,0 @@ -#!/usr/bin/env bash -# test/e2e/phase15_java_time_arith3.sh — Instant / Duration / LocalTime arithmetic -# (D-462), closing the temporal-arithmetic area. Instant plus/minus Seconds/Millis/ -# Nanos (carry into second-aligned epoch_ms); Duration plus/minus Seconds/Minutes/ -# Hours/Days/Millis/Nanos + plus(Duration)/multipliedBy/dividedBy; LocalTime plus/ -# minus Hours/Minutes/Seconds/Nanos (WRAP mod 24h, no day). clj-grounded. `cljw -`. -set -euo pipefail -cd "$(dirname "$0")/../.." -BIN="zig-out/bin/cljw" -[ -n "${CLJW_SKIP_BUILD:-}" ] || zig build -Dwasm -Doptimize="${CLJW_OPT:-ReleaseSafe}" >/dev/null -fail() { echo "FAIL $1" >&2; exit 1; } -eq() { local n="$1" g="$2" w="$3"; [[ "$g" == "$w" ]] || fail "$n: got '$g' want '$w'"; echo "PASS $n -> $w"; } -out() { "$BIN" - ; } - -# --- Instant -A=$(out <<'EOF' 2>&1 -(let [i (java.time.Instant/ofEpochSecond 100 0)] - (println (.getEpochSecond (.plusSeconds i 5)) (.toEpochMilli (.plusMillis i 1500)) - (.getNano (.plusNanos i 1)) (.getEpochSecond (.minusSeconds i 50)))) -EOF -) -eq 'instant' "$A" '105 101500 1 50' - -# --- Duration -B=$(out <<'EOF' 2>&1 -(println (str (.plusSeconds (java.time.Duration/ofSeconds 5) 10)) - (str (.plusMinutes (java.time.Duration/ofSeconds 0) 2)) - (str (.plus (java.time.Duration/ofSeconds 5) (java.time.Duration/ofMillis 500))) - (str (.multipliedBy (java.time.Duration/ofSeconds 3) 4)) - (str (.dividedBy (java.time.Duration/ofSeconds 10) 4)) - (str (.minusSeconds (java.time.Duration/ofSeconds 10) 3))) -EOF -) -eq 'duration' "$B" 'PT15S PT2M PT5.5S PT12S PT2.5S PT7S' - -# --- LocalTime (wraps mod 24h) -C=$(out <<'EOF' 2>&1 -(println (str (.plusHours (java.time.LocalTime/of 23 0) 2)) - (str (.minusHours (java.time.LocalTime/of 1 0) 2)) - (str (.plusMinutes (java.time.LocalTime/of 12 30) 45)) - (str (.plusNanos (java.time.LocalTime/of 12 0 0 0) 1500000000))) -EOF -) -eq 'localtime' "$C" '01:00 23:00 13:15 12:00:01.500' - -echo "OK — phase15_java_time_arith3 (D-462) green" diff --git a/test/e2e/phase15_java_time_compare.sh b/test/e2e/phase15_java_time_compare.sh deleted file mode 100644 index 3ba1df159..000000000 --- a/test/e2e/phase15_java_time_compare.sh +++ /dev/null @@ -1,51 +0,0 @@ -#!/usr/bin/env bash -# test/e2e/phase15_java_time_compare.sh — java.time comparison predicates + -# Duration unary ops (D-462). isBefore/isAfter on the point types (Instant, -# LocalDate, LocalTime, LocalDateTime), isEqual on LocalDate/LocalDateTime, -# and Duration isZero/isNegative/negated/abs. All exact-match vs clj (no AD; -# compareTo's JVM magnitude is a separate follow-up). Uses `cljw -`. Layer 2. -set -euo pipefail -cd "$(dirname "$0")/../.." -BIN="zig-out/bin/cljw" -[ -n "${CLJW_SKIP_BUILD:-}" ] || zig build -Dwasm -Doptimize="${CLJW_OPT:-ReleaseSafe}" >/dev/null -fail() { echo "FAIL $1" >&2; exit 1; } -eq() { local n="$1" g="$2" w="$3"; [[ "$g" == "$w" ]] || fail "$n: got '$g' want '$w'"; echo "PASS $n -> $w"; } -out() { "$BIN" - ; } - -A=$(out <<'EOF' 2>&1 -(let [a (java.time.Instant/ofEpochSecond 5) b (java.time.Instant/ofEpochSecond 9)] - (println (.isBefore a b) (.isAfter a b) (.isBefore a a) (.isAfter b a))) -EOF -) -eq 'instant' "$A" 'true false false true' - -B=$(out <<'EOF' 2>&1 -(let [a (java.time.LocalDate/of 2024 1 1) b (java.time.LocalDate/of 2024 6 1)] - (println (.isBefore a b) (.isAfter a b) (.isEqual a b) (.isEqual a (java.time.LocalDate/of 2024 1 1)))) -EOF -) -eq 'localdate' "$B" 'true false false true' - -C=$(out <<'EOF' 2>&1 -(let [a (java.time.LocalTime/of 6 0) b (java.time.LocalTime/of 18 0)] - (println (.isBefore a b) (.isAfter a b))) -(let [a (java.time.LocalDateTime/of 2024 1 1 6 0) b (java.time.LocalDateTime/of 2024 1 1 18 0)] - (println (.isBefore a b) (.isAfter a b) (.isEqual a b))) -EOF -) -eq 'localtime-ldt' "$C" $'true false\ntrue false false' - -D=$(out <<'EOF' 2>&1 -(println (.isZero (java.time.Duration/ofSeconds 0)) - (.isZero (java.time.Duration/ofSeconds 5)) - (.isNegative (java.time.Duration/ofSeconds -1)) - (.isNegative (java.time.Duration/ofSeconds 1))) -(println (str (.negated (java.time.Duration/ofMillis 1500))) - (str (.abs (java.time.Duration/ofSeconds -30))) - (str (.abs (java.time.Duration/ofSeconds 30))) - (str (.negated (java.time.Duration/ofSeconds 0)))) -EOF -) -eq 'duration-unary' "$D" $'true false true false\nPT-1.5S PT30S PT30S PT0S' - -echo "OK — phase15_java_time_compare (D-462) green" diff --git a/test/e2e/phase15_java_time_dow_month.sh b/test/e2e/phase15_java_time_dow_month.sh deleted file mode 100644 index 174aa51bc..000000000 --- a/test/e2e/phase15_java_time_dow_month.sh +++ /dev/null @@ -1,125 +0,0 @@ -#!/usr/bin/env bash -# test/e2e/phase15_java_time_dow_month.sh — LocalDate/LocalDateTime getDayOfWeek / -# getMonth / getDayOfYear (D-462 / ADR-0161). DayOfWeek + Month are host-enum -# singletons (.host_instance, ordinal in state[0]; (str)/.name = enum NAME, getValue -# → ISO int; static fields DayOfWeek/MONDAY etc; getters return the SAME interned -# singleton as the constant — identical? parity). getDayOfYear returns an int (no -# enum). clj-grounded. Uses `cljw -` (stdin). Layer 2. -set -euo pipefail -cd "$(dirname "$0")/../.." -BIN="zig-out/bin/cljw" -[ -n "${CLJW_SKIP_BUILD:-}" ] || zig build -Dwasm -Doptimize="${CLJW_OPT:-ReleaseSafe}" >/dev/null -fail() { echo "FAIL $1" >&2; exit 1; } -eq() { local n="$1" g="$2" w="$3"; [[ "$g" == "$w" ]] || fail "$n: got '$g' want '$w'"; echo "PASS $n -> $w"; } -out() { "$BIN" - ; } - -# --- LocalDate getDayOfWeek / getMonth / getDayOfYear -A=$(out <<'EOF' 2>&1 -(let [d (java.time.LocalDate/of 2024 3 9)] - (println (str (.getDayOfWeek d)) (.getValue (.getDayOfWeek d)) - (str (.getMonth d)) (.getValue (.getMonth d)) - (.getDayOfYear d))) -EOF -) -eq 'localdate' "$A" 'SATURDAY 6 MARCH 3 69' - -# --- LocalDateTime getDayOfWeek / getMonth -B=$(out <<'EOF' 2>&1 -(let [dt (java.time.LocalDateTime/of 2024 12 25 10 0)] - (println (str (.getDayOfWeek dt)) (str (.getMonth dt)))) -EOF -) -eq 'localdatetime' "$B" 'WEDNESDAY DECEMBER' - -# --- enum value-= (two Saturdays equal; different days not) -C=$(out <<'EOF' 2>&1 -(println (= (.getDayOfWeek (java.time.LocalDate/of 2024 3 9)) (.getDayOfWeek (java.time.LocalDate/of 2024 3 16))) - (= (.getMonth (java.time.LocalDate/of 2024 3 9)) (.getMonth (java.time.LocalDate/of 2024 4 9)))) -EOF -) -eq 'enum-eq' "$C" 'true false' - -# --- Sunday boundary (ISO Mon=1..Sun=7); epoch day 0 = Thursday -D=$(out <<'EOF' 2>&1 -(println (str (.getDayOfWeek (java.time.LocalDate/of 1970 1 1))) ; Thursday - (str (.getDayOfWeek (java.time.LocalDate/of 2024 3 10)))) ; Sunday -EOF -) -eq 'dow-boundary' "$D" 'THURSDAY SUNDAY' - -# --- ChronoUnit enum constants: (str)=display name, (.name)=enum name, = (D-511/D-510) -E=$(out <<'EOF' 2>&1 -(println (str java.time.temporal.ChronoUnit/DAYS) - (.name java.time.temporal.ChronoUnit/DAYS) - (str java.time.temporal.ChronoUnit/HALF_DAYS) - (.name java.time.temporal.ChronoUnit/MILLENNIA) - (= java.time.temporal.ChronoUnit/DAYS java.time.temporal.ChronoUnit/DAYS) - (= java.time.temporal.ChronoUnit/DAYS java.time.temporal.ChronoUnit/WEEKS)) -EOF -) -eq 'chrono-unit' "$E" 'Days DAYS HalfDays MILLENNIA true false' - -# --- LocalDate.until(d2, ChronoUnit) — date-unit diffs; day-of-month boundary -F=$(out <<'EOF' 2>&1 -(let [a (java.time.LocalDate/of 2020 1 1) b (java.time.LocalDate/of 2021 3 15)] - (println (.until a b java.time.temporal.ChronoUnit/DAYS) - (.until a b java.time.temporal.ChronoUnit/WEEKS) - (.until a b java.time.temporal.ChronoUnit/MONTHS) - (.until a b java.time.temporal.ChronoUnit/YEARS) - ; Jan31->Mar1 is NOT a full 2 months (day-of-month boundary) - (.until (java.time.LocalDate/of 2020 1 31) (java.time.LocalDate/of 2020 3 1) java.time.temporal.ChronoUnit/MONTHS) - ; negative direction - (.until (java.time.LocalDate/of 2020 3 1) (java.time.LocalDate/of 2020 1 1) java.time.temporal.ChronoUnit/DAYS))) -EOF -) -eq 'until' "$F" '439 62 14 1 1 -60' - -# --- a time-based unit on a LocalDate raises (JVM UnsupportedTemporalTypeException) -G=$("$BIN" -e '(.until (java.time.LocalDate/of 2020 1 1) (java.time.LocalDate/of 2020 1 2) java.time.temporal.ChronoUnit/HOURS)' 2>&1 || true) -case "$G" in - *"date-based"*|*"ChronoUnit"*) echo "PASS until-unsupported -> raised" ;; - *) fail "until-unsupported: expected a date-based-unit error, got '$G'" ;; -esac - -# --- LocalDateTime.until — date units (time-of-day adjusted) + time units (D-512) -H=$(out <<'EOF' 2>&1 -(let [a (java.time.LocalDateTime/of 2020 1 1 0 0 0) b (java.time.LocalDateTime/of 2021 3 2 6 30 15)] - (println (.until a b java.time.temporal.ChronoUnit/DAYS) - (.until a b java.time.temporal.ChronoUnit/HOURS) - (.until a b java.time.temporal.ChronoUnit/MINUTES) - (.until a b java.time.temporal.ChronoUnit/SECONDS) - (.until a b java.time.temporal.ChronoUnit/MONTHS) - (.until a b java.time.temporal.ChronoUnit/YEARS))) -EOF -) -eq 'ldt-until' "$H" '426 10230 613830 36829815 14 1' - -# --- DayOfWeek/Month static-field enum constants (ADR-0161 / D-510): name / getValue -I=$(out <<'EOF' 2>&1 -(println (str java.time.DayOfWeek/MONDAY) (.getValue java.time.DayOfWeek/SUNDAY) (.name java.time.DayOfWeek/MONDAY) - (str java.time.Month/JANUARY) (.getValue java.time.Month/DECEMBER) (.name java.time.Month/MARCH)) -EOF -) -eq 'enum-const' "$I" 'MONDAY 7 MONDAY JANUARY 12 MARCH' - -# --- getter returns the SAME interned singleton as the constant (clj identical? parity) -J=$(out <<'EOF' 2>&1 -(println (= (.getDayOfWeek (java.time.LocalDate/of 2024 3 9)) java.time.DayOfWeek/SATURDAY) - (identical? (.getDayOfWeek (java.time.LocalDate/of 2024 3 9)) java.time.DayOfWeek/SATURDAY) - (= (.getMonth (java.time.LocalDate/of 2024 3 9)) java.time.Month/MARCH) - (identical? (.getMonth (java.time.LocalDate/of 2024 3 9)) java.time.Month/MARCH)) -EOF -) -eq 'enum-identity' "$J" 'true true true true' - -# --- host-enum Comparable: sign-based compare (AD-043) + sort by ordinal -K=$(out <<'EOF' 2>&1 -(println (compare java.time.DayOfWeek/MONDAY java.time.DayOfWeek/SUNDAY) - (compare java.time.Month/MARCH java.time.Month/JANUARY) - (compare java.math.RoundingMode/UP java.math.RoundingMode/DOWN) - (mapv str (sort [java.time.DayOfWeek/WEDNESDAY java.time.DayOfWeek/MONDAY java.time.DayOfWeek/FRIDAY]))) -EOF -) -eq 'enum-compare' "$K" '-1 1 -1 [MONDAY WEDNESDAY FRIDAY]' - -echo "OK — phase15_java_time_dow_month (D-462 / ADR-0161) green" diff --git a/test/e2e/phase15_java_time_duration.sh b/test/e2e/phase15_java_time_duration.sh deleted file mode 100644 index e33ddddf5..000000000 --- a/test/e2e/phase15_java_time_duration.sh +++ /dev/null @@ -1,55 +0,0 @@ -#!/usr/bin/env bash -# test/e2e/phase15_java_time_duration.sh — java.time.Duration wiring (D-462). -# Duration is a `.typed_instance` (instant_value.zig model: per-Runtime -# descriptor + TWO fields seconds + nanos, NORMALIZED so nanos∈[0,1e9) and -# seconds carries the sign). Factories ofSeconds[/nanos]/ofMillis/ofNanos/ -# ofMinutes/ofHours/ofDays; instance getSeconds/getNano/toMillis/toMinutes. -# `(str d)` = ISO-8601 duration `PT…` (the exact JDK Duration.toString), -# clj-grounded. Uses `cljw -` (stdin). Layer 2. -set -euo pipefail -cd "$(dirname "$0")/../.." -BIN="zig-out/bin/cljw" -[ -n "${CLJW_SKIP_BUILD:-}" ] || zig build -Dwasm -Doptimize="${CLJW_OPT:-ReleaseSafe}" >/dev/null -fail() { echo "FAIL $1" >&2; exit 1; } -eq() { local n="$1" g="$2" w="$3"; [[ "$g" == "$w" ]] || fail "$n: got '$g' want '$w'"; echo "PASS $n -> $w"; } -out() { "$BIN" - ; } - -# --- str: the full PT-format surface (positive, negative, fractional) -A=$(out <<'EOF' 2>&1 -(doseq [d [(java.time.Duration/ofSeconds 90) - (java.time.Duration/ofHours 2) - (java.time.Duration/ofMinutes 90) - (java.time.Duration/ofMillis 1500) - (java.time.Duration/ofSeconds 5 500000000) - (java.time.Duration/ofSeconds 0) - (java.time.Duration/ofSeconds -30) - (java.time.Duration/ofSeconds 3661) - (java.time.Duration/ofSeconds -3661) - (java.time.Duration/ofMillis -1500) - (java.time.Duration/ofSeconds -1 500000000) - (java.time.Duration/ofNanos 123456789) - (java.time.Duration/ofDays 1)]] - (println (str d))) -EOF -) -eq 'str-forms' "$A" $'PT1M30S\nPT2H\nPT1H30M\nPT1.5S\nPT5.5S\nPT0S\nPT-30S\nPT1H1M1S\nPT-1H-1M-1S\nPT-1.5S\nPT-0.5S\nPT0.123456789S\nPT24H' - -# --- readers: normalized fields + toMillis/toMinutes -B=$(out <<'EOF' 2>&1 -(let [d (java.time.Duration/ofMillis -1500)] - (println (.getSeconds d) (.getNano d) (.toMillis d))) -(let [d (java.time.Duration/ofSeconds 90)] - (println (.getSeconds d) (.getNano d) (.toMillis d) (.toMinutes d))) -EOF -) -eq 'readers' "$B" $'-2 500000000 -1500\n90 0 90000 1' - -# --- value =: by normalized seconds + nanos -C=$(out <<'EOF' 2>&1 -(println (= (java.time.Duration/ofSeconds 90) (java.time.Duration/ofMillis 90000))) -(println (= (java.time.Duration/ofSeconds 90) (java.time.Duration/ofSeconds 91))) -EOF -) -eq 'value-eq' "$C" $'true\nfalse' - -echo "OK — phase15_java_time_duration (D-462) green" diff --git a/test/e2e/phase15_java_time_duration_between.sh b/test/e2e/phase15_java_time_duration_between.sh deleted file mode 100644 index 9bcc1aa9c..000000000 --- a/test/e2e/phase15_java_time_duration_between.sh +++ /dev/null @@ -1,41 +0,0 @@ -#!/usr/bin/env bash -# test/e2e/phase15_java_time_duration_between.sh — cross-type Duration ops (D-462): -# (Duration/between a b) (elapsed time; dispatches on the temporal type — Instant/ -# LocalDateTime/LocalTime) and Instant/LocalDateTime .plus(Duration)/.minus(Duration) -# (deadlines, with carry). The two most common temporal computations. clj-grounded. -# Uses `cljw -` (stdin). Layer 2. -set -euo pipefail -cd "$(dirname "$0")/../.." -BIN="zig-out/bin/cljw" -[ -n "${CLJW_SKIP_BUILD:-}" ] || zig build -Dwasm -Doptimize="${CLJW_OPT:-ReleaseSafe}" >/dev/null -fail() { echo "FAIL $1" >&2; exit 1; } -eq() { local n="$1" g="$2" w="$3"; [[ "$g" == "$w" ]] || fail "$n: got '$g' want '$w'"; echo "PASS $n -> $w"; } -out() { "$BIN" - ; } - -# --- Duration/between (elapsed time) across the point types -A=$(out <<'EOF' 2>&1 -(println (str (java.time.Duration/between (java.time.Instant/ofEpochSecond 100) (java.time.Instant/ofEpochSecond 190))) - (str (java.time.Duration/between (java.time.Instant/ofEpochSecond 190) (java.time.Instant/ofEpochSecond 100))) - (str (java.time.Duration/between (java.time.LocalDateTime/of 2024 1 1 12 0) (java.time.LocalDateTime/of 2024 1 1 13 30))) - (str (java.time.Duration/between (java.time.LocalTime/of 6 0) (java.time.LocalTime/of 8 30)))) -EOF -) -eq 'between' "$A" 'PT1M30S PT-1M-30S PT1H30M PT2H30M' - -# --- Instant .plus/.minus (Duration) -B=$(out <<'EOF' 2>&1 -(println (str (.plus (java.time.Instant/ofEpochSecond 100) (java.time.Duration/ofSeconds 5 500000000))) - (str (.minus (java.time.Instant/ofEpochSecond 100) (java.time.Duration/ofSeconds 50)))) -EOF -) -eq 'instant-plus-dur' "$B" '1970-01-01T00:01:45.500Z 1970-01-01T00:00:50Z' - -# --- LocalDateTime .plus (Duration), with day carry -C=$(out <<'EOF' 2>&1 -(println (str (.plus (java.time.LocalDateTime/of 2024 1 1 23 0) (java.time.Duration/ofHours 2))) - (str (.minus (java.time.LocalDateTime/of 2024 1 2 1 0) (java.time.Duration/ofHours 2)))) -EOF -) -eq 'ldt-plus-dur' "$C" '2024-01-02T01:00 2024-01-01T23:00' - -echo "OK — phase15_java_time_duration_between (D-462) green" diff --git a/test/e2e/phase15_java_time_instant.sh b/test/e2e/phase15_java_time_instant.sh deleted file mode 100644 index 7b2cb1f03..000000000 --- a/test/e2e/phase15_java_time_instant.sh +++ /dev/null @@ -1,78 +0,0 @@ -#!/usr/bin/env bash -# test/e2e/phase15_java_time_instant.sh — java.time.Instant wiring (D-462). -# Instant is a `.typed_instance` (timestamp.zig/date.zig model: per-Runtime -# descriptor + epoch-ms + nanos fields; no new NaN-box tag). Static factories -# (now/ofEpochSecond/ofEpochMilli/parse) + instance methods (getEpochSecond/ -# getNano/toEpochMilli). Values are clj-grounded via `(str …)` = ISO_INSTANT -# (variable fraction + Z) and the method results — NOT pr-str (clj prints the -# opaque `#object[…]` identity form, AD). Uses `cljw -` (stdin). Layer 2. -set -euo pipefail -cd "$(dirname "$0")/../.." -BIN="zig-out/bin/cljw" -[ -n "${CLJW_SKIP_BUILD:-}" ] || zig build -Dwasm -Doptimize="${CLJW_OPT:-ReleaseSafe}" >/dev/null -fail() { echo "FAIL $1" >&2; exit 1; } -eq() { local n="$1" g="$2" w="$3"; [[ "$g" == "$w" ]] || fail "$n: got '$g' want '$w'"; echo "PASS $n -> $w"; } - -out() { "$BIN" - ; } - -# --- epoch 0: str = ISO_INSTANT (no fraction) + the three readers -A=$(out <<'EOF' 2>&1 -(let [i (java.time.Instant/ofEpochSecond 0)] - (println (str i)) - (println (.getEpochSecond i)) - (println (.getNano i)) - (println (.toEpochMilli i))) -EOF -) -eq 'epoch0' "$A" $'1970-01-01T00:00:00Z\n0\n0\n0' - -# --- seconds + nanos: str shows the millis fraction; getNano = full nanos -B=$(out <<'EOF' 2>&1 -(let [i (java.time.Instant/ofEpochSecond 1704067200 500000000)] - (println (str i)) - (println (.getNano i)) - (println (.toEpochMilli i))) -EOF -) -eq 'sec-nanos' "$B" $'2024-01-01T00:00:00.500Z\n500000000\n1704067200500' - -# --- parse (ISO-8601) round-trips to the same ISO_INSTANT str -C=$(out <<'EOF' 2>&1 -(println (str (java.time.Instant/parse "2024-01-01T00:00:00Z"))) -EOF -) -eq 'parse' "$C" '2024-01-01T00:00:00Z' - -# --- ofEpochMilli -D=$(out <<'EOF' 2>&1 -(println (str (java.time.Instant/ofEpochMilli 1704067200000))) -EOF -) -eq 'ofEpochMilli' "$D" '2024-01-01T00:00:00Z' - -# --- (class …) is the simple name (AD-003 no-JVM), and = is by value -E=$(out <<'EOF' 2>&1 -(println (= (java.time.Instant/ofEpochSecond 5) (java.time.Instant/ofEpochSecond 5))) -(println (= (java.time.Instant/ofEpochSecond 5) (java.time.Instant/ofEpochSecond 6))) -EOF -) -eq 'value-eq' "$E" $'true\nfalse' - -# --- Instant.until — fixed-duration units NANOS..DAYS (D-512) -F=$(out <<'EOF' 2>&1 -(let [a (java.time.Instant/ofEpochSecond 0) b (java.time.Instant/ofEpochSecond 3661)] - (println (.until a b java.time.temporal.ChronoUnit/SECONDS) - (.until a b java.time.temporal.ChronoUnit/MINUTES) - (.until a b java.time.temporal.ChronoUnit/HOURS) - (.until (java.time.Instant/ofEpochSecond 0) (java.time.Instant/ofEpochSecond 200000) java.time.temporal.ChronoUnit/DAYS))) -EOF -) -eq 'instant-until' "$F" '3661 61 1 2' -# a date-based unit (WEEKS) on an Instant raises (JVM UnsupportedTemporalTypeException) -G=$("$BIN" -e '(.until (java.time.Instant/ofEpochSecond 0) (java.time.Instant/ofEpochSecond 1) java.time.temporal.ChronoUnit/WEEKS)' 2>&1 || true) -case "$G" in - *"time-based"*|*"ChronoUnit"*) echo "PASS instant-until-unsupported -> raised" ;; - *) fail "instant-until-unsupported: expected a time-based-unit error, got '$G'" ;; -esac - -echo "OK — phase15_java_time_instant (D-462) green" diff --git a/test/e2e/phase15_java_time_ldt_arith.sh b/test/e2e/phase15_java_time_ldt_arith.sh deleted file mode 100644 index 5d69fb501..000000000 --- a/test/e2e/phase15_java_time_ldt_arith.sh +++ /dev/null @@ -1,35 +0,0 @@ -#!/usr/bin/env bash -# test/e2e/phase15_java_time_ldt_arith.sh — LocalDateTime arithmetic (D-462). -# plusDays/minusDays/plusWeeks/minusWeeks (epoch_day) + plusHours/plusMinutes/ -# plusSeconds/plusNanos/minus* (nano_of_day with carry/borrow into epoch_day — -# crossing midnight rolls the day). All return a LocalDateTime. clj-grounded -# incl. the midnight-crossing carry/borrow edges. Uses `cljw -`. Layer 2. -set -euo pipefail -cd "$(dirname "$0")/../.." -BIN="zig-out/bin/cljw" -[ -n "${CLJW_SKIP_BUILD:-}" ] || zig build -Dwasm -Doptimize="${CLJW_OPT:-ReleaseSafe}" >/dev/null -fail() { echo "FAIL $1" >&2; exit 1; } -eq() { local n="$1" g="$2" w="$3"; [[ "$g" == "$w" ]] || fail "$n: got '$g' want '$w'"; echo "PASS $n -> $w"; } -out() { "$BIN" - ; } - -# --- day/week (epoch_day) -A=$(out <<'EOF' 2>&1 -(println (str (.plusDays (java.time.LocalDateTime/of 2024 1 1 12 30) 40)) - (str (.minusDays (java.time.LocalDateTime/of 2024 3 1 12 0) 1)) - (str (.plusWeeks (java.time.LocalDateTime/of 2024 1 1 0 0) 2))) -EOF -) -eq 'day-week' "$A" '2024-02-10T12:30 2024-02-29T12:00 2024-01-15T00:00' - -# --- time units with midnight carry/borrow -B=$(out <<'EOF' 2>&1 -(println (str (.plusHours (java.time.LocalDateTime/of 2024 1 1 23 30) 2)) ; -> next day 01:30 - (str (.plusMinutes (java.time.LocalDateTime/of 2024 1 1 23 59) 2)) ; -> next day 00:01 - (str (.minusHours (java.time.LocalDateTime/of 2024 1 1 1 0) 2))) ; -> prev day 23:00 -(println (str (.plusSeconds (java.time.LocalDateTime/of 2024 1 1 0 0 30) 45)) - (str (.plusNanos (java.time.LocalDateTime/of 2024 1 1 12 0 0 0) 1500000000))) -EOF -) -eq 'time-carry' "$B" $'2024-01-02T01:30 2024-01-02T00:01 2023-12-31T23:00\n2024-01-01T00:01:15 2024-01-01T12:00:01.500' - -echo "OK — phase15_java_time_ldt_arith (D-462) green" diff --git a/test/e2e/phase15_java_time_ldt_calendar.sh b/test/e2e/phase15_java_time_ldt_calendar.sh deleted file mode 100644 index 14cd77460..000000000 --- a/test/e2e/phase15_java_time_ldt_calendar.sh +++ /dev/null @@ -1,31 +0,0 @@ -#!/usr/bin/env bash -# test/e2e/phase15_java_time_ldt_calendar.sh — LocalDateTime calendar arithmetic -# (D-462): plusMonths/minusMonths/plusYears/minusYears. Applies the civil -# month/year clamp to the DATE part (epoch_day, shared with LocalDate) and KEEPS -# the time-of-day (nano_of_day). clj-grounded incl. the Feb-clamp + year-rollover -# edges. Uses `cljw -` (stdin). Layer 2. -set -euo pipefail -cd "$(dirname "$0")/../.." -BIN="zig-out/bin/cljw" -[ -n "${CLJW_SKIP_BUILD:-}" ] || zig build -Dwasm -Doptimize="${CLJW_OPT:-ReleaseSafe}" >/dev/null -fail() { echo "FAIL $1" >&2; exit 1; } -eq() { local n="$1" g="$2" w="$3"; [[ "$g" == "$w" ]] || fail "$n: got '$g' want '$w'"; echo "PASS $n -> $w"; } -out() { "$BIN" - ; } - -A=$(out <<'EOF' 2>&1 -(println (str (.plusMonths (java.time.LocalDateTime/of 2024 1 31 14 30) 1)) ; Feb29 clamp, keep 14:30 - (str (.plusMonths (java.time.LocalDateTime/of 2023 1 31 9 0) 1)) ; Feb28 - (str (.plusMonths (java.time.LocalDateTime/of 2024 12 15 6 0) 2)) ; year rollover - (str (.minusMonths (java.time.LocalDateTime/of 2024 3 31 0 0) 1))) ; Feb29 -EOF -) -eq 'months' "$A" '2024-02-29T14:30 2023-02-28T09:00 2025-02-15T06:00 2024-02-29T00:00' - -B=$(out <<'EOF' 2>&1 -(println (str (.plusYears (java.time.LocalDateTime/of 2024 2 29 23 59 59) 1)) ; Feb28 clamp, keep time - (str (.minusYears (java.time.LocalDateTime/of 2024 6 15 12 0) 5))) -EOF -) -eq 'years' "$B" '2025-02-28T23:59:59 2019-06-15T12:00' - -echo "OK — phase15_java_time_ldt_calendar (D-462) green" diff --git a/test/e2e/phase15_java_time_local_date.sh b/test/e2e/phase15_java_time_local_date.sh deleted file mode 100644 index 9283094a3..000000000 --- a/test/e2e/phase15_java_time_local_date.sh +++ /dev/null @@ -1,50 +0,0 @@ -#!/usr/bin/env bash -# test/e2e/phase15_java_time_local_date.sh — java.time.LocalDate + LocalTime -# (D-462). Both are `.typed_instance` values: LocalDate = [epoch_day] (via -# daysFromCivil), LocalTime = [nano_of_day]. They are the two halves of -# LocalDateTime and share its ISO date/time format+parse helpers. Also wires -# LocalDateTime.toLocalDate / toLocalTime (return the new types). clj-grounded. -# Uses `cljw -` (stdin). Layer 2. -set -euo pipefail -cd "$(dirname "$0")/../.." -BIN="zig-out/bin/cljw" -[ -n "${CLJW_SKIP_BUILD:-}" ] || zig build -Dwasm -Doptimize="${CLJW_OPT:-ReleaseSafe}" >/dev/null -fail() { echo "FAIL $1" >&2; exit 1; } -eq() { local n="$1" g="$2" w="$3"; [[ "$g" == "$w" ]] || fail "$n: got '$g' want '$w'"; echo "PASS $n -> $w"; } -out() { "$BIN" - ; } - -# --- LocalDate: str (incl. zero-padded year), readers, parse, plusDays, = -A=$(out <<'EOF' 2>&1 -(doseq [d [(java.time.LocalDate/of 2024 1 1) (java.time.LocalDate/of 2024 12 31) (java.time.LocalDate/of 1 1 1)]] - (println (str d))) -(let [d (java.time.LocalDate/of 2024 3 9)] (println (.getYear d) (.getMonthValue d) (.getDayOfMonth d))) -(println (str (java.time.LocalDate/parse "2024-02-29"))) -(println (str (.plusDays (java.time.LocalDate/of 2024 1 1) 40))) -(println (= (java.time.LocalDate/of 2024 1 1) (java.time.LocalDate/of 2024 1 1))) -(println (= (java.time.LocalDate/of 2024 1 1) (java.time.LocalDate/of 2024 1 2))) -EOF -) -eq 'localdate' "$A" $'2024-01-01\n2024-12-31\n0001-01-01\n2024 3 9\n2024-02-29\n2024-02-10\ntrue\nfalse' - -# --- LocalTime: str (conditional sec + fraction), readers, parse, = -B=$(out <<'EOF' 2>&1 -(doseq [t [(java.time.LocalTime/of 12 30) (java.time.LocalTime/of 12 30 45) (java.time.LocalTime/of 12 30 45 500000000) (java.time.LocalTime/of 0 0)]] - (println (str t))) -(let [t (java.time.LocalTime/of 14 5 45 123456789)] (println (.getHour t) (.getMinute t) (.getSecond t) (.getNano t))) -(println (str (java.time.LocalTime/parse "06:07:08"))) -(println (= (java.time.LocalTime/of 12 30) (java.time.LocalTime/of 12 30 0))) -(println (= (java.time.LocalTime/of 12 30) (java.time.LocalTime/of 12 31))) -EOF -) -eq 'localtime' "$B" $'12:30\n12:30:45\n12:30:45.500\n00:00\n14 5 45 123456789\n06:07:08\ntrue\nfalse' - -# --- LocalDateTime.toLocalDate / toLocalTime return the new types -C=$(out <<'EOF' 2>&1 -(let [dt (java.time.LocalDateTime/of 2024 3 9 14 5 45 500000000)] - (println (str (.toLocalDate dt))) - (println (str (.toLocalTime dt)))) -EOF -) -eq 'to-local' "$C" $'2024-03-09\n14:05:45.500' - -echo "OK — phase15_java_time_local_date (D-462) green" diff --git a/test/e2e/phase15_java_time_local_date_arith.sh b/test/e2e/phase15_java_time_local_date_arith.sh deleted file mode 100644 index eae2263c7..000000000 --- a/test/e2e/phase15_java_time_local_date_arith.sh +++ /dev/null @@ -1,62 +0,0 @@ -#!/usr/bin/env bash -# test/e2e/phase15_java_time_local_date_arith.sh — LocalDate arithmetic (D-462). -# plusDays/minusDays/plusWeeks/minusWeeks (epoch_day math) + plusMonths/ -# minusMonths/plusYears/minusYears (civil math with day-clamping to month length, -# the JVM LocalDate semantics) + isLeapYear/lengthOfMonth. All return a LocalDate -# (or bool/int). clj-grounded incl. the leap/clamp edges. Uses `cljw -`. Layer 2. -set -euo pipefail -cd "$(dirname "$0")/../.." -BIN="zig-out/bin/cljw" -[ -n "${CLJW_SKIP_BUILD:-}" ] || zig build -Dwasm -Doptimize="${CLJW_OPT:-ReleaseSafe}" >/dev/null -fail() { echo "FAIL $1" >&2; exit 1; } -eq() { local n="$1" g="$2" w="$3"; [[ "$g" == "$w" ]] || fail "$n: got '$g' want '$w'"; echo "PASS $n -> $w"; } -out() { "$BIN" - ; } - -# --- day/week arithmetic (epoch_day) -A=$(out <<'EOF' 2>&1 -(println (str (.plusDays (java.time.LocalDate/of 2024 1 1) 40)) - (str (.minusDays (java.time.LocalDate/of 2024 1 1) 1)) - (str (.plusWeeks (java.time.LocalDate/of 2024 1 1) 2)) - (str (.minusWeeks (java.time.LocalDate/of 2024 1 15) 2))) -EOF -) -eq 'day-week' "$A" '2024-02-10 2023-12-31 2024-01-15 2024-01-01' - -# --- month/year arithmetic with day-clamping (civil) -B=$(out <<'EOF' 2>&1 -(println (str (.plusMonths (java.time.LocalDate/of 2024 1 31) 1)) ; -> 2024-02-29 (leap clamp) - (str (.plusMonths (java.time.LocalDate/of 2023 1 31) 1)) ; -> 2023-02-28 - (str (.plusMonths (java.time.LocalDate/of 2024 12 15) 1)) ; -> 2025-01-15 (year rollover) - (str (.minusMonths (java.time.LocalDate/of 2024 3 31) 1))) ; -> 2024-02-29 -(println (str (.plusYears (java.time.LocalDate/of 2024 2 29) 1)) ; -> 2025-02-28 (leap clamp) - (str (.minusYears (java.time.LocalDate/of 2024 6 15) 5))) ; -> 2019-06-15 -EOF -) -eq 'month-year' "$B" $'2024-02-29 2023-02-28 2025-01-15 2024-02-29\n2025-02-28 2019-06-15' - -# --- isLeapYear / lengthOfMonth -C=$(out <<'EOF' 2>&1 -(println (.isLeapYear (java.time.LocalDate/of 2024 1 1)) - (.isLeapYear (java.time.LocalDate/of 2023 1 1)) - (.isLeapYear (java.time.LocalDate/of 1900 1 1)) - (.isLeapYear (java.time.LocalDate/of 2000 1 1))) -(println (.lengthOfMonth (java.time.LocalDate/of 2024 2 1)) - (.lengthOfMonth (java.time.LocalDate/of 2023 2 1)) - (.lengthOfMonth (java.time.LocalDate/of 2024 4 1))) -EOF -) -eq 'leap-length' "$C" $'true false false true\n29 28 30' - -# --- atStartOfDay / atTime → LocalDateTime (D-462 residual 2; clj-grounded) -D=$(out <<'EOF' 2>&1 -(let [d (java.time.LocalDate/of 2026 6 20)] - (println (str (.atStartOfDay d))) ; -> 2026-06-20T00:00 - (println (str (.atTime d 13 30))) ; -> 2026-06-20T13:30 - (println (str (.atTime d 13 30 45))) ; -> 2026-06-20T13:30:45 - (println (str (.atTime d 0 0 0 500000000))) ; -> 2026-06-20T00:00:00.500 - (println (str (.toLocalDate (.atTime d 13 30))))) ; round-trip -> 2026-06-20 -EOF -) -eq 'at-time' "$D" $'2026-06-20T00:00\n2026-06-20T13:30\n2026-06-20T13:30:45\n2026-06-20T00:00:00.500\n2026-06-20' - -echo "OK — phase15_java_time_local_date_arith (D-462) green" diff --git a/test/e2e/phase15_java_time_local_date_time.sh b/test/e2e/phase15_java_time_local_date_time.sh deleted file mode 100644 index 13682d531..000000000 --- a/test/e2e/phase15_java_time_local_date_time.sh +++ /dev/null @@ -1,54 +0,0 @@ -#!/usr/bin/env bash -# test/e2e/phase15_java_time_local_date_time.sh — java.time.LocalDateTime (D-462). -# A `.typed_instance` (instant_value.zig model) carrying TWO fields: -# epoch-day (signed days since 1970-01-01, via daysFromCivil) + nano-of-day -# ([0,86400e9)). Factories of(y,m,d,h,mi[,s[,n]]) (arity 5-7), now, parse; -# instance getYear/getMonthValue/getDayOfMonth/getHour/getMinute/getSecond/ -# getNano. `(str d)` = ISO local date-time (omits :ss when sec=nano=0, variable -# fraction) — clj-grounded. Uses `cljw -` (stdin). Layer 2. -set -euo pipefail -cd "$(dirname "$0")/../.." -BIN="zig-out/bin/cljw" -[ -n "${CLJW_SKIP_BUILD:-}" ] || zig build -Dwasm -Doptimize="${CLJW_OPT:-ReleaseSafe}" >/dev/null -fail() { echo "FAIL $1" >&2; exit 1; } -eq() { local n="$1" g="$2" w="$3"; [[ "$g" == "$w" ]] || fail "$n: got '$g' want '$w'"; echo "PASS $n -> $w"; } -out() { "$BIN" - ; } - -# --- str: ISO local, conditional seconds + variable fraction -A=$(out <<'EOF' 2>&1 -(doseq [d [(java.time.LocalDateTime/of 2024 1 1 12 30) - (java.time.LocalDateTime/of 2024 1 1 12 30 45) - (java.time.LocalDateTime/of 2024 1 1 12 30 45 500000000) - (java.time.LocalDateTime/of 2024 6 15 0 0 0) - (java.time.LocalDateTime/of 2024 12 31 0 0)]] - (println (str d))) -EOF -) -eq 'str-forms' "$A" $'2024-01-01T12:30\n2024-01-01T12:30:45\n2024-01-01T12:30:45.500\n2024-06-15T00:00\n2024-12-31T00:00' - -# --- readers -B=$(out <<'EOF' 2>&1 -(let [d (java.time.LocalDateTime/of 2024 3 9 14 5 45 123456789)] - (println (.getYear d) (.getMonthValue d) (.getDayOfMonth d) - (.getHour d) (.getMinute d) (.getSecond d) (.getNano d))) -EOF -) -eq 'readers' "$B" '2024 3 9 14 5 45 123456789' - -# --- parse (with + without seconds) -C=$(out <<'EOF' 2>&1 -(println (str (java.time.LocalDateTime/parse "2024-01-01T12:30:45"))) -(println (str (java.time.LocalDateTime/parse "2024-01-01T12:30"))) -EOF -) -eq 'parse' "$C" $'2024-01-01T12:30:45\n2024-01-01T12:30' - -# --- value = -D=$(out <<'EOF' 2>&1 -(println (= (java.time.LocalDateTime/of 2024 1 1 12 30 0) (java.time.LocalDateTime/of 2024 1 1 12 30))) -(println (= (java.time.LocalDateTime/of 2024 1 1 12 30) (java.time.LocalDateTime/of 2024 1 1 12 31))) -EOF -) -eq 'value-eq' "$D" $'true\nfalse' - -echo "OK — phase15_java_time_local_date_time (D-462) green" diff --git a/test/e2e/phase15_java_time_sort.sh b/test/e2e/phase15_java_time_sort.sh deleted file mode 100644 index 2770ea514..000000000 --- a/test/e2e/phase15_java_time_sort.sh +++ /dev/null @@ -1,43 +0,0 @@ -#!/usr/bin/env bash -# test/e2e/phase15_java_time_sort.sh — (compare …)/(sort …) on java.time values -# (D-462). compare.zig grew a per-type temporal arm (mirrors the equal.zig arms), -# so temporal values are Comparable. TEMPORAL compare stays SIGN-based (clj -# returns a field-difference MAGNITUDE for LocalDate/LocalDateTime — the -# narrowed AD-043); string/char/keyword/symbol compare now returns clj's -# Java compareTo magnitude (parity, 2026-07-16). Uses `cljw -`. Layer 2. -set -euo pipefail -cd "$(dirname "$0")/../.." -BIN="zig-out/bin/cljw" -[ -n "${CLJW_SKIP_BUILD:-}" ] || zig build -Dwasm -Doptimize="${CLJW_OPT:-ReleaseSafe}" >/dev/null -fail() { echo "FAIL $1" >&2; exit 1; } -eq() { local n="$1" g="$2" w="$3"; [[ "$g" == "$w" ]] || fail "$n: got '$g' want '$w'"; echo "PASS $n -> $w"; } -out() { "$BIN" - ; } - -# --- compare returns the sign (-1/0/1) for every temporal type -A=$(out <<'EOF' 2>&1 -(println (compare (java.time.Instant/ofEpochSecond 5) (java.time.Instant/ofEpochSecond 9)) - (compare (java.time.Instant/ofEpochSecond 9) (java.time.Instant/ofEpochSecond 5)) - (compare (java.time.Instant/ofEpochSecond 5) (java.time.Instant/ofEpochSecond 5))) -(println (compare (java.time.LocalDate/of 2024 6 1) (java.time.LocalDate/of 2024 1 1)) - (compare (java.time.LocalTime/of 6 0) (java.time.LocalTime/of 18 0)) - (compare (java.time.Duration/ofSeconds 1) (java.time.Duration/ofSeconds 9)) - (compare (java.time.LocalDateTime/of 2024 1 1 6 0) (java.time.LocalDateTime/of 2024 1 1 18 0))) -EOF -) -eq 'compare-sign' "$A" $'-1 1 0\n1 -1 -1 -1' - -# --- sort works on temporal values (the headline payoff) -B=$(out <<'EOF' 2>&1 -(println (mapv str (sort [(java.time.LocalDate/of 2024 6 1) (java.time.LocalDate/of 2024 1 1) (java.time.LocalDate/of 2024 3 1)]))) -EOF -) -eq 'sort' "$B" '[2024-01-01 2024-03-01 2024-06-01]' - -# --- PARITY (was in AD-043): strings return clj's compareTo magnitude -C=$(out <<'EOF' 2>&1 -(println (compare "a" "c") (compare "abc" "ab") (compare :a :c) (compare 'aa 'ac) (compare \a \c)) -EOF -) -eq 'string-family-magnitude' "$C" '-2 1 -2 -2 -2' - -echo "OK — phase15_java_time_sort (D-462) green" diff --git a/test/e2e/phase15_literal_roundtrip.sh b/test/e2e/phase15_literal_roundtrip.sh deleted file mode 100755 index c85c7b9eb..000000000 --- a/test/e2e/phase15_literal_roundtrip.sh +++ /dev/null @@ -1,107 +0,0 @@ -#!/usr/bin/env bash -# test/e2e/phase15_literal_roundtrip.sh — literals that must survive a macro -# expansion, plus the two literal/resolution gaps the clojure.core compliance -# suite surfaced. -# -# A macro's arguments arrive as VALUES and its expansion is re-analysed as a -# form, so every value the reader can produce needs an inverse. `#uuid` and -# `#inst` had none: any macro whose body contained one killed the whole -# namespace. `resolve` looked its prefix up as a real namespace name only, so -# an aliased `str/blank?` answered nil while the var plainly existed. And a -# BigDecimal exponent was rejected outright. Layer 2. -set -euo pipefail -cd "$(dirname "$0")/../.." -BIN="zig-out/bin/cljw" -[ -n "${CLJW_SKIP_BUILD:-}" ] || zig build -Dwasm -Doptimize="${CLJW_OPT:-ReleaseSafe}" >/dev/null -fail() { echo "FAIL $1" >&2; exit 1; } -assert_eq() { local n="$1" g="$2" w="$3"; [[ "$g" == "$w" ]] || fail "$n: got '$g' want '$w'"; echo "PASS $n -> $w"; } - -MACRO='(defmacro m [& body] `(do ~@body))' - -# --- reader literals round-trip through a macro expansion --- - -assert_eq 'uuid-through-macro' \ - "$("$BIN" -e "(do $MACRO (m #uuid \"550e8400-e29b-41d4-a716-446655440000\"))" 2>&1 | tail -1)" \ - '#uuid "550e8400-e29b-41d4-a716-446655440000"' - -# the value is a uuid, not a re-read string -assert_eq 'uuid-through-macro-type' \ - "$("$BIN" -e "(do $MACRO (uuid? (m #uuid \"550e8400-e29b-41d4-a716-446655440000\")))" 2>&1 | tail -1)" \ - 'true' - -assert_eq 'inst-through-macro' \ - "$("$BIN" -e "(do $MACRO (inst? (m #inst \"2010-11-12T13:14:15.666-05:00\")))" 2>&1 | tail -1)" \ - 'true' - -# the printer and the analyzer's inverse share one rule, so a value that -# prints as a reader literal reads back as the same value -assert_eq 'inst-round-trip-equal' \ - "$("$BIN" -e "(do $MACRO (= #inst \"2010-11-12T18:14:15.666-00:00\" (m #inst \"2010-11-12T13:14:15.666-05:00\")))" 2>&1 | tail -1)" \ - 'true' - -# --- resolve follows a require alias --- - -assert_eq 'resolve-through-alias' \ - "$("$BIN" -e "(do (require '[clojure.string :as str]) (resolve 'str/blank?))" 2>&1 | tail -1)" \ - "#'clojure.string/blank?" - -assert_eq 'resolve-real-ns-still-works' \ - "$("$BIN" -e "(do (require '[clojure.string :as str]) (resolve 'clojure.string/blank?))" 2>&1 | tail -1)" \ - "#'clojure.string/blank?" - -# an unknown name under a known alias is still nil, not an error -assert_eq 'resolve-alias-miss' \ - "$("$BIN" -e "(do (require '[clojure.string :as str]) (resolve 'str/no-such-var))" 2>&1 | tail -1)" \ - 'nil' - -# ns-resolve reads the symbol's prefix in the CONTEXT of the named ns -assert_eq 'ns-resolve-through-alias' \ - "$("$BIN" -e "(do (require '[clojure.string :as str]) (ns-resolve 'user 'str/upper-case))" 2>&1 | tail -1)" \ - "#'clojure.string/upper-case" - -# --- BigDecimal exponents --- -# `× 10^exp` on a (unscaled, scale) pair is a scale shift; the digits never move - -assert_eq 'bigdec-positive-exponent' \ - "$("$BIN" -e '1e5M' 2>&1 | tail -1)" \ - '1E+5M' - -assert_eq 'bigdec-negative-exponent' \ - "$("$BIN" -e '1E-10M' 2>&1 | tail -1)" \ - '1E-10M' - -assert_eq 'bigdec-signed-mantissa-exponent' \ - "$("$BIN" -e '-1.5E-10M' 2>&1 | tail -1)" \ - '-1.5E-10M' - -assert_eq 'bigdec-exponent-value' \ - "$("$BIN" -e '(= 150000M 1.5e5M)' 2>&1 | tail -1)" \ - 'true' - -assert_eq 'bigdec-plain-unchanged' \ - "$("$BIN" -e '123.456M' 2>&1 | tail -1)" \ - '123.456M' - -# --- when-let / when-first with no body (clj returns nil either way) --- - -assert_eq 'when-let-empty-body-truthy' \ - "$("$BIN" -e '(when-let [x 1])' 2>&1 | tail -1)" \ - 'nil' - -assert_eq 'when-let-empty-body-falsey' \ - "$("$BIN" -e '(when-let [x nil])' 2>&1 | tail -1)" \ - 'nil' - -assert_eq 'when-let-body-still-runs' \ - "$("$BIN" -e '(when-let [x 5] (* x 2))' 2>&1 | tail -1)" \ - '10' - -assert_eq 'when-first-empty-body' \ - "$("$BIN" -e '(when-first [x [1 2]])' 2>&1 | tail -1)" \ - 'nil' - -assert_eq 'when-first-body-still-runs' \ - "$("$BIN" -e '(when-first [x [7 8]] (inc x))' 2>&1 | tail -1)" \ - '8' - -echo "OK — phase15_literal_roundtrip (17 cases) green" diff --git a/test/e2e/phase15_macro_heaplong.sh b/test/e2e/phase15_macro_heaplong.sh deleted file mode 100755 index e3992ea5d..000000000 --- a/test/e2e/phase15_macro_heaplong.sh +++ /dev/null @@ -1,47 +0,0 @@ -#!/usr/bin/env bash -# test/e2e/phase15_macro_heaplong.sh -# -# A macro that returns a large-magnitude i64 (|n| > the i48 fixnum window, so it -# is a heap-boxed Long — big_int tag, origin .long, D-165) must round-trip -# through macroexpansion as a Long, NOT a BigInt. valueToForm feeds the macro -# evaluator; before the fix it emitted a big_int literal for EVERY big_int tag, -# so the heap-Long came back as a BigInt and broke downstream i64-only ops -# (unsigned-bit-shift-right). This is what silently defeated test.check's -# `longify` (0x… wrapped to a negative long) even with *unchecked-math* on. - -set -euo pipefail -cd "$(dirname "$0")/../.." - -BIN="zig-out/bin/cljw" -[ -n "${CLJW_SKIP_BUILD:-}" ] || zig build -Dwasm -Doptimize="${CLJW_OPT:-ReleaseSafe}" >/dev/null - -fail() { echo "FAIL $1" >&2; exit 1; } - -check() { # check