Skip to content

tomli: add build-tomli.yml for riscv64 mypyc wheels - #308

Merged
luhenry merged 2 commits into
mainfrom
tomli
Aug 24, 2026
Merged

tomli: add build-tomli.yml for riscv64 mypyc wheels#308
luhenry merged 2 commits into
mainfrom
tomli

Conversation

@luhenry

@luhenry luhenry commented Aug 24, 2026

Copy link
Copy Markdown
Member

What

Adds .github/workflows/build-tomli.yml to build riscv64 wheels for tomli 2.4.1 and publish them to pypi.riseproject.dev.

tomli ships a pure-Python py3-none-any wheel plus mypyc-compiled binary wheels for common platforms (cp311–cp314/cp314t, x86_64/aarch64 manylinux+musllinux, Windows, macOS) — but none for riscv64. This builds the compiled riscv64 wheels so riscv64 users get the same accelerated parser everyone else does.

Ref: #94

How

Mirrors upstream's own binary-wheels job in tests.yaml:

  • Backend swap. tomli's pyproject.toml declares the flit_core backend (pure-Python only). setup.py invokes mypyc.build.mypycify() only when TOMLI_USE_MYPYC=1. So the workflow runs upstream's own scripts/use_setuptools.py on the host (rewrites [build-system]setuptools>=77.0.3 + mypy[mypyc]>=1.15), then runs cibuildwheel with TOMLI_USE_MYPYC=1 forwarded into the manylinux container via CIBW_ENVIRONMENT_PASS_LINUX.
  • Matrix [cp312, cp313, cp314, cp314t] — mypyc emits per-interpreter (non-abi3) wheels; PyPI confirms separate cpXY wheels with no -abi3- tag. (cp311 dropped per the repo's 3.12+ policy.)
  • Tests run upstream's exact wheel test: python -m unittest discover --start-directory {project}. The full toml-test corpus is committed directly at the tag (no submodules), so a plain checkout is self-contained. tomli has zero runtime and zero test deps, so no registry index env is needed.

Validation (local, aarch64 under QEMU)

  • cp312-manylinux_riscv64 full cibuildwheel build + test cycle green — 17 tests OK; cibuildwheel ran the suite from a temp cwd so import tomli correctly resolved to the installed compiled .so (not ./src).
  • cp314t-manylinux_riscv64 build green (free-threaded + mypyc).
  • ✅ Wheel is genuinely compiled (_parser loads from .cpython-312-riscv64-linux-gnu.so), auditwheel-repairs to manylinux_2_31_riscv64.manylinux_2_39_riscv64, ~286 kB, no vendored external libs.
  • python -c "import yaml; yaml.safe_load(...)" and actionlint clean (only the expected ubuntu-24.04-riscv unknown-label warning).

The publish job dry-runs on this PR branch (off main), so it's safe here.

🤖 Generated with Claude Code

tomli ships a pure-Python py3-none-any wheel plus mypyc-compiled binary
wheels for common platforms, but none for riscv64. This builds the
compiled riscv64 wheels (cp312/cp313/cp314/cp314t).

The compiled build is opt-in: upstream's pyproject.toml uses the flit_core
backend (pure-Python only), and setup.py invokes mypyc only when
TOMLI_USE_MYPYC=1. Mirror upstream's binary-wheels job: run their own
scripts/use_setuptools.py to swap the backend to setuptools + mypy[mypyc]
on the host, then cibuildwheel with TOMLI_USE_MYPYC forwarded into the
container. Tests run upstream's exact wheel test command
(python -m unittest discover); the toml-test corpus is committed at the
tag, so a plain checkout is self-contained.

Validated on aarch64 under QEMU: cp312 full build+test cycle green
(17 tests), cp314t build green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@threexc threexc left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only thing I think should be changed is removal of the comments in the body about the matrix, checkout step, python install, and build backend switch - these are the LLM explaining things in a lot of extra detail, and someone else trying to use our workflows as a reference might think we did more to customize things than is the case.

LGTM otherwise.

Address PR review: remove the verbose narration on the matrix, checkout,
Python install, and backend-switch steps. They over-explained and could
mislead someone using our workflows as a reference into thinking we
customized more than we did — this is a faithful reproduction of upstream's
binary-wheels job. Keep concise one-line "why" notes only.

Comment-only change; build logic is unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
luhenry added a commit that referenced this pull request Aug 24, 2026
Per repo guidance (PR #308 review): workflows are read as reference, so
reserve comments for the non-obvious and cut per-step narration. The full
rationale lives in the commit message and CLAUDE.md gotchas 20-22.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
luhenry added a commit that referenced this pull request Aug 24, 2026
These workflows are read as reference; verbose per-step narration makes it look
like we customized more than we did (PR #308 review). Cut the matrix/step
paragraphs, keep one why-line each for the non-obvious bits (Cython pin, the
easy_install preinstall workaround, the upstream test command).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
luhenry added a commit that referenced this pull request Aug 24, 2026
Per PR #308 review guidance: reserve comments for the non-obvious
(setuptools-rust abi3 flag, no-cp313t, musl skip), one line each; drop
the per-step paragraphs that narrated standard steps.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
luhenry added a commit that referenced this pull request Aug 24, 2026
These workflows are mined as copy-paste reference, so verbose per-step narration
makes it look like we customized more than we did (PR #308 review). Cut the
multi-line paragraphs and step narration down to single "why" lines, keeping only
the non-obvious: BUNDLED-not-vcpkg, the OpenSSL-shared and UNITY_BUILD-off riscv
deviations, registry-only deps, and the pandas test fallback.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@luhenry
luhenry merged commit 5fdc1e8 into main Aug 24, 2026
8 checks passed
@luhenry
luhenry deleted the tomli branch August 24, 2026 14:36
@luhenry luhenry linked an issue Aug 24, 2026 that may be closed by this pull request
luhenry added a commit that referenced this pull request Aug 24, 2026
* fonttools: add build-fonttools.yml for riscv64 manywheel builds

Build compiled riscv64 wheels for fonttools 4.63.0 and publish them to
pypi.riseproject.dev. fonttools is a setuptools project whose six Cython
extensions (cu2qu, qu2cu, bezierTools, momentsPen, varLib.iup, feaLib.lexer)
are compiled from pure-python sources; FONTTOOLS_WITH_CYTHON=1 forces the
compile and makes a failure hard (setup.py otherwise falls back to a pure
wheel). Build-from-checkout with cibuildwheel over the repo-standard
cp312/cp313/cp314/cp314t interpreter set, mirroring upstream's wheels.yml -
including its py-cy-noextra tox test command, which asserts the extensions
compiled and runs the full pytest suite without optional extras.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fonttools: pin cython<3.3.0 to fix qu2cu strict-annotation failures

The first CI run built compiled wheels on all four interpreters but 9 qu2cu
tests failed identically with `TypeError: Expected list, got tuple` at
qu2cu.py:237. Root cause is build-tool version drift, not a riscv64 issue:
fonttools pins Cython nowhere (setup_requires=["cython"], no [build-system]),
so the build pulled the newest Cython, 3.3.0 (released 2026-08-22). 3.3.0
enforces the `quads: List[List[Point]]` argument annotation as a strict runtime
type check; the caller passes a list-of-tuples, so the compiled extension
rejects it. Upstream's 4.63.0 wheels (sdist 2026-05-14) were built with Cython
3.2.x, which ignores the annotation.

Reproduced natively on x86 (not riscv64-specific): Cython 3.3.0 -> 9 failed;
3.2.4 and 3.2.9 -> all pass. Fix pins cython<3.3.0 and preinstalls it with
build isolation disabled, because setup.py's setup_requires fetch is an
easy_install that ignores pip specifiers - the pinned Cython must already be
importable for its has_cython path to use it (gotcha-17 preinstall shape,
applied to a build tool). Verified under QEMU: the pin resolves to 3.2.9, all
six extensions cythonize, and the wheel builds + auditwheel-repairs cleanly.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fonttools: rename matrix.build to matrix.python

Match the build-fastuuid.yml convention: matrix entries are bare interpreter
tags (cp312, ...) and the -manylinux_riscv64 suffix is appended at each use
site (job name, cibuildwheel only:, artifact name). No behavior change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fonttools: trim workflow comments to single why-lines

These workflows are read as reference; verbose per-step narration makes it look
like we customized more than we did (PR #308 review). Cut the matrix/step
paragraphs, keep one why-line each for the non-obvious bits (Cython pin, the
easy_install preinstall workaround, the upstream test command).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
luhenry added a commit that referenced this pull request Aug 24, 2026
* workflows: add build-bcrypt.yml for riscv64 wheels

bcrypt 5.0.0 is a PyO3/Rust extension built with setuptools-rust (not
maturin). The sdist bundles a committed Cargo.lock + tests/ and a plain
`python -m build --sdist` reproduces the PyPI sdist, so this uses the
sdist -> bdist shape: build the sdist once on ubuntu-latest, then feed
the tarball to cibuildwheel on the riscv64 runner.

Two build jobs mirror the two wheel shapes upstream ships:
  * build_abi3: one cp312-abi3 wheel (RISE's min Python floor), built
    once and reused+tested on cp312/cp313/cp314. setuptools-rust only
    emits abi3 when bdist_wheel gets --py-limited-api, which cibuildwheel
    does not set itself, so it is passed via CIBW_CONFIG_SETTINGS.
  * build_freethreaded: a per-interpreter cp314t wheel (abi3 has no
    free-threaded stable ABI; pyo3 auto-disables it under
    Py_GIL_DISABLED). The manylinux_2_39_riscv64 image has no cp313t, so
    cp314t is the only free-threaded target.

Rust is installed in-container via CIBW_BEFORE_ALL_LINUX (rustup); musl
is skipped (no riscv64 musl host toolchain). Each wheel runs bcrypt's
bundled pytest suite. Validated end to end under QEMU on aarch64: both
wheels build for riscv64 and all 147 tests pass (abi3 wheel loaded on
cp314, free-threaded wheel on cp314t).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* workflows: trim build-bcrypt.yml comments to single why-lines

Per PR #308 review guidance: reserve comments for the non-obvious
(setuptools-rust abi3 flag, no-cp313t, musl skip), one line each; drop
the per-step paragraphs that narrated standard steps.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Remove unused variable

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tomli riscv64 support

2 participants