Repository navigation
Conversation
Port home-assistant-chip-core (chip-core/chip-clusters/chip-repl
wheels from home-assistant-libs/chip-wheels, built on top of
project-chip/connectedhomeip) to riscv64 at version 2025.7.0, the
latest chip-wheels release.
Mirrors chip-wheels' own build.yaml: a build_prepare job runs the
x86_64-only ZAP code pre-generation step (project-chip/zap ships no
riscv64 binaries) and tars the patched, bootstrapped Matter SDK tree
for build_wheels to pick up inside the manylinux_2_39_riscv64 image.
Two one-line pigweed/connectedhomeip patches unblock the Pigweed/CIPD
client bootstrap on riscv64: cipd_setup/wrapper.py's arch_normalized()
wrongly maps "riscv64" to "amd64" via its generic endswith("64")
fallback, and its SUPPORTED_PLATFORMS tuple never listed
"linux-riscv64" to begin with, even though the CIPD backend itself
already serves a real linux-riscv64 client. gn, ninja and clang are
pulled out of the CIPD fetch entirely (gen_pigweed_cipd_json.py's
existing exclude-list mechanism, already used for rust/renode) since
fuchsia/third_party/ninja has no riscv64 CIPD build at all; gn is
built from source and ninja comes from its own riscv64 PyPI wheel,
same as this repo's build-skia-python.yml/build-skia-pathops.yml.
clang is dropped outright: nothing in BUILD.gn/python_wheel.gni
references it, and the host toolchain defaults to GCC.
chip_python_platform_tag is set to manylinux_2_39 (this repo's pinned
riscv64 image), correcting upstream's own manylinux_2_31 mislabeling
(their Linux job actually builds in a plain debian:12 container and
never runs auditwheel). auditwheel repair here vendors glib2
(libglib/libgio/libgobject-2.0, pulled in by the Linux BLE stack's
GDBus usage) and libnl3 (libnl-3/libnl-route-3, chip_minmdns_default_
policy="libnl"), both LGPL, hence the added gpl_sources job.
4727eaf to
36c3699
Compare
… on riscv64 gen_pigweed_cipd_json.py runs on both the x86_64 build_prepare job (ZAP codegen, which needs gn/ninja/clang fetched via CIPD) and the riscv64 build_wheels job (no CIPD coverage for them there). Patch 0002 added gn/gn/, fuchsia/third_party/ninja/ and fuchsia/third_party/clang/ to _LIST_OF_PACKAGES_TO_EXCLUDE unconditionally, so build_prepare lost gn/ninja/zap-cli too and failed with FileNotFoundError: 'gn' during pw_env_setup, then FAILED TO EXECUTE ZAP GENERATION: zap-cli during codepregen.py. Gate the three new exclusions on platform.machine() == 'riscv64', leaving the pre-existing rust/renode exclusions unconditional.
…on riscv64 gn's own build (build/gen.py) links with -static-libstdc++, which needs libstdc++.a. The riscv64 manylinux_2_39 image doesn't ship it by default, so the link failed with "/usr/bin/ld: cannot find -lstdc++" (gotcha 77, the same fix build-skia-python.yml/build-skia-pathops.yml already apply).
|
Root-caused the CI failure on check_run 116348086171 (run 38706111004, head Root cause: both existing patches (CIPD riscv64 recognition, excluding gn/ninja/clang from CIPD) worked as intended — the job got past bootstrap and into the "Install gn and ninja" step, which builds
Fix: added This only addresses the |
…tained before tarring Fixes PR #3279's build_wheels failure: actions/checkout's submodules:true leaves connectedhomeip/.git as a gitfile pointing at ../.git/modules/connectedhomeip in the outer chip-wheels checkout. Create Matter SDK tar packages only the connectedhomeip/ directory itself, so that gitlink's target never makes it into the tarball. build_wheels extracts the tar into a completely separate riscv64 container with no outer chip-wheels checkout at all, so every git command inside connectedhomeip/ fails with 'fatal: gitfile does not point to a valid repository' -- including pigweed's bootstrap.sh, which runs 'git submodule status --recursive' there. Move connectedhomeip's real git directory into connectedhomeip/.git itself (and drop the now-stale core.worktree it inherits) right after checkout, so it is a standalone repository before anything packages it. Verified the failure and the fix against a throwaway superproject/ submodule pair: tarring+extracting the submodule alone reproduces 'fatal: not a git repository' on the gitlink target exactly as seen in CI, and the same tar/extract round-trip succeeds once the git directory is relocated this way.
…a -C The previous commit's 'git -C connectedhomeip config --unset core.worktree' itself failed build_prepare: any git invocation resolves core.worktree as part of repository discovery before running the actual command, and the freshly-copied config's worktree setting (relative to its old location, .git/modules/connectedhomeip) now points one level too far up from connectedhomeip/.git -- 'fatal: cannot chdir to ../../../connectedhomeip: No such file or directory'. git config --file edits the given file directly with no repository discovery at all, sidestepping the chicken-and-egg problem. Verified end to end against a throwaway superproject/submodule pair: move the git directory, unset core.worktree this way, tar the submodule alone, extract it fresh elsewhere, and 'git submodule status --recursive' (what pigweed's bootstrap actually runs) succeeds.
…igweed's revision Fixes PR #3279's build_wheels failure in Bootstrap: the `gn gen` that Pigweed's env setup runs to install the Python packages failed with every target reporting "needs //build/toolchain/linux:linux_riscv64_gcc()". connectedhomeip's build/config/BUILDCONFIG.gn selects the default Linux toolchain as //build/toolchain/linux:linux_${target_cpu}_gcc, and build/toolchain/linux/BUILD.gn only defines it for x64, x86, arm and arm64 (still the case on upstream master). New patch 0003 adds a linux_riscv64_gcc toolchain mirroring linux_x64_gcc (native host gcc/g++/ar, no cross prefix; build/config/compiler has no riscv64 flags to add). It also adds riscv64 to the host_cpu list behind chip_enable_python_modules in the root BUILD.gn: without that, //src/controller/python:chip-repl is left out of the graph on a riscv64 host and gn_gen.sh's --fail-on-unused-args rejects chip_python_package_prefix. Separately, GN_REVISION was carried over from build-skia-python.yml and is much newer than the gn/gn git_revision (487f8353) the pinned Pigweed's cipd_setup/pigweed.json gives every other platform. That newer gn checks all targets for duplicated outputs and rejects connectedhomeip's graph with "Two or more targets generate the same output" on pw_protobuf_compiler's *._includes files, on x86_64 as well as riscv64, so it would have failed the same Bootstrap step right after the toolchain fix. Pin the revision Pigweed itself uses instead. Verified with gn built at both revisions from source: the bootstrap `gn gen` and gn_gen.sh's `gn gen --check --fail-on-unused-args` with this workflow's args, run on a riscv64 gn binary under qemu-user (host_cpu = "riscv64") against the pinned connectedhomeip with all three patches plus chip-wheels' own applied, both succeed, emit native gcc/g++ rules with no x86/arm flags, and name the wheel manylinux_2_39_riscv64. Without patch 0003 the graph fails exactly as in CI; with gn 0625da07 even an x86_64 host fails on the duplicate outputs.
|
Root-caused the CI failure on check_run 117319183583 (run 39000908457, head Root cause: the job got through the gn-from-source build and into Bootstrap. There, the Two more issues were behind that one, and I reproduced both locally:
Fix (
Checked locally: I built gn at both revisions and cross-built a riscv64 gn at Not covered: the actual compile and link of the CHIP SDK on riscv64. That needs the ZAP pre-generated tree and the real toolchain, so CI is the first place it runs. |
Fixes PR #3279's build_wheels failure in Bootstrap: with the GN toolchain in place, Pigweed's env setup now gets as far as building :python_packages.install, and every generate_protos.py invocation (pw_protobuf common/codegen protos, pw_protobuf_compiler test protos, pw_system device_service, pw_tokenizer options) dies with "FileNotFoundError: No such file or directory: PosixPath('protoc')". generate_protos.py runs a bare `protoc` from PATH (pigweed only passes --protoc when pw_protobuf_compiler_PROTOC_TARGET is set, and CHIP does not set it). On every other platform that protoc comes from the infra/3pp/tools/protoc CIPD package in the pinned Pigweed's cipd_setup/pigweed.json (version:3@28.2). Its platforms list stops at linux-amd64/linux-arm64, so cipd_setup/update.py silently skips it on riscv64, and the CIPD backend has no infra/3pp/tools/protoc/linux-riscv64 package at all (only linux-amd64/arm64, mac and windows builds exist). The riscv64 alternatives that are already packaged don't work: - Rocky 10 CRB's protobuf-compiler is 3.19.6. generate_protos.py always passes --experimental_editions, which 3.19.6 rejects ("Missing value for flag: --experimental_editions"). - The riscv64 grpcio-tools wheels on pypi.riseproject.dev (1.83.1, 1.84.0) bundle libprotoc 35.1. Its Python gencode requires a protobuf 7.x runtime, but CHIP's constraints.txt pins protobuf==5.28.3. Neither PyPI nor the registry has a riscv64 protoc-wheel-0 or an older grpcio-tools. So build protoc 28.2, the version Pigweed's CIPD pin uses, from the protobuf git tag with its abseil-cpp submodule. The release tarball leaves third_party/abseil-cpp empty. Only the protoc target is built, with tests, libupb and zlib off. It is installed with the well-known .proto files into /opt/protoc, the same bin/ plus include/ layout as the CIPD package, and symlinked into /usr/local/bin like gn. protoc finds <prefix>/include relative to its resolved binary, which field_options.proto and pw_tokenizer's options.proto need for google/protobuf/descriptor.proto. No patch change is needed, because CIPD already skips the package on riscv64. Verified locally with this step's exact cmake build and the two component installs. protoc 28.2 was built natively on x86_64 and cross-built for riscv64 with GCC 14. The riscv64 build was run under qemu-user. In both cases protoc was reached only as a bare `protoc` on PATH, and the five failing generate_protos.py --language python invocations from the CI log were replayed against the pinned Pigweed (3fe6ddb) sources. All five succeed and emit the _pb2.py/.pyi files, including the two that import google/protobuf/descriptor.proto, which resolves from <prefix>/include. The riscv64 and x86_64 outputs are byte-identical. Every generated module imports under protobuf==5.28.3 with mypy-protobuf 3.5.0 present, as in CHIP's venv. The same replay with protoc 3.19.6 fails on --experimental_editions. The configure also passes under CMake 4.4, matching the image's pipx cmake.
|
Root-caused the CI failure on job 117388650530 (run 39029176327, head Root cause: the GN toolchain fix worked. Bootstrap's
None of the prebuilt riscv64 options work:
Fix (
Checked locally: I built protoc 28.2 with this step's exact cmake/install commands, natively on x86_64 and cross-built for riscv64 with GCC 14 (run under qemu-user). I then replayed all five failing
Not covered: the protoc build itself on the native riscv64 runner, which should be about 424 compile steps. Also not covered is anything after Bootstrap, i.e. the CHIP compile and link. |
Fixes PR #3279's build_wheels failure in Bootstrap. With protoc in place, :python_packages.install now gets past --python_out, but the --mypy_out plugin that runs in the same protoc call fails for every proto (pw_protobuf common/codegen protos, pw_protobuf_compiler test protos, pw_system device_service, pw_tokenizer options): mypy_protobuf/extensions_pb2.py, line 23 google_dot_protobuf_dot_descriptor__pb2.FieldOptions.RegisterExtension(options) AttributeError: type object 'FieldOptions' has no attribute 'RegisterExtension' The pins match every other platform. CHIP's constraints.txt has protobuf==5.28.3 and mypy-protobuf==3.5.0, and Pigweed's pw_protobuf_compiler requires mypy-protobuf<3.6.0. The difference is which protobuf wheel gets installed. protobuf 5.28.3 ships binary (upb) wheels for manylinux x86_64/aarch64 only. A riscv64 cp312 pip falls back to protobuf-5.28.3-py3-none-any.whl, the pure-Python backend, and pypi.riseproject.dev only carries riscv64 protobuf 7.x. In 5.28.3 the pure-Python backend's python_message.py has no RegisterExtension, while the upb extension still provides it. mypy-protobuf 3.5.0's prebuilt extensions_pb2.py calls it, so protoc-gen-mypy cannot even import. Set PIP_NO_BINARY=protobuf for build_wheels. pip then builds protobuf 5.28.3 from its sdist, which compiles the same google._upb._message extension the binary wheels ship. The version stays as pinned, and mypy-protobuf's generated code is left untouched. Verified locally with CPython 3.12, the pinned Pigweed (3fe6ddb) protos and protoc 28.2. pip download for manylinux_2_39_riscv64 cp312 resolves both pins to py3-none-any wheels. A venv with exactly those wheels reproduces the CI traceback on all five protos. A venv installed with PIP_NO_BINARY=protobuf against CHIP's constraints.txt builds protobuf-5.28.3-cp312 from the sdist, reports the upb backend, and runs protoc --python_out --mypy_out cleanly on the same five protos. That holds with both the x86_64 protoc and the riscv64 protoc under qemu-user, and every generated _pb2 module imports.
|
Root-caused the CI failure on job 117446257865 (run 39048883450, head Root cause: the protoc build worked. The versions are the same as on every other platform. CHIP's
Fix ( Checked locally with CPython 3.12, the pinned Pigweed (
Not covered: the sdist's C extension build on the native riscv64 runner. Also not covered is anything after Bootstrap, i.e. the CHIP compile and link. |
build_wheels downloaded a ~4.1 GB tar of the whole bootstrapped Matter SDK from build_prepare. The riscv64 runners pull Actions artifacts from Azure blob storage at ~1 MB/s, in a single stream that actions/download-artifact restarts from byte zero on any stall, so the download took 62-68 minutes when it worked and failed after ~60 minutes twice in a row. The tar grew from 1.8 GB to 4.1 GB once connectedhomeip's full-history git directory was copied into it. Only the ZAP pre-generation needs x86_64. build_wheels now checks out chip-wheels and the connectedhomeip submodules itself and applies both patch sets, which takes minutes from GitHub on these runners, and only zzz_pregenerated/ crosses jobs. Since the outer chip-wheels checkout is present again, the self-contained-submodule workaround is gone, and chip_project_config_include_dirs=["//.."] finds chip-wheels' CHIPProjectConfig.h as it does upstream.
Fixes PR #3279's build_wheels failure in :python_packages.install (run 39073160824): Collecting cryptography (from pigweed==0.0.21+...) Using cached cryptography-43.0.0.tar.gz (686 kB) BackendUnavailable: ... ModuleNotFoundError: No module named 'maturin' Pigweed's pip_install_pigweed_package target (pw_python_pip_install in pw_build/python_dist.gni) always runs pip with --no-build-isolation, so any dependency of the generated `pigweed` package that has no wheel for the host is built inside the Pigweed venv, with only what that venv holds. pw_tls_client and pw_software_update require cryptography, which CHIP's constraints.txt pins to 43.0.0. That release has no riscv64 wheel on PyPI, so pip took its sdist, whose maturin backend isn't in the venv. Installing maturin there would not be enough either: the build also needs a Rust toolchain the manylinux image doesn't ship. pypi.riseproject.dev carries cryptography 43.0.1 for riscv64 (cp39-abi3, manylinux_2_35). 43.0.1 only rebuilds the upstream wheels against OpenSSL 3.3.2, with no code change. Patch 0004 bumps the constraint to 43.0.1, and build_wheels sets PIP_EXTRA_INDEX_URL to the registry plus PIP_ONLY_BINARY=cryptography,cffi. With that, pip installs the wheel instead of building, and so does cffi, whose cp312 riscv64 wheel is also only on the registry and whose sdist would need libffi-devel. Checked with a riscv64 cp312 dry-run resolve (PyPI plus the registry, patched constraints) over the install_requires of every Pigweed setup.cfg. cryptography is the only package in that closure with a non-setuptools native build backend. The remaining sdist-only pins (psutil 5.9.4, websockets 10.4, markupsafe 2.1.2, watchdog 2.3.1, docopt) are setuptools builds, which work without build isolation.
Fixes PR #3279's build_wheels failure in :python_packages.install (run 39082118553): ERROR: Cannot install pigweed==0.0.21+20261018014909 because these package versions have conflicting dependencies. pigweed 0.0.21+20261018014909 depends on cryptography The user requested (constraint) cryptography==43.0.1 This is not a version-range conflict. pw_tls_client/py/setup.cfg, the only Pigweed package that needs cryptography at this submodule revision (3fe6ddbf82e6), asks for a bare `cryptography`, with no specifier. pip found no installable candidate for cryptography==43.0.1 at all. Pigweed's virtualenv setup upgrades the bootstrap venv's pip under CHIP's constraints.txt, which pins pip==23.2.1. That pip vendors packaging 21.3, whose manylinux arch allowlist has no riscv64, so it reports only linux_riscv64 tags (gotchas 234/640). It therefore rejects pypi.riseproject.dev's cryptography-43.0.1-cp39-abi3-manylinux_2_35_riscv64 wheel. PIP_ONLY_BINARY=cryptography rules out the sdist, so no candidate is left. Patch 0005 pins pip==24.3.1, the version this Pigweed revision's own python_base_requirements.txt pins and already installs in the GN build venv. pip 24.1 is the first release that knows riscv64 manylinux tags. Checked with a cp312 venv made to report linux-riscv64 (glibc 2.39, the same glibc as manylinux_2_39), against PyPI plus the registry, with build_wheels' PIP_* settings. The resolve covered every third-party install_requires of Pigweed's setup.cfg/pyproject.toml files plus requirements.build.txt: - pip 23.2.1 lists 27 riscv64 tags, all linux_riscv64. The resolve fails with the same ResolutionImpossible as CI. - pip 24.3.1 lists 675 tags, headed by cp312-cp312-manylinux_2_39_riscv64. The resolve succeeds with cryptography 43.0.1 and cffi 2.1.1, both from registry wheels. The only sdists left are setuptools builds (MarkupSafe, docopt, protobuf via PIP_NO_BINARY, psutil, watchdog, websockets). - On x86_64, build_prepare's requirements.all.txt plus the Pigweed set resolve under pip 24.3.1 just as they do under 23.2.1. - pip-tools 7.0.0, still pinned by constraints.txt, installs alongside pip 24.3.1 (it only requires pip>=22.2).
… pygobject Fixes build_wheels' pygobject==3.50.0 build (a home-assistant-chip-repl dependency): meson's cairo-gobject pkg-config/cmake probe reports NO, and pygi-foreign-cairo.c's #include <cairo-gobject.h> then fails with "No such file or directory". cairo-devel alone doesn't ship this header or its .pc file; they're in the separate cairo-gobject-devel package.
Fixes the publish job of run 39097039954, where _publish-wheel.yml's release step stopped with "Wheels contain mixed projects or versions": the wheelhouse held home_assistant_chip_core plus the home_assistant_chip_clusters and home_assistant_chip_repl wheels the CHIP build also emits. The publish workflow and update_doc.py publish exactly one project per run, and docs/packages/ only declares home-assistant-chip-core. Upstream already ships clusters and repl 2025.7.0 on PyPI as pure-Python py3-none-any wheels (Root-Is-Purelib, no extension module), so riscv64 installs them from PyPI with no port needed. Keep building and installing them for the unit tests, but stop copying them into the wheelhouse.
home-assistant-chip-core2025.7.0Builds the Matter (CHIP) Python device controller's native bindings (chip-core) from home-assistant-libs/chip-wheels, which wraps project-chip/connectedhomeip's Pigweed/GN build. Upstream publishes no riscv64 wheel.
Mirrors chip-wheels' build.yaml.
Differs from upstream
zzz_pregenerated/) goes from build_prepare (x86_64, since ZAP has no riscv64 build) to build_wheels, which checks out chip-wheels and the connectedhomeip submodules and applies the patches itself - upstream hands over a tar of the whole bootstrapped SDK (~4 GB here), and the riscv64 runners download Actions artifacts at ~1 MB/s in one stream that restarts from zero on any stall, so that transfer took over an hour and failed when the stream brokepigweed.jsonpins for CIPD) and ninja from its PyPI wheel - fuchsia/third_party/ninja has no riscv64 CIPD buildinfra/3pp/tools/protocversion Pigweed'spigweed.jsonpins for CIPD - that package has no riscv64 CIPD build, CRB's protoc 3.19.6 rejects--experimental_editions, and the riscv64 grpcio-tools bundles protoc 35, whose output won't load under CHIP's protobuf 5.28.3PIP_NO_BINARY=protobufin build_wheels - protobuf 5.28.3 has no riscv64 binary wheel, so pip installs its pure-Python wheel, which lacks theRegisterExtensionthat the pinned mypy-protobuf 3.5.0 calls; the sdist builds the same upb backend the other platforms usePIP_EXTRA_INDEX_URL+PIP_ONLY_BINARY=cryptography,cffiin build_wheels) - Pigweed pip-installs itspigweedpackage with--no-build-isolation, and CHIP's pinned cryptography 43.0.0 has no riscv64 wheel, so pip built its sdist inside the Pigweed venv, which has no maturin (No module named 'maturin') and no Rust toolchain; 43.0.1 only rebuilds the upstream wheels against OpenSSL 3.3.2, and our registry ships it (plus cffi's cp312 wheel) for riscv64cryptography-43.0.1-cp39-abi3-manylinux_2_35_riscv64wheel. With the sdist excluded,pigweed's barecryptographyrequirement had no candidate (ResolutionImpossibleagainst thecryptography==43.0.1constraint). 24.3.1 is the version Pigweed's ownpython_base_requirements.txtpins and already uses in the GN build venvInstall system dependenciesalso installscairo-gobject-devel-home-assistant-chip-repl'spygobject==3.50.0dependency needscairo-gobject.hfor its Cairo integration (pygi-foreign-cairo.c);cairo-develalone doesn't ship that headercipd_setup/wrapper.pypatched for linux-riscv64 - it never recognized riscv64 at alllinux_riscv64_gcctoolchain and enable the Python modules on riscv64 hosts - it only knows x64/x86/arm/arm64chip_python_platform_tagset tomanylinux_2_39(this repo's image) - upstream's ownmanylinux_2_31is mislabeled; their job never runs auditwheelhome-assistant-chip-coreis published - the build also emitshome-assistant-chip-clustersandhome-assistant-chip-repl, which the unit tests still install, but upstream already ships both 2025.7.0 on PyPI as pure-Pythonpy3-none-anywheels (no extension module), so riscv64 installs them from PyPI as is._publish-wheel.ymlalso publishes one project per run and stopped withWheels contain mixed projects or versionswhile they were in the wheelhouseauditwheel repair+ agpl_sourcesjob - vendors glib2/libnl3 (LGPL) that upstream's plaindebian:12container ships unauditedMatrix: single cp37-abi3 build, same as upstream - GN emits one abi3 wheel directly, no cp312/13/14 split.
Testing
License: Wheel vendors glib2 (LGPL-2.1) and libnl3 (LGPL-2.1) via auditwheel;
gpl_sourcescarries their source RPMs.Patches
0001-pigweed-cipd_setup-wrapper-support-linux-riscv64.patch- To upstream. Without it, the CIPD client bootstrap maps riscv64 to amd64 and then rejects it as unsupported.0002-scripts-setup-exclude-gn-ninja-clang-from-cipd.patch- Inappropriate [riscv64 CI infra choice]. Without it, bootstrap tries to CIPD-fetch fuchsia/third_party/ninja, which has no riscv64 build at all.0003-build-support-native-linux-riscv64-hosts.patch- To upstream. Without it,gn genfails on every target withneeds //build/toolchain/linux:linux_riscv64_gcc(), andchip-replis left out of the graph on a riscv64 host.0004-scripts-setup-constrain-cryptography-to-43.0.1.patch- Inappropriate [riscv64 wheel availability]. Without it,:python_packages.installbuilds cryptography 43.0.0 from sdist without build isolation and fails withBackendUnavailable: No module named 'maturin'.0005-scripts-setup-constrain-pip-to-24.3.1.patch- Inappropriate [riscv64 wheel availability]. Without it, the bootstrap venv's pip 23.2.1 can't install anymanylinux_*_riscv64wheel, and:python_packages.installfails withResolutionImpossible: pigweed depends on cryptography / The user requested (constraint) cryptography==43.0.1.Run 39097039954 completed
build_wheels(the full GN/Ninja build and unit tests) andgpl_sources; itspublishjob then failed on the clusters/repl wheels, which the wheelhouse no longer carries.