Repository navigation
Refresh ESP Env Hub fixes onto 11.0.0-alpha.2 and support farm C5/C61 - #31
Draft
tyeth-ai-assisted wants to merge 17 commits into
Draft
tyeth-ai-assisted wants to merge 17 commits into
tyeth-ai-assisted wants to merge 17 commits into
Conversation
With MICROPY_GC_SPLIT_HEAP_AUTO the Python heap grows into the largest free IDF block until nothing is left. The wifi driver, lwIP and NimBLE allocate at runtime (TX buffers, station association, DHCP replies, ESP-NOW frames), so a Python program that fills its heap silently breaks the radio: esp_now_send() fails with ESP_ERR_ESPNOW_NO_MEM, softAP clients associate but never get a DHCP lease (udp_sendto fails), and start_ap() can hard fault. Reserve CIRCUITPY_ESP_RADIO_HEAP_RESERVE (32 KB, board-overridable) of the IDF heap whenever the wifi radio or the BLE adapter is enabled, by capping what port_heap_get_largest_free_size() reports to the GC. Programs that never turn a radio on keep the whole heap. Measured on an ESP32-C6 devkit (no PSRAM) running a wifi+ESP-NOW collector: before, every ESP-NOW send failed 0x3067 with idf_free=7884 largest=7680 and phones stuck at "obtaining IP address" (DHCP ACK udp_sendto -> -1); after, sends succeed (idf_free ~31 KB) and phones get leases.
A server-side SSLSocket did a full handshake for every connection. Browsers open several connections per page, and on a small MCU with an RSA-2048 certificate each handshake costs ~1.3 s of CPU (ESP32-C6), so pages stalled and speculative connections were abandoned. Configure mbedTLS session tickets (AES-256-GCM via PSA, 24 h lifetime, one process-wide key) on server-side contexts when MBEDTLS_SSL_SESSION_TICKETS and MBEDTLS_SSL_TICKET_C are available (ESP-IDF enables them by default), so returning clients resume with an abbreviated handshake and no public-key operation. Client-side behaviour is unchanged.
…and bench devkits The base branch defaults CIRCUITPY_ESP_RADIO_HEAP_RESERVE to 32 KB, which is what a plain softAP + ESP-NOW workload needs. Adding a TLS server on top needs more: at 32 KB the C6 hub still hit MemoryError starting a handshake and lost DHCP replies. Override to 40 KB on the Adafruit ESP32 / ESP32-S3 / ESP32-C6 Feathers, Metro S3, QT Py ESP32 / ESP32-S3, and the two bench devkits. Other Adafruit ESP32/S3 boards (ItsyBitsy, MatrixPortal, Sparkle Motion, Qualia, cameras) are deliberately left on the 32 KB default for now. Replaces the branch's old global 58140f6 ("radio heap reserve 40KB"), which raised the port-wide default instead of overriding per board.
…argets
The ESP-NOW / WiFi / BLE environmental collector ships ~20 libraries in
lib/. On a board without PSRAM that is the difference between running and
not. Measured on a Feather ESP32-S3 No PSRAM (2026-09-13), with the
application itself already cross-compiled to .mpy:
* BLE + ESP-NOW + softAP all come up, and then `import datastore` dies
with `MemoryError: memory allocation failed, allocating 158 bytes`;
* with BLE off it reaches the eInk dashboard and the SD mount fails for
want of 832 bytes, with 15.9 KB free.
Cross-compiling the application removes the on-device *compiler* peak --
which is what made the C6 unbootable -- but not the resident bytecode.
Freezing does: a frozen module executes in place from flash and never
occupies heap.
Twenty-two libraries, on the 13 espressif boards this project targets --
the same 13 the custom-build workflow has been dispatched for, and on the
branch those builds actually used.
Nine were not yet in frozen/ and are added as submodules: IL0373, JD79667,
MAX1704x, NTP, SCD4X, SCD30, SEN6x, PCF8523, DS1307. The other thirteen
were already there.
The 22 lines are repeated verbatim in each board's mpconfigboard.mk rather
than factored into one include. That is not an aesthetic choice: tools/
ci_fetch_deps.py reads each board file line by line looking for
"FROZEN_MPY_DIRS += $(TOP)/" to decide which submodules to check out, and
it does not follow includes. The include version builds fine locally,
FREEZE lists all 22 dirs, and then CI fails at MKMANIFEST with every
frozen directory empty. Each copy of the block carries a comment saying so
before someone tidies it back.
Deliberately not frozen:
* adafruit_pixelbuf -- already a built-in C module.
* sensirion_i2c_driver / sensirion_i2c_sen5x -- a third-party circup
bundle rather than a library repo, and node_sensors.py carries a
minimal built-in SEN5x driver for boards without them.
* PCF85063A -- no Adafruit library exists; the project drives that chip
directly from its own extrtc.py.
Size: ~162 KB of frozen bytecode from 621 KB of source, uniform across all
13. Comfortable on the 8 MB and 16 MB boards; the 4 MB ones are what CI
has to answer, and if one overflows the fix is to split the list rather
than shrink it everywhere.
NOTE for whoever deploys against these builds: sys.path puts /lib ahead of
.frozen, so a copy left in lib/ SHADOWS the frozen one and the board pays
the RAM anyway. Emptying lib/ is what collects the benefit -- the same
trap as leaving a .py beside a .mpy.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
build-board-custom.yml calls .github/actions/deps/submodules without version: true, so steps.set-up-submodules.outputs.version is empty. For a board with frozen modules the mpy_cross action then runs make with CP_VERSION="" in its environment, and get_version_info_from_git() took the presence of the variable as authoritative, returned an empty tag, and makeversionhdr.py aborted with "Cannot determine version". Boards without frozen modules never run that step, which is why the Pico 2 W asset build passed while the Pico W one failed twice on the same commit. Fall back to git describe when CP_VERSION is present but empty. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The hub serves its portal over a Nordic UART service and the nodes take their config the same way, so _bleio is not optional for this project on the boards it is aimed at. Reading ports/espressif/mpconfigport.mk, it should already be on: CIRCUITPY_BLEIO_NATIVE defaults to 1 and is forced off only for the S2 and the P4 (no BLE hardware) and for 2MB flash -- not 4MB, which only loses dualbank and gains -Os. So on these four boards this is a no-op that says so out loud, and it stops a future flash-size trim taking BLE away silently the way the 2MB block already does. Boards: feather_esp32s3_4mbflash_2mbpsram, qtpy_esp32s3_4mbflash_2mbpsram, feather_esp32s3_tft, feather_esp32s3_reverse_tft. Size is not the constraint here, contrary to what a 4MB board suggests: the last green build of feather_esp32s3_4mbflash_2mbpsram used 1,841,040 of 2,883,584 bytes of firmware space, leaving 1,042,544 free -- room for this project's ~162 KB of frozen bytecode several times over.
wrap_socket(server_side=True) configured mbedTLS with VERIFY_REQUIRED whenever the context had the default root bundle attached, which every ssl.SSLContext() does. On a server that means "require a client certificate", so a plain HTTPS server -- load_cert_chain() and nothing else -- failed every handshake with MBEDTLS_ERR_SSL_NO_CLIENT_CERTIFICATE (-0x7480) and the client saw a timeout. Default server-side contexts to VERIFY_NONE, as CPython's do (CERT_NONE for servers); a context that loaded its own CA with load_verify_locations() still verifies clients. Client-side behaviour is unchanged. Seen on a Pico 2 W serving a captive portal over its softAP; the same code runs on every port using the shared mbedTLS module. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
tyeth-ai-assisted
changed the base branch from
main
to
codex/circuitpython-alpha2-base
October 22, 2026 04:36
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Update the Env Hub ESP integration to CircuitPython 11.0.0-alpha.2, retaining the radio heap reserve, TLS tickets/server policy, frozen libraries, small-board BLE configuration and CP_VERSION fix while dropping upstreamed duplicates. C5 N8R8 and C61 N8R2 gain frozen dependencies and optional UART consoles. Board-specific OTA layouts fit the images while preserving Metro TinyUF2; filesystem relocation requires backup and restoration. The nine integration variants use 16 KiB inbound / 8 KiB outbound TLS records to accommodate the complete three-certificate chain.
The
ea06dcb44echange protects complete ESP-NOW enqueue/dequeue and readiness checks across the Wi-Fi task and VM. Python allocation and exceptions remain outside the critical section. Actual-source controls reproduce partial-packet publication on the preceding revision; ten candidate test groups pass, including 20,000 concurrent packets with exact receive/drop accounting. Physical S3-to-ESP32-V2 comparisons reproduce invalid-buffer errors before the fix and pass integrity/order/accounting afterward. Broadcast loss remains; this is not a lossless-delivery claim. Source and physical comparison.Head
e46718f34badds only a documentation correction: omitted-peer sends target registered unicast peers; broadcast/multicast peers must be passed explicitly. A standalone C6 ESP-IDF probe reproduces the broadcast-only NULL error independently of CircuitPython. This is not a new RX synchronization regression. Minimal C reproduction and limits. Installed firmware remainsea06dcb44e; no binary rebuild is claimed for the documentation-only commit.Validation and attribution:
cb5301b6fb; C61 retained 30,272 application bytes of headroom. Freshea06V2/C6 builds and the Metro S3 update have their own artifact and installation evidence.ea06. C6 passes five MCU-reset startup orderings, 10,240-byte connected GATT transfer, bounded disconnect, fresh reconnect, BLE restart, five absent-SD SPI-unlock controls, DHCP and five exact HTTP responses. Concurrent AP/BLE/ESP-NOW measured 93/100 exact round trips with seven sender failures and no corrupt received payloads. It is a bare board: actual display/RTC/SCD41 and low-MTU-23 behavior are not qualified. C6 evidence.cb530, C612a7fand preceding portal assets; older standalone C5/V2 passes do not qualify the current firmware. Failed and interrupted runs remain preserved. Exact soak attribution.Keep draft for the remaining targeted coverage, including multi-sender/sustained coexistence and physical peripherals. USB inventory/load failures are tracked separately; no endpoint-capacity root cause is claimed. Unrelated pre-existing dirty submodules remain preserved. Coverage ledger.