Skip to content

DO NOT MERGE - synthio/audiopwmio-esp32s2-funhouse - #39

Draft
tyeth wants to merge 8 commits into
base/upstream-main-cfe5624efrom
uat/synthio-audiopwmio-esp32s2-funhouse
Draft

tyeth wants to merge 8 commits into
base/upstream-main-cfe5624efrom
uat/synthio-audiopwmio-esp32s2-funhouse

Conversation

@tyeth

@tyeth tyeth commented Oct 23, 2026 •

Copy link
Copy Markdown
Owner

Draft, for CI and hardware UAT only. Do not merge.

This branch exists to run CI and to produce a single FunHouse firmware with all the in-flight work combined. Every commit here is already up for review individually — review those, not this.

What is on it

Base: cfe5624e47 (upstream main, past 11.0.0-alpha.2).

commit reviewed in
espressif: add audiopwmio.PWMAudioOut for the ESP32-S2 #33
synthio: stop MidiTrack treating every 0xF. byte as End of Track #34
synthio: support running status in MidiTrack #35
synthio: treat a Note On of velocity 0 as a Note Off in MidiTrack #36
synthio: keep MidiTrack's timing accurate for long delays #37
synthio: follow the MIDI file's tempo map in MidiTrack #38

Build

adafruit_funhouse builds clean: 1,477,440 bytes flash used, 1,406,144 free; 217,256 bytes internal SRAM.

Status

Not yet working on hardware. The audiopwmio port stalled on a real FunHouse — playing stayed true with no audio and playback never completed. The cause was found by inspection (the sample refill was queued only from the ISR's buffer-exhaustion path, so a tick that found no ready buffer queued nothing and wedged permanently; and the queueing call is taskENTER_CRITICAL(), illegal from an ISR). That is fixed here, along with return-checking on all five gptimer_* calls so a timer that will not configure raises RuntimeError from play() instead of hanging. This has not been confirmed on hardware.

The synthio changes are verified against an independent MIDI parser on the host and build clean on arm and xtensa, but have also not been run on a device.

🤖 Generated with Claude Code

tyeth and others added 7 commits October 23, 2026 01:29
The espressif port had no audiopwmio at all, so boards whose speaker is
not wired to a DAC pin had no way to play samples. On the Adafruit
FunHouse the piezo is on GPIO42, while the ESP32-S2 DAC is hardwired to
GPIO17/GPIO18 (and GPIO18 is the light sensor), so audioio cannot reach
it.

LEDC has no DMA path for its duty registers, and the S2's I2S has no PDM
transmitter (no SOC_I2S_SUPPORTS_PDM_TX), so samples are pushed from a
GPTimer alarm ISR running at the sample rate. The ISR drains one of two
device buffers; audiopwmout_background(), called from
port_background_task(), refills the other from the audiosample protocol.

The ISR deliberately touches nothing but its own object's volatile state.
It must not queue a background callback: on this port that macro is
taskENTER_CRITICAL(), which aborts when called from ISR context. Driving
the refill by polling instead also means a buffer the ISR found empty is
always picked up again, rather than relying on the ISR to have queued the
refill before it stalled.

Every gptimer call is checked, so a timer that cannot be configured
raises instead of leaving `playing` true forever. alarm_config is static
because CircuitPython builds with CONFIG_GPTIMER_CTRL_FUNC_IN_IRAM, under
which gptimer_set_alarm_action() rejects a config outside internal RAM.

The carrier is APB_CLK / 2**10 = 78.125 kHz, which keeps it out of the
audio band while leaving 10 bits of amplitude resolution. The GPTimer
runs at 40 MHz (the finest the timer group divider allows), putting the
sample-rate error below 0.01% at 8k/11k/22k/44.1k.

A single-buffer sample that loops wraps inside the ISR rather than
round-tripping through the background task, so looping RawSamples do not
click.

ledc_ll.h lives in the per-target esp_hal_ledc directory, which was not
on the include path, so add it.

Enabled for esp32s2 only; the S3 has neither DAC nor audiopwmio and could
reuse this later. Costs 2336 bytes of flash and 648 bytes of internal
SRAM on adafruit_funhouse, and nothing on boards that leave it off.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
gptimer_register_event_callbacks() returned ESP_ERR_INVALID_ARG on the
FunHouse, so the alarm callback was never installed: the timer ran, no
alarm was ever delivered, nothing consumed samples, and play() left
`playing` true forever.

The driver is built with CONFIG_GPTIMER_ISR_CACHE_SAFE (selected by
CONFIG_GPTIMER_ISR_HANDLER_IN_IRAM, which defaults on), so
gptimer_register_event_callbacks() requires the callback in IRAM and its
user context in internal RAM. The callback was fine; the user context was
not. It was `self`, a MicroPython heap object, and port_malloc() takes
MALLOC_CAP_SPIRAM first, so on any board with PSRAM the whole object is
external.

Clearing that check alone would not be enough. A cache-safe ISR can run
while the flash cache is disabled, which happens during any flash write,
including the user saving code.py, so everything it dereferences has to be
in internal RAM. Follow the pattern common-hal/frequencyio already uses:
put the ISR-reachable state in an audiopwmio_internal_data_t allocated
with MALLOC_CAP_INTERNAL and pass that as the user context. The LEDC
channel numbers are copied in too, so the ISR no longer reaches into the
pwmio objects. The audio buffers move to heap_caps_malloc(MALLOC_CAP_
INTERNAL) for the same reason, capped so a large sample fails with
ESP_ERR_NO_MEM rather than exhausting internal RAM. The refill path still
runs in task context and reaches the object as before.

Buffers now hold one value per output channel rather than always a stereo
pair, which halves the internal RAM a mono output needs.

play() asserts esp_ptr_in_iram() for the callback and esp_ptr_internal()
for the control block and both buffers, so a regression surfaces there
instead of as a fault at a random later moment. Error paths report the
underlying esp_err_t through raise_esp_error() instead of a generic
message.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The decoder read any byte with a high nibble of 15 as an "End of Track"
event and stopped there. A MIDI track conventionally opens with a Set
Tempo meta event, which begins 0xFF, so for most real files the track
ended before a single note was played and the sample was silent.

Meta and System Exclusive events both carry an explicit length, so skip
exactly that many bytes instead of guessing, and end the track only on an
actual End of Track meta event. Decoding the first 30 seconds of a stock
Bach Toccata now yields 479 and 423 note events on its two note tracks,
where it previously yielded 2 and none.

Also report error_location as the offset where the error was found rather
than the end of the track it was abandoned at.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A MIDI file may omit the status byte when it repeats the previous one, so
a run of notes on the same channel is encoded as data bytes alone. The
decoder read each of those as a status byte, took the invalid-event branch
and ended the track.

Remember the last channel status byte and imply it when the next event
begins with a data byte. System messages cancel it, as the specification
requires.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The MIDI specification lets a Note On with a velocity of zero stand in for
a Note Off, and files commonly use it so that a run of notes can share one
status byte. MidiTrack ignored velocity entirely, so such a note was
started and never stopped, and the voice sounded until the track ended.

Read the velocity that was already being parsed and discarded, and release
the note when it is zero.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
span.dur is a uint16_t, so a delay longer than 65535 samples wrapped: at
44.1kHz that is any gap beyond 1.5 seconds, and a held note or a rest of
ordinary length came out far too short. The conversion overflowed too,
because delta * sample_rate does not fit in 32 bits for a long delay.

Compute the duration in 64 bits and carry whatever does not fit in
span.dur in a separate field, handing it over a chunk at a time.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
MidiTrack played at one fixed rate for the whole track, so a file that
changes tempo, or that simply does not use the 120 BPM default, played at
the wrong speed throughout. The Set Tempo meta event was skipped.

Act on Set Tempo. Its value is in microseconds per quarter note, which can
only be converted to the ticks per second MidiTrack works in if the ticks
per quarter note is known, so take that as a new optional ticks_per_beat
argument. from_file() reads it from the file header; it is left at 0 for
SMPTE timing, where the rate is already absolute and Set Tempo does not
apply, and for callers that pass raw track bytes without it.

The tempo is rewound along with the track when the sample restarts, so a
looping track does not inherit the rate its last loop ended on.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@tyeth

tyeth commented Oct 23, 2026

Copy link
Copy Markdown
Owner Author

Rebuilt on the internal-RAM fix (1f6a9ddbb3, #33).

The first hardware run failed at gptimer_register_event_callbacks with ESP_ERR_INVALID_ARG (0x102). CONFIG_GPTIMER_ISR_CACHE_SAFE is enabled in this build, which requires the callback to be in IRAM and user_data to be in internal RAM; the driver passed self, a MicroPython heap object, and port_malloc() allocates that heap from PSRAM on a 2MB board. All ISR-reachable state now lives in an internal-RAM control block, following the common-hal/frequencyio pattern.

Branch is now 7 commits on cfe5624e47. Builds clean for adafruit_funhouse: 1,477,824 bytes flash used, 1,405,760 free; 217,224 bytes internal SRAM, 77,688 free.

Still unverified on hardware.

@tyeth
tyeth force-pushed the uat/synthio-audiopwmio-esp32s2-funhouse branch from d1c71d0 to 68c4509 Compare October 23, 2026 01:04
@tyeth

tyeth commented Oct 23, 2026

Copy link
Copy Markdown
Owner Author

UAT passed. The combined build works on an Adafruit FunHouse: 440 Hz test tone, then a MIDI track played through audiopwmio on the onboard piezo to completion, clean decode (error_location: None).

Both halves of this branch are therefore exercised together on real hardware. Still a draft and still not for merge — the individual PRs #33–#38 are the reviewable units.

@tyeth

tyeth commented Oct 23, 2026 •

Copy link
Copy Markdown
Owner Author

UF2 for funhouse, with halloween midi file played through synthio using an audiopwmio output, loading the file after fixing a bit of midi from_file stuff.

funhouse_halloween_audiopwmio.zip

Code

# SPDX-FileCopyrightText: Copyright (c) 2026 Tyeth Gundry for Adafruit Industries
# SPDX-License-Identifier: MIT
"""FunHouse: 440 Hz test, then a polyphonic MIDI file via audiopwmio + synthio.

REQUIRES the custom firmware that adds audiopwmio.PWMAudioOut to the ESP32-S2.
Stock CircuitPython has no audiopwmio on espressif and this will raise
ImportError -- use the pwmio/adafruit_midi_parser version instead.

Unlike the piezo/pwmio version this is POLYPHONIC: synthio renders up to 12
voices (CIRCUITPY_SYNTHIO_MAX_CHANNELS on esp32s2) into a sample stream, which
audiopwmio pushes out of GPIO42 on a 78 kHz carrier.

The .mid must be prepared for synthio.from_file(), which is strict:
  * format 0, exactly one track
  * no running status, and no meta events at all (a tempo event is read as
    end-of-track and silently stops playback)
  * tempo is fixed at 2*division ticks/sec, so the real tempo must be baked
    into the delta times
Use make_synthio_midi.py to convert; a normal .mid will not work.
"""

import array
import math
import time

import audiocore
import audiopwmio
import board
import synthio
import time
time.sleep(3)

MIDI_FILE = "/melody.mid"
SAMPLE_RATE = 11025

dac = audiopwmio.PWMAudioOut(board.SPEAKER)

# --- 1. 440 Hz sine --------------------------------------------------------
length = 8000 // 440
sine_wave = array.array("H", [0] * length)
for i in range(length):
    sine_wave[i] = int(math.sin(math.pi * 2 * i / length) * (2**15) + 2**15)

print("440 Hz test tone")
dac.play(audiocore.RawSample(sine_wave, sample_rate=8000), loop=True)
time.sleep(1)
dac.stop()
time.sleep(0.5)

# --- 2. MIDI through synthio ----------------------------------------------
# Single-cycle waveform for the voices. Must be signed 16-bit ('h').
# Delete the waveform= argument for synthio's default 50% square, which is
# closer to a chip-organ and arguably more fitting for a toccata.
WAVE_LENGTH = 128
waveform = array.array("h", [0] * WAVE_LENGTH)
for i in range(WAVE_LENGTH):
    waveform[i] = int(math.sin(math.pi * 2 * i / WAVE_LENGTH) * (2**15 - 1))

envelope = synthio.Envelope(
    attack_time=0.03,
    decay_time=0.05,
    release_time=0.2,
    attack_level=1.0,
    sustain_level=0.8,
)

try:
    midi_data = open(MIDI_FILE, "rb")
except OSError:
    print("no", MIDI_FILE, "- nothing to play")
else:
    midi = synthio.from_file(
        midi_data,
        sample_rate=SAMPLE_RATE,
        waveform=waveform,
        envelope=envelope,
    )
    print("playing", MIDI_FILE, "at", midi.tempo, "ticks/sec")
    dac.play(midi)
    while dac.playing:
        pass
    # -1 means the track decoded cleanly; anything else is the byte offset
    # where MidiTrack gave up (usually running status or a meta event).
    print("error_location:", midi.error_location)
    midi.deinit()

print("stopped")
dac.deinit()

zephyr-cp builds audiomp3 and ulab, but its PORT_DEPS left out both
submodules. Board builds only worked when the scheduler job's
submodules-common cache restored; with a cold cache they fail on
lib/mp3/src/mp3common.h.

Fixes adafruit#11655

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tyeth-ai-assisted

Copy link
Copy Markdown
Collaborator

The zephyr-cp board failures here weren't caused by this PR. zephyr-cp's CI dependency list doesn't fetch lib/mp3 or extmod/ulab, and upstream normally gets them from a submodule cache hit, which this run missed. Tracked in adafruit#11655 and fixed by adafruit#11656. That fix is cherry-picked here (c8554c8), so this run should build those boards.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants