gh-156344: rebuild the selector self-pipe on EOF instead of busy-looping - #156345
Open
aidaodedjl wants to merge 2 commits into
Open
gh-156344: rebuild the selector self-pipe on EOF instead of busy-looping#156345aidaodedjl wants to merge 2 commits into
aidaodedjl wants to merge 2 commits into
Conversation
…y-looping When the self-pipe socketpair of a BaseSelectorEventLoop reaches a clean EOF (e.g. the OS tears the connection down across a power or session state change on Windows), _read_from_self broke out of its read loop but left the reader registered on the dead socket. A closed-for-read socket is permanently readable, so every select() iteration re-fired the callback: one core pinned at 100% CPU with nothing logged, measured at 582k callback invocations during a 3-second idle sleep. Rebuild the pair instead: allocate the replacement before touching the old sockets so an allocation failure leaves the previous state intact, move the process-wide signal wakeup fd to the new socket when (and only when) it is registered on our _csock -- restoring foreign registrations untouched, and keeping the old write end open when it cannot be moved from a worker thread -- then remove the old reader, close the old sockets, and register the reader on the new socket.
aidaodedjl
requested review from
1st1,
asvetlov,
kumaraditya303 and
willingc
as code owners
August 25, 2026 05:30
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Root cause
BaseSelectorEventLoopwakes itself through a socketpair created in_make_self_pipe(). When that pair reaches a clean EOF — on Windows thepair is a loopback TCP connection, and the OS can tear it down across
power or session state changes (sleep/hibernate, RDP disconnect, fast
user switching) —
_read_from_self()breaks out of its drain loop:but the reader stays registered on the dead
_ssock. A peer-closedsocket is permanently readable, so every
select()iteration re-firesthe callback: one core pinned at 100% CPU with nothing logged,
measured at 582,692
_read_from_selfinvocations during a 3-secondidle sleep (1.23s CPU). The loop never recovers.
This is the selector-side twin of gh-156333 (proactor loop, fixed in
#156343); the reporter of that issue hit the same teardown on a
WindowsSelectorEventLoopat 3.13.15.The fix
_rebuild_self_pipe(), called from_read_from_self()on clean EOF,mirrors the proactor-side approach with selector-specific care:
socket.socketpair(), both endsnon-blocking. An allocation failure leaves the previous state intact.
signal.set_wakeup_fd(-1)returns the previously registered fd; movethe registration to the new
_csockonly when it was pointed at ourold
_csock. A foreign registration (another loop's self-pipe, atest fixture) is restored untouched via
set_wakeup_fd(old_fd).set_wakeup_fdonly works on themain thread; a rebuild triggered from a worker thread keeps the old
_csockopen so signal delivery keeps working (one socket leakedbeats silently breaking Python's signal handling), and skips the
wakeup-fd move for the same reason.
sockets (old
_csockclosed only when it was safe to move), thenregister the reader on the new
_ssock— restoring thecross-thread wakeup invariant
_make_self_pipe()established.ConnectionResetErrorstill propagates to the caller unchanged — thatis
sock_recv's documented contract and the loop's error-handling path(
call_exception_handler) remains the right place for it.Tests
Six new tests in
test_selector_events.py:test_read_from_self_eof_rebuilds_self_pipetest_read_from_self_blocking_is_not_eofBlockingIOErrordoes NOT trigger a rebuild (guard)test_self_pipe_eof_rebuild_functional_csock.shutdown()→ rebuild fires exactly once, reader lands on the new fd, cross-threadcall_soon_threadsafestill wakes the looptest_rebuild_self_pipe_moves_wakeup_fd_csockis migrated to the new onetest_rebuild_self_pipe_leaves_foreign_wakeup_fdtest_rebuild_self_pipe_no_signalsset_wakeup_fdnever calledAll six fail on the pre-fix code (verified by checking out the original
selector_events.py); after the fix the fulltest_asynciosuitepasses on Windows (34/34 files, incl.
test_selector_eventsandtest_windows_events) and on macOS.Measured on the Windows repro from the issue: 582,692 callback
invocations / 1.23s CPU during 3s idle → 1 invocation (the rebuild
itself) / 0.00s CPU, with cross-thread wakeup intact.