Skip to content

gh-156333: rebuild the proactor self-pipe on EOF instead of busy-looping - #156343

Open
aidaodedjl wants to merge 1 commit into
python:mainfrom
aidaodedjl:gh-156333-proactor-self-pipe-eof
Open

gh-156333: rebuild the proactor self-pipe on EOF instead of busy-looping#156343
aidaodedjl wants to merge 1 commit into
python:mainfrom
aidaodedjl:gh-156333-proactor-self-pipe-eof

Conversation

@aidaodedjl

@aidaodedjl aidaodedjl commented Aug 25, 2026

Copy link
Copy Markdown

Summary

On Windows, BaseProactorEventLoop wakes itself through a self-pipe — a loopback TCP socketpair created by socket.socketpair(). If that connection reaches a clean EOF while the loop is running — for example the OS tears the idle loopback connection down across a power/session state change — _loop_self_reading re-armed recv() on the dead socket, which completed immediately and rescheduled the callback forever: one core pinned at 100% CPU, no exception raised, nothing logged, the process never recovers. Reproduced deterministically on main (3.16.0a0, self-built): 346,702 re-arms of _loop_self_reading during a 3-second idle sleep (a graceful loop._csock.shutdown(socket.SHUT_WR) models the OS teardown).

At EOF f.result() returns b'', which is not an exception, so control fell through to the else branch and armed a read that could never block again.

Fix: when the recv result is empty, rebuild the socketpair instead of re-arming on the dead one:

  • allocate the replacement pair first, so a failure inside the rebuild leaves the previous state untouched and the error propagates instead of being swallowed by the catch-all handler;
  • re-register signal.set_wakeup_fd() on the new socket before closing the old sockets, mirroring the ordering used by close();
  • arm the next read on the new socket, so cross-thread wakeups (call_soon_threadsafe) keep working after the rebuild.

Verification (real Windows machine, self-built 3.16.0a0 from the commit this PR is based on)

  • The issue's reproducer: 346,702 re-arms / 1.42s CPU during a 3s sleep before → 0 re-arms / 0.00s CPU after; the loop still completes cross-thread wakeups after the rebuild.
  • Full test_asyncio suite: 35/35 files, 2,625 tests, 0 failures (both new tests included).
  • Both new regression tests were verified to fail on unpatched main (git checkout of the pristine proactor_events.py, tests re-run: mock test FAIL, functional test FAIL), then pass with the patch.

Out of scope / follow-ups

  • The same OS teardown can also surface as ConnectionResetError from the pending recv instead of a clean EOF; that pre-existing path is not handled here.
  • BaseSelectorEventLoop._read_from_self (used by WindowsSelectorEventLoop) has the same EOF busy-loop shape: measured 582,692 _read_from_self calls during a 3s idle sleep on the same machine/reproducer shape. Will be reported separately.

…y-looping

When the self-pipe socketpair of a BaseProactorEventLoop reaches a clean
EOF (e.g. the OS tears the loopback connection down across a power or
session state change on Windows), _loop_self_reading re-armed recv() on
the dead socket, which completed immediately and rescheduled the callback
forever, pinning one core at 100% CPU with nothing logged.

Detect the EOF via the empty recv result and rebuild the socketpair
instead: allocate the replacement first (so a failure leaves the previous
state untouched), re-register signal.set_wakeup_fd on the new socket
before closing the old one (mirroring close()), then arm the next read on
the new socket so cross-thread wakeups keep working.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant