Skip to content

microvm-worker: a failed timer start logs under the listener lock #832

Closed

Description

@ParidelPooya

Problem

  1. The MicroVM worker's listener holds self._lock while it starts its idle timer (_start_idle_time) and its grace timer (_start_grace). Every caller of both methods holds the lock.
  2. When threading.Timer.start() raises, each method logs an error through the caller's logger, with describe(error), while it still holds the lock.
  3. The caller passes that logger. safe_logger catches its exceptions, but it does not move the call outside the lock.
  4. The lock is not reentrant.
  5. So a logger that is slow blocks every hook and job request. A logger that calls back into the listener, for example a handler that calls close(), deadlocks it.

The error itself comes from threading.Timer.start(), not from a caller's client. So the case needs a failed thread start and a logger that blocks or calls the listener. #831 fixed the same pattern for the failed-SuspendMicrovm warning.

Where

packages/aws-durable-execution-sdk-python-microvm-worker/src/aws_durable_execution_sdk_python_microvm_worker/worker.py, in _start_idle_time ("could not start the idle time") and _start_grace ("could not start the end of the refusal").

Proposed fix

Record the failure under the lock, and log it after the caller releases the lock. Add a test whose logger calls listener.close() when a timer cannot start, bounded by a timeout.

Activity

  1. added a commit that references this issue on Oct 25, 2026
    ed1dbb1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions