Skip to content

Make stale worker.pid reclaim atomic so two workers cannot own a channel #166

Description

@iamnbutler

Code-review finding, not reproduced at runtime. An independent static review of #159 found this. The algorithm predates #159: #159 moved it unchanged from apps/host/src/worker.ts into catalog.lock.

catalog.lock at a476c32 (

export function lock(id: string): boolean {
const { pid } = paths(id);
try {
writeFileSync(pid, String(process.pid), { flag: "wx" });
return true;
} catch (error) {
if ((error as NodeJS.ErrnoException).code !== "EEXIST") throw error;
if (isAlive(Number(readFileSync(pid, "utf8")))) return false;
rmSync(pid, { force: true });
return lock(id);
}
}
):

writeFileSync(pid, String(process.pid), { flag: "wx" });   // claim
...
if (isAlive(Number(readFileSync(pid, "utf8")))) return false;
rmSync(pid, { force: true });                                // reclaim a dead owner's file
return lock(id);

Interleaving, with a stale worker.pid holding dead PID D and contenders A and B (for example two clients starting the same dormant channel after a crash):

  1. A and B both get EEXIST, both read D, and both see it dead.
  2. A runs rmSync, retries, and its wx write succeeds: A holds the lock.
  3. B runs rmSync, removing A's fresh lock file, then its own wx write succeeds: B holds the lock too.

Two workers then own one channel's SQLite storage and socket, which breaks the single-owner invariant in docs/architecture.md. The same window applies to remove()'s use of the lock during delete.

Acceptance: a single exclusive owner of a channel's storage, held across crash recovery and concurrent starts and deletes. Verify it with real contenders started at the same moment after a worker died and left a stale worker.pid: exactly one worker may open the channel each time.

Found while reviewing #159 (#153). Dogfooding #5.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions