The vitest-orphan-leak

A real incident from building this fusion. Two harmless-looking facts combined into 16 immortal processes pinning the machine. The fix is a small classic of defensive systems engineering — and a lesson in fixing the interaction, not just one cause.

The two causes — neither fatal alone

This is the heart of the lesson. The leak needed both of these to be true at once:

The fix — two layers, one per cause

A single patch wouldn't be robust: harden the config and a different hang still orphans; add the wrapper and a hang inside the group still pins a core until the wall-clock timeout. Both layers ship.

Layer 1 — make a hang fail, not hang (cause 1)

Bounded timeouts in vitest.config.ts (applied to both root projects via extends) turn an infinite hang into a bounded failure, and the forks pool isolates a stuck file in a child process Vitest force-kills on teardown — so a stuck file can't pin a core indefinitely.

// vitest.config.ts:20-27
// Anti-orphan hardening: a hung test (server/socket/MCP/interval/unresolved
// promise with no teardown) must FAIL on a bounded timeout, never hang a
// worker forever. The `forks` pool isolates hangs in child processes that
// Vitest force-kills on teardown, so a stuck file cannot pin a CPU core.
testTimeout: 15_000,
hookTimeout: 15_000,
teardownTimeout: 10_000,
pool: 'forks',

Layer 2 — make orphaning impossible (cause 2)

scripts/safe-test.mjs (exposed as pnpm test:safe) runs the suite in its own process group, under a hard wall-clock timeout, then kills the whole group — not just the parent PID — and sweeps any stray vitest as a last-resort net. Three techniques, each load-bearing:

// scripts/safe-test.mjs:34-44 — detached group + negative-PID kill
// detached:true => POSIX setsid => the child leads a NEW process group, so
// `kill(-pid)` reaches every descendant (vitest main + all tinypool forks).
const child = spawn(bin, args, { stdio: 'inherit', detached: true });

const killGroup = (signal) => {
  try { process.kill(-child.pid, signal); }   // NEGATIVE pid = the whole group
  catch { /* group already gone */ }
};

// scripts/safe-test.mjs:26-32 — the last-resort sweep
const sweep = () => {
  try { execFileSync('pkill', ['-9', '-f', 'vitest'], { stdio: 'ignore' }); }
  catch { /* nothing left to kill — pkill exits non-zero when no match */ }
};

• detached:true → the child leads a new process group (POSIX setsid), so a single signal can reach every descendant.
• process.kill(-child.pid, …) → the negative PID signals the whole group, including tinypool forks — never a bare PID.
• pkill -9 -f vitest sweep → catches anything that already escaped to PID 1 (the orphaning case), so leakage can't accumulate across runs.
• A hard SAFE_TEST_TIMEOUT_MS (default 600000 = 10 min) bounds the whole run; a timeout exits 124. The same kill+sweep runs on normal exit, on SIGINT/SIGTERM, and on spawn error.

A third belt-and-braces measure: .factory/run.ts sweeps stray host vitest at the start and end of each iteration — so an automated loop never inherits a previous run's leak.

How it was verified — the Proof Gate

# the verification, conceptually
pnpm test:safe            # bounded run in its own group → 565 passed
pgrep -f vitest           # → (no output): nothing leaked. THIS is the proof.

The operating rules that fell out of it

Common confusions