sorokin.oleg.edu/learnharness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

practice bug fix — the boot sweep stamped a LIVE run (owner live test 2026-09-29)

Browse Source 
Owner report: created a set, 'Interrupted (server restarted)' showed
on list + set views, generation proceeded and overwrote it. The
sweep's 'generating + no registry row' test had a false-positive
mode: it ran at EVERY pipeline instantiation, and an instance could
evaluate against a registry whose fresh row it couldn't observe.

Three guards, per ruling:

1. HEARTBEAT: ProblemSet.updatedAt? (contract + doc) — bumped on
   every store write; createSet stamps it = createdAt; saveSet
   re-stamps (validator tolerates absence for legacy rows).

2. ONCE PER PROCESS: the sweep guards on globalThis
   .__lhPracticeSweepDone — no instance re-evaluates after the
   process's first sweep.

3. GRACE WINDOW: only sets whose heartbeat (updatedAt ?? createdAt)
   is older than 120s are stamped. A live run saves at every phase/
   problem transition, so it is structurally unreachable; a true
   orphan is minutes stale at next boot. One [practice <id>] log
   line per stamped set (the extraction/jobs console.error idiom).

Tests: stale orphan stamped; FRESH generating set untouched (grace);
once-guard (second factory instantiation never re-sweeps); live-row
protection re-armed via resetSweep() test seam; store pins for
updatedAt birth-stamp + re-stamp + legacy tolerance. Architecture.md
sweep sentence updated (once-per-process + grace semantics).

1142/1142, tsc 0, lint 0, build 0.
This commit is contained in:
amosblomqvist
2026-09-29 20:19:46 +03:00
parent e92679254d
commit 8d39f3717c
6 changed files with 116 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/agent/subagents/practice-pipeline.ts
+23 -9
View File
@@ -264,15 +264,29 @@ export interface PracticePipelineDeps extends PracticeModels {
export function createPracticePipeline(deps: PracticePipelineDeps) {
const agents = createPracticeAgents(deps);
// BOOT SWEEP: a persisted partitioning/generating set with no live row
// is a crash/restart orphan (the registry is memory-only) — mark it,
// idempotently. Runs at every pipeline instantiation (dev route
// entries re-run it; live rows are globalThis-shared and skipped).
for (const set of deps.practiceStore.listSets()) {
if (
(set.status === "partitioning" || set.status === "generating") &&
!practiceRuns().has(`${set.id}:generation`)
) {
// BOOT SWEEP (owner bug fix, live test 2026-09-29 — the sweep once
// stamped a LIVE run): a persisted partitioning/generating set with no
// live row AND a heartbeat older than the grace window is a crash/
// restart orphan (the registry is memory-only) — mark it, idempotently.
// THREE guards against stamping the living:
// 1. ONCE PER PROCESS (globalThis flag) — an instance could otherwise
// evaluate against a registry whose fresh row it can't observe
// (dev route bundles, restart paths).
// 2. GRACE WINDOW (120s over updatedAt ?? createdAt) — a live run
// saves at every phase/problem transition, so its heartbeat is
// structurally never 120s stale; a true orphan is minutes stale at
// the next boot.
// 3. Live-row check (globalThis-shared registry) as before.
const sweepFlag = globalThis as { __lhPracticeSweepDone?: boolean };
if (!sweepFlag.__lhPracticeSweepDone) {
sweepFlag.__lhPracticeSweepDone = true;
const staleBefore = Date.now() - 120_000;
for (const set of deps.practiceStore.listSets()) {
if (set.status !== "partitioning" && set.status !== "generating") continue;
if (practiceRuns().has(`${set.id}:generation`)) continue;
const heartbeat = Date.parse(set.updatedAt ?? set.createdAt);
if (Number.isFinite(heartbeat) && heartbeat > staleBefore) continue; // grace
console.error(`[practice ${set.id}] boot sweep: stale ${set.status} set marked interrupted`);
set.status = "error";
set.error = "Interrupted (server restarted)";
deps.practiceStore.saveSet(set);