serialized audit: the audit+merge critical section is process-wide, one run at a time (owner 2026-09-28)
Extraction may run in parallel across sources (extractors are source-local and graph-blind), but the auditor explores the LIVE graph through its tools and stamps identity verdicts against what it finds — two concurrent audits would each link into a graph the other is about to change, the structural mint of duplicate nodes. withAuditLock: a FIFO promise-chain mutex on globalThis (the same module-instance idiom as the run registries — a module-level lock would be per-route-instance and useless in dev). The check runs before any LLM is invoked or the graph read; the lock spans audit through merge inclusive (the merge is the graph mutation) and is reacquired per batch, so interleaved multi-batch runs each audit against the latest graph instead of starving each other. Applied at both front doors — chat batches and notes imports. New 'waiting' phase (extracting → waiting → auditing → merging): visible in the status feed so a queued run never looks stalled in extracting. Abort-aware on both sides of the queue: a run aborted while waiting leaves without auditing (its chain link still releases — successors never hang); a run aborted mid-audit releases in finally. Tests pin: a second run queues at the gate and its auditor is invoked only after the first run's merge is in the ledgers (the post-merge-graph invariant itself); a run aborted while waiting never audits, checkpoints nothing, and leaves a healthy chain for a third run. Gates: typecheck, 1061/1061, lint, build.
This commit is contained in:
@@ -340,6 +340,8 @@ flowchart TB
|
||||
|
||||
**Raw reasoning chains only** (2026-09-27 cleanup): the extractor reads ONLY the session — the whole thing, rendered as readable markdown by the shared engine (`core/session-markdown.ts`; the same renderer will serve future user-facing session export) — and proposes free-form propositions (true statements; a misconception is stated as its true correction) and `follows`/`supports` edges with verbatim evidence. Every node is an objective, world-facing fact. No quiz outcomes, no understanding assessments — the events ledger and `UnderstandingLevel` are deleted; the learner's relationship to a proposition becomes practice-module territory (assessment records, later).
|
||||
|
||||
**Serialized audit** (owner ruling 2026-09-28): extraction may run in parallel across sources, but the audit→merge section is a process-wide critical section — a FIFO mutex on `globalThis` (the same module-instance idiom as the run registries). Before any auditor LLM call or graph read, a run checks the gate and waits (`waiting` phase, visible in the status feed) until every earlier run's merge has landed, so each auditor explores the graph as it actually stands — two concurrent audits would each link into a graph the other is about to change, the structural mint of duplicate nodes. The lock is reacquired per batch (interleaved runs each see the latest graph instead of starving) and is abort-aware on both sides of the queue.
|
||||
|
||||
**Division of trust** (the pipeline's core rule): the extractor is fully trusted and is the only stage that ever sees the source. The auditor never sees the transcript — it receives the findings JSON alone and explores the existing graph through read-only tools (`find`, `list_sessions`, `list_propositions` — statement text included, no timestamps) to reach identity verdicts: `linkTo` an existing node or let the merger create one. Equivalence bias (owner ruling 2026-09-27): paraphrases, rewordings, and compressions of the same claim MERGE; only genuinely distinct propositions stay apart. The auditor may also draw bridging edges — typed `follows`/`supports` connections between existing graph nodes and the batch's propositions (the auditor is the asserter: chat-scoped provenance, no message anchor). The deterministic merger validates and appends: linkTo resolution, dedupe by `(from, to, type)`, coverage checkpoints. There is no drop channel — nothing rejects the extractor's output.
|
||||
|
||||
**Coverage, not quiz-counting** (supersedes the v1 every-N-quiz-answers trigger): each message's *view* (the compact line format `core/coverage.ts` has always hashed) is hashed into a per-chat fingerprint ledger (`coverage.jsonl`); "covered" means exactly "a run committed on this content." The prompt view is the richer markdown document — a deliberate 2026-09-27 split: re-formatting presentation never re-opens covered history. Uncovered = fingerprint diff, never dual-tracked; runs are greedy token-budgeted batches (`graph.extractionChunkTokens` — a settings knob, default 4096, floor 500, read fresh each turn) of the oldest uncovered messages — quiz-less sessions extract too. A coarse `sessions.jsonl` checkpoint still marks each merged run. The trigger seam (`turn-service.ts`) is trigger-agnostic from the pipeline's side: when a better mechanism lands (session end, idle, explicit flush), only the seam changes. Single-flight per source; the run registry powers the coverage UI and true abort.
|
||||
|
||||
Reference in New Issue
Block a user