Sync main (graph redesign b07c907): extraction observability ported onto the redesigned pipeline

Conflict resolution: extraction.ts takes main's redesign wholesale and
re-applies 072671f's observability deltas (stage durations, pipeline
failure line) — log shape only, no behavior change. graph-view.tsx's
legend/filters (094bafa) rekeyed from node.name (removed) to
statement/description, matching the new GraphNode shape.
This commit is contained in:
amosblomqvist
2026-09-08 22:23:07 +03:00
14 changed files with 954 additions and 819 deletions
+2 -2
View File
@@ -114,7 +114,7 @@ graph TB
LEDG --> SNAP --> RGRAPH
```
**Core invariant:** the extraction pipeline (extractor → auditor → merger) is the sole graph writer. The LLM stages propose and verify (propositions, edges, probe attribution — each with verbatim evidence); the deterministic merger owns every ledger append and rebuilds anything the model could silently mis-copy (event kinds, timestamps, idempotency refs) from the transcript. The teacher stays a teacher; everything below the ledgers is derived.
**Core invariant:** the extraction pipeline (extractor → auditor → merger) is the sole graph writer — tree-blind and post-hoc. The LLM stages propose, verify, and judge identity (link-or-create against the current graph; each finding carries verbatim evidence and message-granular provenance); the deterministic merger owns every ledger append and rebuilds anything the model could silently mis-copy (event kinds, timestamps, idempotency refs, provenance) from the transcript. Every node is an objective, world-facing fact; the learner's relationship to it lives entirely in the event ledger. The teacher stays a teacher; everything below the ledgers is derived.
## Layout
@@ -165,7 +165,7 @@ data/ chats/ · graph/ · jobs/ — runtime state, git
**The async sub-agent protocol (ack-and-steer).** `research` / `demo` / `visual` are server-executed tools: the call resolves immediately mid-turn with a fire-and-forget ack and the teacher keeps teaching; a background runner works in parallel (the job store tracks a live activity line; the client polls `/api/jobs`). On completion the runner builds a **steer message** — a user-role `UIMessage` whose text part is the model-facing result (data parts are dropped by provider conversion) and whose `data-subagent-artifact` part carries the full result for the client — and the **wake pipeline** (`server/jobs.ts`) delivers it: a per-chat serialized queue waits for the turn to end and no question to be pending, then fires a server-side teacher turn that weaves the result into the lesson. The client adopts the wake turn through the same `turnInFlight` machinery as returning mid-stream (busy shimmer, composer gate) and renders steers as artifacts — never a bubble. Sub-agents **verify, never one-shot**: the demo-builder headlessly executes its HTML in system Chrome before delivery; the visual-maker renders each SVG and looks at it (needs a vision-capable model — `MODELS.visualMaker` in config.ts).
**The knowledge graph (the two channels).** The write channel: the extraction pipeline — extractor (LLM: propositions, edges, probe attribution, each with verbatim evidence) → auditor (LLM: drops anything unbacked) → merger (`mergeFindings`, deterministic: exact/alias identity resolution, event rebuild from the transcript's tool parts, dedupe, checkpoint). Triggered once a chat accumulates `EXTRACTION.everyNQuizAnswers` (default 3, config.ts) new answered quizzes — a v1 placeholder trigger isolated in one seam (`turn-service.ts`), swappable without touching the pipeline. Failure checkpoints nothing; the next trigger re-runs. The read channel: the home view derives node states from the ledgers (`GET /api/graph` is a pure read — forceAtlas2 run-then-freeze, no idle simulation), and at session start the derived graph is rendered into a compact markdown block appended to the teacher's instructions (`buildGraphContext`) — whole graph while small, hotspots + 1-hop neighborhood past 150 nodes.
**The knowledge graph (the two channels).** The write channel: the extraction pipeline — extractor (LLM, tree-blind: propositions as true statements + descriptions, follows/supports edges, quiz attribution, understanding notes) → auditor (LLM: drops anything unbacked, stamps each survivor with an identity verdict — link to an existing node or create) → merger (`mergeFindings`, deterministic: applies the verdicts, link-appends phrasings/provenance, event rebuild from the transcript's tool parts, dedupe, checkpoint). Triggered once a chat accumulates `EXTRACTION.everyNQuizAnswers` (default 3, config.ts) new answered quizzes — a v1 placeholder trigger isolated in one seam (`turn-service.ts`), swappable without touching the pipeline. Failure checkpoints nothing; the next trigger re-runs. The read channel: the home view derives node states from the ledgers (`GET /api/graph` is a pure read — forceAtlas2 run-then-freeze, no idle simulation), and at session start the derived graph is rendered into a compact markdown block appended to the teacher's instructions (`buildGraphContext`) — whole graph while small, hotspots + 1-hop neighborhood past 150 nodes.
## Testing
+6 -6
View File
@@ -81,11 +81,11 @@ const STATES_ORDER: NodeStateClass[] = ["taught", "known", "missed", "dont-know"
/**
* Pure filter predicate for the legend/filters panel: a node is visible
* when its state is toggled on AND (no query OR name/statement matches).
* when its state is toggled on AND (no query OR statement matches).
* Exported for tests; the sigma reducers just consume its verdict.
*/
export function isNodeVisible(
node: Pick<GraphNode, "name" | "statement">,
node: Pick<GraphNode, "statement" | "description">,
state: NodeStateClass,
hiddenStates: ReadonlySet<NodeStateClass>,
query: string,
@@ -93,7 +93,7 @@ export function isNodeVisible(
if (hiddenStates.has(state)) return false;
const q = query.trim().toLowerCase();
if (!q) return true;
return node.name.toLowerCase().includes(q) || node.statement.toLowerCase().includes(q);
return node.statement.toLowerCase().includes(q) || node.description.toLowerCase().includes(q);
}
// ---- Label drawing (canvas) ---------------------------------------------
@@ -434,7 +434,7 @@ export function GraphView() {
// frozen layout stable for a given node set across remounts.
const rand = seededRandom(node.id);
graph.addNode(node.id, {
label: node.name,
label: node.statement,
size: STATE_TOKENS[state].size,
color: stateColors.get(state) ?? edgeColor,
x: (rand() - 0.5) * 100,
@@ -694,8 +694,8 @@ export function GraphView() {
style={{ visibility: "hidden", left: 0, top: 0 }}
className="absolute z-10 w-80 max-w-[calc(100%-1rem)] rounded-xl border bg-card p-4 text-left shadow-lg"
>
<div className="text-sm font-medium leading-snug">{selected.node.name}</div>
<div className="mt-1 text-xs leading-relaxed text-muted-foreground">{selected.node.statement}</div>
<div className="text-sm font-medium leading-snug">{selected.node.statement}</div>
<div className="mt-1 text-xs leading-relaxed text-muted-foreground">{selected.node.description}</div>
<div className="mt-1 text-xs uppercase tracking-wide text-muted-foreground">
{STATE_LABELS[selected.state]}
</div>
+3 -3
View File
@@ -75,11 +75,11 @@ chat record ──▶ EXTRACTOR (LLM) ──▶ AUDITOR (LLM) ──▶ MERGER (
proposes verifies applies
```
**Division of trust** (the pipeline's core rule): the LLM stages supply only judgment — *what was learned* (propositions, incl. misconceptions and prose-only `supports` edges), *what depends on what*, and *which proposition an untagged probe tested*. Everything the model could silently mis-copy — event kinds, timestamps, idempotency refs, notes, question texts — is rebuilt deterministically by the merger from the transcript's tool parts. The extractor files verbatim evidence quotes; the auditor (the faithfulness net) drops anything unbacked and logs every drop. Conservative throughout: duplicates over wrong merges, drop over invent.
**Tree-blind and post-hoc** (2026-09-02 redesign, docs/vision.md §1): the extractor reads ONLY the transcript — never the session's plan tree — and proposes free-form propositions (true statements with descriptions, misconception-free: a misconception is stated as its true correction), `follows`/`supports` edges, per-proposition understanding notes, and message-granular provenance. Every node is an objective, world-facing fact; the learner's relationship to it lives entirely in the event ledger.
**Merger invariants** (`src/core/graph.ts`, `mergeFindings`): idempotent and incremental — statement/alias exact-match identity resolution (fuzzy/embedding adjudication deferred), one mapping line per (chat, graph node) as the merge audit trail, edge dedupe by `(from, to, type)` with per-chat provenance, event dedupe by chat-scoped ref, and a sessions.jsonl checkpoint (`foldedAt`) marking which turns are processed. Failure anywhere before the merger checkpoints nothing; the next trigger re-runs.
**Division of trust** (the pipeline's core rule): the LLM stages supply only judgment — *what was learned*, *what follows/supports what*, *which proposition each quiz tested* (retroactively, against the whole transcript), and — the auditor's — *identity*: whether a proposition IS an existing graph node (link-or-create). Everything the model could silently mis-copy — event kinds (`correct` / `miss` / `dont-know`), timestamps, idempotency refs, notes, provenance points — is rebuilt deterministically by the merger from the transcript's tool parts. The extractor files verbatim evidence quotes; the auditor (which sees the extractor's output PLUS the current graph) drops anything unbacked, logs every drop, and stamps each surviving proposition with its identity verdict (`linkTo`). Conservative throughout: duplicates over wrong merges, drop over invent.
**Probe attribution** (vision open question, resolved structurally): tagged events (quiz `nodeId`, `plan_mark_done`'s tree node) resolve deterministically through the proposition mapping. The extractor's judgment is consulted only for **untagged probe quizzes** — retroactively, against the whole transcript, so a pre-tree probe maps to a proposition the tree only articulates later. Attribution can also rescue a tagged-but-unknown id; genuinely unattributable probes land `nodeId: null` with the question text, exactly as they always have.
**Merger invariants** (`src/core/graph.ts`, `mergeFindings`): idempotent and incremental — identity comes from the auditor's `linkTo` verdict alone (validated against the ledger; never string matching), one mapping line per (chat, sourceId) as the verdict audit trail (which also makes re-merging the same chat idempotent), link-appends: a linked proposition appends a node line whose phrasings join the node's `aliases` and whose provenance unions (reads merge by id), edge dedupe by `(from, to, type)`-already-asserted-by-this-chat with message-granular provenance, event dedupe by chat-scoped ref (quiz toolCallId, `obs:<sourceId>` for understanding observations), and a sessions.jsonl checkpoint (`foldedAt`) marking which turns are processed. Failure anywhere before the merger checkpoints nothing; the next trigger re-runs.
**The trigger is a v1 placeholder, isolated in ONE seam.** `src/server/turn-service.ts` (`triggerExtraction`) counts answered quizzes not yet in the ledger (`pendingQuizAnswers`, core) and fires the pipeline async once a chat crosses `EXTRACTION.everyNQuizAnswers` (default 3, config.ts) — single-flight per chat. The pipeline is trigger-agnostic: when a better mechanism lands (session end, idle, explicit flush), only this seam changes.
+4 -5
View File
@@ -1,20 +1,19 @@
# The learner's knowledge graph
After these instruction fragments you may see a fenced block titled **LEARNER KNOWLEDGE GRAPH (auto-generated)**. It is the accumulated record of what this learner has actually demonstrated, across every past session — rebuilt fresh from quiz outcomes and plan trees, every turn. Each line is a proposition in its current state (in brackets), and `a → b` under **Requires** means *a* must be understood before *b*.
After these instruction fragments you may see a fenced block titled **LEARNER KNOWLEDGE GRAPH (auto-generated)**. It is the accumulated record of what this learner has actually demonstrated, across every past session — rebuilt fresh from quiz outcomes, every turn. Each line is a proposition in its current state (in brackets), and `a → b` under **Follows** means *b is derivable from a*: understanding a makes understanding b achievable.
The states, exactly:
- **[missed]** — answered wrong. Treat as a live misconception site until a check proves otherwise.
- **[dont-know]** — the learner declined to guess. Unexplored, not broken.
- **[known]** — answered correctly. Foundation you can build on and refer to by name.
- **[taught]** — a lesson covered it. Check that it stuck before stacking anything heavy on it.
- **[candidate]** — asserted into a plan but never probed. Assume nothing.
- **[candidate]** — present in the graph but never probed. Assume nothing.
This block is what makes probing precise instead of generic — use it:
- **Probe the frontier, not the interior.** [known] and [taught] nodes near the session's topic are the learner's foundation: reference them by name and build on them — never re-teach what the graph says he holds. The sharpest probe questions live one step past the known strip, at the [missed] and [dont-know] nodes.
- **Probe the frontier, not the interior.** [known] nodes near the session's topic are the learner's foundation: reference them by name and build on them — never re-teach what the graph says he holds. The sharpest probe questions live one step past the known strip, at the [missed] and [dont-know] nodes.
- **[missed] nodes are your first stops.** A wrong answer is a misconception until shown otherwise. Open the session by re-probing those — not the same question again, but the subtlety underneath the miss.
- **Read the Requires chains as scaffolding.** They show what the learner's knowledge rests on. If a prerequisite under the session's topic is [missed] or [dont-know], that is where teaching starts — not at the topic itself.
- **Read the Follows chains as scaffolding.** They show what the learner's knowledge rests on. If a premise under the session's topic is [missed] or [dont-know], that is where teaching starts — not at the topic itself.
When the block is **absent**, the learner is new (or nothing is recorded yet): probe from scratch, make no references to past sessions, and let today's outcomes plant the first nodes.
+156 -118
View File
@@ -1,23 +1,28 @@
import { ToolLoopAgent, stepCountIs, tool, type LanguageModel, type UIMessage } from "ai";
import { z } from "zod";
import { modelFor } from "../models.ts";
import type { AuditResult, ChatRecord, SessionFindings } from "../../core/contracts.ts";
import { mergeFindings, type EmissionStats, type GraphDeps } from "../../core/graph.ts";
import type { AuditResult, ChatRecord, GraphNode, SessionFindings } from "../../core/contracts.ts";
import { getGraph, mergeFindings, type EmissionStats, type GraphDeps } from "../../core/graph.ts";
/**
* The extraction pipeline (vision §1 construction) — the sole graph writer.
* Three stages, one orchestrator:
* The extraction pipeline (vision §1 construction, 2026-09-02 redesign) — the
* sole graph writer. Three stages, one orchestrator:
*
* extractor (LLM) transcript → SessionFindings (propositions, edges,
* event attribution — each with verbatim evidence)
* auditor (LLM) findings → AuditResult (the verified subset + drops)
* extractor (LLM) transcript → SessionFindings (free-form propositions,
* follows/supports edges, quiz attribution, understanding
* notes — each with verbatim evidence). TREE-BLIND: it
* sees only the transcript, never the plan tree.
* auditor (LLM) findings + THE CURRENT GRAPH → AuditResult: the
* verified subset with an identity verdict per
* proposition (linkTo an existing node, or create), plus
* the drops for the log.
* merger (core) mergeFindings() → deterministic ledger appends
*
* The LLM stages supply only the judgment calls; the deterministic rebuild
* in the merger owns everything that must never be mis-copied (event kinds,
* timestamps, refs). Both agents are single-tool-call workers: their whole
* deliverable is one zod-validated submit call — a ToolLoopAgent purely so
* the model can retry a schema-invalid call.
* timestamps, refs, provenance points). Both agents are single-tool-call
* workers: their whole deliverable is one zod-validated submit call — a
* ToolLoopAgent purely so the model can retry a schema-invalid call.
*
* Created via createExtractionPipeline(deps) — the composition root
* (chat-handler) injects the storage ports + models; this module never
@@ -32,7 +37,7 @@ import { mergeFindings, type EmissionStats, type GraphDeps } from "../../core/gr
* stats (or failed) for every run.
*/
// ─── the transcript view (what the LLM sees) ────────────────────────────────
// ─── the transcript view (what the extractor + auditor see) ─────────────────
/**
* A quiz tool part's outcome, one line: the attribution evidence.
@@ -49,46 +54,35 @@ function quizLine(part: { toolCallId: string; input: unknown; output: unknown })
}
/**
* The compact transcript the extractor + auditor both see: user/teacher
* text verbatim (evidence quotes must match it), the final plan tree, and
* the structured tool events (quiz outcomes, mark_done). Job tool parts
* (research/demo/visual) are noise for extraction — their outcomes are
* summarized as one line each.
* The compact transcript both LLM stages see: user/teacher text VERBATIM,
* each tagged with its message id (provenance anchors — the extractor
* reports them back per proposition/edge), and quiz outcome lines.
* TREE-BLIND by design (2026-09-02): no plan tree, no mark_done —
* extraction is free-form, decoupled from the session's plan. Job tool
* parts (research/demo/visual) are noise for extraction — their outcomes
* are summarized as one line each.
*/
export function transcriptView(record: ChatRecord): string {
const lines: string[] = [];
lines.push(`SESSION: ${record.title}`);
if (record.tree && record.tree.nodes.length > 0) {
const nodes = record.tree.nodes.map((n) => `${n.id} = ${n.title}`).join("; ");
lines.push(`FINAL PLAN TREE (node ids): ${nodes}`);
if (record.tree.edges.length > 0) {
const edges = record.tree.edges.map((e) => `${e.from} requires ${e.to}`).join("; ");
lines.push(`TREE EDGES: ${edges}`);
}
}
for (const message of record.messages as UIMessage[]) {
record.messages.forEach((message, index) => {
const who = message.role === "user" ? "LEARNER" : "TEACHER";
for (const part of message.parts) {
const mid = (message as UIMessage).id || `m${index}`;
for (const part of (message as UIMessage).parts) {
if (part.type === "text" && part.text.trim()) {
lines.push(`${who}: ${part.text.trim()}`);
lines.push(`${who} [${mid}] ${part.text.trim()}`);
} else if (part.type.startsWith("tool-")) {
const name = part.type.slice("tool-".length);
const p = part as unknown as { toolCallId: string; state: string; input: unknown; output: unknown };
if (name === "quiz" && p.state === "output-available") {
lines.push(quizLine(p));
} else if (name === "plan_mark_done" && p.state === "output-available") {
const input = p.input as { id?: unknown } | undefined;
lines.push(`TEACHER marked tree node done [${p.toolCallId}]: ${typeof input?.id === "string" ? input.id : "?"}`);
} else if (name.startsWith("plan_")) {
// plan ops are folded into the FINAL PLAN TREE line above
} else if (p.state === "output-available") {
lines.push(`(${name} tool result delivered)`);
}
}
}
}
});
return lines.join("\n");
}
@@ -99,87 +93,98 @@ const evidence = z
.min(1)
.describe("Verbatim quotes from the transcript that back this. The auditor will check every quote.");
const messagesField = z
.array(z.string())
.describe("The message ids (from the [id] tags in the transcript) where this came up — provenance.");
const propositionSchema = z.object({
id: z.string().describe("Local id: \"p1\", \"p2\", … — referenced by edges and attribution."),
statement: z
.string()
.describe(
"The proposition as one reasoning step — a TRUE, self-contained, world-facing statement. " +
"Never reference the learner, the teacher, or the session: not \"the learner understands X\" but " +
'"X". A misconception is stated as the TRUE correction ("Nietzsche did not endorse ' +
'might-makes-right social Darwinism") — the wrongness lives in the quiz outcomes, never here.',
),
description: z
.string()
.describe(
"The elaborated version of the same proposition — more words: scope, precision, what exactly it means. Still true, still world-facing.",
),
aliases: z.array(z.string()).describe("Verbatim phrasings this session used for the same proposition, if any."),
messages: messagesField,
evidence,
understanding: z
.string()
.optional()
.describe(
'One note about THIS session\'s evidence of the learner\'s grasp of this proposition, on a named-word scale ' +
'("solid", "shaky", "accepted", "resisted", "not yet engaged") — no numbers, no predictions.',
),
});
const edgeSchema = z.object({
from: z.string().describe("Proposition id that is followed / supported."),
to: z.string().describe("Proposition id that follows / is supported by `from`."),
type: z
.enum(["follows", "supports"])
.describe('"follows" = to is derivable from from; "supports" = from grounds to without entailing it.'),
messages: messagesField,
evidence,
});
const findingsSchema = z.object({
propositions: z
.array(
z.object({
id: z.string().describe("Local id: \"p1\", \"p2\", … — referenced by edges and attribution."),
name: z.string().describe(
"A short 2-6 word name for the fact — its header, used for rendering. Not a sentence.",
),
statement: z
.string()
.describe(
"The proposition as one reasoning step — an OBJECTIVE, world-facing fact. " +
"Never reference the learner, the teacher, or the session: not \"the learner " +
'understands X\" but \"X\"; not \"the learner thinks Y\" but the wrong idea itself, ' +
'stated as a claim (\"Reading: Nietzsche endorsed might-makes-right social Darwinism\"), ' +
"with kind \"misconception\" carrying that it is wrong.",
),
aliases: z.array(z.string()).describe("Verbatim alternate phrasings from this session, if any."),
kind: z.enum(["truth", "misconception"]).describe("misconception = a wrong idea the learner holds or was warned about."),
treeNodeId: z.string().optional().describe("The plan-tree node id this came from, when it did."),
evidence,
}),
)
.describe("Every distinct proposition the session touched — from the plan tree AND from prose."),
.array(propositionSchema)
.describe("Every distinct proposition the session touched — true statements, misconception-free."),
edges: z
.array(
z.object({
from: z.string().describe("Proposition id that is the prerequisite / support."),
to: z.string().describe("Proposition id that depends on / is supported by `from`."),
type: z.enum(["requires", "supports"]),
evidence,
}),
)
.describe("Relations between propositions. `requires` = prerequisite; `supports` = helpful but not prerequisite. Include relations the plan tree does NOT show when the transcript articulates them."),
.array(edgeSchema)
.describe("Relations between propositions: follows (derivable) / supports (grounds without entailing)."),
attribution: z
.object({
quiz: z
.record(z.string(), z.string().nullable())
.describe("toolCallId of every UNTAGGED quiz line → the proposition id it was really testing. Probe questions asked BEFORE the tree existed are attributed RETROACTIVELY — the whole transcript is visible; map them to the proposition they were probing (often articulated only later). null when genuinely unattributable. Tagged quizzes (with a tree nodeId in the transcript view) resolve automatically — do not include them."),
.describe(
"toolCallId of EVERY quiz line → the proposition id it was really testing. Attribute RETROACTIVELY — the whole transcript is visible; map each quiz to the proposition it tests (often articulated elsewhere in the session). null when genuinely unattributable.",
),
})
.describe("Probe attribution — your only event judgment."),
.describe("Quiz attribution — your only event judgment."),
});
const EXTRACTOR_INSTRUCTIONS = `You are the extraction specialist in a learning application. You read ONE session
transcript and report what was learned — as structured findings for the learner's global knowledge graph.
The transcript view shows: the session's final plan tree (dependency map, with node ids), the conversation
verbatim (LEARNER / TEACHER), and structured events (QUIZ lines with the learner's outcome, mark_done lines).
The transcript view shows: the conversation verbatim (LEARNER / TEACHER lines, each tagged with its message id)
and QUIZ outcome lines.
Report, with verbatim evidence quotes for everything:
NODE THEORY — the graph is a set of objective facts, full stop. A node is what is true (or falsely
believed) about the world, never anything about the learner or the session. The learner's
relationship to a fact — whether they know it, missed it, or hold it as a misconception — is
expressed ENTIRELY by the quiz-outcome events you attribute; it never leaks into the node itself.
So no "the learner understands X", no "discussed in this session": the node is just X. A
misconception node states the wrong idea itself as a claim, with kind "misconception" marking it
wrong — never "the learner believes Y".
NODE THEORY — the graph is a set of TRUE propositions about the world, full stop. A node is never anything
about the learner or the session. The learner's relationship to a proposition — whether they know it, missed
it, resisted it — is expressed ENTIRELY by the quiz outcomes and your understanding note; it never leaks into
the node itself. So no "the learner understands X", no "discussed in this session": the node is just X. A
misconception is NOT a node of its own wrong claim — state the TRUE correction as the proposition, and let the
quiz outcomes carry that the learner believed the wrong version.
1. PROPOSITIONS — every distinct reasoning-step-sized proposition the session touched:
- Each carries a "name": a short 2-6 word header (the rendering handle), and a "statement": the
full fact as one objective reasoning step.
- Plan-tree nodes become propositions (pass their treeNodeId) — statement from the node title, enriched to a
complete proposition if the prose elaborates it.
- Propositions that appeared only in prose (teacher explanations, learner questions) get fresh ids.
- Include MISCONCEPTIONS — wrong ideas the learner showed or the teacher warned against — with kind
"misconception".
- Each is a self-contained TRUE statement ("statement"), plus its elaboration ("description" — more
words, same proposition: scope, precision, what exactly it means).
- Include corrections of misconceptions the teacher articulated, and true facts explained in prose.
- One reasoning step per proposition: never a topic blob, never a sub-step.
- aliases: verbatim alternate phrasings this session used for the same thing.
- "messages": the [id] tags of every message where the proposition came up — provenance, report it.
- "aliases": verbatim phrasings this session used for the same proposition.
- "understanding": ONE named-word-scale note on the evidence of the learner's grasp, for this session
only ("solid", "shaky", "accepted", "resisted", "not yet engaged"). No numbers, no predictions.
2. EDGES between propositions:
- The tree's dependency edges are the backbone: from = prerequisite, to = dependent, type "requires".
- Also include relations articulated only in prose. Use "supports" when the transcript presents something as
helpful but NOT a prerequisite.
- "follows": to is DERIVABLE from from (understanding from makes understanding to achievable).
- "supports": from GROUNDS to without entailing it (helpful context, not a prerequisite).
- Only relations the transcript actually articulates — with evidence.
3. ATTRIBUTION — UNTAGGED quizzes only:
- QUIZ lines whose question carries a tree nodeId resolve automatically; do not attribute those.
- The others are PROBE-phase questions (asked before the tree was initialized) that test prerequisite
understanding. Attribute them RETROACTIVELY to the proposition they were really probing — even if that
proposition is only articulated later in the session. The QUIZ question text is your guide. If you truly
cannot tell, use null.
3. ATTRIBUTION — every QUIZ line gets a mapping: toolCallId → the proposition id it was really testing.
Attribute RETROACTIVELY — a quiz often tests a proposition the session only articulates later; the
question text is your guide. If you truly cannot tell, use null.
Evidence discipline: every quote must appear VERBATIM in the transcript view (a QUIZ line counts as the
transcript). The auditor rejects anything unbacked. When in doubt, report; the auditor decides.
@@ -210,6 +215,16 @@ function createExtractor(model: LanguageModel) {
// ─── the auditor ───────────────────────────────────────────────────────────
const auditSchema = findingsSchema.extend({
propositions: z.array(
propositionSchema.extend({
linkTo: z
.string()
.optional()
.describe(
"The id of the EXISTING graph node this proposition IS — your identity verdict. Absent = create a new node. When unsure, omit: duplicates over wrong merges.",
),
}),
),
dropped: z
.array(
z.object({
@@ -221,27 +236,46 @@ const auditSchema = findingsSchema.extend({
});
const AUDITOR_INSTRUCTIONS = `You are the audit stage of a learning application's extraction pipeline. You receive
ONE session transcript and a set of proposed findings (propositions, edges, event attribution). Your job is the
faithfulness net: verify every candidate against the transcript, keep only what is backed, and return the
verified subset.
ONE session transcript, a set of proposed findings (propositions, edges, quiz attribution), and the CURRENT
knowledge graph. Your jobs: the faithfulness net, and identity.
Verify:
- PROPOSITIONS: every evidence quote must appear (near-)verbatim in the transcript view, and the statement must
be a faithful reading of it. The node theory applies: a statement must be an OBJECTIVE, world-facing fact —
reword learner- or session-relative statements ("the learner understands X" → "X") or drop them when the
objective fact is not actually backed. Drop invented or bloated statements. Conservative: when two proposed propositions
are really one, keep the better-stated one and drop the other.
- EDGES: the evidence must actually articulate the relation, in the direction claimed (from = prerequisite).
Drop direction-swapped or speculative edges.
- ATTRIBUTION: only UNTAGGED quizzes carry attribution — the QUIZ line's question must genuinely test the
attributed proposition. Drop wrong attributions to null, or re-point them to the proposition the question
actually tests.
FAITHFULNESS — verify every candidate against the transcript:
- PROPOSITIONS: every evidence quote must appear (near-)verbatim in the transcript view, and the statement
must be a faithful reading of it. The node theory applies: a statement must be a TRUE, self-contained,
world-facing fact — reword learner- or session-relative statements ("the learner understands X" → "X"), state
misconceptions as the true correction, or drop them when no objective fact is backed. Drop invented or
bloated statements. Conservative: when two proposed propositions are really one, keep the better-stated one
and drop the other.
- EDGES: the evidence must actually articulate the relation, in the direction claimed. Drop
direction-swapped or speculative edges.
- ATTRIBUTION: the QUIZ line's question must genuinely test the attributed proposition. Drop wrong
attributions to null, or re-point them to the proposition the question actually tests.
- UNDERSTANDING notes: verify against the quiz outcomes and the session's evidence; reword to a named-word
scale or drop.
IDENTITY — link-or-create, per proposition:
- You see the current graph (every node: id, statement, description, aliases). For each verified proposition,
decide: does an EXISTING node already state this same proposition?
- If yes: set "linkTo" to that node's id. The session's phrasings will join its aliases and its provenance.
- If no: omit "linkTo" and a new node is created.
- When unsure, omit — duplicates are recoverable later; wrong merges are not.
Also drop propositions not referenced by anything else ONLY if they have no other backing — an unreferenced but
well-evidenced proposition is still knowledge.
Never invent new candidates. Your entire deliverable is ONE submit_review call containing the verified findings
(the same shapes you received) plus the dropped list.`;
(the same shapes you received, with linkTo added) plus the dropped list.`;
/** The current graph, compact — the auditor's identity context. */
function graphView(nodes: GraphNode[]): string {
if (nodes.length === 0) return "(the graph is currently empty — every proposition is a new node)";
const lines = nodes.map(
(n) =>
`- ${n.id} :: ${n.statement}\n description: ${n.description}` +
(n.aliases.length > 0 ? `\n aliases: ${n.aliases.join(" | ")}` : ""),
);
return lines.join("\n");
}
function createAuditor(model: LanguageModel) {
return new ToolLoopAgent({
@@ -249,7 +283,7 @@ function createAuditor(model: LanguageModel) {
instructions: AUDITOR_INSTRUCTIONS,
tools: {
submit_review: tool({
description: "Submit the verified findings subset + the dropped list.",
description: "Submit the verified findings subset (with identity verdicts) + the dropped list.",
inputSchema: auditSchema,
execute: async (review) => review,
}),
@@ -302,18 +336,21 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
throw new Error("the extractor never submitted findings");
}
async function audit(chatId: string, findings: SessionFindings): Promise<SessionFindings> {
async function audit(chatId: string, findings: SessionFindings): Promise<AuditResult> {
const record = chatStore.getChat(chatId);
if (!record) throw new Error(`Chat not found: ${chatId}`);
const view = transcriptView(record);
const graph = getGraph(graphStore);
console.info(
`[extraction ${chatId}] auditor starting (${findings.propositions.length} propositions, ${findings.edges.length} edges, ` +
`${Object.keys(findings.attribution.quiz).length} attributions)`,
`${Object.keys(findings.attribution.quiz).length} attributions, ${graph.nodes.length} graph nodes)`,
);
const t0 = performance.now();
const review = await auditorAgent.generate({
prompt: `Transcript:\n\n${view}\n\nProposed findings:\n\n${JSON.stringify(findings, null, 2)}`,
prompt:
`Transcript:\n\n${view}\n\nProposed findings:\n\n${JSON.stringify(findings, null, 2)}\n\n` +
`Current knowledge graph (${graph.nodes.length} nodes):\n\n${graphView(graph.nodes)}`,
});
let auditResult: AuditResult | null = null;
for (const part of review.content) {
@@ -328,13 +365,13 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
auditResult.dropped.map((d) => `${d.ref} (${d.reason})`).join("; "),
);
}
const { dropped: _dropped, ...verified } = auditResult;
console.info(
`[extraction ${chatId}] audit kept ${verified.propositions.length}/${findings.propositions.length} propositions, ` +
`${verified.edges.length}/${findings.edges.length} edges`,
`[extraction ${chatId}] audit kept ${auditResult.propositions.length}/${findings.propositions.length} propositions, ` +
`${auditResult.edges.length}/${findings.edges.length} edges, ` +
`${auditResult.propositions.filter((p) => p.linkTo !== undefined).length} linked to existing nodes`,
);
console.info(`[extraction ${chatId}] auditor done in ${Math.round(performance.now() - t0)}ms`);
return verified;
return auditResult;
}
async function runExtraction(chatId: string): Promise<EmissionStats> {
@@ -344,10 +381,10 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
try {
console.info(`[extraction ${chatId}] pipeline start`);
const findings = await extract(chatId);
const verified = await audit(chatId, findings);
const auditResult = await audit(chatId, findings);
// The merger re-reads the chat record itself — events are rebuilt
// from the transcript, never from stale copies.
const stats = mergeFindings(chatId, verified, { chatStore, graphStore });
const stats = mergeFindings(chatId, auditResult, { chatStore, graphStore });
console.info(
`[extraction ${chatId}] merged: ${stats.nodes} nodes, ${stats.edges} edges, ${stats.events} events ` +
`(${Math.round(performance.now() - t0)}ms total)`,
@@ -365,3 +402,4 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
return { runExtraction };
}
+130 -79
View File
@@ -1,50 +1,69 @@
/**
* Knowledge-graph contracts — nodes, ledgers, extraction findings
*
* The minimal implementation of the graph vision (docs/vision.md §1, the
* 2026-09-02 redesign): every node is a TRUE proposition; edges are follows
* (derivable) and supports (grounds without entailing); provenance is
* message-granular; extraction is free-form and post-hoc, fully decoupled
* from the session's plan tree; the event ledger records quiz outcomes plus
* the extractor's per-proposition understanding observations. Anything not
* yet decided (the state model, strength derivation, understanding
* treatment, selection strategies) is deliberately NOT here — transitional
* stubs carry the renderer until those land.
*
* Part of the domain-split contracts (src/core/contracts/): each domain file
* is owned outright by its stream — cross-domain schema changes go through
* master. Pure types + protocol docs + tiny pure helpers; type-only imports
* of "ai" (safe everywhere: server, client, tests). See docs/vision.md.
* master. Pure types + protocol docs + tiny pure helpers. See docs/vision.md.
*/
// ─── Knowledge graph (vision §1 — global, cross-session) ────────────────────
export type EdgeType = "requires" | "supports";
/** Two relations only (DECIDED 2026-09-02): follows = b is derivable from a;
* supports = a grounds b without entailing it. No scalars, no encoded
* strength — strength is derived at read time from topology. */
export type EdgeType = "follows" | "supports";
/** One provenance point — message-granular (DECIDED 2026-09-02): nodes and
* edges record every (chat, message) where they came up, deep-linkable to
* the exact transcript point. */
export interface ProvenancePoint {
chatId: string;
messageId: string;
}
export interface GraphNode {
/** Opaque stable id, store-assigned: "n_" + nanoid-like. */
id: string;
/** Short name / header for the fact (2-6 words) — the rendering handle.
* NODE THEORY: a node is an OBJECTIVE fact, never learner- or
* session-relative. No "the learner understands…", no "this session
* covered…" — just the fact, world-facing. The learner's relationship
* to the fact lives ENTIRELY in the event ledger: state labels are
* derived from quiz outcomes, never written into the node. */
name: string;
/** Canonical proposition statement — the full fact, one reasoning step,
* also objective (no learner / teacher / session references). */
/** The minimal proposition — one true, self-contained statement. This is
* BOTH the rendering label AND the identity key: the auditor recognizes
* a re-encounter by judgment (link-or-create), not by string matching. */
statement: string;
/** Verbatim alternate statements from sessions (merge target audit trail). */
/** The elaborated version — required. Same proposition, more words:
* scope, precision, the "what exactly does it mean". */
description: string;
/** Verbatim phrasings sessions used for this proposition (the identity
* audit trail — how the world said what the node says). */
aliases: string[];
kind: "truth" | "misconception";
createdAt: string;
/** Every (chat, message) point where this proposition came up. */
provenance: ProvenancePoint[];
}
export interface GraphEdge {
/** Prerequisite GraphNode id. */
from: string;
to: string;
type: EdgeType;
/** Chat ids that asserted this edge (teacher may simplify — auditable). */
provenance: string[];
/** Every (chat, message) point where this relation was asserted. */
provenance: ProvenancePoint[];
}
/** mappings.jsonl line — the session→graph identity map: the extraction
* pipeline's proposition (identified by its source id) → the graph node it
* merged into. The merge audit trail: one line per (chat, graph node). */
* pipeline's proposition (identified by its run-local source id) → the
* graph node it resolved to (created or linked). The audit trail of the
* auditor's identity verdicts; also the chat-scoped resolution table the
* merger uses to point events and edges at node ids. */
export interface NodeMapping {
chatId: string;
/** The writer's local id for the proposition — a plan-tree node id when
* tree-derived, else the extractor's proposition id ("p1", …). */
/** The extractor's run-local proposition id ("p1", …). */
sourceId: string;
graphNodeId: string;
}
@@ -57,23 +76,26 @@ export interface SessionCheckpoint {
foldedAt: string;
}
export type LedgerEventKind =
| "probe-correct"
| "probe-miss"
| "dont-know"
| "misconception-signaled"
| "taught"
| "check-correct"
| "check-incorrect";
/**
* Event kinds (DECIDED 2026-09-02): quiz outcomes only — correct / miss /
* dont-know — plus `observation`, the extractor's per-proposition
* understanding note joining the ledger as a session-scoped observation
* (named-word scale; treatment pending the state-model ideation). No
* `taught` events at all; no misconception kind — a misconception is a
* TRUE correction in the graph, and the wrongness lives here, in the
* ledger, until the state model exists.
*/
export type LedgerEventKind = "correct" | "miss" | "dont-know" | "observation";
export interface LedgerEvent {
/** null = not yet attributed (untagged probe events; consolidation assigns). */
/** null = not yet attributed. */
nodeId: string | null;
kind: LedgerEventKind;
chatId: string;
at: string;
/** Idempotency key: the quiz part's toolCallId or `taught:<treeNodeId>` —
* re-emission of the same chat must never double-record an event. */
/** Idempotency key: the quiz part's toolCallId, or `obs:<sourceId>` for
* understanding observations — re-emission of the same chat must never
* double-record an event. */
ref?: string;
/** The learner's quiz note — diagnostic gold (vision §1 calibration). */
note?: string;
@@ -83,91 +105,121 @@ export interface LedgerEvent {
/**
* Store format (src/store/graph.ts): JSONL ledgers at config.LEDGER_FILES
* (nodes/edges/events/mappings/sessions). The derived graph is rebuilt by
* reading the ledgers — store events, derive state. sessions.jsonl is the
* extraction checkpoint: which chats the extraction pipeline has processed,
* and through which updatedAt (never extract the same turns twice).
* (nodes/edges/events/mappings/sessions), all append-only.
*
* nodes.jsonl one line per node — the FIRST line for an id defines
* statement/description/createdAt; later lines for the
* same id are link-appends (a session's re-encounter):
* their statements/aliases join `aliases`, their
* provenance unions. getGraph merges by id.
* edges.jsonl one line per edge ASSERTION; readers merge by
* (from,to,type), unioning provenance.
*
* The derived graph is rebuilt by reading the ledgers — store events,
* derive state. sessions.jsonl is the extraction checkpoint: which chats
* the extraction pipeline has processed, and through which updatedAt
* (never extract the same turns twice).
*
* The EXTRACTION PIPELINE is the sole graph writer (extractor → auditor →
* merger): the extractor proposes propositions/edges/attribution with
* verbatim evidence, the auditor drops anything unbacked, and the merger
* (core/graph.ts, deterministic) resolves identity against the existing
* ledgers and appends. See SessionFindings below.
* merger), tree-blind and post-hoc: the extractor reads only the transcript
* and proposes propositions/edges/quiz-attribution with verbatim evidence;
* the auditor sees the extractor's output PLUS the current graph, owns
* faithfulness (evidence check) and identity (link-or-create: point each
* proposition at the existing node it is, or let it be created — recording
* the session's phrasings as aliases); the merger (core/graph.ts,
* deterministic) applies the verdicts and appends.
*/
// ─── Extraction findings (extractor output → audit → merger input) ───────────
/** One proposed proposition — a node candidate. */
/** One proposed proposition — a node candidate (or a re-encounter of one). */
export interface ExtractedProposition {
/** Extractor-local id ("p1", …) referenced by edges + attribution. */
id: string;
/** Short name / header for the fact (2-6 words) — the rendering handle. */
name: string;
/** Canonical one-reasoning-step statement — an objective, world-facing
* fact (no learner / teacher / session references; the learner's
* relationship to it is carried only by the event attribution). */
/** The minimal proposition — a true, self-contained, world-facing
* statement. No learner / teacher / session references: misconceptions
* are stated as the true correction, and wrongness lives in the event
* ledger, never in the node. */
statement: string;
/** Verbatim alternate phrasings from the session (merge aliases). */
/** The elaborated version — required. */
description: string;
/** Verbatim phrasings the session used. */
aliases: string[];
kind: "truth" | "misconception";
/** The plan-tree node this came from, when it did (mapping provenance). */
treeNodeId?: string;
/** The message ids where this proposition came up — provenance. */
messages: string[];
/** Verbatim transcript quotes backing the proposition. */
evidence: string[];
/** The extractor's understanding note for this proposition IN THIS
* session — a named-word scale ("solid", "shaky", "accepted",
* "resisted"), no numbers. Joins the ledger as an observation event;
* treatment pending the state-model ideation. */
understanding?: string;
}
/** One proposed relation between two propositions. */
export interface ExtractedEdge {
/** Proposition id that is the prerequisite / support. */
/** Proposition id that is followed / supported. */
from: string;
/** Proposition id that depends on / is supported by `from`. */
to: string;
type: EdgeType;
/** The message ids where this relation was asserted — provenance. */
messages: string[];
/** Verbatim transcript quotes backing the relation. */
evidence: string[];
}
/** The extractor's event attribution — its ONLY say about events. The kinds,
* timestamps, idempotency refs, notes and question texts are all rebuilt
* deterministically by the merger from the transcript's quiz / mark_done
* tool parts; the LLM supplies exactly the judgment call: WHICH proposition
* each one is about (vision open question: probes attribute retroactively,
* against the whole transcript — a pre-tree probe can map to a proposition
* the tree only articulates later). */
/** The extractor's event attribution — its ONLY say about events. Event
* kinds, timestamps, idempotency refs, notes and question texts are all
* rebuilt deterministically by the merger from the transcript's tool
* parts, and TAGGED events (quiz nodeId, mark_done tree node) resolve
* deterministically through the proposition mapping. The LLM supplies
* exactly the one judgment call that needs it — the vision's open
* probe-attribution question: WHICH proposition did an UNTAGGED probe
* test? Answered retroactively against the whole transcript — a pre-tree
* probe can map to a proposition the tree only articulates later. */
/**
* The extractor's quiz attribution — its ONLY say about quiz events. Event
* kinds, timestamps, idempotency refs, notes and question texts are all
* rebuilt deterministically by the merger from the transcript's quiz tool
* parts; the LLM supplies exactly the judgment call: WHICH proposition each
* quiz tested. Tree-blind, so EVERY quiz attributes here (there is no tree
* mapping to lean on) — retroactively against the whole transcript.
*/
export interface ExtractionAttribution {
/** UNTAGGED quiz toolCallId → proposition id; missing / null =
* unattributed (the event lands with nodeId null + the question text,
* exactly like unattributable probes always did). */
/** quiz toolCallId → proposition id; null / missing = unattributed (the
* event lands with nodeId null + the question text, a later-
* attribution input). */
quiz: Record<string, string | null>;
}
/** The extractor's — and, filtered, the auditor's — output shape. */
/** The extractor's output shape. */
export interface SessionFindings {
propositions: ExtractedProposition[];
edges: ExtractedEdge[];
attribution: ExtractionAttribution;
}
/** The auditor's verdict: the verified subset, plus the drops (for the log —
* the audit trail of what was rejected and why). */
export interface AuditResult extends SessionFindings {
/** The auditor's verdict on one proposition: the findings, plus `linkTo` —
* the identity judgment. Absent = create a new node. */
export interface AuditedProposition extends ExtractedProposition {
/** The existing GraphNode id this proposition IS — the auditor's
* identity verdict. On link, the session's phrasings (statement,
* aliases) join the node's aliases and the provenance unions.
* Conservative: when unsure, leave it out — duplicates over wrong
* merges. */
linkTo?: string;
}
/** The auditor's output: the verified findings (with identity verdicts),
* plus the drops (the audit trail of what was rejected and why). */
export interface AuditResult {
propositions: AuditedProposition[];
edges: ExtractedEdge[];
attribution: ExtractionAttribution;
/** Dropped candidates: proposition id, edge "from→to", or attribution
* toolCallId — with the reason. */
dropped: { ref: string; reason: string }[];
}
/** Per-node state class for the home view (vision §1 "Home page") — derived,
* never stored: taught if ANY taught event; else the LATEST quiz event decides
* (known / missed / dont-know); a node with no events is a candidate. */
/**
* Per-node state class for the home view — TRANSITIONAL (kept so the
* renderer survives the contract change untouched). Derived, never stored:
* the LATEST quiz event decides (known / missed / dont-know); a node with
* no quiz events is a candidate. `taught` is now unreachable (no taught
* events are emitted anymore) — the real state model (dependency depth +
* topology-derived strength + quiz events) is pending ideation.
*/
export type NodeStateClass = "taught" | "known" | "missed" | "dont-know" | "candidate";
/** The GET /api/graph payload — everything the home view's PURE renderer needs. */
@@ -187,4 +239,3 @@ export function titleFromPrompt(prompt: string): string {
const oneLine = prompt.replace(/\s+/g, " ").trim();
return oneLine.length > 72 ? `${oneLine.slice(0, 72)}…` : oneLine;
}
+16 -22
View File
@@ -15,10 +15,9 @@
* the block is absent (new learner).
*
* Selection strategy (§1 "the selection problem", simplest first): the WHOLE
* graph while it is small; past MAX_NODES, hotspots (missed / dont-know /
* misconception nodes) plus their 1-hop `requires` neighborhood — a live
* misconception is always relevant context, and the frontier is where the
* next session should probe.
* graph while it is small; past MAX_NODES, hotspots (missed / dont-know
* nodes) plus their 1-hop `follows` neighborhood — the frontier is where
* the next session should probe.
*/
import type { GraphEdge, GraphNode } from "./contracts.ts";
@@ -31,22 +30,18 @@ export const GRAPH_CONTEXT_HEADER = "LEARNER KNOWLEDGE GRAPH (auto-generated)";
/** Above this node count only hotspots + their 1-hop neighborhood are shown. */
export const MAX_CONTEXT_NODES = 150;
/** Group order for node lines: diagnostic value first (vision §1 hotspots).
* A misconception-kind node is a hotspot whatever its state — a live
* misconception is always relevant context. */
/** Group order for node lines: diagnostic value first (vision §1 hotspots). */
const HOTSPOT_STATES: ReadonlySet<NodeStateClass> = new Set(["missed", "dont-know"]);
function groupOf(node: GraphNode, state: NodeStateClass): number {
if (HOTSPOT_STATES.has(state) || node.kind === "misconception") return 0;
if (HOTSPOT_STATES.has(state)) return 0;
if (state === "known") return 1;
if (state === "taught") return 2;
return 3; // candidate
return 2; // candidate
}
const GROUP_LABELS = [
"Hotspots (missed / dont-know / misconceptions) — likely misconception sites and unexplored ground:",
"Hotspots (missed / dont-know) — likely misconception sites and unexplored ground:",
"Known (answered correctly):",
"Taught (lessons covered them):",
"Candidates (never probed — assume nothing):",
] as const;
@@ -76,9 +71,9 @@ export function renderGraphContext(
truncated = true;
const hotspots = nodes.filter((n) => isHotspot(n, stateOf(n)));
const keep = new Set(hotspots.map((n) => n.id));
// 1-hop requires neighborhood: prerequisites AND dependents.
// 1-hop follows neighborhood: premises AND derivations.
for (const edge of edges) {
if (edge.type !== "requires") continue;
if (edge.type !== "follows") continue;
if (keep.has(edge.from) || keep.has(edge.to)) {
keep.add(edge.from);
keep.add(edge.to);
@@ -91,7 +86,7 @@ export function renderGraphContext(
// ── node lines, grouped/sorted by diagnostic priority ──────────────────────────
const lines: string[] = [
`===== ${GRAPH_CONTEXT_HEADER} =====`,
`${truncated ? `showing ${selected.length} of ${nodes.length} nodes (hotspots + their 1-hop neighborhood)` : `${nodes.length} nodes`}, ${edges.filter((e) => e.type === "requires").length} requires edges`,
`${truncated ? `showing ${selected.length} of ${nodes.length} nodes (hotspots + their 1-hop neighborhood)` : `${nodes.length} nodes`}, ${edges.filter((e) => e.type === "follows").length} follows edges`,
];
for (const [group, label] of GROUP_LABELS.entries()) {
@@ -101,19 +96,18 @@ export function renderGraphContext(
if (groupNodes.length === 0) continue;
lines.push("", label);
for (const node of groupNodes) {
const kind = node.kind === "misconception" ? " · misconception" : "";
lines.push(`• [${stateOf(node)}] ${node.statement} (${node.id})${kind}`);
lines.push(`• [${stateOf(node)}] ${node.statement} (${node.id})`);
}
}
// ── requires edges, merged into chains ──────────────────────────────────
const byId = new Map(selected.map((n) => [n.id, n.statement]));
const requires = edges.filter(
(e) => e.type === "requires" && byId.has(e.from) && byId.has(e.to),
const follows = edges.filter(
(e) => e.type === "follows" && byId.has(e.from) && byId.has(e.to),
);
if (requires.length > 0) {
lines.push("", "Requires (prerequisite → dependent):");
for (const chain of edgeChains(requires)) {
if (follows.length > 0) {
lines.push("", "Follows (premise → derivation):");
for (const chain of edgeChains(follows)) {
lines.push(chain.map((id) => byId.get(id) ?? id).join(" → "));
}
}
+176 -172
View File
@@ -2,22 +2,34 @@
* The knowledge graph: merger (the deterministic final stage of the
* extraction pipeline — vision §1 construction) + derivation (reads).
*
* The extraction pipeline is the SOLE graph writer:
* The extraction pipeline is the SOLE graph writer, tree-blind and post-hoc:
*
* extractor (LLM) → SessionFindings: propositions, edges, and event
* attribution, each with verbatim evidence
* auditor (LLM) → the verified subset (unbacked candidates dropped)
* extractor (LLM) → SessionFindings: propositions, edges, quiz
* attribution, per-proposition understanding notes —
* each with verbatim evidence, from the transcript ALONE
* auditor (LLM) → AuditResult: the verified subset, with the IDENTITY
* VERDICT per proposition (linkTo an existing node, or
* create), and the drops for the log
* merger (HERE) → mergeFindings(): deterministic ledger appends
*
* The merger is deliberately the ONLY stage that touches the ledgers, and it
* is pure code: the LLM supplies exactly the judgment calls (what was
* taught, what depends on what, which proposition a quiz tested), while
* event kinds/timestamps/refs — anything the model could silently
* mis-copy — are rebuilt deterministically from the transcript's tool
* parts. Idempotent + incremental: statement/alias matching dedupes nodes,
* (from,to,type) dedupes edge assertions, chat-scoped refs dedupe events,
* one mapping line per (chat, graph node), and sessions.jsonl checkpoints
* which chats have been extracted (never the same turns twice).
* is pure code: the LLM supplies exactly the judgment calls (what the
* propositions are, what follows/supports what, which proposition a quiz
* tested, whether a proposition IS an existing node), while event
* kinds/timestamps/refs/provenance — anything the model could silently
* mis-copy — are rebuilt deterministically from the transcript's tool parts
* and the message ids in the findings. Idempotent + incremental: refs are
* chat-scoped idempotency keys, edges dedupe by (from,to,type)-already-
* asserted-by-this-chat, mapping lines dedupe by (chat, sourceId), and
* sessions.jsonl checkpoints which chats have been extracted (never the
* same turns twice).
*
* Identity across re-encounters is the AUDITOR's judgment (link-or-create),
* not string matching: the conservative v1 exact-statement merge was
* replaced by the 2026-09-02 redesign. A re-extracted session's
* propositions link to the existing nodes; the link appends a node line
* whose phrasings join `aliases` and whose provenance unions (getGraph
* merges lines by id).
*
* Derivation below (getGraph / deriveNodeStates / graphSnapshot) is reads
* only — everything is recomputed from the ledgers. Every function takes
@@ -29,14 +41,14 @@
import { randomBytes } from "node:crypto";
import { isToolUIPart, type UIMessage } from "ai";
import type {
AuditResult,
GraphEdge,
GraphNode,
ProvenancePoint,
GraphSnapshot,
LedgerEvent,
LedgerEventKind,
NodeStateClass,
PlanToolResult,
SessionFindings,
} from "./contracts.ts";
import type { QuestionAnswerData } from "./question-core.ts";
import type { ChatStore, GraphStore } from "./ports.ts";
@@ -80,19 +92,25 @@ function toolParts(messages: UIMessage[]): ToolPartView[] {
return out;
}
/** A plan tool part's output, if it is a well-formed PlanToolResult. */
function planResult(part: ToolPartView): PlanToolResult | null {
if (part.state !== "output-available") return null;
const out = part.output;
if (typeof out !== "object" || out === null) return null;
if (!("tree" in out) || typeof (out as PlanToolResult).ok !== "boolean") return null;
return out as PlanToolResult;
/** (chatId, messageIds) → provenance points; order-preserving, deduped. */
function pointsOf(chatId: string, messages: string[]): ProvenancePoint[] {
const seen = new Set<string>();
const out: ProvenancePoint[] = [];
for (const messageId of messages) {
if (messageId.length === 0 || seen.has(messageId)) continue;
seen.add(messageId);
out.push({ chatId, messageId });
}
return out;
}
const samePoint = (a: ProvenancePoint, b: ProvenancePoint): boolean =>
a.chatId === b.chatId && a.messageId === b.messageId;
// ─── the merger (extraction pipeline stage 3 — deterministic) ────────────────
export interface EmissionStats {
/** New graph nodes appended (and mappings with them). */
/** New graph nodes appended. */
nodes: number;
/** New edge assertions appended. */
edges: number;
@@ -102,156 +120,135 @@ export interface EmissionStats {
/**
* Merge one chat's audited findings into the graph ledgers — the pipeline's
* apply stage. Deterministic + idempotent: re-merging the same findings (or
* apply stage. Deterministic + idempotent: re-merging the same verdicts (or
* a later re-extraction of the same chat) appends nothing but the new.
*
* Identity resolution (v1: conservative): a proposition whose statement or
* alias EXACTLY matches (case-insensitive, trimmed) an existing node's
* statement or alias merges onto it; everything else is a new node. One
* mapping line per (chat, graph node) — the merge audit trail. Fuzzy /
* embedding adjudication stays deferred (vision §1 consolidation).
* Identity is the auditor's link-or-create verdict: `linkTo` an existing
* node (validated against the ledger — an unknown id falls back to create)
* appends a LINK line (the session's phrasings join the node's aliases and
* the provenance unions; getGraph merges lines by id); absent → a new node.
* Mapping lines record each verdict: one per (chat, sourceId). Idempotent:
* a proposition id already mapped for this chat reuses its resolution.
*
* Events are rebuilt deterministically from the transcript's quiz and
* plan_mark_done parts (kinds, timestamps, notes, refs). TAGGED events
* (quiz nodeId, mark_done tree node) resolve through the proposition
* mapping — deterministic; the findings' attribution is consulted ONLY for
* untagged probe quizzes: the retroactive probe-attribution resolution (a
* pre-tree probe maps to a proposition the tree only articulates later).
* Unattributed quizzes land with nodeId null + the question text, exactly
* like the unattributable ones always have.
* Events are rebuilt deterministically from the transcript: quiz outcomes
* (correct / miss / dont-know; kinds + timestamps + refs from the tool
* parts, nodeId from the extractor's attribution through this run's
* mapping) plus one observation event per proposition carrying an
* understanding note (ref `obs:<sourceId>`). No taught events, ever.
*/
export function mergeFindings(chatId: string, findings: SessionFindings, deps: GraphDeps): EmissionStats {
export function mergeFindings(chatId: string, audit: AuditResult, deps: GraphDeps): EmissionStats {
const { graphStore: ledgers, chatStore } = deps;
const record = chatStore.getChat(chatId);
if (!record) return { nodes: 0, edges: 0, events: 0 };
const stats: EmissionStats = { nodes: 0, edges: 0, events: 0 };
// 1. Identity resolution. mapping: proposition id (and tree node id)
// → graph node id — from this chat's existing mapping lines, then
// extended as findings resolve. chatNodeIds: graph nodes this chat has
// already mapped onto — the (chat, graph node) mapping dedupe.
// 1. Resolution: this run's propositions → graph node ids. Existing
// mapping lines for THIS chat are loaded first — the audit trail doubles
// as the resolution table, which makes re-merging the same chat
// idempotent (a re-extraction re-merges through the auditor's linkTo
// verdicts instead). Fresh sourceIds resolve as the verdicts say.
const existingIds = new Set(ledgers.readNodes().map((n) => n.id));
const mapping = new Map<string, string>();
const chatNodeIds = new Set<string>();
const seenMappingKeys = new Set<string>();
for (const m of ledgers.readMappings()) {
if (m.chatId !== chatId || typeof m.sourceId !== "string") continue;
if (m.chatId !== chatId) continue;
mapping.set(m.sourceId, m.graphNodeId);
chatNodeIds.add(m.graphNodeId);
}
// 2. Propositions → graph nodes. Exact statement/alias match against the
// whole ledger first (cross-session reuse), then against propositions
// created earlier in THIS merge (intra-findings dedupe).
const norm = (s: string): string => s.trim().toLowerCase();
const existingByTitle = new Map<string, string>();
for (const node of ledgers.readNodes()) {
for (const title of [node.statement, ...node.aliases]) {
const key = norm(title);
if (key && !existingByTitle.has(key)) existingByTitle.set(key, node.id);
}
seenMappingKeys.add(m.sourceId);
}
const now = new Date().toISOString();
for (const p of findings.propositions) {
if (mapping.has(p.id)) continue;
const keys = [p.statement, ...p.aliases].map(norm).filter((k) => k.length > 0);
const matchId = keys.map((k) => existingByTitle.get(k)).find((id) => id !== undefined);
for (const p of audit.propositions) {
if (mapping.has(p.id)) continue; // already resolved for this chat — idempotent
const points = pointsOf(chatId, p.messages);
let graphNodeId: string;
if (matchId !== undefined) {
graphNodeId = matchId; // reuse — conservative merge, duplicates over wrong merges
} else {
const graphNode: GraphNode = {
id: newGraphNodeId(),
name: p.name,
if (p.linkTo !== undefined && existingIds.has(p.linkTo)) {
// Identity verdict: this proposition IS that node. Append a link
// line — the session's phrasings (statement + aliases) join the
// node's aliases, the provenance unions. Reads merge by id.
graphNodeId = p.linkTo;
ledgers.appendNode({
id: graphNodeId,
statement: p.statement,
description: p.description,
aliases: [...p.aliases],
kind: p.kind,
createdAt: now,
provenance: points,
});
} else {
const node: GraphNode = {
id: newGraphNodeId(),
statement: p.statement,
description: p.description,
aliases: [...p.aliases],
createdAt: now,
provenance: points,
};
ledgers.appendNode(graphNode);
ledgers.appendNode(node);
stats.nodes++;
graphNodeId = graphNode.id;
}
for (const key of keys) if (!existingByTitle.has(key)) existingByTitle.set(key, graphNodeId);
if (!chatNodeIds.has(graphNodeId)) {
ledgers.appendMapping({ chatId, sourceId: p.treeNodeId ?? p.id, graphNodeId });
chatNodeIds.add(graphNodeId);
graphNodeId = node.id;
existingIds.add(graphNodeId);
}
mapping.set(p.id, graphNodeId);
if (p.treeNodeId !== undefined) mapping.set(p.treeNodeId, graphNodeId);
if (!seenMappingKeys.has(p.id)) {
ledgers.appendMapping({ chatId, sourceId: p.id, graphNodeId });
seenMappingKeys.add(p.id);
}
}
// 3. Edges. Dedupe by (from,to,type) across the whole ledger: an existing
// assertion just gains this chatId as provenance (appended as a new
// line; reads merge).
const asserted = new Map<string, Set<string>>();
// 2. Edges. Resolve endpoints through the mapping; dedupe by
// (from,to,type) already asserted by THIS chat (an existing assertion
// by another chat stays untouched — provenance unions at read time).
const assertedByChat = new Set<string>();
for (const line of ledgers.readEdges()) {
const key = `${line.from}\u0000${line.to}\u0000${line.type}`;
let provenance = asserted.get(key);
if (!provenance) {
provenance = new Set();
asserted.set(key, provenance);
}
for (const c of line.provenance) provenance.add(c);
if (line.provenance.some((pt) => pt.chatId === chatId)) assertedByChat.add(key);
}
for (const edge of findings.edges) {
for (const edge of audit.edges) {
const from = mapping.get(edge.from);
const to = mapping.get(edge.to);
if (!from || !to || from === to) continue; // unmapped endpoint — skip
const key = `${from}\u0000${to}\u0000${edge.type}`;
const provenance = asserted.get(key);
if (provenance?.has(chatId)) continue; // already asserted by this chat
ledgers.appendEdge({ from, to, type: edge.type, provenance: [chatId] });
if (provenance) provenance.add(chatId);
else asserted.set(key, new Set([chatId]));
if (assertedByChat.has(key)) continue;
ledgers.appendEdge({ from, to, type: edge.type, provenance: pointsOf(chatId, edge.messages) });
assertedByChat.add(key);
stats.edges++;
}
// 4. Events, rebuilt deterministically in transcript order so the
// probe/check boundary is exact: quizzes BEFORE the first plan_init
// output are probes. Refs are chat-scoped (a quiz toolCallId or
// taught:<treeNodeId>) — a ref already in the ledger means this turn
// was extracted before. The findings' attribution supplies the
// nodeId, nothing else.
// 3. Events, rebuilt deterministically in transcript order. Refs are
// chat-scoped idempotency keys (quiz toolCallId, obs:<sourceId>) —
// a ref already in the ledger means this evidence was extracted before.
const seenRefs = new Set<string>();
for (const event of ledgers.readEvents()) {
if (event.chatId === chatId && event.ref) seenRefs.add(event.ref);
}
let planSeen = false;
for (const part of toolParts(record.messages)) {
// The boundary flips on the first plan_init OUTPUT, whatever it says
// (an ok:false "tree already initialized" still means a tree exists).
if (part.name === "plan_init" && part.state === "output-available") planSeen = true;
if (part.name === "quiz" && part.state === "output-available") {
const event = quizEvent(part, chatId, mapping, findings, planSeen, now);
if (event?.ref !== undefined && !seenRefs.has(event.ref)) {
ledgers.appendEvent(event);
seenRefs.add(event.ref);
stats.events++;
}
}
if (part.name === "plan_mark_done" && part.state === "output-available") {
const result = planResult(part);
const input = part.input as { id?: unknown } | undefined;
if (result?.ok && typeof input?.id === "string") {
const ref = `taught:${input.id}`;
if (!seenRefs.has(ref)) {
// mark_done is tagged with the tree node id — deterministic.
ledgers.appendEvent({
nodeId: mapping.get(input.id) ?? null,
kind: "taught",
chatId,
at: now,
ref,
});
seenRefs.add(ref);
stats.events++;
}
}
if (part.name !== "quiz" || part.state !== "output-available") continue;
const event = quizEvent(part, chatId, audit, mapping, now);
if (event?.ref !== undefined && !seenRefs.has(event.ref)) {
ledgers.appendEvent(event);
seenRefs.add(event.ref);
stats.events++;
}
}
// Understanding observations: one per proposition carrying a note,
// resolved to the node the verdict mapped it to.
for (const p of audit.propositions) {
if (p.understanding === undefined || p.understanding.length === 0) continue;
const ref = `obs:${p.id}`;
if (seenRefs.has(ref)) continue;
ledgers.appendEvent({
nodeId: mapping.get(p.id) ?? null,
kind: "observation",
chatId,
at: now,
ref,
note: p.understanding,
});
seenRefs.add(ref);
stats.events++;
}
// 5. Checkpoint: this chat is extracted through its current updatedAt.
// 4. Checkpoint: this chat is extracted through its current updatedAt.
ledgers.appendCheckpoint({ chatId, foldedAt: record.updatedAt });
return stats;
}
@@ -261,9 +258,8 @@ export function mergeFindings(chatId: string, findings: SessionFindings, deps: G
function quizEvent(
part: ToolPartView,
chatId: string,
audit: AuditResult,
mapping: Map<string, string>,
findings: SessionFindings,
planSeen: boolean,
now: string,
): LedgerEvent | null {
const out = part.output;
@@ -274,16 +270,13 @@ function quizEvent(
let kind: LedgerEventKind;
if (answer.dontKnow === true) kind = "dont-know";
else if (answer.correct === true) kind = planSeen ? "check-correct" : "probe-correct";
else kind = planSeen ? "check-incorrect" : "probe-miss";
else if (answer.correct === true) kind = "correct";
else kind = "miss";
// Tagged (input.nodeId) resolves deterministically through the mapping;
// attribution is the LLM judgment call for UNTAGGED probes only — the
// vision's open probe-attribution question, resolved retroactively.
const input = part.input as { nodeId?: unknown } | undefined;
const tagged = typeof input?.nodeId === "string" ? mapping.get(input.nodeId) : undefined;
const attributedId = findings.attribution.quiz[part.toolCallId];
const nodeId = tagged ?? (attributedId ? (mapping.get(attributedId) ?? null) : null);
// Tree-blind: EVERY quiz attributes through the extractor's judgment —
// the vision's retroactive attribution, with no tree mapping to lean on.
const attributedId = audit.attribution.quiz[part.toolCallId];
const nodeId = attributedId ? (mapping.get(attributedId) ?? null) : null;
const event: LedgerEvent = {
nodeId,
@@ -293,7 +286,7 @@ function quizEvent(
ref: part.toolCallId,
};
if (typeof answer.note === "string" && answer.note.length > 0) event.note = answer.note;
// The question text is the attribution input for unattributed probes.
// The question text is the attribution input for unattributed quizzes.
if (nodeId === null && typeof answer.questionText === "string") event.question = answer.questionText;
return event;
}
@@ -335,20 +328,39 @@ export interface DerivedGraph {
}
/**
* The whole graph from the ledgers. Edge LINES are assertions; the returned
* edges are merged by (from,to,type) with unioned provenance, first-seen
* order. `store events, derive state` — this is the only reader that matters.
* The whole graph from the ledgers. Node LINES may repeat an id (link-
* appends): the FIRST line defines statement/description/createdAt; later
* lines' statements and aliases join `aliases`, provenance unions. Edge
* LINES are assertions, merged by (from,to,type) with unioned provenance,
* first-seen order. `store events, derive state` — this is the only reader
* that matters.
*/
export function getGraph(graphStore: GraphStore): DerivedGraph {
const nodes = graphStore.readNodes();
// Merge node lines by id (first line defines; later lines add phrasings).
const byId = new Map<string, GraphNode>();
for (const line of graphStore.readNodes()) {
const existing = byId.get(line.id);
if (!existing) {
byId.set(line.id, { ...line, aliases: [...line.aliases], provenance: [...line.provenance] });
continue;
}
// A later line's statement is a phrasing, not a replacement.
for (const phrasing of [line.statement, ...line.aliases]) {
if (phrasing.length > 0 && !existing.aliases.includes(phrasing)) existing.aliases.push(phrasing);
}
for (const pt of line.provenance) {
if (!existing.provenance.some((q) => samePoint(q, pt))) existing.provenance.push(pt);
}
}
const merged: GraphEdge[] = [];
const byKey = new Map<string, GraphEdge>();
for (const line of graphStore.readEdges()) {
const key = `${line.from}\u0000${line.to}\u0000${line.type}`;
const existing = byKey.get(key);
if (existing) {
for (const c of line.provenance) {
if (!existing.provenance.includes(c)) existing.provenance.push(c);
for (const pt of line.provenance) {
if (!existing.provenance.some((q) => samePoint(q, pt))) existing.provenance.push(pt);
}
} else {
const edge: GraphEdge = { ...line, provenance: [...line.provenance] };
@@ -356,34 +368,27 @@ export function getGraph(graphStore: GraphStore): DerivedGraph {
merged.push(edge);
}
}
return { nodes, edges: merged, events: graphStore.readEvents() };
return { nodes: [...byId.values()], edges: merged, events: graphStore.readEvents() };
}
const QUIZ_EVENT_KINDS: ReadonlySet<string> = new Set([
"probe-correct",
"check-correct",
"probe-miss",
"check-incorrect",
"dont-know",
]);
const QUIZ_EVENT_KINDS: ReadonlySet<string> = new Set(["correct", "miss", "dont-know"]);
/**
* Derive node states from events (never stored): taught if ANY taught event;
* else the LATEST quiz event decides (known / missed / dont-know); a node
* with no events is a candidate.
* Derive node states from events (never stored): the LATEST quiz event
* decides (known / missed / dont-know); a node with no quiz events is a
* candidate. TRANSITIONAL (2026-09-02 redesign): `taught` is unreachable
* now that no taught events are emitted, and observation events are NOT
* state inputs — the real state model (dependency depth + topology-derived
* strength + quiz events) is pending ideation; this keeps the renderer
* alive until it lands.
*/
export function deriveNodeStates(nodes: GraphNode[], events: LedgerEvent[]): Record<string, NodeStateClass> {
const states: Record<string, NodeStateClass> = {};
for (const node of nodes) states[node.id] = "candidate";
const taught = new Set<string>();
const latest = new Map<string, { at: string; kind: LedgerEventKind; index: number }>();
events.forEach((event, index) => {
if (event.nodeId === null) return;
if (event.kind === "taught") {
taught.add(event.nodeId);
return;
}
if (!QUIZ_EVENT_KINDS.has(event.kind)) return;
const current = latest.get(event.nodeId);
if (!current || event.at > current.at || (event.at === current.at && index > current.index)) {
@@ -392,13 +397,9 @@ export function deriveNodeStates(nodes: GraphNode[], events: LedgerEvent[]): Rec
});
for (const node of nodes) {
if (taught.has(node.id)) {
states[node.id] = "taught";
continue;
}
const last = latest.get(node.id);
if (!last) continue; // candidate
if (last.kind === "probe-correct" || last.kind === "check-correct") states[node.id] = "known";
if (last.kind === "correct") states[node.id] = "known";
else if (last.kind === "dont-know") states[node.id] = "dont-know";
else states[node.id] = "missed";
}
@@ -422,10 +423,13 @@ export function graphSnapshot(deps: GraphDeps): GraphSnapshot {
chats.add(chatId);
};
for (const m of deps.graphStore.readMappings()) touch(m.graphNodeId, m.chatId);
for (const node of nodes) {
for (const pt of node.provenance) touch(node.id, pt.chatId);
}
for (const edge of edges) {
for (const chatId of edge.provenance) {
touch(edge.from, chatId);
touch(edge.to, chatId);
for (const pt of edge.provenance) {
touch(edge.from, pt.chatId);
touch(edge.to, pt.chatId);
}
}
for (const event of events) {
+1 -1
View File
@@ -56,7 +56,7 @@ export interface GraphStore {
appendEdge(assertion: GraphEdge): void;
readEdges(): GraphEdge[];
/** Append one quiz outcome / taught event. */
/** Append one quiz outcome / understanding observation. */
appendEvent(event: LedgerEvent): void;
readEvents(): LedgerEvent[];
+28 -35
View File
@@ -1,20 +1,26 @@
/**
* GraphStore — the fs JSONL adapter: five append-only ledgers at
* config.LEDGER_FILES (contracts.ts "v1 emission").
* nodes.jsonl GraphNode — one line per graph node
* nodes.jsonl GraphNode — one line per node write: the FIRST
* line for an id defines it; a later
* line with a REPEATED id is a
* link-append (session phrasings join
* aliases, provenance unions); reads
* merge by id (core/graph.ts getGraph)
* edges.jsonl GraphEdge — one line per edge ASSERTION (a second
* session asserting the same (from,to,type)
* appends another line; reads merge)
* events.jsonl LedgerEvent — quiz outcomes + taught events
* events.jsonl LedgerEvent — quiz outcomes + understanding observations
* mappings.jsonl NodeMapping — the session→graph identity map:
* (chatId, sourceId) → graphNodeId
* sessions.jsonl SessionCheckpoint— emission checkpoints (foldedAt)
*
* Implements core/ports.ts GraphStore. data/graph/* is exclusively this
* adapter's. Reads are strict against the contracts.ts shapes: corrupt lines
* are skipped, never guessed at — the ledgers are the source of truth and
* everything else is derived (core/graph.ts, which takes this port as a
* parameter). Same idioms as src/store/chats.ts: sync fs, no framework.
* adapter's. Reads are strict against the contracts.ts shapes (the 2026-09-02
* graph redesign): corrupt lines are skipped, never guessed at — the ledgers
* are the source of truth and everything else is derived (core/graph.ts,
* which takes this port as a parameter). Same idioms as src/store/chats.ts:
* sync fs, no framework.
*/
import * as fs from "node:fs";
@@ -26,6 +32,7 @@ import type {
LedgerEvent,
LedgerEventKind,
NodeMapping,
ProvenancePoint,
SessionCheckpoint,
} from "../core/contracts.ts";
import type { GraphStore } from "../core/ports.ts";
@@ -69,26 +76,22 @@ function readValid<T>(file: string, isValid: (v: unknown) => v is T): T[] {
return readRawLines(file).filter(isValid);
}
/** Pre-name ledger line (the first extraction runs) — read tolerantly. */
type StoredGraphNode = Omit<GraphNode, "name"> & { name?: string };
/** Provenance points are (chatId, messageId) pairs — message-granular. */
const isProvenancePoint = (v: unknown): v is ProvenancePoint =>
isRecord(v) && isString(v.chatId) && isString(v.messageId);
const isStoredGraphNode = (v: unknown): v is StoredGraphNode =>
const isGraphNode = (v: unknown): v is GraphNode =>
isRecord(v) &&
isString(v.id) &&
isString(v.statement) &&
isString(v.description) &&
Array.isArray(v.aliases) &&
(v.kind === "truth" || v.kind === "misconception") &&
v.aliases.every(isString) &&
isString(v.createdAt) &&
(v.name === undefined || isString(v.name));
Array.isArray(v.provenance) &&
v.provenance.every(isProvenancePoint);
/** Derived header for legacy node lines written before `name` existed:
* the first clause of the statement, capped — good enough to render. */
function nameFromStatement(statement: string): string {
const clause = statement.split(/[.;:—]/)[0]!.trim();
return clause.length > 60 ? `${clause.slice(0, 57)}…` : clause || statement;
}
const EDGE_TYPES = ["requires", "supports"] as const;
const EDGE_TYPES = ["follows", "supports"] as const;
const isGraphEdge = (v: unknown): v is GraphEdge =>
isRecord(v) &&
@@ -96,17 +99,9 @@ const isGraphEdge = (v: unknown): v is GraphEdge =>
isString(v.to) &&
(EDGE_TYPES as readonly string[]).includes(v.type as string) &&
Array.isArray(v.provenance) &&
v.provenance.every(isString);
v.provenance.every(isProvenancePoint);
const EVENT_KINDS: readonly LedgerEventKind[] = [
"probe-correct",
"probe-miss",
"dont-know",
"misconception-signaled",
"taught",
"check-correct",
"check-incorrect",
];
const EVENT_KINDS: readonly LedgerEventKind[] = ["correct", "miss", "dont-know", "observation"];
const isLedgerEvent = (v: unknown): v is LedgerEvent =>
isRecord(v) &&
@@ -127,13 +122,11 @@ export const graphStore: GraphStore = {
appendNode(node: GraphNode): void {
appendLine(LEDGER_FILES.nodes, node);
},
/** Raw node lines in append order (ids are unique — no merging needed).
* Legacy lines without a `name` get one derived from the statement. */
/** Raw node lines in append order. A node id may appear on more than
* one line (a later session's LINK-append: phrasings + provenance); the
* derivation (core/graph.ts getGraph) merges lines by id. */
readNodes(): GraphNode[] {
return readValid(LEDGER_FILES.nodes, isStoredGraphNode).map((n) => ({
...n,
name: n.name ?? nameFromStatement(n.statement),
}));
return readValid(LEDGER_FILES.nodes, isGraphNode);
},
/** Append one edge ASSERTION; readers merge lines by (from,to,type). */
+89 -71
View File
@@ -1,13 +1,16 @@
/**
* The extraction pipeline tests (vision §1 construction): extractor →
* auditor → merger, the sole graph writer.
* The extraction pipeline tests (vision §1 construction, 2026-09-02 redesign):
* extractor → auditor → merger, the sole graph writer — tree-blind,
* free-form, with the auditor owning faithfulness and identity.
*
* transcriptView — the compact view both LLM stages see (pure)
* transcriptView — the compact view both LLM stages see (pure):
* message-id-tagged verbatim text + quiz lines, NO tree
* pipeline — MockLanguageModelV4 drives both agents: findings out,
* audit verdict in, deterministic merge applied; failure
* paths checkpoint nothing; single-flight per chat
* audit verdict (with linkTo identity) in, deterministic
* merge applied; failure paths checkpoint nothing;
* single-flight per chat
*
* The merger itself (identity resolution, event rebuild, idempotency) is
* The merger itself (identity verdicts, event rebuild, idempotency) is
* tested in tests/graph.test.ts directly against mergeFindings — here it is
* the integration: what one full pipeline run does to the ledgers.
*
@@ -21,9 +24,16 @@ import * as fs from "node:fs";
import { after, describe, it } from "node:test";
import { MockLanguageModelV4 } from "ai/test";
import type { UIMessage } from "ai";
import type { GraphEdge, GraphNode, LedgerEvent, NodeMapping, SessionCheckpoint, SessionFindings } from "../src/core/contracts.ts";
import type {
AuditResult,
GraphEdge,
GraphNode,
LedgerEvent,
NodeMapping,
SessionCheckpoint,
SessionFindings,
} from "../src/core/contracts.ts";
import type { GraphStore } from "../src/core/ports.ts";
import { createTree, markDone } from "../src/core/plan-tree.ts";
import { createExtractionPipeline, transcriptView } from "../src/agent/subagents/extraction.ts";
import { chatStore } from "../src/store/chats.ts";
import { PATHS } from "../config.ts";
@@ -72,7 +82,7 @@ function quizPart(toolCallId: string, input: Record<string, unknown>, output: Re
return { type: "tool-quiz", toolCallId, state: "output-available", input, output };
}
function planPart(toolCallId: string, name: string, input: Record<string, unknown>, tree: ReturnType<typeof createTree>): unknown {
function planPart(toolCallId: string, name: string, input: Record<string, unknown>, tree: unknown): unknown {
return {
type: `tool-${name}`,
toolCallId,
@@ -82,21 +92,11 @@ function planPart(toolCallId: string, name: string, input: Record<string, unknow
};
}
/** A session-shaped fixture chat: probe → plan → check → mark_done. */
/** A session-shaped fixture chat: probe → teaching text → check. Tree-blind
* world: the plan tree is NOT part of extraction — it is noise. */
function fixtureChat() {
const tree = markDone(
createTree(
[
{ id: "cr", title: "Chain rule intuition" },
{ id: "apply", title: "Applying the chain rule" },
],
[{ from: "cr", to: "apply" }],
),
"cr",
);
const chat = chatStore.createChat("Extraction fixture");
createdChats.push(chat.id);
chatStore.saveTree(chat.id, tree);
chatStore.saveMessages(chat.id, [
user("u1", "teach me the chain rule"),
assistant("a1", [
@@ -107,15 +107,14 @@ function fixtureChat() {
]),
assistant("a2", [
{ type: "text", text: "The derivative measures how fast a function changes." },
planPart("p1", "plan_init", { nodes: [], edges: [] }, tree),
planPart("p1", "plan_init", { nodes: [], edges: [] }, { nodes: [], edges: [], currentNodeId: null, doneNodeIds: [], version: 1 }),
]),
assistant("a3", [
quizPart("check-1", { question: "Check", nodeId: "apply" }, {
quizPart("check-1", { question: "Check" }, {
questionId: "check-1", kind: "quiz", questionText: "Check", correct: true, dontKnow: false,
answeredAt: "2026-08-22T02:00:00.000Z",
}),
]),
assistant("a4", [planPart("p2", "plan_mark_done", { id: "cr" }, tree)]),
]);
return { chatId: chat.id, record: chatStore.getChat(chat.id)! };
}
@@ -130,30 +129,38 @@ after(() => {
}
});
/** What the extractor would produce for the fixture session. */
/** What the extractor would produce for the fixture session — tree-blind,
* free-form, with message-granular provenance + an understanding note. */
const FINDINGS: SessionFindings = {
propositions: [
{
id: "p1",
name: "Chain rule",
statement: "Chain rule intuition",
statement: "The derivative measures how fast a function changes",
description: "The derivative of a function at a point is its instantaneous rate of change there.",
aliases: [],
kind: "truth",
treeNodeId: "cr",
evidence: ["TEACHER: The derivative measures how fast a function changes."],
messages: ["a2"],
evidence: ["TEACHER [a2]: The derivative measures how fast a function changes."],
understanding: "solid",
},
{
id: "p2",
name: "Applying chain rule",
statement: "Applying the chain rule",
statement: "Chain rule intuition",
description: "The chain rule differentiates composed functions by multiplying local rates.",
aliases: [],
kind: "truth",
treeNodeId: "apply",
evidence: ["FINAL PLAN TREE (node ids): cr = Chain rule intuition; apply = Applying the chain rule"],
messages: ["a2"],
evidence: ["LEARNER [u1]: teach me the chain rule"],
},
],
edges: [{ from: "p1", to: "p2", type: "requires", evidence: ["TREE EDGES: cr requires apply"] }],
attribution: { quiz: { "probe-1": "p1" } },
edges: [
{
from: "p1",
to: "p2",
type: "follows",
messages: ["a2"],
evidence: ["LEARNER [u1]: teach me the chain rule"],
},
],
attribution: { quiz: { "probe-1": "p1", "check-1": "p2" } },
};
/** A model whose ONE tool call is `name` with the given input; then text. */
@@ -202,28 +209,26 @@ function silentModel(): MockLanguageModelV4 {
// ─── transcriptView (pure) ───────────────────────────────────────────────────
describe("transcriptView (pure)", () => {
it("shows the final tree, verbatim conversation, structured events — plan ops folded", () => {
it("shows message-id-tagged verbatim conversation + quiz lines — tree-blind, plan ops folded", () => {
const { record } = fixtureChat();
const view = transcriptView(record);
// The final tree with node ids (the extractor's backbone + the
// pruned-out stays pruned — only the FINAL tree is shown).
assert.match(view, /SESSION: Extraction fixture/);
assert.match(view, /FINAL PLAN TREE \(node ids\): cr = Chain rule intuition; apply = Applying the chain rule/);
assert.match(view, /TREE EDGES: cr requires apply/);
// Conversation verbatim — evidence quotes can come from it.
assert.match(view, /LEARNER: teach me the chain rule/);
assert.match(view, /TEACHER: The derivative measures how fast a function changes\./);
// Conversation verbatim, each line tagged with its message id —
// the provenance anchors the extractor reports back.
assert.match(view, /LEARNER \[u1\] teach me the chain rule/);
assert.match(view, /TEACHER \[a2\] The derivative measures how fast a function changes\./);
// Structured events, attribution-ready.
assert.match(view, /QUIZ \[probe-1\] What is a derivative, roughly\? — the learner answered incorrectly/);
assert.match(view, /QUIZ \[check-1\] Check — the learner answered correctly/);
assert.match(view, /TEACHER marked tree node done \[p2\]: cr/);
// plan_init / plan_mark_done tool parts themselves are folded into
// the tree line — no raw JSON noise.
assert.doesNotMatch(view, /plan_init/);
// TREE-BLIND: no plan tree, no tree edges, no mark_done lines — plan
// ops fold to the generic one-liner like every other non-quiz tool.
assert.doesNotMatch(view, /FINAL PLAN TREE/);
assert.doesNotMatch(view, /TREE EDGES/);
assert.match(view, /\(plan_init tool result delivered\)/);
assert.doesNotMatch(view, /output-available/);
});
@@ -253,48 +258,61 @@ describe("transcriptView (pure)", () => {
// ─── the pipeline ───────────────────────────────────────────────────────────
describe("the extraction pipeline (extractor → auditor → merger)", () => {
it("runs end to end: findings → audit verdict → deterministic merge + checkpoint", async () => {
it("runs end to end: findings → audit verdict (identity) → deterministic merge + checkpoint", async () => {
const { chatId } = fixtureChat();
const ledgers = memoryLedgers();
// The auditor passes p1 through, drops p2 (unbacked), re-points the
// probe attribution to p1, and records the drop.
const verified: SessionFindings = {
propositions: [FINDINGS.propositions[0]!],
// The auditor passes p1 through and LINKS it to a pre-existing node
// (identity verdict), drops p2 (unbacked) + the edge with it.
const existingId = "n_existing";
ledgers.appendNode({
id: existingId,
statement: "Derivatives measure rates of change",
description: "Pre-existing node from an earlier session.",
aliases: [],
createdAt: "2026-08-01T00:00:00.000Z",
provenance: [],
});
const verdict: AuditResult = {
propositions: [
{ ...FINDINGS.propositions[0]!, linkTo: existingId },
],
edges: [],
attribution: { quiz: { "probe-1": "p1" } },
dropped: [{ ref: "p2", reason: "evidence not found" }, { ref: "p1→p2", reason: "endpoint dropped" }],
};
const pipeline = createExtractionPipeline({
chatStore,
graphStore: ledgers,
extractor: submitModel("submit_findings", FINDINGS),
auditor: submitModel("submit_review", { ...verified, dropped: [{ ref: "p2", reason: "evidence not found" }] }),
auditor: submitModel("submit_review", verdict),
});
const stats = await pipeline.runExtraction(chatId);
assert.deepEqual(stats, { nodes: 1, edges: 0, events: 3 });
// The linked proposition created NO new node; the understanding note
// became an observation; both quizzes became events.
assert.deepEqual(stats, { nodes: 0, edges: 0, events: 3 });
// Only the audited proposition became a node; mapping provenance kept.
assert.equal(ledgers.readNodes().length, 1);
assert.equal(ledgers.readNodes()[0]!.statement, "Chain rule intuition");
assert.deepEqual(ledgers.readMappings(), [{ chatId, sourceId: "cr", graphNodeId: ledgers.readNodes()[0]!.id }]);
// Link-append: the phrasing joined the existing node's aliases and
// the provenance unioned; no second node line defines a new node.
assert.equal(ledgers.readNodes().length, 2); // the pre-seeded line + the link line
assert.deepEqual(ledgers.readMappings(), [{ chatId, sourceId: "p1", graphNodeId: existingId }]);
// Events rebuilt deterministically from the transcript — the probe
// retroactively attributed, the check tagged, mark_done mapped.
// Events rebuilt deterministically: quizzes in transcript order, then
// the observation. check-1's attribution pointed at the DROPPED p2 →
// unresolvable → null (the question text is kept for later attribution).
const events = ledgers.readEvents();
assert.deepEqual(
events.map((e) => [e.kind, e.ref]),
events.map((e) => [e.kind, e.ref, e.nodeId]),
[
["probe-miss", "probe-1"],
["check-correct", "check-1"],
["taught", "taught:cr"],
["miss", "probe-1", existingId],
["correct", "check-1", null],
["observation", "obs:p1", existingId],
],
);
const nodeId = ledgers.readNodes()[0]!.id;
assert.equal(events[0]!.nodeId, nodeId, "probe attributed via audit");
assert.equal(events[1]!.nodeId, null, "tagged to a dropped proposition → null");
assert.equal(events[2]!.nodeId, nodeId, "taught via tree-node mapping");
assert.equal(events[0]!.nodeId, existingId, "probe attributed via the verdict's mapping");
assert.equal(events[2]!.note, "solid", "the understanding note joins the ledger");
// Checkpointed through the record's updatedAt — never the same turns twice.
assert.equal(ledgers.readCheckpoints().length, 1);
@@ -318,7 +336,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
chatStore,
graphStore: ledgers,
extractor: silentModel(),
auditor: submitModel("submit_review", FINDINGS),
auditor: submitModel("submit_review", { ...FINDINGS, dropped: [] }),
});
await assert.rejects(failingExtractor.runExtraction(chatId), /the extractor never submitted findings/);
assert.equal(ledgers.readCheckpoints().length, 0);
+117 -138
View File
@@ -1,13 +1,14 @@
/**
* Graph-as-context tests (W8; vision §1 "the graph as session context").
* Graph-as-context tests (vision §1 "the graph as session context", 2026-09-02
* redesign vocabulary: follows edges, no node kinds).
*
* renderGraphContext — pure: empty / small / cap-truncation rendering
* buildGraphContext — per-request entry point (derived-graph → block)
* injection plumbing — chat-flow style: the teacher's instructions carry
* the block (MockLanguageModelV4 captures the prompt)
* exact-match merge — the merger (mergeFindings) dedupes
* case/whitespace-identical statements onto existing
* nodes (no fuzzy matching)
* cross-chat edges — the merger (mergeFindings) appends one ASSERTION
* line per session; reads merge with unioned
* message-granular provenance
*
* Ledger isolation: NONE needed for the merge tests — the merger's ledger is
* injectable (in-memory), so the live data/graph files are only ever READ
@@ -22,7 +23,14 @@ import * as path from "node:path";
import { after, describe, it } from "node:test";
import { MockLanguageModelV4, simulateReadableStream } from "ai/test";
import type { LanguageModelV4StreamPart } from "@ai-sdk/provider";
import type { GraphEdge, GraphNode, LedgerEvent, SessionFindings } from "../src/core/contracts.ts";
import type {
AuditResult,
GraphEdge,
GraphNode,
LedgerEvent,
NodeMapping,
SessionCheckpoint,
} from "../src/core/contracts.ts";
import type { NodeStateClass } from "../src/core/contracts.ts";
import { getGraph, mergeFindings } from "../src/core/graph.ts";
import type { GraphStore } from "../src/core/ports.ts";
@@ -35,19 +43,25 @@ import {
import { createTeacherAgent } from "../src/agent/teacher-agent.ts";
import { handleChatRequest } from "../src/server/chat-handler.ts";
import { chatStore } from "../src/store/chats.ts";
import type { NodeMapping, SessionCheckpoint } from "../src/core/contracts.ts";
import { graphStore } from "../src/store/graph.ts";
import { createTree } from "../src/core/plan-tree.ts";
import { PATHS } from "../config.ts";
// ─── fixtures ────────────────────────────────────────────────────────────────
function node(id: string, statement: string, over: Partial<GraphNode> = {}): GraphNode {
return { id, name: statement.split(";")[0]!, statement, aliases: [], kind: "truth", createdAt: "2026-08-22T00:00:00.000Z", ...over };
return {
id,
statement,
description: `${statement} — elaborated.`,
aliases: [],
createdAt: "2026-08-22T00:00:00.000Z",
provenance: [],
...over,
};
}
function edge(from: string, to: string): GraphEdge {
return { from, to, type: "requires", provenance: ["chat_x"] };
return { from, to, type: "follows", provenance: [{ chatId: "chat_x", messageId: "m1" }] };
}
function states(...entries: [string, NodeStateClass][]): Record<string, NodeStateClass> {
@@ -61,7 +75,7 @@ describe("renderGraphContext (pure)", () => {
assert.equal(renderGraphContext([], [], {}), null);
});
it("renders a header, state-grouped node lines (hotspots first), and chained requires edges", () => {
it("renders a header, state-grouped node lines (hotspots first), and chained follows edges", () => {
const nodes = [
node("n_a", "Chain rule intuition"),
node("n_b", "Quotient rule"),
@@ -73,26 +87,26 @@ describe("renderGraphContext (pure)", () => {
["n_a", "known"],
["n_b", "missed"],
["n_c", "candidate"],
["n_d", "taught"],
["n_d", "dont-know"],
);
const out = renderGraphContext(nodes, edges, st)!;
assert.ok(out.includes(GRAPH_CONTEXT_HEADER), "header line present");
assert.match(out, /4 nodes, 3 requires edges/);
assert.match(out, /4 nodes, 3 follows edges/);
// Node lines: "• [state] statement (id)".
assert.ok(out.includes("• [missed] Quotient rule (n_b)"));
assert.ok(out.includes("• [known] Chain rule intuition (n_a)"));
assert.ok(out.includes("• [taught] Composite functions (n_d)"));
assert.ok(out.includes("• [dont-know] Composite functions (n_d)"));
assert.ok(out.includes("• [candidate] Product rule (n_c)"));
// Groups in the decided order: hotspots, known, taught, candidates.
const order = ["Hotspots", "Known", "Taught", "Candidates"].map((g) => out.indexOf(g));
// Groups in the decided order: hotspots, known, candidates.
const order = ["Hotspots", "Known", "Candidates"].map((g) => out.indexOf(g));
assert.deepEqual([...order].sort((a, b) => a - b), order, "groups appear in priority order");
// Edges merged into chains: d → a → b → c (one line, three arrows).
assert.ok(out.includes("Composite functions → Chain rule intuition → Quotient rule → Product rule"));
assert.ok(out.includes("Requires (prerequisite → dependent)"));
assert.ok(out.includes("Follows (premise → derivation)"));
assert.ok(out.includes("===== end of knowledge graph ====="), "clear end delimiter");
});
@@ -104,15 +118,8 @@ describe("renderGraphContext (pure)", () => {
assert.ok(!out.includes("B → C"), "no edge was invented between siblings");
});
it("marks misconception-kind nodes as hotspots even when their state is candidate", () => {
const nodes = [node("n_m", "Division by zero is fine", { kind: "misconception" })];
const out = renderGraphContext(nodes, [], states(["n_m", "candidate"]))!;
assert.match(out, /• \[candidate\] Division by zero is fine \(n_m\) · misconception/);
assert.ok(out.includes("Hotspots"), "misconception nodes join the hotspot group");
});
it("caps at MAX_CONTEXT_NODES: hotspots + their 1-hop requires neighborhood only, with a truncation note", () => {
// 3 hotspots, each with a prerequisite and a dependent (1-hop), plus a
it("caps at MAX_CONTEXT_NODES: hotspots + their 1-hop follows neighborhood only, with a truncation note", () => {
// 3 hotspots, each with a premise and a derivation (1-hop), plus a
// long tail of disconnected candidates that must drop out.
const nodes: GraphNode[] = [];
const edges: GraphEdge[] = [];
@@ -121,7 +128,7 @@ describe("renderGraphContext (pure)", () => {
const h = `n_h${i}`;
const pre = `n_p${i}`;
const dep = `n_d${i}`;
nodes.push(node(pre, `Prerequisite ${i}`), node(h, `Hotspot ${i}`), node(dep, `Dependent ${i}`));
nodes.push(node(pre, `Premise ${i}`), node(h, `Hotspot ${i}`), node(dep, `Derivation ${i}`));
edges.push(edge(pre, h), edge(h, dep));
st[pre] = "known";
st[h] = "missed";
@@ -139,11 +146,11 @@ describe("renderGraphContext (pure)", () => {
const out = renderGraphContext(nodes, edges, st)!;
assert.ok(out.includes("• [missed] Hotspot 1 (n_h1)"), "hotspots stay");
assert.ok(out.includes("• [known] Prerequisite 1 (n_p1)"), "1-hop prerequisites stay");
assert.ok(out.includes("• [candidate] Dependent 1 (n_d1)"), "1-hop dependents stay");
assert.ok(out.includes("• [known] Premise 1 (n_p1)"), "1-hop premises stay");
assert.ok(out.includes("• [candidate] Derivation 1 (n_d1)"), "1-hop derivations stay");
assert.ok(!out.includes("Far away"), "disconnected nodes drop out");
assert.match(out, new RegExp(`showing 9 of ${nodes.length} nodes`), "truncation is noted");
assert.ok(out.includes(", 6 requires edges"), "edges among the selected nodes stay");
assert.ok(out.includes(", 6 follows edges"), "edges among the selected nodes stay");
});
it("keeps the full graph at exactly the cap (no truncation note)", () => {
@@ -169,11 +176,11 @@ describe("buildGraphContext", () => {
it("derives states from events and renders the block", () => {
const nodes = [node("n_a", "Solo node"), node("n_b", "Checked node")];
const events: LedgerEvent[] = [
{ nodeId: "n_b", kind: "check-correct", chatId: "c1", at: "2026-08-22T00:00:00.000Z", ref: "r1" },
{ nodeId: "n_b", kind: "correct", chatId: "c1", at: "2026-08-22T00:00:00.000Z", ref: "r1" },
];
const out = buildGraphContext({ nodes, edges: [], events })!;
assert.ok(out.includes(GRAPH_CONTEXT_HEADER));
assert.ok(out.includes("• [known] Checked node (n_b)"), "check-correct → known");
assert.ok(out.includes("• [known] Checked node (n_b)"), "correct → known");
assert.ok(out.includes("• [candidate] Solo node (n_a)"), "no events → candidate");
});
@@ -244,7 +251,7 @@ describe("graph-context injection (chat-flow style)", () => {
nodes: [node("n_ctx", "Chain rule intuition"), node("n_ctx2", "Derivative as a limit")],
edges: [edge("n_ctx2", "n_ctx")],
events: [
{ nodeId: "n_ctx2", kind: "check-correct", chatId: "c1", at: "2026-08-22T00:00:00.000Z", ref: "r1" },
{ nodeId: "n_ctx2", kind: "correct", chatId: "c1", at: "2026-08-22T00:00:00.000Z", ref: "r1" },
] as LedgerEvent[],
};
const context = buildGraphContext(graph);
@@ -296,7 +303,7 @@ describe("graph-context injection (chat-flow style)", () => {
});
});
// ─── exact-match merge (merger identity resolution) ──────────────────────────
// ─── cross-chat edge assertions (merger + read-time merging) ────────────────
/** An in-memory stand-in for the five ledgers (the GraphStore port, core/ports.ts). */
function memoryLedgers() {
@@ -317,128 +324,100 @@ function memoryLedgers() {
appendCheckpoint: (c: SessionCheckpoint) => void checkpoints.push(c),
readCheckpoints: () => checkpoints.map((c) => ({ ...c })),
// Test accessors (not part of the GraphStore surface).
counts: { get nodes() { return nodes.length; }, get mappings() { return mappings.length; } },
} satisfies GraphStore & { counts: { nodes: number; mappings: number } };
counts: { get nodes() { return nodes.length; } },
} satisfies GraphStore & { counts: { nodes: number } };
}
/** Findings as the extractor would file them for a plan tree: one
* proposition per node (statement = title, treeNodeId set), one `requires`
* edge per tree edge — the merge tests drive the merger with these. */
function treeFindings(tree: ReturnType<typeof createTree>): SessionFindings {
const idByNodeId = new Map(tree.nodes.map((n, i) => [n.id, `p${i + 1}`]));
return {
propositions: tree.nodes.map((n, i) => ({
id: `p${i + 1}`,
name: n.title,
statement: n.title,
aliases: [],
kind: "truth" as const,
treeNodeId: n.id,
evidence: ["fixture evidence"],
})),
edges: tree.edges.map((e) => ({
from: idByNodeId.get(e.from)!,
to: idByNodeId.get(e.to)!,
type: "requires" as const,
evidence: ["fixture evidence"],
})),
attribution: { quiz: {} },
};
}
/** A real chat record with a stored tree (the merger reads the real chat store). */
function fixtureChat(title: string, tree: ReturnType<typeof createTree>) {
/** A minimal chat record with one user message (the merger reads the real chat store). */
function fixtureChat(title: string) {
const chat = chatStore.createChat(title);
createdChats.push(chat.id);
chatStore.saveTree(chat.id, tree);
chatStore.saveMessages(chat.id, [{ id: "u1", role: "user", parts: [{ type: "text", text: "go" }] }]);
return chatStore.getChat(chat.id)!;
}
describe("exact-match merge (merger identity resolution)", () => {
it("maps a case/whitespace-identical title onto the existing node instead of duplicating", () => {
describe("cross-chat edge assertions (merger + read-time merging)", () => {
it("a second session asserting the same edge appends a second line; reads merge with unioned provenance", () => {
const ledgers = memoryLedgers();
const tree1 = createTree([{ id: "cr", title: "Chain rule intuition" }], []);
const chat1 = fixtureChat("Merge A", tree1);
let stats = mergeFindings(chat1.id, treeFindings(tree1), { chatStore, graphStore: ledgers });
assert.deepEqual(stats, { nodes: 1, edges: 0, events: 0 });
const firstId = ledgers.readNodes()[0]!.id;
// Second session, same proposition: different casing + padding.
const tree2 = createTree([{ id: "cr2", title: " chain Rule INTUITION " }], []);
const chat2 = fixtureChat("Merge B", tree2);
stats = mergeFindings(chat2.id, treeFindings(tree2), { chatStore, graphStore: ledgers });
assert.deepEqual(stats, { nodes: 0, edges: 0, events: 0 }, "no duplicate node appended");
assert.equal(ledgers.counts.nodes, 1, "still exactly one graph node");
assert.equal(ledgers.readNodes()[0]!.id, firstId);
// Provenance stays honest: one mapping line per (chat, graph node).
assert.equal(ledgers.counts.mappings, 2);
assert.deepEqual(ledgers.readMappings()[1], { chatId: chat2.id, sourceId: "cr2", graphNodeId: firstId });
});
it("matches aliases too (exact, case-insensitive)", () => {
const ledgers = memoryLedgers();
ledgers.appendNode({
id: "n_alias",
name: "Differentiation",
statement: "Differentiation basics",
aliases: ["the derivative"],
kind: "truth",
createdAt: "2026-08-22T00:00:00.000Z",
});
const tree = createTree([{ id: "d", title: "The Derivative" }], []);
const chat = fixtureChat("Merge alias", tree);
const stats = mergeFindings(chat.id, treeFindings(tree), { chatStore, graphStore: ledgers });
assert.deepEqual(stats, { nodes: 0, edges: 0, events: 0 }, "alias match → no new node");
assert.deepEqual(ledgers.readMappings(), [{ chatId: chat.id, sourceId: "d", graphNodeId: "n_alias" }]);
});
it("does NOT fuzzy-match near-identical titles (that stays deferred)", () => {
const ledgers = memoryLedgers();
const tree1 = createTree([{ id: "cr", title: "Chain rule intuition" }], []);
const chat1 = fixtureChat("Fuzzy A", tree1);
mergeFindings(chat1.id, treeFindings(tree1), { chatStore, graphStore: ledgers });
const tree2 = createTree([{ id: "cr2", title: "Chain rules intuition" }], []);
const chat2 = fixtureChat("Fuzzy B", tree2);
const stats = mergeFindings(chat2.id, treeFindings(tree2), { chatStore, graphStore: ledgers });
assert.equal(stats.nodes, 1, "near-miss statements stay separate nodes");
assert.equal(ledgers.counts.nodes, 2);
});
it("merged nodes unify the second chat's edges onto the existing graph node", () => {
const ledgers = memoryLedgers();
const tree1 = createTree(
[
{ id: "base", title: "Derivative as a limit" },
{ id: "chain", title: "Chain rule intuition" },
// Session 1: two fresh propositions + a follows edge between them.
const chat1 = fixtureChat("Edge A");
const verdict1: AuditResult = {
propositions: [
{
id: "p1",
statement: "Derivative as a limit",
description: "The derivative is the limit of difference quotients.",
aliases: [],
messages: ["u1"],
evidence: ["e"],
},
{
id: "p2",
statement: "Chain rule intuition",
description: "Composed functions differentiate by multiplying local rates.",
aliases: [],
messages: ["u1"],
evidence: ["e"],
},
],
[{ from: "base", to: "chain" }],
);
const chat1 = fixtureChat("Edge A", tree1);
mergeFindings(chat1.id, treeFindings(tree1), { chatStore, graphStore: ledgers });
edges: [{ from: "p1", to: "p2", type: "follows", messages: ["u1"], evidence: ["e"] }],
attribution: { quiz: {} },
dropped: [],
};
let stats = mergeFindings(chat1.id, verdict1, { chatStore, graphStore: ledgers });
assert.deepEqual(stats, { nodes: 2, edges: 1, events: 0 });
const base = ledgers.readNodes()[0]!.id;
const chain = ledgers.readNodes()[1]!.id;
// Second session re-teaches the same two propositions (exact statements
// modulo case) and asserts the same edge — it must dedupe onto the
// existing nodes.
const tree2 = createTree(
[
{ id: "base", title: "derivative as a limit" },
{ id: "chain", title: "chain rule intuition" },
// Session 2: the auditor links both propositions to the SAME nodes
// (identity verdicts) and asserts the same relation.
const chat2 = fixtureChat("Edge B");
const verdict2: AuditResult = {
propositions: [
{
id: "q1",
statement: "The derivative as a limit of quotients", // different phrasing
description: "…",
aliases: [],
messages: ["u1"],
evidence: ["e"],
linkTo: base,
},
{
id: "q2",
statement: "Chain rule, intuitively",
description: "…",
aliases: [],
messages: ["u1"],
evidence: ["e"],
linkTo: chain,
},
],
[{ from: "base", to: "chain" }],
);
const chat2 = fixtureChat("Edge B", tree2);
const stats = mergeFindings(chat2.id, treeFindings(tree2), { chatStore, graphStore: ledgers });
// A new ASSERTION line is appended (provenance is honest) — reads merge
// by (from,to,type), so the derived graph still has one edge with both
// sessions in its provenance.
edges: [{ from: "q1", to: "q2", type: "follows", messages: ["u1"], evidence: ["e"] }],
attribution: { quiz: {} },
dropped: [],
};
stats = mergeFindings(chat2.id, verdict2, { chatStore, graphStore: ledgers });
// No new nodes; ONE new assertion line (per-session honest provenance).
assert.deepEqual(stats, { nodes: 0, edges: 1, events: 0 }, "no new nodes, one re-assertion line");
assert.equal(ledgers.counts.nodes, 2);
assert.equal(ledgers.counts.nodes, 4, "2 defining lines + 2 link-appends for the linked propositions");
const lines = ledgers.readEdges();
assert.equal(lines.length, 2, "two assertion lines for the same edge");
assert.equal(lines[0]!.from, lines[1]!.from);
assert.equal(lines[0]!.to, lines[1]!.to);
assert.deepEqual(lines.map((l) => l.provenance), [[chat1.id], [chat2.id]]);
assert.deepEqual(lines.map((l) => l.provenance), [
[{ chatId: chat1.id, messageId: "u1" }],
[{ chatId: chat2.id, messageId: "u1" }],
]);
// Read-time merging: one derived edge, both sessions in its provenance.
const derived = getGraph(ledgers).edges;
assert.equal(derived.length, 1);
assert.deepEqual(derived[0]!.provenance, [
{ chatId: chat1.id, messageId: "u1" },
{ chatId: chat2.id, messageId: "u1" },
]);
});
});
+6 -3
View File
@@ -9,7 +9,10 @@ import { isNodeVisible } from "../components/graph-view.tsx";
import type { NodeStateClass } from "../src/core/index.ts";
const none: ReadonlySet<NodeStateClass> = new Set();
const node = { name: "Money multiplier", statement: "The money multiplier is the maximum ratio of broad money to base money." };
const node = {
statement: "The money multiplier is the maximum ratio of broad money to base money.",
description: "How much broad money banks create from each unit of base money.",
};
describe("isNodeVisible (legend/filters predicate)", () => {
it("is visible with no filters", () => {
@@ -26,11 +29,11 @@ describe("isNodeVisible (legend/filters predicate)", () => {
assert.equal(isNodeVisible(node, "taught", hidden, ""), true);
});
it("search matches the name case-insensitively", () => {
it("search matches the statement case-insensitively", () => {
assert.equal(isNodeVisible(node, "candidate", none, "MONEY multiplier"), true);
});
it("search matches the statement when the name does not", () => {
it("search matches the description when the statement does not", () => {
assert.equal(isNodeVisible(node, "candidate", none, "maximum ratio"), true);
assert.equal(isNodeVisible(node, "candidate", none, " velocity"), false);
});
+220 -164
View File
@@ -1,9 +1,9 @@
/**
* Graph merger + trigger + derivation tests (vision §1; contracts.ts
* "extraction pipeline"). The merger (core/graph.ts mergeFindings) is the
* pipeline's deterministic apply stage: findings in, ledger appends out —
* identity resolution, retroactive probe attribution, event rebuild, and
* the idempotency guarantees.
* Graph merger + trigger + derivation tests (vision §1, 2026-09-02 redesign).
* The merger (core/graph.ts mergeFindings) is the pipeline's deterministic
* apply stage: AUDITED VERDICTS in, ledger appends out — link-or-create
* identity (the auditor's linkTo, never string matching), deterministic quiz
* events, understanding observations, and the idempotency guarantees.
*
* Ledger isolation: PATHS is fixed at import time, so the five data/graph
* ledgers are backed up and cleared before the suite and restored after
@@ -18,9 +18,8 @@ import assert from "node:assert/strict";
import * as fs from "node:fs";
import { after, describe, it } from "node:test";
import type { UIMessage } from "ai";
import type { PlanTree, PlanToolResult, SessionFindings } from "../src/core/contracts.ts";
import type { AuditResult } from "../src/core/contracts.ts";
import type { QuestionAnswerData } from "../src/core/question-core.ts";
import { createTree, markDone } from "../src/core/plan-tree.ts";
import { getGraph, graphSnapshot, mergeFindings, pendingQuizAnswers, type GraphDeps } from "../src/core/graph.ts";
import { graphStore } from "../src/store/graph.ts";
import { chatStore } from "../src/store/chats.ts";
@@ -80,16 +79,6 @@ function quizPart(
return { type: "tool-quiz", toolCallId, state: "output-available", input, output };
}
function planPart(toolCallId: string, name: string, input: Record<string, unknown>, tree: PlanTree): unknown {
return {
type: `tool-${name}`,
toolCallId,
state: "output-available",
input,
output: { ok: true, tree } satisfies PlanToolResult,
};
}
function answer(over: Partial<QuestionAnswerData>): QuestionAnswerData {
return {
questionId: "q",
@@ -102,59 +91,17 @@ function answer(over: Partial<QuestionAnswerData>): QuestionAnswerData {
};
}
/** Create a chat in the real store with a tree and messages; returns the record. */
function fixtureChat(title: string, tree: PlanTree | null, messages: UIMessage[]) {
/** Create a chat in the real store with messages; returns the record.
* Tree-blind world: the plan tree is irrelevant to the merger — no tree. */
function fixtureChat(title: string, messages: UIMessage[]) {
const chat = chatStore.createChat(title);
createdChats.push(chat.id);
if (tree) chatStore.saveTree(chat.id, tree);
chatStore.saveMessages(chat.id, messages);
return chatStore.getChat(chat.id)!;
}
/** The fixture tree: alpha → beta, gamma free-floating, alpha done. */
function fixtureTree(): PlanTree {
return markDone(
createTree(
[
{ id: "alpha", title: "Alpha intuition" },
{ id: "beta", title: "Beta rule" },
{ id: "gamma", title: "Gamma application" },
],
[{ from: "alpha", to: "beta" }],
),
"alpha",
);
}
/**
* Findings shaped exactly like what the extractor would produce for a
* session's plan tree: one proposition per tree node (statement = title,
* treeNodeId set), one `requires` edge per tree edge.
*/
function treeFindings(tree: PlanTree): SessionFindings {
const idByNodeId = new Map(tree.nodes.map((n, i) => [n.id, `p${i + 1}`]));
return {
propositions: tree.nodes.map((n, i) => ({
id: `p${i + 1}`,
name: n.title,
statement: n.title,
aliases: [],
kind: "truth" as const,
treeNodeId: n.id,
evidence: ["fixture evidence"],
})),
edges: tree.edges.map((e) => ({
from: idByNodeId.get(e.from)!,
to: idByNodeId.get(e.to)!,
type: "requires" as const,
evidence: ["fixture evidence"],
})),
attribution: { quiz: {} },
};
}
/** The full fixture transcript: probes → plan → checks → mark_done → noise. */
function fixtureMessages(tree: PlanTree): UIMessage[] {
/** The full fixture transcript: five graded quizzes + a dismissed one + an ask. */
function fixtureMessages(): UIMessage[] {
return [
user("u1", "teach me the fixture topic"),
assistant("a1", [
@@ -171,32 +118,30 @@ function fixtureMessages(tree: PlanTree): UIMessage[] {
answer({ questionId: "call-2", questionText: "Q2 text", dontKnow: true, correct: false, answeredAt: "2026-08-22T01:01:00.000Z" }),
),
]),
assistant("a3", [planPart("p1", "plan_init", { nodes: [], edges: [] }, tree)]),
assistant("a4", [
assistant("a3", [
// input.nodeId "alpha" is deliberately present and IGNORED —
// attribution is the only quiz→node path in the tree-blind world.
quizPart(
"call-3",
{ question: "Q3", nodeId: "alpha", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
answer({ questionId: "call-3", questionText: "Q3 text", correct: true, note: "recalled the diagram", answeredAt: "2026-08-22T02:00:00.000Z" }),
),
]),
assistant("a5", [
assistant("a4", [
quizPart(
"call-4",
{ question: "Q4", nodeId: "beta", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
{ question: "Q4", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
answer({ questionId: "call-4", questionText: "Q4 text", correct: false, answeredAt: "2026-08-22T02:01:00.000Z" }),
),
]),
assistant("a6", [
// nodeId present but NOT in the tree → tagged-but-unmapped: the
// attribution can rescue it, or it lands null + question text.
assistant("a5", [
quizPart(
"call-5",
{ question: "Q5", nodeId: "ghost", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
{ question: "Q5", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
answer({ questionId: "call-5", questionText: "Q5 text", dontKnow: true, correct: false, answeredAt: "2026-08-22T02:02:00.000Z" }),
),
]),
assistant("a7", [planPart("p2", "plan_mark_done", { id: "alpha" }, tree)]),
assistant("a8", [
assistant("a6", [
// dismissed quiz + an answered ask — neither becomes an event
quizPart(
"call-6",
@@ -214,25 +159,58 @@ function fixtureMessages(tree: PlanTree): UIMessage[] {
];
}
/**
* The extractor's findings for the fixture chat: the tree as propositions +
* the probe attribution. The audit stage is a no-op pass-through in these
* merger tests — its behavior is tested with the pipeline (extraction.test).
*/
function fixtureFindings(): SessionFindings {
const findings = treeFindings(fixtureTree());
// Untagged probe quizzes: call-1 attributed retroactively to Alpha (the
// vision's open question — resolved by the extractor); call-2 genuinely
// unattributable; call-5's ghost nodeId is rescued onto Gamma.
findings.attribution.quiz = { "call-1": "p1", "call-2": null, "call-5": "p3" };
return findings;
/** The auditor's verdict for the fixture chat: three verified propositions
* (no linkTo — the graph starts empty), one follows edge, all quizzes
* attributed, one understanding note. */
function fixtureAudit(): AuditResult {
return {
propositions: [
{
id: "p1",
statement: "Alpha intuition",
description: "Alpha intuition, elaborated: what alpha means and how it behaves in practice.",
aliases: ["alpha feel"],
messages: ["a1", "a3"],
evidence: ["fixture evidence"],
understanding: "solid after the check",
},
{
id: "p2",
statement: "Beta rule",
description: "The beta rule and when it applies.",
aliases: [],
messages: ["a4"],
evidence: ["fixture evidence"],
},
{
id: "p3",
statement: "Gamma application",
description: "Applying the whole framework to gamma cases.",
aliases: [],
messages: ["a5"],
evidence: ["fixture evidence"],
},
],
edges: [
{
from: "p1",
to: "p2",
type: "follows",
messages: ["a4"],
evidence: ["fixture evidence"],
},
],
attribution: { quiz: { "call-1": "p1", "call-2": null, "call-3": "p1", "call-4": "p2", "call-5": "p3" } },
dropped: [],
};
}
/** The chat every merger test folds (shared, so ordering is intentional). */
const chat1 = fixtureChat("Graph fixture", fixtureTree(), fixtureMessages(fixtureTree()));
const chat1 = fixtureChat("Graph fixture", fixtureMessages());
/** Set by the cross-session test; the snapshot test (which runs after it) reads it. */
/** Set by the cross-session tests; the snapshot test (which runs after) reads them. */
let chat2Id = "";
let chat3Id = "";
// ─── the merger ──────────────────────────────────────────────────────────────
@@ -248,71 +226,83 @@ function scopedGraph(chatIds: string[]) {
);
return {
nodes: graph.nodes.filter((n) => nodeIds.has(n.id)),
edges: graph.edges.filter((e) => e.provenance.some((c) => chatIds.includes(c))),
edges: graph.edges.filter((e) => e.provenance.some((pt) => chatIds.includes(pt.chatId))),
events: graph.events.filter((e) => chatIds.includes(e.chatId)),
};
}
describe("the merger (mergeFindings)", () => {
it("merges propositions/edges by identity and rebuilds events with retroactive probe attribution", () => {
const stats = mergeFindings(chat1.id, fixtureFindings(), graphDeps);
it("creates nodes from the verdicts, rebuilds quiz events deterministically, records observations", () => {
const stats = mergeFindings(chat1.id, fixtureAudit(), graphDeps);
assert.deepEqual(stats, { nodes: 3, edges: 1, events: 6 });
const graph = scopedGraph([chat1.id]);
assert.equal(graph.nodes.length, 3);
const idByStatement = new Map(graph.nodes.map((n) => [n.statement, n.id]));
for (const node of graph.nodes) {
assert.equal(node.kind, "truth");
assert.deepEqual(node.aliases, []);
assert.match(node.id, /^n_/);
assert.ok(node.description.length > 0, "description is required");
assert.equal("kind" in node, false, "no kind field");
assert.ok(Array.isArray(node.provenance), "message-granular provenance");
}
// Provenance is message-granular: p1 came up in a1 and a3.
const alpha = graph.nodes.find((n) => n.statement === "Alpha intuition")!;
assert.deepEqual(alpha.provenance, [
{ chatId: chat1.id, messageId: "a1" },
{ chatId: chat1.id, messageId: "a3" },
]);
// The tree edge alpha→beta becomes a `requires` edge on the mapped nodes.
// The follows edge, provenance attached.
assert.equal(graph.edges.length, 1);
const edge = graph.edges[0]!;
assert.equal(edge.from, idByStatement.get("Alpha intuition"));
assert.equal(edge.to, idByStatement.get("Beta rule"));
assert.equal(edge.type, "requires");
assert.deepEqual(edge.provenance, [chat1.id]);
assert.equal(edge.type, "follows");
assert.deepEqual(edge.provenance, [{ chatId: chat1.id, messageId: "a4" }]);
// Mapping lines: one per (chat, sourceId).
assert.deepEqual(
graphStore.readMappings().map((m) => m.sourceId),
["p1", "p2", "p3"],
);
// Events — kinds, refs, notes, and the attribution paths.
assert.equal(graph.events.length, 6);
const byRef = new Map(graph.events.map((e) => [e.ref, e]));
const probeMiss = byRef.get("call-1")!;
assert.equal(probeMiss.kind, "probe-miss"); // quiz BEFORE plan_init
assert.equal(probeMiss.nodeId, idByStatement.get("Alpha intuition"), "retroactive probe attribution");
assert.equal(probeMiss.question, undefined, "attributed → no question text needed");
const miss = byRef.get("call-1")!;
assert.equal(miss.kind, "miss");
assert.equal(miss.nodeId, idByStatement.get("Alpha intuition"), "attributed quiz");
const probeDontKnow = byRef.get("call-2")!;
assert.equal(probeDontKnow.kind, "dont-know"); // same kind, both phases
assert.equal(probeDontKnow.nodeId, null, "unattributable probe → null");
assert.equal(probeDontKnow.question, "Q2 text");
const dontKnow = byRef.get("call-2")!;
assert.equal(dontKnow.kind, "dont-know");
assert.equal(dontKnow.nodeId, null, "unattributable → null");
assert.equal(dontKnow.question, "Q2 text");
const check = byRef.get("call-3")!;
assert.equal(check.kind, "check-correct"); // quiz AFTER plan_init
assert.equal(check.nodeId, idByStatement.get("Alpha intuition"), "tagged quiz → deterministic");
assert.equal(check.note, "recalled the diagram");
const correct = byRef.get("call-3")!;
assert.equal(correct.kind, "correct");
assert.equal(correct.nodeId, idByStatement.get("Alpha intuition"), "attribution wins over the ignored input.nodeId");
assert.equal(correct.note, "recalled the diagram");
const checkMiss = byRef.get("call-4")!;
assert.equal(checkMiss.kind, "check-incorrect");
assert.equal(checkMiss.nodeId, idByStatement.get("Beta rule"));
const miss2 = byRef.get("call-4")!;
assert.equal(miss2.kind, "miss");
assert.equal(miss2.nodeId, idByStatement.get("Beta rule"));
const ghost = byRef.get("call-5")!;
assert.equal(ghost.kind, "dont-know");
assert.equal(ghost.nodeId, idByStatement.get("Gamma application"), "tagged-but-unmapped rescued by attribution");
assert.equal(ghost.question, undefined);
const dontKnow2 = byRef.get("call-5")!;
assert.equal(dontKnow2.kind, "dont-know");
assert.equal(dontKnow2.nodeId, idByStatement.get("Gamma application"));
const taught = byRef.get("taught:alpha")!;
assert.equal(taught.kind, "taught");
assert.equal(taught.nodeId, idByStatement.get("Alpha intuition"), "mark_done → tree-node mapping");
const observation = byRef.get("obs:p1")!;
assert.equal(observation.kind, "observation");
assert.equal(observation.nodeId, idByStatement.get("Alpha intuition"));
assert.equal(observation.note, "solid after the check");
// The dismissed quiz and the ask question became nothing.
assert.equal([...byRef.keys()].filter((r) => r === "call-6" || r === "call-7").length, 0);
});
it("is idempotent — re-merging the same findings appends nothing", () => {
const stats = mergeFindings(chat1.id, fixtureFindings(), graphDeps);
it("is idempotent — re-merging the same verdicts appends nothing", () => {
const stats = mergeFindings(chat1.id, fixtureAudit(), graphDeps);
assert.deepEqual(stats, { nodes: 0, edges: 0, events: 0 });
const graph = scopedGraph([chat1.id]);
assert.equal(graph.nodes.length, 3);
@@ -322,38 +312,63 @@ describe("the merger (mergeFindings)", () => {
assert.equal(graphStore.readCheckpoints().filter((c) => c.chatId === chat1.id).length, 2);
});
it("merges a second session onto the same nodes and accumulates edge provenance", () => {
// chat2 re-teaches the same two propositions (identical statements) —
// the conservative identity resolution recognizes them.
const tree2 = createTree(
[
{ id: "a", title: "Alpha intuition" },
{ id: "b", title: "Beta rule" },
],
[{ from: "a", to: "b" }],
);
const chat2 = fixtureChat("Graph fixture 2", tree2, [
it("applies a linkTo verdict: phrasings join aliases, provenance unions, no duplicate node", () => {
const alphaId = scopedGraph([chat1.id]).nodes.find((n) => n.statement === "Alpha intuition")!.id;
const chat2 = fixtureChat("Graph fixture 2", [
user("u1", "again"),
assistant("a1", [planPart("p1", "plan_init", { nodes: [], edges: [] }, tree2)]),
assistant("a1", [
quizPart(
"call-9",
{ question: "Q9", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
answer({ questionId: "call-9", questionText: "Q9 text", correct: true, answeredAt: "2026-08-22T03:00:00.000Z" }),
),
]),
]);
const stats = mergeFindings(chat2.id, treeFindings(tree2), graphDeps);
assert.deepEqual(stats, { nodes: 0, edges: 1, events: 0 });
chat2Id = chat2.id;
// The auditor's identity verdict: this proposition IS the alpha node.
const stats = mergeFindings(
chat2.id,
{
propositions: [
{
id: "q1",
statement: "Alpha feel", // a DIFFERENT phrasing — identity is the verdict, not strings
description: "The intuitive grasp of alpha.",
aliases: ["alpha intuition"],
messages: ["a1"],
evidence: ["fixture evidence"],
linkTo: alphaId,
},
],
edges: [],
attribution: { quiz: { "call-9": "q1" } },
dropped: [],
},
graphDeps,
);
assert.deepEqual(stats, { nodes: 0, edges: 0, events: 1 });
const after = scopedGraph([chat1.id, chat2.id]);
assert.equal(after.nodes.length, 3); // no duplicate nodes
assert.equal(after.edges.length, 1); // one merged edge…
assert.deepEqual(after.edges[0]!.provenance, [chat1.id, chat2.id]); // …two sessions
assert.equal(after.nodes.length, 3, "no duplicate node");
const alpha = getGraph(graphStore).nodes.find((n) => n.id === alphaId)!;
assert.ok(alpha.aliases.includes("Alpha feel"), "the session's phrasing joins the aliases");
assert.ok(alpha.aliases.includes("alpha intuition"));
assert.ok(
alpha.provenance.some((pt) => pt.chatId === chat2.id && pt.messageId === "a1"),
"provenance unions",
);
// The mapping records the verdict: (chat2, q1) → the linked node.
const m = graphStore.readMappings().find((x) => x.chatId === chat2.id)!;
assert.deepEqual(m, { chatId: chat2.id, sourceId: "q1", graphNodeId: alphaId });
});
it("derives node states, sessions, and node provenance in graphSnapshot", () => {
it("derives node states and sessions in graphSnapshot", () => {
const snap = graphSnapshot(graphDeps);
const idByStatement = new Map(snap.nodes.map((n) => [n.statement, n.id]));
assert.equal(snap.nodeStates[idByStatement.get("Alpha intuition")!], "taught");
// Alpha's latest quiz is call-9 (chat2, correct) → known.
assert.equal(snap.nodeStates[idByStatement.get("Alpha intuition")!], "known");
assert.equal(snap.nodeStates[idByStatement.get("Beta rule")!], "missed");
// Gamma: no taught/checks — but the rescued dont-know probe (call-5)
// makes its latest event a dont-know.
assert.equal(snap.nodeStates[idByStatement.get("Gamma application")!], "dont-know");
assert.equal(snap.sessions[chat1.id], "Graph fixture");
@@ -366,8 +381,35 @@ describe("the merger (mergeFindings)", () => {
assert.deepEqual(snap.nodeSessions[idByStatement.get("Gamma application")!], [chat1.id]);
});
it("never string-matches without a verdict — an identical statement without linkTo is a NEW node", () => {
const chat3 = fixtureChat("Graph fixture 3", [user("u1", "third")]);
chat3Id = chat3.id;
const stats = mergeFindings(
chat3.id,
{
propositions: [
{
id: "r1",
statement: "Alpha intuition", // IDENTICAL to the existing node's statement
description: "Same words, no verdict.",
aliases: [],
messages: ["u1"],
evidence: ["fixture evidence"],
// no linkTo — identity is the auditor's call alone
},
],
edges: [],
attribution: { quiz: {} },
dropped: [],
},
graphDeps,
);
assert.equal(stats.nodes, 1, "created — the merger does not match strings");
assert.equal(getGraph(graphStore).nodes.filter((n) => n.statement === "Alpha intuition").length, 2);
});
it("returns zeros for unknown chats", () => {
assert.deepEqual(mergeFindings("chat_does_not_exist", treeFindings(fixtureTree()), graphDeps), {
assert.deepEqual(mergeFindings("chat_does_not_exist", fixtureAudit(), graphDeps), {
nodes: 0,
edges: 0,
events: 0,
@@ -379,14 +421,12 @@ describe("the merger (mergeFindings)", () => {
describe("the trigger helper (pendingQuizAnswers)", () => {
it("counts answered quizzes not yet in the ledger — dismissed/ask never count", () => {
const tree = createTree([{ id: "solo", title: "Solo node" }], []);
const chat = fixtureChat("Trigger fixture", tree, [
const chat = fixtureChat("Trigger fixture", [
user("u1", "go"),
assistant("a1", [planPart("p1", "plan_init", { nodes: [], edges: [] }, tree)]),
assistant("a2", [
assistant("a1", [
quizPart(
"call-s",
{ question: "QS", nodeId: "solo", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
{ question: "QS", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
answer({ questionId: "call-s", questionText: "QS text", correct: true, answeredAt: "2026-08-22T03:00:00.000Z" }),
),
quizPart(
@@ -407,7 +447,9 @@ describe("the trigger helper (pendingQuizAnswers)", () => {
assert.equal(pendingQuizAnswers(chat.id, graphDeps), 1, "one answered quiz pending");
assert.equal(pendingQuizAnswers("chat_does_not_exist", graphDeps), 0);
mergeFindings(chat.id, treeFindings(tree), graphDeps);
mergeFindings(chat.id, fixtureAudit(), graphDeps);
// call-s is unattributed here (fixtureAudit has no call-s attribution)
// — but it still becomes an event, so nothing stays pending.
assert.equal(pendingQuizAnswers(chat.id, graphDeps), 0, "extracted → nothing pending");
// A NEW answered quiz raises the count again (the re-trigger path).
@@ -416,7 +458,7 @@ describe("the trigger helper (pendingQuizAnswers)", () => {
assistant("a3", [
quizPart(
"call-s4",
{ question: "QS4", nodeId: "solo", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
{ question: "QS4", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
answer({ questionId: "call-s4", questionText: "QS4 text", correct: false, answeredAt: "2026-08-22T03:01:00.000Z" }),
),
]),
@@ -429,29 +471,43 @@ describe("the trigger helper (pendingQuizAnswers)", () => {
describe("graph ledger store", () => {
it("round-trips lines and skips corrupt / foreign-shaped lines (strict reads)", () => {
const node = { id: "n_storetest", name: "Store test", statement: "Store test", aliases: [], kind: "truth" as const, createdAt: "2026-08-22T00:00:00.000Z" };
const node = {
id: "n_storetest",
statement: "Store test",
description: "Store test, elaborated.",
aliases: [] as string[],
createdAt: "2026-08-22T00:00:00.000Z",
provenance: [{ chatId: "chat_c", messageId: "m1" }],
};
graphStore.appendNode(node);
// A torn append and a legacy V0-shaped node (different schema) — both skipped.
// A torn append and a legacy pre-redesign node (old schema) — both skipped.
fs.appendFileSync(
LEDGER_FILES.nodes,
`{not json}\n${JSON.stringify({ id: "legacy", statement: "Legacy shape", domain: "x", state: "known" })}\n`,
`{not json}\n${JSON.stringify({ id: "legacy", statement: "Legacy shape", kind: "truth", name: "Legacy", createdAt: "2026-08-22T00:00:00.000Z" })}\n`,
"utf-8",
);
const nodes = graphStore.readNodes().filter((n) => n.id === "n_storetest" || n.id === "legacy");
assert.equal(nodes.length, 1);
assert.equal(nodes[0]!.statement, "Store test");
assert.equal(nodes[0]!.name, "Store test");
assert.deepEqual(nodes[0]!.provenance, [{ chatId: "chat_c", messageId: "m1" }]);
});
it("derives a name for pre-name ledger lines (legacy reads)", () => {
// A full, valid node line written before `name` existed.
fs.appendFileSync(
LEDGER_FILES.nodes,
`${JSON.stringify({ id: "n_prename", statement: "Derivatives measure instantaneous change; they are limits.", aliases: [], kind: "truth", createdAt: "2026-08-22T00:00:00.000Z" })}\n`,
"utf-8",
);
const node = graphStore.readNodes().find((n) => n.id === "n_prename");
assert.ok(node);
assert.equal(node.name, "Derivatives measure instantaneous change");
it("returns RAW node lines — repeated ids (link-appends) are each their own line; getGraph merges", () => {
const line = (msgId: string) =>
JSON.stringify({
id: "n_linktest",
statement: `Phrasing ${msgId}`,
description: "Elaboration.",
aliases: [],
createdAt: "2026-08-22T00:00:00.000Z",
provenance: [{ chatId: "chat_c", messageId: msgId }],
});
fs.appendFileSync(LEDGER_FILES.nodes, `${line("m1")}\n${line("m2")}\n`, "utf-8");
const lines = graphStore.readNodes().filter((n) => n.id === "n_linktest");
assert.equal(lines.length, 2, "the store returns raw lines, one per append");
const merged = getGraph(graphStore).nodes.find((n) => n.id === "n_linktest")!;
assert.equal(merged.statement, "Phrasing m1", "the FIRST line defines the node");
assert.ok(merged.aliases.includes("Phrasing m2"), "later phrasings join the aliases");
assert.deepEqual(merged.provenance.map((p) => p.messageId), ["m1", "m2"]);
});
});