diff --git a/README.md b/README.md
index 47011e5..228c9e1 100644
--- a/README.md
+++ b/README.md
@@ -114,7 +114,7 @@ graph TB
LEDG --> SNAP --> RGRAPH
```
-**Core invariant:** the extraction pipeline (extractor → auditor → merger) is the sole graph writer. The LLM stages propose and verify (propositions, edges, probe attribution — each with verbatim evidence); the deterministic merger owns every ledger append and rebuilds anything the model could silently mis-copy (event kinds, timestamps, idempotency refs) from the transcript. The teacher stays a teacher; everything below the ledgers is derived.
+**Core invariant:** the extraction pipeline (extractor → auditor → merger) is the sole graph writer — tree-blind and post-hoc. The LLM stages propose, verify, and judge identity (link-or-create against the current graph; each finding carries verbatim evidence and message-granular provenance); the deterministic merger owns every ledger append and rebuilds anything the model could silently mis-copy (event kinds, timestamps, idempotency refs, provenance) from the transcript. Every node is an objective, world-facing fact; the learner's relationship to it lives entirely in the event ledger. The teacher stays a teacher; everything below the ledgers is derived.
## Layout
@@ -165,7 +165,7 @@ data/ chats/ · graph/ · jobs/ — runtime state, git
**The async sub-agent protocol (ack-and-steer).** `research` / `demo` / `visual` are server-executed tools: the call resolves immediately mid-turn with a fire-and-forget ack and the teacher keeps teaching; a background runner works in parallel (the job store tracks a live activity line; the client polls `/api/jobs`). On completion the runner builds a **steer message** — a user-role `UIMessage` whose text part is the model-facing result (data parts are dropped by provider conversion) and whose `data-subagent-artifact` part carries the full result for the client — and the **wake pipeline** (`server/jobs.ts`) delivers it: a per-chat serialized queue waits for the turn to end and no question to be pending, then fires a server-side teacher turn that weaves the result into the lesson. The client adopts the wake turn through the same `turnInFlight` machinery as returning mid-stream (busy shimmer, composer gate) and renders steers as artifacts — never a bubble. Sub-agents **verify, never one-shot**: the demo-builder headlessly executes its HTML in system Chrome before delivery; the visual-maker renders each SVG and looks at it (needs a vision-capable model — `MODELS.visualMaker` in config.ts).
-**The knowledge graph (the two channels).** The write channel: the extraction pipeline — extractor (LLM: propositions, edges, probe attribution, each with verbatim evidence) → auditor (LLM: drops anything unbacked) → merger (`mergeFindings`, deterministic: exact/alias identity resolution, event rebuild from the transcript's tool parts, dedupe, checkpoint). Triggered once a chat accumulates `EXTRACTION.everyNQuizAnswers` (default 3, config.ts) new answered quizzes — a v1 placeholder trigger isolated in one seam (`turn-service.ts`), swappable without touching the pipeline. Failure checkpoints nothing; the next trigger re-runs. The read channel: the home view derives node states from the ledgers (`GET /api/graph` is a pure read — forceAtlas2 run-then-freeze, no idle simulation), and at session start the derived graph is rendered into a compact markdown block appended to the teacher's instructions (`buildGraphContext`) — whole graph while small, hotspots + 1-hop neighborhood past 150 nodes.
+**The knowledge graph (the two channels).** The write channel: the extraction pipeline — extractor (LLM, tree-blind: propositions as true statements + descriptions, follows/supports edges, quiz attribution, understanding notes) → auditor (LLM: drops anything unbacked, stamps each survivor with an identity verdict — link to an existing node or create) → merger (`mergeFindings`, deterministic: applies the verdicts, link-appends phrasings/provenance, event rebuild from the transcript's tool parts, dedupe, checkpoint). Triggered once a chat accumulates `EXTRACTION.everyNQuizAnswers` (default 3, config.ts) new answered quizzes — a v1 placeholder trigger isolated in one seam (`turn-service.ts`), swappable without touching the pipeline. Failure checkpoints nothing; the next trigger re-runs. The read channel: the home view derives node states from the ledgers (`GET /api/graph` is a pure read — forceAtlas2 run-then-freeze, no idle simulation), and at session start the derived graph is rendered into a compact markdown block appended to the teacher's instructions (`buildGraphContext`) — whole graph while small, hotspots + 1-hop neighborhood past 150 nodes.
## Testing
diff --git a/components/graph-view.tsx b/components/graph-view.tsx
index 99666be..dc0b4dd 100644
--- a/components/graph-view.tsx
+++ b/components/graph-view.tsx
@@ -363,7 +363,7 @@ export function GraphView() {
// frozen layout stable for a given node set across remounts.
const rand = seededRandom(node.id);
graph.addNode(node.id, {
- label: node.name,
+ label: node.statement,
size: STATE_TOKENS[state].size,
color: stateColors.get(state) ?? edgeColor,
x: (rand() - 0.5) * 100,
@@ -552,8 +552,8 @@ export function GraphView() {
style={{ visibility: "hidden", left: 0, top: 0 }}
className="absolute z-10 w-80 max-w-[calc(100%-1rem)] rounded-xl border bg-card p-4 text-left shadow-lg"
>
- {selected.node.name}
- {selected.node.statement}
+ {selected.node.statement}
+ {selected.node.description}
{STATE_LABELS[selected.state]}
diff --git a/docs/architecture.md b/docs/architecture.md
index d221ae3..67f46a5 100644
--- a/docs/architecture.md
+++ b/docs/architecture.md
@@ -75,11 +75,11 @@ chat record ──▶ EXTRACTOR (LLM) ──▶ AUDITOR (LLM) ──▶ MERGER (
proposes verifies applies
```
-**Division of trust** (the pipeline's core rule): the LLM stages supply only judgment — *what was learned* (propositions, incl. misconceptions and prose-only `supports` edges), *what depends on what*, and *which proposition an untagged probe tested*. Everything the model could silently mis-copy — event kinds, timestamps, idempotency refs, notes, question texts — is rebuilt deterministically by the merger from the transcript's tool parts. The extractor files verbatim evidence quotes; the auditor (the faithfulness net) drops anything unbacked and logs every drop. Conservative throughout: duplicates over wrong merges, drop over invent.
+**Tree-blind and post-hoc** (2026-09-02 redesign, docs/vision.md §1): the extractor reads ONLY the transcript — never the session's plan tree — and proposes free-form propositions (true statements with descriptions, misconception-free: a misconception is stated as its true correction), `follows`/`supports` edges, per-proposition understanding notes, and message-granular provenance. Every node is an objective, world-facing fact; the learner's relationship to it lives entirely in the event ledger.
-**Merger invariants** (`src/core/graph.ts`, `mergeFindings`): idempotent and incremental — statement/alias exact-match identity resolution (fuzzy/embedding adjudication deferred), one mapping line per (chat, graph node) as the merge audit trail, edge dedupe by `(from, to, type)` with per-chat provenance, event dedupe by chat-scoped ref, and a sessions.jsonl checkpoint (`foldedAt`) marking which turns are processed. Failure anywhere before the merger checkpoints nothing; the next trigger re-runs.
+**Division of trust** (the pipeline's core rule): the LLM stages supply only judgment — *what was learned*, *what follows/supports what*, *which proposition each quiz tested* (retroactively, against the whole transcript), and — the auditor's — *identity*: whether a proposition IS an existing graph node (link-or-create). Everything the model could silently mis-copy — event kinds (`correct` / `miss` / `dont-know`), timestamps, idempotency refs, notes, provenance points — is rebuilt deterministically by the merger from the transcript's tool parts. The extractor files verbatim evidence quotes; the auditor (which sees the extractor's output PLUS the current graph) drops anything unbacked, logs every drop, and stamps each surviving proposition with its identity verdict (`linkTo`). Conservative throughout: duplicates over wrong merges, drop over invent.
-**Probe attribution** (vision open question, resolved structurally): tagged events (quiz `nodeId`, `plan_mark_done`'s tree node) resolve deterministically through the proposition mapping. The extractor's judgment is consulted only for **untagged probe quizzes** — retroactively, against the whole transcript, so a pre-tree probe maps to a proposition the tree only articulates later. Attribution can also rescue a tagged-but-unknown id; genuinely unattributable probes land `nodeId: null` with the question text, exactly as they always have.
+**Merger invariants** (`src/core/graph.ts`, `mergeFindings`): idempotent and incremental — identity comes from the auditor's `linkTo` verdict alone (validated against the ledger; never string matching), one mapping line per (chat, sourceId) as the verdict audit trail (which also makes re-merging the same chat idempotent), link-appends: a linked proposition appends a node line whose phrasings join the node's `aliases` and whose provenance unions (reads merge by id), edge dedupe by `(from, to, type)`-already-asserted-by-this-chat with message-granular provenance, event dedupe by chat-scoped ref (quiz toolCallId, `obs:` for understanding observations), and a sessions.jsonl checkpoint (`foldedAt`) marking which turns are processed. Failure anywhere before the merger checkpoints nothing; the next trigger re-runs.
**The trigger is a v1 placeholder, isolated in ONE seam.** `src/server/turn-service.ts` (`triggerExtraction`) counts answered quizzes not yet in the ledger (`pendingQuizAnswers`, core) and fires the pipeline async once a chat crosses `EXTRACTION.everyNQuizAnswers` (default 3, config.ts) — single-flight per chat. The pipeline is trigger-agnostic: when a better mechanism lands (session end, idle, explicit flush), only this seam changes.
diff --git a/docs/vision.md b/docs/vision.md
index 0a93ae1..695d9b4 100644
--- a/docs/vision.md
+++ b/docs/vision.md
@@ -2,7 +2,7 @@
Living document. Started 2026-08-22, capturing the current vision and all agreed decisions. Amend as things evolve.
-Prior art: `knowledge graph design.md` in the Obsidian vault (2025-08-20 ideation) — still canonical for the epistemic model (known vs understood, requires/supports, event ledger, calibration rubric, merging policy). This document records what today's decisions **amend** in it, and adds the session-experience and presentation layers the original doc deferred.
+Prior art: `knowledge graph design.md` in the Obsidian vault (2025-08-20 ideation) — was canonical for the epistemic model; the 2026-09-02 node-theory + construction redesign in §1 amends it further (no misconception nodes, `follows`/`supports`, no taught events, message-granular provenance). This document records what today's decisions **amend** in it, and adds the session-experience and presentation layers the original doc deferred.
---
@@ -10,33 +10,55 @@ Prior art: `knowledge graph design.md` in the Obsidian vault (2025-08-20 ideatio
Everything the learner learns accumulates into one persistent, cross-domain graph. It is the home page of the app and the heart of the system: structure builds out visibly over time, and the data supports frontier expansion, what-next queries, repairs, unlock-value (per the original design doc).
+**Node theory + construction redesigned 2026-09-02** — the subsections below from "What the graph is" through "Understanding is derived" are the current philosophy, superseding both the original design doc's epistemic model in part and the 2026-08-22 live-emission construction decision (see the amendments table).
+
### Critical requirement: frequent updates
-The graph updates **continuously, not per-session**. A learner who answers a few questions and leaves without "finishing" a session must still see something new on the home page. Partial sessions deposit value. This requirement drove the construction decision below.
+The graph updates **continuously, not per-session**. A learner who answers a few questions and leaves without "finishing" a session must still see something new on the home page. Partial sessions deposit value. The background extraction pipeline's checkpoint mechanism satisfies this (fold on home/idle, never the same turns twice) — not live emission.
-### Construction: live events + background consolidation (DECIDED)
+### What the graph is — node theory (DECIDED 2026-09-02)
-**Amendment to the original design doc.** The original doc stated "the teacher agent is never modified; all graph-building is post-hoc," and rejected live structured emission as too much cognitive load. That is now **reversed**, because the teacher will be maintaining a formal plan tree anyway (§2) — once it does, live emission is nearly free, and it's the only way to satisfy the frequent-updates requirement.
+The global graph is an **objective world-model of the domains the learner has encountered**: the facts, their structure (what follows from / is supported by what), and — inferred from that structure plus the event ledger — the state of the learner's understanding. Nothing learner- or session-relative is ever stored **in** the graph; understanding is **inferred, never encoded**.
-- **Live:** the teacher maintains the session's dependency tree via structured tools and tags quiz outcomes to tree nodes as they happen. The home graph reflects this in near real time.
-- **Background consolidation (lightweight, frequent):** cross-session node merging (embed → kNN → LLM adjudication, conservative — duplicates over wrong merges), calibration rubric application, and edges that were articulated only in prose. Runs on navigation home / idle — not as a heavy per-session batch.
-- The original three-stage transcript-extraction pipeline survives only as (a) the retroactive bootstrap over old session files, (b) the audit/faithfulness net.
+- **Every node is a true proposition.** Worded objectively, in minimal form as it exists w.r.t. its context. Truth is context-relative and that is fine: "Gravity is a force" and "Gravity is not a force" coexist as nodes — each true in its context (classical mechanics vs. GR), with **the surrounding subgraph as the context**. No context field, no pedantic self-containment of facts.
+- **Node schema:** `statement` — the proposition in minimal form (a short sentence; the rendering label and identity key); `description` — the elaborated version (required); `aliases` — the verbatim phrasings sessions used (merge keys + audit trail). There is **no `kind`** field.
+- **Misconceptions are not a node category.** A myth enters the graph as its true correction ("The 10%-of-brain claim is a myth"); the learner's holding of the wrong idea lives entirely in the event ledger. Wrongness is about the learner, and the learner lives in the ledger.
-### One substrate, merged across sessions (DECIDED)
+### Edges (DECIDED 2026-09-02)
-A single node model: propositions at one-reasoning-step grain, `requires`/`supports` edges, no edge scalars. A session's plan tree **is** a session-scoped subgraph. The global graph is the merge of all session trees:
+Two kinds, no scalars, no encoded strength — prerequisiteness is a spectrum, and we deliberately refuse to formalize the spectrum:
-- **v1 (bare bones):** merge = naive union. No adjudication yet.
-- **later:** the embed + LLM-adjudication merge pipeline from the design doc (the quiet superpower: cross-session/cross-domain edges only ever get created here).
+- **`follows`** — `A → B`: B is *derivable* from A (A is a premise of B). If C follows from both A and B, both `A → C` and `B → C` exist; joint requirements are just sibling edges.
+- **`supports`** — `A → B`: A *grounds or justifies* B without entailing it.
-Carried over unchanged from the design doc: known (evidence property) vs understood (derived structural property); event ledger as source of truth with derived state recomputed; anchored 5-level calibration rubric; misconception nodes as first-class structure; candidate node class for frontier territory.
+Strength/depth of dependencies is **derived at read time** from topology (chains, corroboration), never stored — this is the input the future understanding model consumes.
+
+### Provenance — message granularity (DECIDED 2026-09-02)
+
+Every node records every **(chat, message) point** where it came up; every edge records every message that asserted it. Deep-linkable to the exact transcript points — the derivation audit trail is a first-class feature, not a byproduct.
+
+### Construction: free-form extraction, fully decoupled from sessions (DECIDED 2026-09-02 — supersedes live emission)
+
+**Second amendment, reversing the 2026-08-22 live-emission decision.** The session's plan tree is the teacher's scaffolding for the session experience (§2); the global graph does not depend on it — structurally or otherwise. What the teacher formalizes is session-relative; the graph must not inherit those choices. (The original design doc's post-hoc stance was right after all.)
+
+Three stages, one pipeline — the sole graph writer:
+
+- **Extractor (LLM) — session-only scope.** Sees ONLY the session transcript (tree-blind, with message ids). Reports freely, with verbatim evidence: propositions from anywhere in the session (one reasoning step each; prose counts as much as anything the teacher formalized), edges as the transcript articulates them, message-granular provenance, retroactive quiz attribution, and per proposition an **understanding note** — how strongly the learner accepted/understood it in this session, a small named-word scale, no numbers. The notes are data for the future state model; their treatment is TBD.
+- **Auditor (LLM) — the global-graph scope.** Sees the extractor's output AND the existing graph. Two jobs: the faithfulness net (every candidate verified against verbatim evidence — unchanged), and the **identity judgment**: link each proposition to an existing node (recording the session's phrasing as an alias) or create it new. Conservative: duplicates over wrong merges.
+- **Merger (deterministic).** Append-only ledger writes; quiz event kinds/refs rebuilt mechanically from quiz tool parts. Every judgment is an LLM's; everything mechanical stays code.
+
+**Event ledger (amended):** quiz outcomes are rebuilt deterministically from quiz tool parts (correct / miss / don't-know, plus the learner's note). **There is no `taught` event** — no marking-things-done. The extractor's per-proposition understanding notes join the ledger as session-scoped observations. Identity is no longer exact-string matching — it is the auditor's judgment, with aliases as the phrasing record.
+
+### Understanding is derived (amended 2026-09-02)
+
+Known vs. understood, event ledger as source of truth, derived state recomputed on read — carried over from the design doc in spirit. The state model itself is now an explicit design discussion: **node states should be session-independent and computed from the depth and strength of dependencies** (the `follows`/`supports` topology) plus quiz events and understanding notes. That redesign is pending ideation; the current five-class derivation (taught / known / missed / don't-know / candidate) is transitional. The anchored calibration rubric stays parked until real data.
### Rendering (DECIDED — research report 2026-08-22)
- **Library: sigma.js v3 + graphology + graphology-layout-forceatlas2.** ~40 KB gzipped all-in — half of vis-network, a third of cytoscape, far below react-force-graph (~250 KB deps) and cosmos.gl (luma.gl stack).
- **Layout: synchronous run-then-freeze.** `forceAtlas2.assign(graph, { iterations: 100, settings: inferSettings(graph) })` runs once, writes x/y, and no animation loop ever starts — near-zero idle CPU (sigma only redraws on camera/graph events). <150 ms at hundreds of nodes; at ~2k enable `barnesHutOptimize`, then the bundled Web Worker, then server-cached positions — in that order, only if latency bites.
- **Labels at zoom:** sigma's `labelRenderedSizeThreshold` replicates Obsidian's "text fade threshold" exactly.
-- **Pan/zoom** built in; hover/click events (`enterNode`, `clickNode`) free. Typed edges: `EdgeArrowProgram` and/or two edge colors for `requires` vs `supports`.
+- **Pan/zoom** built in; hover/click events (`enterNode`, `clickNode`) free. Typed edges: `EdgeArrowProgram` and/or two edge colors for `follows` vs `supports`.
- **React integration:** thin ~40-line `useEffect` wrapper (sigma is imperative; `@react-sigma/core` exists but is optional).
- **Obsidian internals (confirmed by research):** PixiJS **WebGL** renderer (canvas fallback), custom d3-force-style simulation (its four force sliders map 1:1 onto d3-force primitives), zoom-gated label fade, alpha-decay settling. Our chosen stack reproduces that feel with a documented, maintained library.
- Rejected: vis-network / cytoscape.js (Canvas 2D, 3× bundle, editing features we don't need), react-force-graph-2d (best freeze API but heaviest dep tree; its strength is *animated* simulation, which we explicitly don't want), cosmos.gl (GPU forces — overkill below ~10k nodes).
@@ -47,11 +69,11 @@ Carried over unchanged from the design doc: known (evidence property) vs underst
Every session starts with the learner's knowledge graph as base context for the teacher. This is what makes probing *precise* — probe questions can target subtleties and nuances at the boundary of what the learner actually holds, instead of re-discovering it from scratch each session. Personalization compounds across sessions.
-**The selection problem (open):** the whole graph may eventually be too much context, and pedagogical relevance ≠ semantic closeness. What matters for a new topic: (a) its **structural neighborhood** — the `requires`-edge subtree the topic stands on, regardless of surface similarity; (b) **diagnostic hotspots** — misconception / shaky / floating nodes anywhere in the graph (a live misconception is always relevant context); (c) the frontier strip near the topic. Candidate strategies, to tune on real data: whole graph while it's small; k-hop structural neighborhood around the topic's nodes; ledger-informed slices (weak/misconceived/floating first); a cheap routing pass that picks subtrees given the session topic.
+**The selection problem (open):** the whole graph may eventually be too much context, and pedagogical relevance ≠ semantic closeness. What matters for a new topic: (a) its **structural neighborhood** — the `follows`-edge subtree the topic stands on, regardless of surface similarity; (b) **diagnostic hotspots** — nodes the ledger marks as shaky / misunderstood / misconceived, anywhere in the graph (a live misconception — events showing the learner holds a node wrong — is always relevant context); (c) the frontier strip near the topic. Candidate strategies, to tune on real data: whole graph while it's small; k-hop structural neighborhood around the topic's nodes; ledger-informed slices (weak/misconceived/floating first); a cheap routing pass that picks subtrees given the session topic.
### Home page
-The graph view replaces the current placeholder. v1: all nodes from all sessions (naive union), node color/shape by state class (taught / probed / known / not-held / candidate later), pan/zoom, labels at zoom. Clicking a node → the session(s) that touched it (cheap version: title list). Everything richer (frontier glow, rot map, blast radius) comes after the consolidation pipeline exists.
+The graph view replaces the current placeholder. v1: all nodes from all sessions, node color/shape by state class (the current five classes are transitional — see "Understanding is derived"), pan/zoom, labels at zoom. Clicking a node → the exact (session, message) points where it came up (message-granular provenance, deep-linked). Everything richer (frontier glow, rot map, blast radius) comes after the state model exists.
---
@@ -63,6 +85,7 @@ Goal: a learning session must look and feel structurally different from a ChatGP
- The agent authors the tree via **structured tools** — `add_node`, `add_edge`, `reparent`, `remove_node`, `set_current`, … — schema-validated, with stable node identity across edits. Mermaid text as the source of truth was rejected: edits would be text surgery and identity would drift.
- The agent may edit the tree mid-session at its own initiative (detour when the learner asks something worth exploring, go deeper, prune) — that power is explicit.
+- The tree is **session scaffolding only**: the global graph is built by free-form post-hoc extraction that is deliberately tree-blind (§1, construction). Nothing structural rides from the tree into the graph.
- **"Where we are"** is tracked as structured state (a `set_current` tool), not inferred from the transcript.
- **Mermaid survives only for ad-hoc lesson diagrams** (flows, cycles, comparisons) that are not the plan tree — and the teacher is explicitly **encouraged to diagram freely in explanations**; diagrams are first-class teaching output, not decoration. Its rendering quality must improve (integrated display, working zoom/pan — current code-block-ish rendering is unacceptable), but it's now a peripheral concern.
@@ -173,7 +196,7 @@ The teacher's context window and attention are precious — it has the lesson to
2. **Bare-bones global graph + home view** — live emission from tree tools into a graph store, naive-union merge, home page renders it with the chosen lightweight renderer. *This is the "see something appear even from a partial session" milestone.* **✅ Wave 2 (W4): JSONL ledgers, idempotent emission, sigma.js + forceAtlas2 home view, foldStaleChats catch-up. Live-verified.**
3. **Phase separation** — probe/teach marker, transcript divider, collapsed teach-phase checks. **✅ Wave 2 (W5): phaseAt from first plan_init, "Teaching · N nodes" divider, invitation bar + reveal. Live-verified.**
4. **Right-panel tree view** — persistent, current-position highlight. **✅ Wave 3 (W7): topo order, done/current/next/upcoming, progress header, collapse-to-rail persisted. Live-verified.**
-5. **Background consolidation** — merge adjudication, calibration, prose-only edges. **◐ Wave 3 (W8, lite): exact/alias-match merge + graph-as-context injection (whole graph while small, 150-node cap with hotspot+1-hop truncation). Live-verified personalization. Full embed+adjudication merge and calibration rubric remain deferred.**
+5. **Background consolidation → free-form extraction pipeline** — merge adjudication, calibration, prose-only edges. **◐ Wave 3 (W8, lite) shipped exact/alias-match merge + graph-as-context injection. Superseded direction (2026-09-02): the pipeline becomes free-form post-hoc extraction (extractor session-only → auditor with global-graph identity judgment → deterministic merger), tree-blind, message-granular provenance, no taught events, per-proposition understanding notes. Minimal implementation of the §1 philosophy next.**
6. **Demo capability** — the `demo` artifact contract (sandboxed iframe, error-fix loop) + the **demo-builder sub-agent** that tests code before delivery (§2.5). High value; slots in any time after step 1. **✅ Wave 2 (W6): demo + visual tools, verifying builders (headless Chrome), sandboxed demo card, SVG card. Live-verified (visual needs a vision-capable LH_VISUAL_MODEL).**
6b. **Researcher tool** — fact verification for the teacher (§2.5); cheap port of proven pi pieces. **✅ Wave 1 (W2): async job protocol, live activity card, sourced brief, teacher continuation. Live-verified.**
7. **Presentation fixes** — LaTeX delimiters, line breaks, mermaid integration. **✅ Wave 1 (W3): \(…\)/\[…\] contract ($ literal forever), remark-breaks, dark mermaid + styled streamdown chrome. Live-verified.**
@@ -185,8 +208,8 @@ The teacher's context window and attention are precious — it has the lesson to
## 5. Open questions
1. ~~Graph rendering library + layout architecture~~ — **decided 2026-08-22: sigma.js + forceatlas2, run-then-freeze** (see §1). Remaining: run the 2k-node synthetic benchmark before implementation.
-2. What exactly the naive-union v1 shows for overlapping topics across sessions (duplicate nodes acceptable until consolidation? — presumed yes, cosmetic only).
-3. How the probe phase's questions map onto tree nodes when no tree exists yet (probe happens *before* the plan is written — do probe outcomes attach to a provisional node set, or to the eventual tree retroactively?).
+2. ~~Overlapping topics across sessions~~ — resolved by the 2026-09-02 model: identity is the auditor's judgment (link-or-create, aliases as phrasings); duplicates are acceptable-and-correctable, wrong merges are not.
+3. ~~Probe questions when no tree exists yet~~ — resolved by free-form extraction: quiz attribution is retroactive against the whole transcript (extractor judgment, auditor-verified), with no dependence on the tree at all.
4. Home-graph v1 interactions beyond pan/zoom (node click → session list? — presumed cheap yes).
5. Grain mismatch frequency (`subsumes`-style relations) — per original doc, measured by data, not pre-solved.
6. Aggregation semantics for understanding (min / mean / two-level) — fit on real data, per original doc.
@@ -197,7 +220,12 @@ The teacher's context window and attention are precious — it has the lesson to
| Original | Now |
|---|---|
-| Teacher never modified; graph built post-hoc | Teacher emits structured data live; background pass consolidates |
+| Teacher never modified; graph built post-hoc | Post-hoc again (2026-09-02): free-form extraction, fully decoupled from the session's formalism — the 2026-08-22 live-emission reversal is itself reversed |
+| Misconception nodes as first-class structure | No node `kind`: every node is a true proposition; misconceptions enter as true corrections, wrongness lives in the event ledger |
+| `requires` edges | `follows` / `supports` — derivability vs. grounding; strength derived at read time, never stored |
+| `taught` events (marking things done) | Dropped; the extractor records per-proposition understanding notes (named-word scale) as session-scoped observations instead |
+| Chat-granular node/edge provenance | Message-granular (chat, message) provenance on nodes and edges — the exact derivation points |
+| `name` + `statement` on nodes | `statement` (minimal proposition, the rendering label) + `description` (required elaboration) + `aliases` (verbatim phrasings) |
| Updates per completed session | Continuous; partial sessions deposit value |
| Mermaid DAG as the plan artifact | Structured tools as source of truth; mermaid only for ad-hoc diagrams |
| UI deferred, "not important" | UI exists (Next.js, v3.4); the graph view is the app's home page |
diff --git a/src/agent/prompts/50-graph-context.md b/src/agent/prompts/50-graph-context.md
index 85d3ac2..da05b3d 100644
--- a/src/agent/prompts/50-graph-context.md
+++ b/src/agent/prompts/50-graph-context.md
@@ -1,20 +1,19 @@
# The learner's knowledge graph
-After these instruction fragments you may see a fenced block titled **LEARNER KNOWLEDGE GRAPH (auto-generated)**. It is the accumulated record of what this learner has actually demonstrated, across every past session — rebuilt fresh from quiz outcomes and plan trees, every turn. Each line is a proposition in its current state (in brackets), and `a → b` under **Requires** means *a* must be understood before *b*.
+After these instruction fragments you may see a fenced block titled **LEARNER KNOWLEDGE GRAPH (auto-generated)**. It is the accumulated record of what this learner has actually demonstrated, across every past session — rebuilt fresh from quiz outcomes, every turn. Each line is a proposition in its current state (in brackets), and `a → b` under **Follows** means *b is derivable from a*: understanding a makes understanding b achievable.
The states, exactly:
- **[missed]** — answered wrong. Treat as a live misconception site until a check proves otherwise.
- **[dont-know]** — the learner declined to guess. Unexplored, not broken.
- **[known]** — answered correctly. Foundation you can build on and refer to by name.
-- **[taught]** — a lesson covered it. Check that it stuck before stacking anything heavy on it.
-- **[candidate]** — asserted into a plan but never probed. Assume nothing.
+- **[candidate]** — present in the graph but never probed. Assume nothing.
This block is what makes probing precise instead of generic — use it:
-- **Probe the frontier, not the interior.** [known] and [taught] nodes near the session's topic are the learner's foundation: reference them by name and build on them — never re-teach what the graph says he holds. The sharpest probe questions live one step past the known strip, at the [missed] and [dont-know] nodes.
+- **Probe the frontier, not the interior.** [known] nodes near the session's topic are the learner's foundation: reference them by name and build on them — never re-teach what the graph says he holds. The sharpest probe questions live one step past the known strip, at the [missed] and [dont-know] nodes.
- **[missed] nodes are your first stops.** A wrong answer is a misconception until shown otherwise. Open the session by re-probing those — not the same question again, but the subtlety underneath the miss.
-- **Read the Requires chains as scaffolding.** They show what the learner's knowledge rests on. If a prerequisite under the session's topic is [missed] or [dont-know], that is where teaching starts — not at the topic itself.
+- **Read the Follows chains as scaffolding.** They show what the learner's knowledge rests on. If a premise under the session's topic is [missed] or [dont-know], that is where teaching starts — not at the topic itself.
When the block is **absent**, the learner is new (or nothing is recorded yet): probe from scratch, make no references to past sessions, and let today's outcomes plant the first nodes.
diff --git a/src/agent/subagents/extraction.ts b/src/agent/subagents/extraction.ts
index dc9edaf..57db55a 100644
--- a/src/agent/subagents/extraction.ts
+++ b/src/agent/subagents/extraction.ts
@@ -1,23 +1,28 @@
import { ToolLoopAgent, stepCountIs, tool, type LanguageModel, type UIMessage } from "ai";
import { z } from "zod";
import { modelFor } from "../models.ts";
-import type { AuditResult, ChatRecord, SessionFindings } from "../../core/contracts.ts";
-import { mergeFindings, type EmissionStats, type GraphDeps } from "../../core/graph.ts";
+import type { AuditResult, ChatRecord, GraphNode, SessionFindings } from "../../core/contracts.ts";
+import { getGraph, mergeFindings, type EmissionStats, type GraphDeps } from "../../core/graph.ts";
/**
- * The extraction pipeline (vision §1 construction) — the sole graph writer.
- * Three stages, one orchestrator:
+ * The extraction pipeline (vision §1 construction, 2026-09-02 redesign) — the
+ * sole graph writer. Three stages, one orchestrator:
*
- * extractor (LLM) transcript → SessionFindings (propositions, edges,
- * event attribution — each with verbatim evidence)
- * auditor (LLM) findings → AuditResult (the verified subset + drops)
+ * extractor (LLM) transcript → SessionFindings (free-form propositions,
+ * follows/supports edges, quiz attribution, understanding
+ * notes — each with verbatim evidence). TREE-BLIND: it
+ * sees only the transcript, never the plan tree.
+ * auditor (LLM) findings + THE CURRENT GRAPH → AuditResult: the
+ * verified subset with an identity verdict per
+ * proposition (linkTo an existing node, or create), plus
+ * the drops for the log.
* merger (core) mergeFindings() → deterministic ledger appends
*
* The LLM stages supply only the judgment calls; the deterministic rebuild
* in the merger owns everything that must never be mis-copied (event kinds,
- * timestamps, refs). Both agents are single-tool-call workers: their whole
- * deliverable is one zod-validated submit call — a ToolLoopAgent purely so
- * the model can retry a schema-invalid call.
+ * timestamps, refs, provenance points). Both agents are single-tool-call
+ * workers: their whole deliverable is one zod-validated submit call — a
+ * ToolLoopAgent purely so the model can retry a schema-invalid call.
*
* Created via createExtractionPipeline(deps) — the composition root
* (chat-handler) injects the storage ports + models; this module never
@@ -27,7 +32,7 @@ import { mergeFindings, type EmissionStats, type GraphDeps } from "../../core/gr
* the next trigger re-runs the whole thing.
*/
-// ─── the transcript view (what the LLM sees) ────────────────────────────────
+// ─── the transcript view (what the extractor + auditor see) ─────────────────
/**
* A quiz tool part's outcome, one line: the attribution evidence.
@@ -44,46 +49,35 @@ function quizLine(part: { toolCallId: string; input: unknown; output: unknown })
}
/**
- * The compact transcript the extractor + auditor both see: user/teacher
- * text verbatim (evidence quotes must match it), the final plan tree, and
- * the structured tool events (quiz outcomes, mark_done). Job tool parts
- * (research/demo/visual) are noise for extraction — their outcomes are
- * summarized as one line each.
+ * The compact transcript both LLM stages see: user/teacher text VERBATIM,
+ * each tagged with its message id (provenance anchors — the extractor
+ * reports them back per proposition/edge), and quiz outcome lines.
+ * TREE-BLIND by design (2026-09-02): no plan tree, no mark_done —
+ * extraction is free-form, decoupled from the session's plan. Job tool
+ * parts (research/demo/visual) are noise for extraction — their outcomes
+ * are summarized as one line each.
*/
export function transcriptView(record: ChatRecord): string {
const lines: string[] = [];
lines.push(`SESSION: ${record.title}`);
- if (record.tree && record.tree.nodes.length > 0) {
- const nodes = record.tree.nodes.map((n) => `${n.id} = ${n.title}`).join("; ");
- lines.push(`FINAL PLAN TREE (node ids): ${nodes}`);
- if (record.tree.edges.length > 0) {
- const edges = record.tree.edges.map((e) => `${e.from} requires ${e.to}`).join("; ");
- lines.push(`TREE EDGES: ${edges}`);
- }
- }
-
- for (const message of record.messages as UIMessage[]) {
+ record.messages.forEach((message, index) => {
const who = message.role === "user" ? "LEARNER" : "TEACHER";
- for (const part of message.parts) {
+ const mid = (message as UIMessage).id || `m${index}`;
+ for (const part of (message as UIMessage).parts) {
if (part.type === "text" && part.text.trim()) {
- lines.push(`${who}: ${part.text.trim()}`);
+ lines.push(`${who} [${mid}] ${part.text.trim()}`);
} else if (part.type.startsWith("tool-")) {
const name = part.type.slice("tool-".length);
const p = part as unknown as { toolCallId: string; state: string; input: unknown; output: unknown };
if (name === "quiz" && p.state === "output-available") {
lines.push(quizLine(p));
- } else if (name === "plan_mark_done" && p.state === "output-available") {
- const input = p.input as { id?: unknown } | undefined;
- lines.push(`TEACHER marked tree node done [${p.toolCallId}]: ${typeof input?.id === "string" ? input.id : "?"}`);
- } else if (name.startsWith("plan_")) {
- // plan ops are folded into the FINAL PLAN TREE line above
} else if (p.state === "output-available") {
lines.push(`(${name} tool result delivered)`);
}
}
}
- }
+ });
return lines.join("\n");
}
@@ -94,87 +88,98 @@ const evidence = z
.min(1)
.describe("Verbatim quotes from the transcript that back this. The auditor will check every quote.");
+const messagesField = z
+ .array(z.string())
+ .describe("The message ids (from the [id] tags in the transcript) where this came up — provenance.");
+
+const propositionSchema = z.object({
+ id: z.string().describe("Local id: \"p1\", \"p2\", … — referenced by edges and attribution."),
+ statement: z
+ .string()
+ .describe(
+ "The proposition as one reasoning step — a TRUE, self-contained, world-facing statement. " +
+ "Never reference the learner, the teacher, or the session: not \"the learner understands X\" but " +
+ '"X". A misconception is stated as the TRUE correction ("Nietzsche did not endorse ' +
+ 'might-makes-right social Darwinism") — the wrongness lives in the quiz outcomes, never here.',
+ ),
+ description: z
+ .string()
+ .describe(
+ "The elaborated version of the same proposition — more words: scope, precision, what exactly it means. Still true, still world-facing.",
+ ),
+ aliases: z.array(z.string()).describe("Verbatim phrasings this session used for the same proposition, if any."),
+ messages: messagesField,
+ evidence,
+ understanding: z
+ .string()
+ .optional()
+ .describe(
+ 'One note about THIS session\'s evidence of the learner\'s grasp of this proposition, on a named-word scale ' +
+ '("solid", "shaky", "accepted", "resisted", "not yet engaged") — no numbers, no predictions.',
+ ),
+});
+
+const edgeSchema = z.object({
+ from: z.string().describe("Proposition id that is followed / supported."),
+ to: z.string().describe("Proposition id that follows / is supported by `from`."),
+ type: z
+ .enum(["follows", "supports"])
+ .describe('"follows" = to is derivable from from; "supports" = from grounds to without entailing it.'),
+ messages: messagesField,
+ evidence,
+});
+
const findingsSchema = z.object({
propositions: z
- .array(
- z.object({
- id: z.string().describe("Local id: \"p1\", \"p2\", … — referenced by edges and attribution."),
- name: z.string().describe(
- "A short 2-6 word name for the fact — its header, used for rendering. Not a sentence.",
- ),
- statement: z
- .string()
- .describe(
- "The proposition as one reasoning step — an OBJECTIVE, world-facing fact. " +
- "Never reference the learner, the teacher, or the session: not \"the learner " +
- 'understands X\" but \"X\"; not \"the learner thinks Y\" but the wrong idea itself, ' +
- 'stated as a claim (\"Reading: Nietzsche endorsed might-makes-right social Darwinism\"), ' +
- "with kind \"misconception\" carrying that it is wrong.",
- ),
- aliases: z.array(z.string()).describe("Verbatim alternate phrasings from this session, if any."),
- kind: z.enum(["truth", "misconception"]).describe("misconception = a wrong idea the learner holds or was warned about."),
- treeNodeId: z.string().optional().describe("The plan-tree node id this came from, when it did."),
- evidence,
- }),
- )
- .describe("Every distinct proposition the session touched — from the plan tree AND from prose."),
+ .array(propositionSchema)
+ .describe("Every distinct proposition the session touched — true statements, misconception-free."),
edges: z
- .array(
- z.object({
- from: z.string().describe("Proposition id that is the prerequisite / support."),
- to: z.string().describe("Proposition id that depends on / is supported by `from`."),
- type: z.enum(["requires", "supports"]),
- evidence,
- }),
- )
- .describe("Relations between propositions. `requires` = prerequisite; `supports` = helpful but not prerequisite. Include relations the plan tree does NOT show when the transcript articulates them."),
+ .array(edgeSchema)
+ .describe("Relations between propositions: follows (derivable) / supports (grounds without entailing)."),
attribution: z
.object({
quiz: z
.record(z.string(), z.string().nullable())
- .describe("toolCallId of every UNTAGGED quiz line → the proposition id it was really testing. Probe questions asked BEFORE the tree existed are attributed RETROACTIVELY — the whole transcript is visible; map them to the proposition they were probing (often articulated only later). null when genuinely unattributable. Tagged quizzes (with a tree nodeId in the transcript view) resolve automatically — do not include them."),
+ .describe(
+ "toolCallId of EVERY quiz line → the proposition id it was really testing. Attribute RETROACTIVELY — the whole transcript is visible; map each quiz to the proposition it tests (often articulated elsewhere in the session). null when genuinely unattributable.",
+ ),
})
- .describe("Probe attribution — your only event judgment."),
+ .describe("Quiz attribution — your only event judgment."),
});
const EXTRACTOR_INSTRUCTIONS = `You are the extraction specialist in a learning application. You read ONE session
transcript and report what was learned — as structured findings for the learner's global knowledge graph.
-The transcript view shows: the session's final plan tree (dependency map, with node ids), the conversation
-verbatim (LEARNER / TEACHER), and structured events (QUIZ lines with the learner's outcome, mark_done lines).
+The transcript view shows: the conversation verbatim (LEARNER / TEACHER lines, each tagged with its message id)
+and QUIZ outcome lines.
Report, with verbatim evidence quotes for everything:
-NODE THEORY — the graph is a set of objective facts, full stop. A node is what is true (or falsely
-believed) about the world, never anything about the learner or the session. The learner's
-relationship to a fact — whether they know it, missed it, or hold it as a misconception — is
-expressed ENTIRELY by the quiz-outcome events you attribute; it never leaks into the node itself.
-So no "the learner understands X", no "discussed in this session": the node is just X. A
-misconception node states the wrong idea itself as a claim, with kind "misconception" marking it
-wrong — never "the learner believes Y".
+NODE THEORY — the graph is a set of TRUE propositions about the world, full stop. A node is never anything
+about the learner or the session. The learner's relationship to a proposition — whether they know it, missed
+it, resisted it — is expressed ENTIRELY by the quiz outcomes and your understanding note; it never leaks into
+the node itself. So no "the learner understands X", no "discussed in this session": the node is just X. A
+misconception is NOT a node of its own wrong claim — state the TRUE correction as the proposition, and let the
+quiz outcomes carry that the learner believed the wrong version.
1. PROPOSITIONS — every distinct reasoning-step-sized proposition the session touched:
- - Each carries a "name": a short 2-6 word header (the rendering handle), and a "statement": the
- full fact as one objective reasoning step.
- - Plan-tree nodes become propositions (pass their treeNodeId) — statement from the node title, enriched to a
- complete proposition if the prose elaborates it.
- - Propositions that appeared only in prose (teacher explanations, learner questions) get fresh ids.
- - Include MISCONCEPTIONS — wrong ideas the learner showed or the teacher warned against — with kind
- "misconception".
+ - Each is a self-contained TRUE statement ("statement"), plus its elaboration ("description" — more
+ words, same proposition: scope, precision, what exactly it means).
+ - Include corrections of misconceptions the teacher articulated, and true facts explained in prose.
- One reasoning step per proposition: never a topic blob, never a sub-step.
- - aliases: verbatim alternate phrasings this session used for the same thing.
+ - "messages": the [id] tags of every message where the proposition came up — provenance, report it.
+ - "aliases": verbatim phrasings this session used for the same proposition.
+ - "understanding": ONE named-word-scale note on the evidence of the learner's grasp, for this session
+ only ("solid", "shaky", "accepted", "resisted", "not yet engaged"). No numbers, no predictions.
2. EDGES between propositions:
- - The tree's dependency edges are the backbone: from = prerequisite, to = dependent, type "requires".
- - Also include relations articulated only in prose. Use "supports" when the transcript presents something as
- helpful but NOT a prerequisite.
+ - "follows": to is DERIVABLE from from (understanding from makes understanding to achievable).
+ - "supports": from GROUNDS to without entailing it (helpful context, not a prerequisite).
+ - Only relations the transcript actually articulates — with evidence.
-3. ATTRIBUTION — UNTAGGED quizzes only:
- - QUIZ lines whose question carries a tree nodeId resolve automatically; do not attribute those.
- - The others are PROBE-phase questions (asked before the tree was initialized) that test prerequisite
- understanding. Attribute them RETROACTIVELY to the proposition they were really probing — even if that
- proposition is only articulated later in the session. The QUIZ question text is your guide. If you truly
- cannot tell, use null.
+3. ATTRIBUTION — every QUIZ line gets a mapping: toolCallId → the proposition id it was really testing.
+ Attribute RETROACTIVELY — a quiz often tests a proposition the session only articulates later; the
+ question text is your guide. If you truly cannot tell, use null.
Evidence discipline: every quote must appear VERBATIM in the transcript view (a QUIZ line counts as the
transcript). The auditor rejects anything unbacked. When in doubt, report; the auditor decides.
@@ -205,6 +210,16 @@ function createExtractor(model: LanguageModel) {
// ─── the auditor ───────────────────────────────────────────────────────────
const auditSchema = findingsSchema.extend({
+ propositions: z.array(
+ propositionSchema.extend({
+ linkTo: z
+ .string()
+ .optional()
+ .describe(
+ "The id of the EXISTING graph node this proposition IS — your identity verdict. Absent = create a new node. When unsure, omit: duplicates over wrong merges.",
+ ),
+ }),
+ ),
dropped: z
.array(
z.object({
@@ -216,27 +231,46 @@ const auditSchema = findingsSchema.extend({
});
const AUDITOR_INSTRUCTIONS = `You are the audit stage of a learning application's extraction pipeline. You receive
-ONE session transcript and a set of proposed findings (propositions, edges, event attribution). Your job is the
-faithfulness net: verify every candidate against the transcript, keep only what is backed, and return the
-verified subset.
+ONE session transcript, a set of proposed findings (propositions, edges, quiz attribution), and the CURRENT
+knowledge graph. Your jobs: the faithfulness net, and identity.
-Verify:
-- PROPOSITIONS: every evidence quote must appear (near-)verbatim in the transcript view, and the statement must
- be a faithful reading of it. The node theory applies: a statement must be an OBJECTIVE, world-facing fact —
- reword learner- or session-relative statements ("the learner understands X" → "X") or drop them when the
- objective fact is not actually backed. Drop invented or bloated statements. Conservative: when two proposed propositions
- are really one, keep the better-stated one and drop the other.
-- EDGES: the evidence must actually articulate the relation, in the direction claimed (from = prerequisite).
- Drop direction-swapped or speculative edges.
-- ATTRIBUTION: only UNTAGGED quizzes carry attribution — the QUIZ line's question must genuinely test the
- attributed proposition. Drop wrong attributions to null, or re-point them to the proposition the question
- actually tests.
+FAITHFULNESS — verify every candidate against the transcript:
+- PROPOSITIONS: every evidence quote must appear (near-)verbatim in the transcript view, and the statement
+ must be a faithful reading of it. The node theory applies: a statement must be a TRUE, self-contained,
+ world-facing fact — reword learner- or session-relative statements ("the learner understands X" → "X"), state
+ misconceptions as the true correction, or drop them when no objective fact is backed. Drop invented or
+ bloated statements. Conservative: when two proposed propositions are really one, keep the better-stated one
+ and drop the other.
+- EDGES: the evidence must actually articulate the relation, in the direction claimed. Drop
+ direction-swapped or speculative edges.
+- ATTRIBUTION: the QUIZ line's question must genuinely test the attributed proposition. Drop wrong
+ attributions to null, or re-point them to the proposition the question actually tests.
+- UNDERSTANDING notes: verify against the quiz outcomes and the session's evidence; reword to a named-word
+ scale or drop.
+
+IDENTITY — link-or-create, per proposition:
+- You see the current graph (every node: id, statement, description, aliases). For each verified proposition,
+ decide: does an EXISTING node already state this same proposition?
+- If yes: set "linkTo" to that node's id. The session's phrasings will join its aliases and its provenance.
+- If no: omit "linkTo" and a new node is created.
+- When unsure, omit — duplicates are recoverable later; wrong merges are not.
Also drop propositions not referenced by anything else ONLY if they have no other backing — an unreferenced but
well-evidenced proposition is still knowledge.
Never invent new candidates. Your entire deliverable is ONE submit_review call containing the verified findings
-(the same shapes you received) plus the dropped list.`;
+(the same shapes you received, with linkTo added) plus the dropped list.`;
+
+/** The current graph, compact — the auditor's identity context. */
+function graphView(nodes: GraphNode[]): string {
+ if (nodes.length === 0) return "(the graph is currently empty — every proposition is a new node)";
+ const lines = nodes.map(
+ (n) =>
+ `- ${n.id} :: ${n.statement}\n description: ${n.description}` +
+ (n.aliases.length > 0 ? `\n aliases: ${n.aliases.join(" | ")}` : ""),
+ );
+ return lines.join("\n");
+}
function createAuditor(model: LanguageModel) {
return new ToolLoopAgent({
@@ -244,7 +278,7 @@ function createAuditor(model: LanguageModel) {
instructions: AUDITOR_INSTRUCTIONS,
tools: {
submit_review: tool({
- description: "Submit the verified findings subset + the dropped list.",
+ description: "Submit the verified findings subset (with identity verdicts) + the dropped list.",
inputSchema: auditSchema,
execute: async (review) => review,
}),
@@ -295,17 +329,20 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
throw new Error("the extractor never submitted findings");
}
- async function audit(chatId: string, findings: SessionFindings): Promise {
+ async function audit(chatId: string, findings: SessionFindings): Promise {
const record = chatStore.getChat(chatId);
if (!record) throw new Error(`Chat not found: ${chatId}`);
const view = transcriptView(record);
+ const graph = getGraph(graphStore);
console.info(
`[extraction ${chatId}] auditor starting (${findings.propositions.length} propositions, ${findings.edges.length} edges, ` +
- `${Object.keys(findings.attribution.quiz).length} attributions)`,
+ `${Object.keys(findings.attribution.quiz).length} attributions, ${graph.nodes.length} graph nodes)`,
);
const review = await auditorAgent.generate({
- prompt: `Transcript:\n\n${view}\n\nProposed findings:\n\n${JSON.stringify(findings, null, 2)}`,
+ prompt:
+ `Transcript:\n\n${view}\n\nProposed findings:\n\n${JSON.stringify(findings, null, 2)}\n\n` +
+ `Current knowledge graph (${graph.nodes.length} nodes):\n\n${graphView(graph.nodes)}`,
});
let auditResult: AuditResult | null = null;
for (const part of review.content) {
@@ -320,12 +357,12 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
auditResult.dropped.map((d) => `${d.ref} (${d.reason})`).join("; "),
);
}
- const { dropped: _dropped, ...verified } = auditResult;
console.info(
- `[extraction ${chatId}] audit kept ${verified.propositions.length}/${findings.propositions.length} propositions, ` +
- `${verified.edges.length}/${findings.edges.length} edges`,
+ `[extraction ${chatId}] audit kept ${auditResult.propositions.length}/${findings.propositions.length} propositions, ` +
+ `${auditResult.edges.length}/${findings.edges.length} edges, ` +
+ `${auditResult.propositions.filter((p) => p.linkTo !== undefined).length} linked to existing nodes`,
);
- return verified;
+ return auditResult;
}
async function runExtraction(chatId: string): Promise {
@@ -334,10 +371,10 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
try {
console.info(`[extraction ${chatId}] pipeline start`);
const findings = await extract(chatId);
- const verified = await audit(chatId, findings);
+ const auditResult = await audit(chatId, findings);
// The merger re-reads the chat record itself — events are rebuilt
// from the transcript, never from stale copies.
- const stats = mergeFindings(chatId, verified, { chatStore, graphStore });
+ const stats = mergeFindings(chatId, auditResult, { chatStore, graphStore });
console.info(`[extraction ${chatId}] merged: ${stats.nodes} nodes, ${stats.edges} edges, ${stats.events} events`);
return stats;
} finally {
@@ -347,3 +384,4 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
return { runExtraction };
}
+
diff --git a/src/core/contracts/graph.ts b/src/core/contracts/graph.ts
index ff5fc51..bb0d8e1 100644
--- a/src/core/contracts/graph.ts
+++ b/src/core/contracts/graph.ts
@@ -1,50 +1,69 @@
/**
* Knowledge-graph contracts — nodes, ledgers, extraction findings
*
+ * The minimal implementation of the graph vision (docs/vision.md §1, the
+ * 2026-09-02 redesign): every node is a TRUE proposition; edges are follows
+ * (derivable) and supports (grounds without entailing); provenance is
+ * message-granular; extraction is free-form and post-hoc, fully decoupled
+ * from the session's plan tree; the event ledger records quiz outcomes plus
+ * the extractor's per-proposition understanding observations. Anything not
+ * yet decided (the state model, strength derivation, understanding
+ * treatment, selection strategies) is deliberately NOT here — transitional
+ * stubs carry the renderer until those land.
+ *
* Part of the domain-split contracts (src/core/contracts/): each domain file
* is owned outright by its stream — cross-domain schema changes go through
- * master. Pure types + protocol docs + tiny pure helpers; type-only imports
- * of "ai" (safe everywhere: server, client, tests). See docs/vision.md.
+ * master. Pure types + protocol docs + tiny pure helpers. See docs/vision.md.
*/
// ─── Knowledge graph (vision §1 — global, cross-session) ────────────────────
-export type EdgeType = "requires" | "supports";
+/** Two relations only (DECIDED 2026-09-02): follows = b is derivable from a;
+ * supports = a grounds b without entailing it. No scalars, no encoded
+ * strength — strength is derived at read time from topology. */
+export type EdgeType = "follows" | "supports";
+
+/** One provenance point — message-granular (DECIDED 2026-09-02): nodes and
+ * edges record every (chat, message) where they came up, deep-linkable to
+ * the exact transcript point. */
+export interface ProvenancePoint {
+ chatId: string;
+ messageId: string;
+}
export interface GraphNode {
/** Opaque stable id, store-assigned: "n_" + nanoid-like. */
id: string;
- /** Short name / header for the fact (2-6 words) — the rendering handle.
- * NODE THEORY: a node is an OBJECTIVE fact, never learner- or
- * session-relative. No "the learner understands…", no "this session
- * covered…" — just the fact, world-facing. The learner's relationship
- * to the fact lives ENTIRELY in the event ledger: state labels are
- * derived from quiz outcomes, never written into the node. */
- name: string;
- /** Canonical proposition statement — the full fact, one reasoning step,
- * also objective (no learner / teacher / session references). */
+ /** The minimal proposition — one true, self-contained statement. This is
+ * BOTH the rendering label AND the identity key: the auditor recognizes
+ * a re-encounter by judgment (link-or-create), not by string matching. */
statement: string;
- /** Verbatim alternate statements from sessions (merge target audit trail). */
+ /** The elaborated version — required. Same proposition, more words:
+ * scope, precision, the "what exactly does it mean". */
+ description: string;
+ /** Verbatim phrasings sessions used for this proposition (the identity
+ * audit trail — how the world said what the node says). */
aliases: string[];
- kind: "truth" | "misconception";
createdAt: string;
+ /** Every (chat, message) point where this proposition came up. */
+ provenance: ProvenancePoint[];
}
export interface GraphEdge {
- /** Prerequisite GraphNode id. */
from: string;
to: string;
type: EdgeType;
- /** Chat ids that asserted this edge (teacher may simplify — auditable). */
- provenance: string[];
+ /** Every (chat, message) point where this relation was asserted. */
+ provenance: ProvenancePoint[];
}
/** mappings.jsonl line — the session→graph identity map: the extraction
- * pipeline's proposition (identified by its source id) → the graph node it
- * merged into. The merge audit trail: one line per (chat, graph node). */
+ * pipeline's proposition (identified by its run-local source id) → the
+ * graph node it resolved to (created or linked). The audit trail of the
+ * auditor's identity verdicts; also the chat-scoped resolution table the
+ * merger uses to point events and edges at node ids. */
export interface NodeMapping {
chatId: string;
- /** The writer's local id for the proposition — a plan-tree node id when
- * tree-derived, else the extractor's proposition id ("p1", …). */
+ /** The extractor's run-local proposition id ("p1", …). */
sourceId: string;
graphNodeId: string;
}
@@ -57,23 +76,26 @@ export interface SessionCheckpoint {
foldedAt: string;
}
-export type LedgerEventKind =
- | "probe-correct"
- | "probe-miss"
- | "dont-know"
- | "misconception-signaled"
- | "taught"
- | "check-correct"
- | "check-incorrect";
+/**
+ * Event kinds (DECIDED 2026-09-02): quiz outcomes only — correct / miss /
+ * dont-know — plus `observation`, the extractor's per-proposition
+ * understanding note joining the ledger as a session-scoped observation
+ * (named-word scale; treatment pending the state-model ideation). No
+ * `taught` events at all; no misconception kind — a misconception is a
+ * TRUE correction in the graph, and the wrongness lives here, in the
+ * ledger, until the state model exists.
+ */
+export type LedgerEventKind = "correct" | "miss" | "dont-know" | "observation";
export interface LedgerEvent {
- /** null = not yet attributed (untagged probe events; consolidation assigns). */
+ /** null = not yet attributed. */
nodeId: string | null;
kind: LedgerEventKind;
chatId: string;
at: string;
- /** Idempotency key: the quiz part's toolCallId or `taught:` —
- * re-emission of the same chat must never double-record an event. */
+ /** Idempotency key: the quiz part's toolCallId, or `obs:` for
+ * understanding observations — re-emission of the same chat must never
+ * double-record an event. */
ref?: string;
/** The learner's quiz note — diagnostic gold (vision §1 calibration). */
note?: string;
@@ -83,91 +105,121 @@ export interface LedgerEvent {
/**
* Store format (src/store/graph.ts): JSONL ledgers at config.LEDGER_FILES
- * (nodes/edges/events/mappings/sessions). The derived graph is rebuilt by
- * reading the ledgers — store events, derive state. sessions.jsonl is the
- * extraction checkpoint: which chats the extraction pipeline has processed,
- * and through which updatedAt (never extract the same turns twice).
+ * (nodes/edges/events/mappings/sessions), all append-only.
+ *
+ * nodes.jsonl one line per node — the FIRST line for an id defines
+ * statement/description/createdAt; later lines for the
+ * same id are link-appends (a session's re-encounter):
+ * their statements/aliases join `aliases`, their
+ * provenance unions. getGraph merges by id.
+ * edges.jsonl one line per edge ASSERTION; readers merge by
+ * (from,to,type), unioning provenance.
+ *
+ * The derived graph is rebuilt by reading the ledgers — store events,
+ * derive state. sessions.jsonl is the extraction checkpoint: which chats
+ * the extraction pipeline has processed, and through which updatedAt
+ * (never extract the same turns twice).
*
* The EXTRACTION PIPELINE is the sole graph writer (extractor → auditor →
- * merger): the extractor proposes propositions/edges/attribution with
- * verbatim evidence, the auditor drops anything unbacked, and the merger
- * (core/graph.ts, deterministic) resolves identity against the existing
- * ledgers and appends. See SessionFindings below.
+ * merger), tree-blind and post-hoc: the extractor reads only the transcript
+ * and proposes propositions/edges/quiz-attribution with verbatim evidence;
+ * the auditor sees the extractor's output PLUS the current graph, owns
+ * faithfulness (evidence check) and identity (link-or-create: point each
+ * proposition at the existing node it is, or let it be created — recording
+ * the session's phrasings as aliases); the merger (core/graph.ts,
+ * deterministic) applies the verdicts and appends.
*/
// ─── Extraction findings (extractor output → audit → merger input) ───────────
-/** One proposed proposition — a node candidate. */
+/** One proposed proposition — a node candidate (or a re-encounter of one). */
export interface ExtractedProposition {
/** Extractor-local id ("p1", …) referenced by edges + attribution. */
id: string;
- /** Short name / header for the fact (2-6 words) — the rendering handle. */
- name: string;
- /** Canonical one-reasoning-step statement — an objective, world-facing
- * fact (no learner / teacher / session references; the learner's
- * relationship to it is carried only by the event attribution). */
+ /** The minimal proposition — a true, self-contained, world-facing
+ * statement. No learner / teacher / session references: misconceptions
+ * are stated as the true correction, and wrongness lives in the event
+ * ledger, never in the node. */
statement: string;
- /** Verbatim alternate phrasings from the session (merge aliases). */
+ /** The elaborated version — required. */
+ description: string;
+ /** Verbatim phrasings the session used. */
aliases: string[];
- kind: "truth" | "misconception";
- /** The plan-tree node this came from, when it did (mapping provenance). */
- treeNodeId?: string;
+ /** The message ids where this proposition came up — provenance. */
+ messages: string[];
/** Verbatim transcript quotes backing the proposition. */
evidence: string[];
+ /** The extractor's understanding note for this proposition IN THIS
+ * session — a named-word scale ("solid", "shaky", "accepted",
+ * "resisted"), no numbers. Joins the ledger as an observation event;
+ * treatment pending the state-model ideation. */
+ understanding?: string;
}
/** One proposed relation between two propositions. */
export interface ExtractedEdge {
- /** Proposition id that is the prerequisite / support. */
+ /** Proposition id that is followed / supported. */
from: string;
- /** Proposition id that depends on / is supported by `from`. */
to: string;
type: EdgeType;
+ /** The message ids where this relation was asserted — provenance. */
+ messages: string[];
/** Verbatim transcript quotes backing the relation. */
evidence: string[];
}
-/** The extractor's event attribution — its ONLY say about events. The kinds,
- * timestamps, idempotency refs, notes and question texts are all rebuilt
- * deterministically by the merger from the transcript's quiz / mark_done
- * tool parts; the LLM supplies exactly the judgment call: WHICH proposition
- * each one is about (vision open question: probes attribute retroactively,
- * against the whole transcript — a pre-tree probe can map to a proposition
- * the tree only articulates later). */
-/** The extractor's event attribution — its ONLY say about events. Event
- * kinds, timestamps, idempotency refs, notes and question texts are all
- * rebuilt deterministically by the merger from the transcript's tool
- * parts, and TAGGED events (quiz nodeId, mark_done tree node) resolve
- * deterministically through the proposition mapping. The LLM supplies
- * exactly the one judgment call that needs it — the vision's open
- * probe-attribution question: WHICH proposition did an UNTAGGED probe
- * test? Answered retroactively against the whole transcript — a pre-tree
- * probe can map to a proposition the tree only articulates later. */
+/**
+ * The extractor's quiz attribution — its ONLY say about quiz events. Event
+ * kinds, timestamps, idempotency refs, notes and question texts are all
+ * rebuilt deterministically by the merger from the transcript's quiz tool
+ * parts; the LLM supplies exactly the judgment call: WHICH proposition each
+ * quiz tested. Tree-blind, so EVERY quiz attributes here (there is no tree
+ * mapping to lean on) — retroactively against the whole transcript.
+ */
export interface ExtractionAttribution {
- /** UNTAGGED quiz toolCallId → proposition id; missing / null =
- * unattributed (the event lands with nodeId null + the question text,
- * exactly like unattributable probes always did). */
+ /** quiz toolCallId → proposition id; null / missing = unattributed (the
+ * event lands with nodeId null + the question text, a later-
+ * attribution input). */
quiz: Record;
}
-/** The extractor's — and, filtered, the auditor's — output shape. */
+/** The extractor's output shape. */
export interface SessionFindings {
propositions: ExtractedProposition[];
edges: ExtractedEdge[];
attribution: ExtractionAttribution;
}
-/** The auditor's verdict: the verified subset, plus the drops (for the log —
- * the audit trail of what was rejected and why). */
-export interface AuditResult extends SessionFindings {
+/** The auditor's verdict on one proposition: the findings, plus `linkTo` —
+ * the identity judgment. Absent = create a new node. */
+export interface AuditedProposition extends ExtractedProposition {
+ /** The existing GraphNode id this proposition IS — the auditor's
+ * identity verdict. On link, the session's phrasings (statement,
+ * aliases) join the node's aliases and the provenance unions.
+ * Conservative: when unsure, leave it out — duplicates over wrong
+ * merges. */
+ linkTo?: string;
+}
+
+/** The auditor's output: the verified findings (with identity verdicts),
+ * plus the drops (the audit trail of what was rejected and why). */
+export interface AuditResult {
+ propositions: AuditedProposition[];
+ edges: ExtractedEdge[];
+ attribution: ExtractionAttribution;
/** Dropped candidates: proposition id, edge "from→to", or attribution
* toolCallId — with the reason. */
dropped: { ref: string; reason: string }[];
}
-/** Per-node state class for the home view (vision §1 "Home page") — derived,
- * never stored: taught if ANY taught event; else the LATEST quiz event decides
- * (known / missed / dont-know); a node with no events is a candidate. */
+/**
+ * Per-node state class for the home view — TRANSITIONAL (kept so the
+ * renderer survives the contract change untouched). Derived, never stored:
+ * the LATEST quiz event decides (known / missed / dont-know); a node with
+ * no quiz events is a candidate. `taught` is now unreachable (no taught
+ * events are emitted anymore) — the real state model (dependency depth +
+ * topology-derived strength + quiz events) is pending ideation.
+ */
export type NodeStateClass = "taught" | "known" | "missed" | "dont-know" | "candidate";
/** The GET /api/graph payload — everything the home view's PURE renderer needs. */
@@ -187,4 +239,3 @@ export function titleFromPrompt(prompt: string): string {
const oneLine = prompt.replace(/\s+/g, " ").trim();
return oneLine.length > 72 ? `${oneLine.slice(0, 72)}…` : oneLine;
}
-
diff --git a/src/core/graph-context.ts b/src/core/graph-context.ts
index 2bf6c68..9f1bf6d 100644
--- a/src/core/graph-context.ts
+++ b/src/core/graph-context.ts
@@ -15,10 +15,9 @@
* the block is absent (new learner).
*
* Selection strategy (§1 "the selection problem", simplest first): the WHOLE
- * graph while it is small; past MAX_NODES, hotspots (missed / dont-know /
- * misconception nodes) plus their 1-hop `requires` neighborhood — a live
- * misconception is always relevant context, and the frontier is where the
- * next session should probe.
+ * graph while it is small; past MAX_NODES, hotspots (missed / dont-know
+ * nodes) plus their 1-hop `follows` neighborhood — the frontier is where
+ * the next session should probe.
*/
import type { GraphEdge, GraphNode } from "./contracts.ts";
@@ -31,22 +30,18 @@ export const GRAPH_CONTEXT_HEADER = "LEARNER KNOWLEDGE GRAPH (auto-generated)";
/** Above this node count only hotspots + their 1-hop neighborhood are shown. */
export const MAX_CONTEXT_NODES = 150;
-/** Group order for node lines: diagnostic value first (vision §1 hotspots).
- * A misconception-kind node is a hotspot whatever its state — a live
- * misconception is always relevant context. */
+/** Group order for node lines: diagnostic value first (vision §1 hotspots). */
const HOTSPOT_STATES: ReadonlySet = new Set(["missed", "dont-know"]);
function groupOf(node: GraphNode, state: NodeStateClass): number {
- if (HOTSPOT_STATES.has(state) || node.kind === "misconception") return 0;
+ if (HOTSPOT_STATES.has(state)) return 0;
if (state === "known") return 1;
- if (state === "taught") return 2;
- return 3; // candidate
+ return 2; // candidate
}
const GROUP_LABELS = [
- "Hotspots (missed / dont-know / misconceptions) — likely misconception sites and unexplored ground:",
+ "Hotspots (missed / dont-know) — likely misconception sites and unexplored ground:",
"Known (answered correctly):",
- "Taught (lessons covered them):",
"Candidates (never probed — assume nothing):",
] as const;
@@ -76,9 +71,9 @@ export function renderGraphContext(
truncated = true;
const hotspots = nodes.filter((n) => isHotspot(n, stateOf(n)));
const keep = new Set(hotspots.map((n) => n.id));
- // 1-hop requires neighborhood: prerequisites AND dependents.
+ // 1-hop follows neighborhood: premises AND derivations.
for (const edge of edges) {
- if (edge.type !== "requires") continue;
+ if (edge.type !== "follows") continue;
if (keep.has(edge.from) || keep.has(edge.to)) {
keep.add(edge.from);
keep.add(edge.to);
@@ -91,7 +86,7 @@ export function renderGraphContext(
// ── node lines, grouped/sorted by diagnostic priority ──────────────────────────
const lines: string[] = [
`===== ${GRAPH_CONTEXT_HEADER} =====`,
- `${truncated ? `showing ${selected.length} of ${nodes.length} nodes (hotspots + their 1-hop neighborhood)` : `${nodes.length} nodes`}, ${edges.filter((e) => e.type === "requires").length} requires edges`,
+ `${truncated ? `showing ${selected.length} of ${nodes.length} nodes (hotspots + their 1-hop neighborhood)` : `${nodes.length} nodes`}, ${edges.filter((e) => e.type === "follows").length} follows edges`,
];
for (const [group, label] of GROUP_LABELS.entries()) {
@@ -101,19 +96,18 @@ export function renderGraphContext(
if (groupNodes.length === 0) continue;
lines.push("", label);
for (const node of groupNodes) {
- const kind = node.kind === "misconception" ? " · misconception" : "";
- lines.push(`• [${stateOf(node)}] ${node.statement} (${node.id})${kind}`);
+ lines.push(`• [${stateOf(node)}] ${node.statement} (${node.id})`);
}
}
// ── requires edges, merged into chains ──────────────────────────────────
const byId = new Map(selected.map((n) => [n.id, n.statement]));
- const requires = edges.filter(
- (e) => e.type === "requires" && byId.has(e.from) && byId.has(e.to),
+ const follows = edges.filter(
+ (e) => e.type === "follows" && byId.has(e.from) && byId.has(e.to),
);
- if (requires.length > 0) {
- lines.push("", "Requires (prerequisite → dependent):");
- for (const chain of edgeChains(requires)) {
+ if (follows.length > 0) {
+ lines.push("", "Follows (premise → derivation):");
+ for (const chain of edgeChains(follows)) {
lines.push(chain.map((id) => byId.get(id) ?? id).join(" → "));
}
}
diff --git a/src/core/graph.ts b/src/core/graph.ts
index dc21609..1501078 100644
--- a/src/core/graph.ts
+++ b/src/core/graph.ts
@@ -2,22 +2,34 @@
* The knowledge graph: merger (the deterministic final stage of the
* extraction pipeline — vision §1 construction) + derivation (reads).
*
- * The extraction pipeline is the SOLE graph writer:
+ * The extraction pipeline is the SOLE graph writer, tree-blind and post-hoc:
*
- * extractor (LLM) → SessionFindings: propositions, edges, and event
- * attribution, each with verbatim evidence
- * auditor (LLM) → the verified subset (unbacked candidates dropped)
+ * extractor (LLM) → SessionFindings: propositions, edges, quiz
+ * attribution, per-proposition understanding notes —
+ * each with verbatim evidence, from the transcript ALONE
+ * auditor (LLM) → AuditResult: the verified subset, with the IDENTITY
+ * VERDICT per proposition (linkTo an existing node, or
+ * create), and the drops for the log
* merger (HERE) → mergeFindings(): deterministic ledger appends
*
* The merger is deliberately the ONLY stage that touches the ledgers, and it
- * is pure code: the LLM supplies exactly the judgment calls (what was
- * taught, what depends on what, which proposition a quiz tested), while
- * event kinds/timestamps/refs — anything the model could silently
- * mis-copy — are rebuilt deterministically from the transcript's tool
- * parts. Idempotent + incremental: statement/alias matching dedupes nodes,
- * (from,to,type) dedupes edge assertions, chat-scoped refs dedupe events,
- * one mapping line per (chat, graph node), and sessions.jsonl checkpoints
- * which chats have been extracted (never the same turns twice).
+ * is pure code: the LLM supplies exactly the judgment calls (what the
+ * propositions are, what follows/supports what, which proposition a quiz
+ * tested, whether a proposition IS an existing node), while event
+ * kinds/timestamps/refs/provenance — anything the model could silently
+ * mis-copy — are rebuilt deterministically from the transcript's tool parts
+ * and the message ids in the findings. Idempotent + incremental: refs are
+ * chat-scoped idempotency keys, edges dedupe by (from,to,type)-already-
+ * asserted-by-this-chat, mapping lines dedupe by (chat, sourceId), and
+ * sessions.jsonl checkpoints which chats have been extracted (never the
+ * same turns twice).
+ *
+ * Identity across re-encounters is the AUDITOR's judgment (link-or-create),
+ * not string matching: the conservative v1 exact-statement merge was
+ * replaced by the 2026-09-02 redesign. A re-extracted session's
+ * propositions link to the existing nodes; the link appends a node line
+ * whose phrasings join `aliases` and whose provenance unions (getGraph
+ * merges lines by id).
*
* Derivation below (getGraph / deriveNodeStates / graphSnapshot) is reads
* only — everything is recomputed from the ledgers. Every function takes
@@ -29,14 +41,14 @@
import { randomBytes } from "node:crypto";
import { isToolUIPart, type UIMessage } from "ai";
import type {
+ AuditResult,
GraphEdge,
GraphNode,
+ ProvenancePoint,
GraphSnapshot,
LedgerEvent,
LedgerEventKind,
NodeStateClass,
- PlanToolResult,
- SessionFindings,
} from "./contracts.ts";
import type { QuestionAnswerData } from "./question-core.ts";
import type { ChatStore, GraphStore } from "./ports.ts";
@@ -80,19 +92,25 @@ function toolParts(messages: UIMessage[]): ToolPartView[] {
return out;
}
-/** A plan tool part's output, if it is a well-formed PlanToolResult. */
-function planResult(part: ToolPartView): PlanToolResult | null {
- if (part.state !== "output-available") return null;
- const out = part.output;
- if (typeof out !== "object" || out === null) return null;
- if (!("tree" in out) || typeof (out as PlanToolResult).ok !== "boolean") return null;
- return out as PlanToolResult;
+/** (chatId, messageIds) → provenance points; order-preserving, deduped. */
+function pointsOf(chatId: string, messages: string[]): ProvenancePoint[] {
+ const seen = new Set();
+ const out: ProvenancePoint[] = [];
+ for (const messageId of messages) {
+ if (messageId.length === 0 || seen.has(messageId)) continue;
+ seen.add(messageId);
+ out.push({ chatId, messageId });
+ }
+ return out;
}
+const samePoint = (a: ProvenancePoint, b: ProvenancePoint): boolean =>
+ a.chatId === b.chatId && a.messageId === b.messageId;
+
// ─── the merger (extraction pipeline stage 3 — deterministic) ────────────────
export interface EmissionStats {
- /** New graph nodes appended (and mappings with them). */
+ /** New graph nodes appended. */
nodes: number;
/** New edge assertions appended. */
edges: number;
@@ -102,156 +120,135 @@ export interface EmissionStats {
/**
* Merge one chat's audited findings into the graph ledgers — the pipeline's
- * apply stage. Deterministic + idempotent: re-merging the same findings (or
+ * apply stage. Deterministic + idempotent: re-merging the same verdicts (or
* a later re-extraction of the same chat) appends nothing but the new.
*
- * Identity resolution (v1: conservative): a proposition whose statement or
- * alias EXACTLY matches (case-insensitive, trimmed) an existing node's
- * statement or alias merges onto it; everything else is a new node. One
- * mapping line per (chat, graph node) — the merge audit trail. Fuzzy /
- * embedding adjudication stays deferred (vision §1 consolidation).
+ * Identity is the auditor's link-or-create verdict: `linkTo` an existing
+ * node (validated against the ledger — an unknown id falls back to create)
+ * appends a LINK line (the session's phrasings join the node's aliases and
+ * the provenance unions; getGraph merges lines by id); absent → a new node.
+ * Mapping lines record each verdict: one per (chat, sourceId). Idempotent:
+ * a proposition id already mapped for this chat reuses its resolution.
*
- * Events are rebuilt deterministically from the transcript's quiz and
- * plan_mark_done parts (kinds, timestamps, notes, refs). TAGGED events
- * (quiz nodeId, mark_done tree node) resolve through the proposition
- * mapping — deterministic; the findings' attribution is consulted ONLY for
- * untagged probe quizzes: the retroactive probe-attribution resolution (a
- * pre-tree probe maps to a proposition the tree only articulates later).
- * Unattributed quizzes land with nodeId null + the question text, exactly
- * like the unattributable ones always have.
+ * Events are rebuilt deterministically from the transcript: quiz outcomes
+ * (correct / miss / dont-know; kinds + timestamps + refs from the tool
+ * parts, nodeId from the extractor's attribution through this run's
+ * mapping) plus one observation event per proposition carrying an
+ * understanding note (ref `obs:`). No taught events, ever.
*/
-export function mergeFindings(chatId: string, findings: SessionFindings, deps: GraphDeps): EmissionStats {
+export function mergeFindings(chatId: string, audit: AuditResult, deps: GraphDeps): EmissionStats {
const { graphStore: ledgers, chatStore } = deps;
const record = chatStore.getChat(chatId);
if (!record) return { nodes: 0, edges: 0, events: 0 };
const stats: EmissionStats = { nodes: 0, edges: 0, events: 0 };
- // 1. Identity resolution. mapping: proposition id (and tree node id)
- // → graph node id — from this chat's existing mapping lines, then
- // extended as findings resolve. chatNodeIds: graph nodes this chat has
- // already mapped onto — the (chat, graph node) mapping dedupe.
+ // 1. Resolution: this run's propositions → graph node ids. Existing
+ // mapping lines for THIS chat are loaded first — the audit trail doubles
+ // as the resolution table, which makes re-merging the same chat
+ // idempotent (a re-extraction re-merges through the auditor's linkTo
+ // verdicts instead). Fresh sourceIds resolve as the verdicts say.
+ const existingIds = new Set(ledgers.readNodes().map((n) => n.id));
const mapping = new Map();
- const chatNodeIds = new Set();
+ const seenMappingKeys = new Set();
for (const m of ledgers.readMappings()) {
- if (m.chatId !== chatId || typeof m.sourceId !== "string") continue;
+ if (m.chatId !== chatId) continue;
mapping.set(m.sourceId, m.graphNodeId);
- chatNodeIds.add(m.graphNodeId);
- }
-
- // 2. Propositions → graph nodes. Exact statement/alias match against the
- // whole ledger first (cross-session reuse), then against propositions
- // created earlier in THIS merge (intra-findings dedupe).
- const norm = (s: string): string => s.trim().toLowerCase();
- const existingByTitle = new Map();
- for (const node of ledgers.readNodes()) {
- for (const title of [node.statement, ...node.aliases]) {
- const key = norm(title);
- if (key && !existingByTitle.has(key)) existingByTitle.set(key, node.id);
- }
+ seenMappingKeys.add(m.sourceId);
}
const now = new Date().toISOString();
- for (const p of findings.propositions) {
- if (mapping.has(p.id)) continue;
- const keys = [p.statement, ...p.aliases].map(norm).filter((k) => k.length > 0);
- const matchId = keys.map((k) => existingByTitle.get(k)).find((id) => id !== undefined);
+
+ for (const p of audit.propositions) {
+ if (mapping.has(p.id)) continue; // already resolved for this chat — idempotent
+ const points = pointsOf(chatId, p.messages);
let graphNodeId: string;
- if (matchId !== undefined) {
- graphNodeId = matchId; // reuse — conservative merge, duplicates over wrong merges
- } else {
- const graphNode: GraphNode = {
- id: newGraphNodeId(),
- name: p.name,
+ if (p.linkTo !== undefined && existingIds.has(p.linkTo)) {
+ // Identity verdict: this proposition IS that node. Append a link
+ // line — the session's phrasings (statement + aliases) join the
+ // node's aliases, the provenance unions. Reads merge by id.
+ graphNodeId = p.linkTo;
+ ledgers.appendNode({
+ id: graphNodeId,
statement: p.statement,
+ description: p.description,
aliases: [...p.aliases],
- kind: p.kind,
createdAt: now,
+ provenance: points,
+ });
+ } else {
+ const node: GraphNode = {
+ id: newGraphNodeId(),
+ statement: p.statement,
+ description: p.description,
+ aliases: [...p.aliases],
+ createdAt: now,
+ provenance: points,
};
- ledgers.appendNode(graphNode);
+ ledgers.appendNode(node);
stats.nodes++;
- graphNodeId = graphNode.id;
- }
- for (const key of keys) if (!existingByTitle.has(key)) existingByTitle.set(key, graphNodeId);
- if (!chatNodeIds.has(graphNodeId)) {
- ledgers.appendMapping({ chatId, sourceId: p.treeNodeId ?? p.id, graphNodeId });
- chatNodeIds.add(graphNodeId);
+ graphNodeId = node.id;
+ existingIds.add(graphNodeId);
}
mapping.set(p.id, graphNodeId);
- if (p.treeNodeId !== undefined) mapping.set(p.treeNodeId, graphNodeId);
+ if (!seenMappingKeys.has(p.id)) {
+ ledgers.appendMapping({ chatId, sourceId: p.id, graphNodeId });
+ seenMappingKeys.add(p.id);
+ }
}
- // 3. Edges. Dedupe by (from,to,type) across the whole ledger: an existing
- // assertion just gains this chatId as provenance (appended as a new
- // line; reads merge).
- const asserted = new Map>();
+ // 2. Edges. Resolve endpoints through the mapping; dedupe by
+ // (from,to,type) already asserted by THIS chat (an existing assertion
+ // by another chat stays untouched — provenance unions at read time).
+ const assertedByChat = new Set();
for (const line of ledgers.readEdges()) {
const key = `${line.from}\u0000${line.to}\u0000${line.type}`;
- let provenance = asserted.get(key);
- if (!provenance) {
- provenance = new Set();
- asserted.set(key, provenance);
- }
- for (const c of line.provenance) provenance.add(c);
+ if (line.provenance.some((pt) => pt.chatId === chatId)) assertedByChat.add(key);
}
- for (const edge of findings.edges) {
+ for (const edge of audit.edges) {
const from = mapping.get(edge.from);
const to = mapping.get(edge.to);
if (!from || !to || from === to) continue; // unmapped endpoint — skip
const key = `${from}\u0000${to}\u0000${edge.type}`;
- const provenance = asserted.get(key);
- if (provenance?.has(chatId)) continue; // already asserted by this chat
- ledgers.appendEdge({ from, to, type: edge.type, provenance: [chatId] });
- if (provenance) provenance.add(chatId);
- else asserted.set(key, new Set([chatId]));
+ if (assertedByChat.has(key)) continue;
+ ledgers.appendEdge({ from, to, type: edge.type, provenance: pointsOf(chatId, edge.messages) });
+ assertedByChat.add(key);
stats.edges++;
}
- // 4. Events, rebuilt deterministically in transcript order so the
- // probe/check boundary is exact: quizzes BEFORE the first plan_init
- // output are probes. Refs are chat-scoped (a quiz toolCallId or
- // taught:) — a ref already in the ledger means this turn
- // was extracted before. The findings' attribution supplies the
- // nodeId, nothing else.
+ // 3. Events, rebuilt deterministically in transcript order. Refs are
+ // chat-scoped idempotency keys (quiz toolCallId, obs:) —
+ // a ref already in the ledger means this evidence was extracted before.
const seenRefs = new Set();
for (const event of ledgers.readEvents()) {
if (event.chatId === chatId && event.ref) seenRefs.add(event.ref);
}
- let planSeen = false;
for (const part of toolParts(record.messages)) {
- // The boundary flips on the first plan_init OUTPUT, whatever it says
- // (an ok:false "tree already initialized" still means a tree exists).
- if (part.name === "plan_init" && part.state === "output-available") planSeen = true;
-
- if (part.name === "quiz" && part.state === "output-available") {
- const event = quizEvent(part, chatId, mapping, findings, planSeen, now);
- if (event?.ref !== undefined && !seenRefs.has(event.ref)) {
- ledgers.appendEvent(event);
- seenRefs.add(event.ref);
- stats.events++;
- }
- }
-
- if (part.name === "plan_mark_done" && part.state === "output-available") {
- const result = planResult(part);
- const input = part.input as { id?: unknown } | undefined;
- if (result?.ok && typeof input?.id === "string") {
- const ref = `taught:${input.id}`;
- if (!seenRefs.has(ref)) {
- // mark_done is tagged with the tree node id — deterministic.
- ledgers.appendEvent({
- nodeId: mapping.get(input.id) ?? null,
- kind: "taught",
- chatId,
- at: now,
- ref,
- });
- seenRefs.add(ref);
- stats.events++;
- }
- }
+ if (part.name !== "quiz" || part.state !== "output-available") continue;
+ const event = quizEvent(part, chatId, audit, mapping, now);
+ if (event?.ref !== undefined && !seenRefs.has(event.ref)) {
+ ledgers.appendEvent(event);
+ seenRefs.add(event.ref);
+ stats.events++;
}
}
+ // Understanding observations: one per proposition carrying a note,
+ // resolved to the node the verdict mapped it to.
+ for (const p of audit.propositions) {
+ if (p.understanding === undefined || p.understanding.length === 0) continue;
+ const ref = `obs:${p.id}`;
+ if (seenRefs.has(ref)) continue;
+ ledgers.appendEvent({
+ nodeId: mapping.get(p.id) ?? null,
+ kind: "observation",
+ chatId,
+ at: now,
+ ref,
+ note: p.understanding,
+ });
+ seenRefs.add(ref);
+ stats.events++;
+ }
- // 5. Checkpoint: this chat is extracted through its current updatedAt.
+ // 4. Checkpoint: this chat is extracted through its current updatedAt.
ledgers.appendCheckpoint({ chatId, foldedAt: record.updatedAt });
return stats;
}
@@ -261,9 +258,8 @@ export function mergeFindings(chatId: string, findings: SessionFindings, deps: G
function quizEvent(
part: ToolPartView,
chatId: string,
+ audit: AuditResult,
mapping: Map,
- findings: SessionFindings,
- planSeen: boolean,
now: string,
): LedgerEvent | null {
const out = part.output;
@@ -274,16 +270,13 @@ function quizEvent(
let kind: LedgerEventKind;
if (answer.dontKnow === true) kind = "dont-know";
- else if (answer.correct === true) kind = planSeen ? "check-correct" : "probe-correct";
- else kind = planSeen ? "check-incorrect" : "probe-miss";
+ else if (answer.correct === true) kind = "correct";
+ else kind = "miss";
- // Tagged (input.nodeId) resolves deterministically through the mapping;
- // attribution is the LLM judgment call for UNTAGGED probes only — the
- // vision's open probe-attribution question, resolved retroactively.
- const input = part.input as { nodeId?: unknown } | undefined;
- const tagged = typeof input?.nodeId === "string" ? mapping.get(input.nodeId) : undefined;
- const attributedId = findings.attribution.quiz[part.toolCallId];
- const nodeId = tagged ?? (attributedId ? (mapping.get(attributedId) ?? null) : null);
+ // Tree-blind: EVERY quiz attributes through the extractor's judgment —
+ // the vision's retroactive attribution, with no tree mapping to lean on.
+ const attributedId = audit.attribution.quiz[part.toolCallId];
+ const nodeId = attributedId ? (mapping.get(attributedId) ?? null) : null;
const event: LedgerEvent = {
nodeId,
@@ -293,7 +286,7 @@ function quizEvent(
ref: part.toolCallId,
};
if (typeof answer.note === "string" && answer.note.length > 0) event.note = answer.note;
- // The question text is the attribution input for unattributed probes.
+ // The question text is the attribution input for unattributed quizzes.
if (nodeId === null && typeof answer.questionText === "string") event.question = answer.questionText;
return event;
}
@@ -335,20 +328,39 @@ export interface DerivedGraph {
}
/**
- * The whole graph from the ledgers. Edge LINES are assertions; the returned
- * edges are merged by (from,to,type) with unioned provenance, first-seen
- * order. `store events, derive state` — this is the only reader that matters.
+ * The whole graph from the ledgers. Node LINES may repeat an id (link-
+ * appends): the FIRST line defines statement/description/createdAt; later
+ * lines' statements and aliases join `aliases`, provenance unions. Edge
+ * LINES are assertions, merged by (from,to,type) with unioned provenance,
+ * first-seen order. `store events, derive state` — this is the only reader
+ * that matters.
*/
export function getGraph(graphStore: GraphStore): DerivedGraph {
- const nodes = graphStore.readNodes();
+ // Merge node lines by id (first line defines; later lines add phrasings).
+ const byId = new Map();
+ for (const line of graphStore.readNodes()) {
+ const existing = byId.get(line.id);
+ if (!existing) {
+ byId.set(line.id, { ...line, aliases: [...line.aliases], provenance: [...line.provenance] });
+ continue;
+ }
+ // A later line's statement is a phrasing, not a replacement.
+ for (const phrasing of [line.statement, ...line.aliases]) {
+ if (phrasing.length > 0 && !existing.aliases.includes(phrasing)) existing.aliases.push(phrasing);
+ }
+ for (const pt of line.provenance) {
+ if (!existing.provenance.some((q) => samePoint(q, pt))) existing.provenance.push(pt);
+ }
+ }
+
const merged: GraphEdge[] = [];
const byKey = new Map();
for (const line of graphStore.readEdges()) {
const key = `${line.from}\u0000${line.to}\u0000${line.type}`;
const existing = byKey.get(key);
if (existing) {
- for (const c of line.provenance) {
- if (!existing.provenance.includes(c)) existing.provenance.push(c);
+ for (const pt of line.provenance) {
+ if (!existing.provenance.some((q) => samePoint(q, pt))) existing.provenance.push(pt);
}
} else {
const edge: GraphEdge = { ...line, provenance: [...line.provenance] };
@@ -356,34 +368,27 @@ export function getGraph(graphStore: GraphStore): DerivedGraph {
merged.push(edge);
}
}
- return { nodes, edges: merged, events: graphStore.readEvents() };
+ return { nodes: [...byId.values()], edges: merged, events: graphStore.readEvents() };
}
-const QUIZ_EVENT_KINDS: ReadonlySet = new Set([
- "probe-correct",
- "check-correct",
- "probe-miss",
- "check-incorrect",
- "dont-know",
-]);
+const QUIZ_EVENT_KINDS: ReadonlySet = new Set(["correct", "miss", "dont-know"]);
/**
- * Derive node states from events (never stored): taught if ANY taught event;
- * else the LATEST quiz event decides (known / missed / dont-know); a node
- * with no events is a candidate.
+ * Derive node states from events (never stored): the LATEST quiz event
+ * decides (known / missed / dont-know); a node with no quiz events is a
+ * candidate. TRANSITIONAL (2026-09-02 redesign): `taught` is unreachable
+ * now that no taught events are emitted, and observation events are NOT
+ * state inputs — the real state model (dependency depth + topology-derived
+ * strength + quiz events) is pending ideation; this keeps the renderer
+ * alive until it lands.
*/
export function deriveNodeStates(nodes: GraphNode[], events: LedgerEvent[]): Record {
const states: Record = {};
for (const node of nodes) states[node.id] = "candidate";
- const taught = new Set();
const latest = new Map();
events.forEach((event, index) => {
if (event.nodeId === null) return;
- if (event.kind === "taught") {
- taught.add(event.nodeId);
- return;
- }
if (!QUIZ_EVENT_KINDS.has(event.kind)) return;
const current = latest.get(event.nodeId);
if (!current || event.at > current.at || (event.at === current.at && index > current.index)) {
@@ -392,13 +397,9 @@ export function deriveNodeStates(nodes: GraphNode[], events: LedgerEvent[]): Rec
});
for (const node of nodes) {
- if (taught.has(node.id)) {
- states[node.id] = "taught";
- continue;
- }
const last = latest.get(node.id);
if (!last) continue; // candidate
- if (last.kind === "probe-correct" || last.kind === "check-correct") states[node.id] = "known";
+ if (last.kind === "correct") states[node.id] = "known";
else if (last.kind === "dont-know") states[node.id] = "dont-know";
else states[node.id] = "missed";
}
@@ -422,10 +423,13 @@ export function graphSnapshot(deps: GraphDeps): GraphSnapshot {
chats.add(chatId);
};
for (const m of deps.graphStore.readMappings()) touch(m.graphNodeId, m.chatId);
+ for (const node of nodes) {
+ for (const pt of node.provenance) touch(node.id, pt.chatId);
+ }
for (const edge of edges) {
- for (const chatId of edge.provenance) {
- touch(edge.from, chatId);
- touch(edge.to, chatId);
+ for (const pt of edge.provenance) {
+ touch(edge.from, pt.chatId);
+ touch(edge.to, pt.chatId);
}
}
for (const event of events) {
diff --git a/src/core/ports.ts b/src/core/ports.ts
index bed10f2..5cc264a 100644
--- a/src/core/ports.ts
+++ b/src/core/ports.ts
@@ -56,7 +56,7 @@ export interface GraphStore {
appendEdge(assertion: GraphEdge): void;
readEdges(): GraphEdge[];
- /** Append one quiz outcome / taught event. */
+ /** Append one quiz outcome / understanding observation. */
appendEvent(event: LedgerEvent): void;
readEvents(): LedgerEvent[];
diff --git a/src/store/graph.ts b/src/store/graph.ts
index 064e9c0..8bc9353 100644
--- a/src/store/graph.ts
+++ b/src/store/graph.ts
@@ -1,20 +1,26 @@
/**
* GraphStore — the fs JSONL adapter: five append-only ledgers at
* config.LEDGER_FILES (contracts.ts "v1 emission").
- * nodes.jsonl GraphNode — one line per graph node
+ * nodes.jsonl GraphNode — one line per node write: the FIRST
+ * line for an id defines it; a later
+ * line with a REPEATED id is a
+ * link-append (session phrasings join
+ * aliases, provenance unions); reads
+ * merge by id (core/graph.ts getGraph)
* edges.jsonl GraphEdge — one line per edge ASSERTION (a second
* session asserting the same (from,to,type)
* appends another line; reads merge)
- * events.jsonl LedgerEvent — quiz outcomes + taught events
+ * events.jsonl LedgerEvent — quiz outcomes + understanding observations
* mappings.jsonl NodeMapping — the session→graph identity map:
* (chatId, sourceId) → graphNodeId
* sessions.jsonl SessionCheckpoint— emission checkpoints (foldedAt)
*
* Implements core/ports.ts GraphStore. data/graph/* is exclusively this
- * adapter's. Reads are strict against the contracts.ts shapes: corrupt lines
- * are skipped, never guessed at — the ledgers are the source of truth and
- * everything else is derived (core/graph.ts, which takes this port as a
- * parameter). Same idioms as src/store/chats.ts: sync fs, no framework.
+ * adapter's. Reads are strict against the contracts.ts shapes (the 2026-09-02
+ * graph redesign): corrupt lines are skipped, never guessed at — the ledgers
+ * are the source of truth and everything else is derived (core/graph.ts,
+ * which takes this port as a parameter). Same idioms as src/store/chats.ts:
+ * sync fs, no framework.
*/
import * as fs from "node:fs";
@@ -26,6 +32,7 @@ import type {
LedgerEvent,
LedgerEventKind,
NodeMapping,
+ ProvenancePoint,
SessionCheckpoint,
} from "../core/contracts.ts";
import type { GraphStore } from "../core/ports.ts";
@@ -69,26 +76,22 @@ function readValid(file: string, isValid: (v: unknown) => v is T): T[] {
return readRawLines(file).filter(isValid);
}
-/** Pre-name ledger line (the first extraction runs) — read tolerantly. */
-type StoredGraphNode = Omit & { name?: string };
+/** Provenance points are (chatId, messageId) pairs — message-granular. */
+const isProvenancePoint = (v: unknown): v is ProvenancePoint =>
+ isRecord(v) && isString(v.chatId) && isString(v.messageId);
-const isStoredGraphNode = (v: unknown): v is StoredGraphNode =>
+const isGraphNode = (v: unknown): v is GraphNode =>
isRecord(v) &&
isString(v.id) &&
isString(v.statement) &&
+ isString(v.description) &&
Array.isArray(v.aliases) &&
- (v.kind === "truth" || v.kind === "misconception") &&
+ v.aliases.every(isString) &&
isString(v.createdAt) &&
- (v.name === undefined || isString(v.name));
+ Array.isArray(v.provenance) &&
+ v.provenance.every(isProvenancePoint);
-/** Derived header for legacy node lines written before `name` existed:
- * the first clause of the statement, capped — good enough to render. */
-function nameFromStatement(statement: string): string {
- const clause = statement.split(/[.;:—]/)[0]!.trim();
- return clause.length > 60 ? `${clause.slice(0, 57)}…` : clause || statement;
-}
-
-const EDGE_TYPES = ["requires", "supports"] as const;
+const EDGE_TYPES = ["follows", "supports"] as const;
const isGraphEdge = (v: unknown): v is GraphEdge =>
isRecord(v) &&
@@ -96,17 +99,9 @@ const isGraphEdge = (v: unknown): v is GraphEdge =>
isString(v.to) &&
(EDGE_TYPES as readonly string[]).includes(v.type as string) &&
Array.isArray(v.provenance) &&
- v.provenance.every(isString);
+ v.provenance.every(isProvenancePoint);
-const EVENT_KINDS: readonly LedgerEventKind[] = [
- "probe-correct",
- "probe-miss",
- "dont-know",
- "misconception-signaled",
- "taught",
- "check-correct",
- "check-incorrect",
-];
+const EVENT_KINDS: readonly LedgerEventKind[] = ["correct", "miss", "dont-know", "observation"];
const isLedgerEvent = (v: unknown): v is LedgerEvent =>
isRecord(v) &&
@@ -127,13 +122,11 @@ export const graphStore: GraphStore = {
appendNode(node: GraphNode): void {
appendLine(LEDGER_FILES.nodes, node);
},
- /** Raw node lines in append order (ids are unique — no merging needed).
- * Legacy lines without a `name` get one derived from the statement. */
+ /** Raw node lines in append order. A node id may appear on more than
+ * one line (a later session's LINK-append: phrasings + provenance); the
+ * derivation (core/graph.ts getGraph) merges lines by id. */
readNodes(): GraphNode[] {
- return readValid(LEDGER_FILES.nodes, isStoredGraphNode).map((n) => ({
- ...n,
- name: n.name ?? nameFromStatement(n.statement),
- }));
+ return readValid(LEDGER_FILES.nodes, isGraphNode);
},
/** Append one edge ASSERTION; readers merge lines by (from,to,type). */
diff --git a/tests/extraction.test.ts b/tests/extraction.test.ts
index 8fa812e..d275ffc 100644
--- a/tests/extraction.test.ts
+++ b/tests/extraction.test.ts
@@ -1,13 +1,16 @@
/**
- * The extraction pipeline tests (vision §1 construction): extractor →
- * auditor → merger, the sole graph writer.
+ * The extraction pipeline tests (vision §1 construction, 2026-09-02 redesign):
+ * extractor → auditor → merger, the sole graph writer — tree-blind,
+ * free-form, with the auditor owning faithfulness and identity.
*
- * transcriptView — the compact view both LLM stages see (pure)
+ * transcriptView — the compact view both LLM stages see (pure):
+ * message-id-tagged verbatim text + quiz lines, NO tree
* pipeline — MockLanguageModelV4 drives both agents: findings out,
- * audit verdict in, deterministic merge applied; failure
- * paths checkpoint nothing; single-flight per chat
+ * audit verdict (with linkTo identity) in, deterministic
+ * merge applied; failure paths checkpoint nothing;
+ * single-flight per chat
*
- * The merger itself (identity resolution, event rebuild, idempotency) is
+ * The merger itself (identity verdicts, event rebuild, idempotency) is
* tested in tests/graph.test.ts directly against mergeFindings — here it is
* the integration: what one full pipeline run does to the ledgers.
*
@@ -21,9 +24,16 @@ import * as fs from "node:fs";
import { after, describe, it } from "node:test";
import { MockLanguageModelV4 } from "ai/test";
import type { UIMessage } from "ai";
-import type { GraphEdge, GraphNode, LedgerEvent, NodeMapping, SessionCheckpoint, SessionFindings } from "../src/core/contracts.ts";
+import type {
+ AuditResult,
+ GraphEdge,
+ GraphNode,
+ LedgerEvent,
+ NodeMapping,
+ SessionCheckpoint,
+ SessionFindings,
+} from "../src/core/contracts.ts";
import type { GraphStore } from "../src/core/ports.ts";
-import { createTree, markDone } from "../src/core/plan-tree.ts";
import { createExtractionPipeline, transcriptView } from "../src/agent/subagents/extraction.ts";
import { chatStore } from "../src/store/chats.ts";
import { PATHS } from "../config.ts";
@@ -72,7 +82,7 @@ function quizPart(toolCallId: string, input: Record, output: Re
return { type: "tool-quiz", toolCallId, state: "output-available", input, output };
}
-function planPart(toolCallId: string, name: string, input: Record, tree: ReturnType): unknown {
+function planPart(toolCallId: string, name: string, input: Record, tree: unknown): unknown {
return {
type: `tool-${name}`,
toolCallId,
@@ -82,21 +92,11 @@ function planPart(toolCallId: string, name: string, input: Record {
}
});
-/** What the extractor would produce for the fixture session. */
+/** What the extractor would produce for the fixture session — tree-blind,
+ * free-form, with message-granular provenance + an understanding note. */
const FINDINGS: SessionFindings = {
propositions: [
{
id: "p1",
- name: "Chain rule",
- statement: "Chain rule intuition",
+ statement: "The derivative measures how fast a function changes",
+ description: "The derivative of a function at a point is its instantaneous rate of change there.",
aliases: [],
- kind: "truth",
- treeNodeId: "cr",
- evidence: ["TEACHER: The derivative measures how fast a function changes."],
+ messages: ["a2"],
+ evidence: ["TEACHER [a2]: The derivative measures how fast a function changes."],
+ understanding: "solid",
},
{
id: "p2",
- name: "Applying chain rule",
- statement: "Applying the chain rule",
+ statement: "Chain rule intuition",
+ description: "The chain rule differentiates composed functions by multiplying local rates.",
aliases: [],
- kind: "truth",
- treeNodeId: "apply",
- evidence: ["FINAL PLAN TREE (node ids): cr = Chain rule intuition; apply = Applying the chain rule"],
+ messages: ["a2"],
+ evidence: ["LEARNER [u1]: teach me the chain rule"],
},
],
- edges: [{ from: "p1", to: "p2", type: "requires", evidence: ["TREE EDGES: cr requires apply"] }],
- attribution: { quiz: { "probe-1": "p1" } },
+ edges: [
+ {
+ from: "p1",
+ to: "p2",
+ type: "follows",
+ messages: ["a2"],
+ evidence: ["LEARNER [u1]: teach me the chain rule"],
+ },
+ ],
+ attribution: { quiz: { "probe-1": "p1", "check-1": "p2" } },
};
/** A model whose ONE tool call is `name` with the given input; then text. */
@@ -202,28 +209,26 @@ function silentModel(): MockLanguageModelV4 {
// ─── transcriptView (pure) ───────────────────────────────────────────────────
describe("transcriptView (pure)", () => {
- it("shows the final tree, verbatim conversation, structured events — plan ops folded", () => {
+ it("shows message-id-tagged verbatim conversation + quiz lines — tree-blind, plan ops folded", () => {
const { record } = fixtureChat();
const view = transcriptView(record);
- // The final tree with node ids (the extractor's backbone + the
- // pruned-out stays pruned — only the FINAL tree is shown).
assert.match(view, /SESSION: Extraction fixture/);
- assert.match(view, /FINAL PLAN TREE \(node ids\): cr = Chain rule intuition; apply = Applying the chain rule/);
- assert.match(view, /TREE EDGES: cr requires apply/);
- // Conversation verbatim — evidence quotes can come from it.
- assert.match(view, /LEARNER: teach me the chain rule/);
- assert.match(view, /TEACHER: The derivative measures how fast a function changes\./);
+ // Conversation verbatim, each line tagged with its message id —
+ // the provenance anchors the extractor reports back.
+ assert.match(view, /LEARNER \[u1\] teach me the chain rule/);
+ assert.match(view, /TEACHER \[a2\] The derivative measures how fast a function changes\./);
// Structured events, attribution-ready.
assert.match(view, /QUIZ \[probe-1\] What is a derivative, roughly\? — the learner answered incorrectly/);
assert.match(view, /QUIZ \[check-1\] Check — the learner answered correctly/);
- assert.match(view, /TEACHER marked tree node done \[p2\]: cr/);
- // plan_init / plan_mark_done tool parts themselves are folded into
- // the tree line — no raw JSON noise.
- assert.doesNotMatch(view, /plan_init/);
+ // TREE-BLIND: no plan tree, no tree edges, no mark_done lines — plan
+ // ops fold to the generic one-liner like every other non-quiz tool.
+ assert.doesNotMatch(view, /FINAL PLAN TREE/);
+ assert.doesNotMatch(view, /TREE EDGES/);
+ assert.match(view, /\(plan_init tool result delivered\)/);
assert.doesNotMatch(view, /output-available/);
});
@@ -253,48 +258,61 @@ describe("transcriptView (pure)", () => {
// ─── the pipeline ───────────────────────────────────────────────────────────
describe("the extraction pipeline (extractor → auditor → merger)", () => {
- it("runs end to end: findings → audit verdict → deterministic merge + checkpoint", async () => {
+ it("runs end to end: findings → audit verdict (identity) → deterministic merge + checkpoint", async () => {
const { chatId } = fixtureChat();
const ledgers = memoryLedgers();
- // The auditor passes p1 through, drops p2 (unbacked), re-points the
- // probe attribution to p1, and records the drop.
- const verified: SessionFindings = {
- propositions: [FINDINGS.propositions[0]!],
+ // The auditor passes p1 through and LINKS it to a pre-existing node
+ // (identity verdict), drops p2 (unbacked) + the edge with it.
+ const existingId = "n_existing";
+ ledgers.appendNode({
+ id: existingId,
+ statement: "Derivatives measure rates of change",
+ description: "Pre-existing node from an earlier session.",
+ aliases: [],
+ createdAt: "2026-08-01T00:00:00.000Z",
+ provenance: [],
+ });
+ const verdict: AuditResult = {
+ propositions: [
+ { ...FINDINGS.propositions[0]!, linkTo: existingId },
+ ],
edges: [],
attribution: { quiz: { "probe-1": "p1" } },
+ dropped: [{ ref: "p2", reason: "evidence not found" }, { ref: "p1→p2", reason: "endpoint dropped" }],
};
const pipeline = createExtractionPipeline({
chatStore,
graphStore: ledgers,
extractor: submitModel("submit_findings", FINDINGS),
- auditor: submitModel("submit_review", { ...verified, dropped: [{ ref: "p2", reason: "evidence not found" }] }),
+ auditor: submitModel("submit_review", verdict),
});
const stats = await pipeline.runExtraction(chatId);
- assert.deepEqual(stats, { nodes: 1, edges: 0, events: 3 });
+ // The linked proposition created NO new node; the understanding note
+ // became an observation; both quizzes became events.
+ assert.deepEqual(stats, { nodes: 0, edges: 0, events: 3 });
- // Only the audited proposition became a node; mapping provenance kept.
- assert.equal(ledgers.readNodes().length, 1);
- assert.equal(ledgers.readNodes()[0]!.statement, "Chain rule intuition");
- assert.deepEqual(ledgers.readMappings(), [{ chatId, sourceId: "cr", graphNodeId: ledgers.readNodes()[0]!.id }]);
+ // Link-append: the phrasing joined the existing node's aliases and
+ // the provenance unioned; no second node line defines a new node.
+ assert.equal(ledgers.readNodes().length, 2); // the pre-seeded line + the link line
+ assert.deepEqual(ledgers.readMappings(), [{ chatId, sourceId: "p1", graphNodeId: existingId }]);
- // Events rebuilt deterministically from the transcript — the probe
- // retroactively attributed, the check tagged, mark_done mapped.
+ // Events rebuilt deterministically: quizzes in transcript order, then
+ // the observation. check-1's attribution pointed at the DROPPED p2 →
+ // unresolvable → null (the question text is kept for later attribution).
const events = ledgers.readEvents();
assert.deepEqual(
- events.map((e) => [e.kind, e.ref]),
+ events.map((e) => [e.kind, e.ref, e.nodeId]),
[
- ["probe-miss", "probe-1"],
- ["check-correct", "check-1"],
- ["taught", "taught:cr"],
+ ["miss", "probe-1", existingId],
+ ["correct", "check-1", null],
+ ["observation", "obs:p1", existingId],
],
);
- const nodeId = ledgers.readNodes()[0]!.id;
- assert.equal(events[0]!.nodeId, nodeId, "probe attributed via audit");
- assert.equal(events[1]!.nodeId, null, "tagged to a dropped proposition → null");
- assert.equal(events[2]!.nodeId, nodeId, "taught via tree-node mapping");
+ assert.equal(events[0]!.nodeId, existingId, "probe attributed via the verdict's mapping");
+ assert.equal(events[2]!.note, "solid", "the understanding note joins the ledger");
// Checkpointed through the record's updatedAt — never the same turns twice.
assert.equal(ledgers.readCheckpoints().length, 1);
@@ -318,7 +336,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
chatStore,
graphStore: ledgers,
extractor: silentModel(),
- auditor: submitModel("submit_review", FINDINGS),
+ auditor: submitModel("submit_review", { ...FINDINGS, dropped: [] }),
});
await assert.rejects(failingExtractor.runExtraction(chatId), /the extractor never submitted findings/);
assert.equal(ledgers.readCheckpoints().length, 0);
diff --git a/tests/graph-context.test.ts b/tests/graph-context.test.ts
index 7156dc5..2611de0 100644
--- a/tests/graph-context.test.ts
+++ b/tests/graph-context.test.ts
@@ -1,13 +1,14 @@
/**
- * Graph-as-context tests (W8; vision §1 "the graph as session context").
+ * Graph-as-context tests (vision §1 "the graph as session context", 2026-09-02
+ * redesign vocabulary: follows edges, no node kinds).
*
* renderGraphContext — pure: empty / small / cap-truncation rendering
* buildGraphContext — per-request entry point (derived-graph → block)
* injection plumbing — chat-flow style: the teacher's instructions carry
* the block (MockLanguageModelV4 captures the prompt)
- * exact-match merge — the merger (mergeFindings) dedupes
- * case/whitespace-identical statements onto existing
- * nodes (no fuzzy matching)
+ * cross-chat edges — the merger (mergeFindings) appends one ASSERTION
+ * line per session; reads merge with unioned
+ * message-granular provenance
*
* Ledger isolation: NONE needed for the merge tests — the merger's ledger is
* injectable (in-memory), so the live data/graph files are only ever READ
@@ -22,7 +23,14 @@ import * as path from "node:path";
import { after, describe, it } from "node:test";
import { MockLanguageModelV4, simulateReadableStream } from "ai/test";
import type { LanguageModelV4StreamPart } from "@ai-sdk/provider";
-import type { GraphEdge, GraphNode, LedgerEvent, SessionFindings } from "../src/core/contracts.ts";
+import type {
+ AuditResult,
+ GraphEdge,
+ GraphNode,
+ LedgerEvent,
+ NodeMapping,
+ SessionCheckpoint,
+} from "../src/core/contracts.ts";
import type { NodeStateClass } from "../src/core/contracts.ts";
import { getGraph, mergeFindings } from "../src/core/graph.ts";
import type { GraphStore } from "../src/core/ports.ts";
@@ -35,19 +43,25 @@ import {
import { createTeacherAgent } from "../src/agent/teacher-agent.ts";
import { handleChatRequest } from "../src/server/chat-handler.ts";
import { chatStore } from "../src/store/chats.ts";
-import type { NodeMapping, SessionCheckpoint } from "../src/core/contracts.ts";
import { graphStore } from "../src/store/graph.ts";
-import { createTree } from "../src/core/plan-tree.ts";
import { PATHS } from "../config.ts";
// ─── fixtures ────────────────────────────────────────────────────────────────
function node(id: string, statement: string, over: Partial = {}): GraphNode {
- return { id, name: statement.split(";")[0]!, statement, aliases: [], kind: "truth", createdAt: "2026-08-22T00:00:00.000Z", ...over };
+ return {
+ id,
+ statement,
+ description: `${statement} — elaborated.`,
+ aliases: [],
+ createdAt: "2026-08-22T00:00:00.000Z",
+ provenance: [],
+ ...over,
+ };
}
function edge(from: string, to: string): GraphEdge {
- return { from, to, type: "requires", provenance: ["chat_x"] };
+ return { from, to, type: "follows", provenance: [{ chatId: "chat_x", messageId: "m1" }] };
}
function states(...entries: [string, NodeStateClass][]): Record {
@@ -61,7 +75,7 @@ describe("renderGraphContext (pure)", () => {
assert.equal(renderGraphContext([], [], {}), null);
});
- it("renders a header, state-grouped node lines (hotspots first), and chained requires edges", () => {
+ it("renders a header, state-grouped node lines (hotspots first), and chained follows edges", () => {
const nodes = [
node("n_a", "Chain rule intuition"),
node("n_b", "Quotient rule"),
@@ -73,26 +87,26 @@ describe("renderGraphContext (pure)", () => {
["n_a", "known"],
["n_b", "missed"],
["n_c", "candidate"],
- ["n_d", "taught"],
+ ["n_d", "dont-know"],
);
const out = renderGraphContext(nodes, edges, st)!;
assert.ok(out.includes(GRAPH_CONTEXT_HEADER), "header line present");
- assert.match(out, /4 nodes, 3 requires edges/);
+ assert.match(out, /4 nodes, 3 follows edges/);
// Node lines: "• [state] statement (id)".
assert.ok(out.includes("• [missed] Quotient rule (n_b)"));
assert.ok(out.includes("• [known] Chain rule intuition (n_a)"));
- assert.ok(out.includes("• [taught] Composite functions (n_d)"));
+ assert.ok(out.includes("• [dont-know] Composite functions (n_d)"));
assert.ok(out.includes("• [candidate] Product rule (n_c)"));
- // Groups in the decided order: hotspots, known, taught, candidates.
- const order = ["Hotspots", "Known", "Taught", "Candidates"].map((g) => out.indexOf(g));
+ // Groups in the decided order: hotspots, known, candidates.
+ const order = ["Hotspots", "Known", "Candidates"].map((g) => out.indexOf(g));
assert.deepEqual([...order].sort((a, b) => a - b), order, "groups appear in priority order");
// Edges merged into chains: d → a → b → c (one line, three arrows).
assert.ok(out.includes("Composite functions → Chain rule intuition → Quotient rule → Product rule"));
- assert.ok(out.includes("Requires (prerequisite → dependent)"));
+ assert.ok(out.includes("Follows (premise → derivation)"));
assert.ok(out.includes("===== end of knowledge graph ====="), "clear end delimiter");
});
@@ -104,15 +118,8 @@ describe("renderGraphContext (pure)", () => {
assert.ok(!out.includes("B → C"), "no edge was invented between siblings");
});
- it("marks misconception-kind nodes as hotspots even when their state is candidate", () => {
- const nodes = [node("n_m", "Division by zero is fine", { kind: "misconception" })];
- const out = renderGraphContext(nodes, [], states(["n_m", "candidate"]))!;
- assert.match(out, /• \[candidate\] Division by zero is fine \(n_m\) · misconception/);
- assert.ok(out.includes("Hotspots"), "misconception nodes join the hotspot group");
- });
-
- it("caps at MAX_CONTEXT_NODES: hotspots + their 1-hop requires neighborhood only, with a truncation note", () => {
- // 3 hotspots, each with a prerequisite and a dependent (1-hop), plus a
+ it("caps at MAX_CONTEXT_NODES: hotspots + their 1-hop follows neighborhood only, with a truncation note", () => {
+ // 3 hotspots, each with a premise and a derivation (1-hop), plus a
// long tail of disconnected candidates that must drop out.
const nodes: GraphNode[] = [];
const edges: GraphEdge[] = [];
@@ -121,7 +128,7 @@ describe("renderGraphContext (pure)", () => {
const h = `n_h${i}`;
const pre = `n_p${i}`;
const dep = `n_d${i}`;
- nodes.push(node(pre, `Prerequisite ${i}`), node(h, `Hotspot ${i}`), node(dep, `Dependent ${i}`));
+ nodes.push(node(pre, `Premise ${i}`), node(h, `Hotspot ${i}`), node(dep, `Derivation ${i}`));
edges.push(edge(pre, h), edge(h, dep));
st[pre] = "known";
st[h] = "missed";
@@ -139,11 +146,11 @@ describe("renderGraphContext (pure)", () => {
const out = renderGraphContext(nodes, edges, st)!;
assert.ok(out.includes("• [missed] Hotspot 1 (n_h1)"), "hotspots stay");
- assert.ok(out.includes("• [known] Prerequisite 1 (n_p1)"), "1-hop prerequisites stay");
- assert.ok(out.includes("• [candidate] Dependent 1 (n_d1)"), "1-hop dependents stay");
+ assert.ok(out.includes("• [known] Premise 1 (n_p1)"), "1-hop premises stay");
+ assert.ok(out.includes("• [candidate] Derivation 1 (n_d1)"), "1-hop derivations stay");
assert.ok(!out.includes("Far away"), "disconnected nodes drop out");
assert.match(out, new RegExp(`showing 9 of ${nodes.length} nodes`), "truncation is noted");
- assert.ok(out.includes(", 6 requires edges"), "edges among the selected nodes stay");
+ assert.ok(out.includes(", 6 follows edges"), "edges among the selected nodes stay");
});
it("keeps the full graph at exactly the cap (no truncation note)", () => {
@@ -169,11 +176,11 @@ describe("buildGraphContext", () => {
it("derives states from events and renders the block", () => {
const nodes = [node("n_a", "Solo node"), node("n_b", "Checked node")];
const events: LedgerEvent[] = [
- { nodeId: "n_b", kind: "check-correct", chatId: "c1", at: "2026-08-22T00:00:00.000Z", ref: "r1" },
+ { nodeId: "n_b", kind: "correct", chatId: "c1", at: "2026-08-22T00:00:00.000Z", ref: "r1" },
];
const out = buildGraphContext({ nodes, edges: [], events })!;
assert.ok(out.includes(GRAPH_CONTEXT_HEADER));
- assert.ok(out.includes("• [known] Checked node (n_b)"), "check-correct → known");
+ assert.ok(out.includes("• [known] Checked node (n_b)"), "correct → known");
assert.ok(out.includes("• [candidate] Solo node (n_a)"), "no events → candidate");
});
@@ -244,7 +251,7 @@ describe("graph-context injection (chat-flow style)", () => {
nodes: [node("n_ctx", "Chain rule intuition"), node("n_ctx2", "Derivative as a limit")],
edges: [edge("n_ctx2", "n_ctx")],
events: [
- { nodeId: "n_ctx2", kind: "check-correct", chatId: "c1", at: "2026-08-22T00:00:00.000Z", ref: "r1" },
+ { nodeId: "n_ctx2", kind: "correct", chatId: "c1", at: "2026-08-22T00:00:00.000Z", ref: "r1" },
] as LedgerEvent[],
};
const context = buildGraphContext(graph);
@@ -296,7 +303,7 @@ describe("graph-context injection (chat-flow style)", () => {
});
});
-// ─── exact-match merge (merger identity resolution) ──────────────────────────
+// ─── cross-chat edge assertions (merger + read-time merging) ────────────────
/** An in-memory stand-in for the five ledgers (the GraphStore port, core/ports.ts). */
function memoryLedgers() {
@@ -317,128 +324,100 @@ function memoryLedgers() {
appendCheckpoint: (c: SessionCheckpoint) => void checkpoints.push(c),
readCheckpoints: () => checkpoints.map((c) => ({ ...c })),
// Test accessors (not part of the GraphStore surface).
- counts: { get nodes() { return nodes.length; }, get mappings() { return mappings.length; } },
- } satisfies GraphStore & { counts: { nodes: number; mappings: number } };
+ counts: { get nodes() { return nodes.length; } },
+ } satisfies GraphStore & { counts: { nodes: number } };
}
-/** Findings as the extractor would file them for a plan tree: one
- * proposition per node (statement = title, treeNodeId set), one `requires`
- * edge per tree edge — the merge tests drive the merger with these. */
-function treeFindings(tree: ReturnType): SessionFindings {
- const idByNodeId = new Map(tree.nodes.map((n, i) => [n.id, `p${i + 1}`]));
- return {
- propositions: tree.nodes.map((n, i) => ({
- id: `p${i + 1}`,
- name: n.title,
- statement: n.title,
- aliases: [],
- kind: "truth" as const,
- treeNodeId: n.id,
- evidence: ["fixture evidence"],
- })),
- edges: tree.edges.map((e) => ({
- from: idByNodeId.get(e.from)!,
- to: idByNodeId.get(e.to)!,
- type: "requires" as const,
- evidence: ["fixture evidence"],
- })),
- attribution: { quiz: {} },
- };
-}
-
-/** A real chat record with a stored tree (the merger reads the real chat store). */
-function fixtureChat(title: string, tree: ReturnType) {
+/** A minimal chat record with one user message (the merger reads the real chat store). */
+function fixtureChat(title: string) {
const chat = chatStore.createChat(title);
createdChats.push(chat.id);
- chatStore.saveTree(chat.id, tree);
chatStore.saveMessages(chat.id, [{ id: "u1", role: "user", parts: [{ type: "text", text: "go" }] }]);
return chatStore.getChat(chat.id)!;
}
-describe("exact-match merge (merger identity resolution)", () => {
- it("maps a case/whitespace-identical title onto the existing node instead of duplicating", () => {
+describe("cross-chat edge assertions (merger + read-time merging)", () => {
+ it("a second session asserting the same edge appends a second line; reads merge with unioned provenance", () => {
const ledgers = memoryLedgers();
- const tree1 = createTree([{ id: "cr", title: "Chain rule intuition" }], []);
- const chat1 = fixtureChat("Merge A", tree1);
- let stats = mergeFindings(chat1.id, treeFindings(tree1), { chatStore, graphStore: ledgers });
- assert.deepEqual(stats, { nodes: 1, edges: 0, events: 0 });
- const firstId = ledgers.readNodes()[0]!.id;
- // Second session, same proposition: different casing + padding.
- const tree2 = createTree([{ id: "cr2", title: " chain Rule INTUITION " }], []);
- const chat2 = fixtureChat("Merge B", tree2);
- stats = mergeFindings(chat2.id, treeFindings(tree2), { chatStore, graphStore: ledgers });
- assert.deepEqual(stats, { nodes: 0, edges: 0, events: 0 }, "no duplicate node appended");
-
- assert.equal(ledgers.counts.nodes, 1, "still exactly one graph node");
- assert.equal(ledgers.readNodes()[0]!.id, firstId);
- // Provenance stays honest: one mapping line per (chat, graph node).
- assert.equal(ledgers.counts.mappings, 2);
- assert.deepEqual(ledgers.readMappings()[1], { chatId: chat2.id, sourceId: "cr2", graphNodeId: firstId });
- });
-
- it("matches aliases too (exact, case-insensitive)", () => {
- const ledgers = memoryLedgers();
- ledgers.appendNode({
- id: "n_alias",
- name: "Differentiation",
- statement: "Differentiation basics",
- aliases: ["the derivative"],
- kind: "truth",
- createdAt: "2026-08-22T00:00:00.000Z",
- });
- const tree = createTree([{ id: "d", title: "The Derivative" }], []);
- const chat = fixtureChat("Merge alias", tree);
- const stats = mergeFindings(chat.id, treeFindings(tree), { chatStore, graphStore: ledgers });
- assert.deepEqual(stats, { nodes: 0, edges: 0, events: 0 }, "alias match → no new node");
- assert.deepEqual(ledgers.readMappings(), [{ chatId: chat.id, sourceId: "d", graphNodeId: "n_alias" }]);
- });
-
- it("does NOT fuzzy-match near-identical titles (that stays deferred)", () => {
- const ledgers = memoryLedgers();
- const tree1 = createTree([{ id: "cr", title: "Chain rule intuition" }], []);
- const chat1 = fixtureChat("Fuzzy A", tree1);
- mergeFindings(chat1.id, treeFindings(tree1), { chatStore, graphStore: ledgers });
- const tree2 = createTree([{ id: "cr2", title: "Chain rules intuition" }], []);
- const chat2 = fixtureChat("Fuzzy B", tree2);
- const stats = mergeFindings(chat2.id, treeFindings(tree2), { chatStore, graphStore: ledgers });
- assert.equal(stats.nodes, 1, "near-miss statements stay separate nodes");
- assert.equal(ledgers.counts.nodes, 2);
- });
-
- it("merged nodes unify the second chat's edges onto the existing graph node", () => {
- const ledgers = memoryLedgers();
- const tree1 = createTree(
- [
- { id: "base", title: "Derivative as a limit" },
- { id: "chain", title: "Chain rule intuition" },
+ // Session 1: two fresh propositions + a follows edge between them.
+ const chat1 = fixtureChat("Edge A");
+ const verdict1: AuditResult = {
+ propositions: [
+ {
+ id: "p1",
+ statement: "Derivative as a limit",
+ description: "The derivative is the limit of difference quotients.",
+ aliases: [],
+ messages: ["u1"],
+ evidence: ["e"],
+ },
+ {
+ id: "p2",
+ statement: "Chain rule intuition",
+ description: "Composed functions differentiate by multiplying local rates.",
+ aliases: [],
+ messages: ["u1"],
+ evidence: ["e"],
+ },
],
- [{ from: "base", to: "chain" }],
- );
- const chat1 = fixtureChat("Edge A", tree1);
- mergeFindings(chat1.id, treeFindings(tree1), { chatStore, graphStore: ledgers });
+ edges: [{ from: "p1", to: "p2", type: "follows", messages: ["u1"], evidence: ["e"] }],
+ attribution: { quiz: {} },
+ dropped: [],
+ };
+ let stats = mergeFindings(chat1.id, verdict1, { chatStore, graphStore: ledgers });
+ assert.deepEqual(stats, { nodes: 2, edges: 1, events: 0 });
+ const base = ledgers.readNodes()[0]!.id;
+ const chain = ledgers.readNodes()[1]!.id;
- // Second session re-teaches the same two propositions (exact statements
- // modulo case) and asserts the same edge — it must dedupe onto the
- // existing nodes.
- const tree2 = createTree(
- [
- { id: "base", title: "derivative as a limit" },
- { id: "chain", title: "chain rule intuition" },
+ // Session 2: the auditor links both propositions to the SAME nodes
+ // (identity verdicts) and asserts the same relation.
+ const chat2 = fixtureChat("Edge B");
+ const verdict2: AuditResult = {
+ propositions: [
+ {
+ id: "q1",
+ statement: "The derivative as a limit of quotients", // different phrasing
+ description: "…",
+ aliases: [],
+ messages: ["u1"],
+ evidence: ["e"],
+ linkTo: base,
+ },
+ {
+ id: "q2",
+ statement: "Chain rule, intuitively",
+ description: "…",
+ aliases: [],
+ messages: ["u1"],
+ evidence: ["e"],
+ linkTo: chain,
+ },
],
- [{ from: "base", to: "chain" }],
- );
- const chat2 = fixtureChat("Edge B", tree2);
- const stats = mergeFindings(chat2.id, treeFindings(tree2), { chatStore, graphStore: ledgers });
- // A new ASSERTION line is appended (provenance is honest) — reads merge
- // by (from,to,type), so the derived graph still has one edge with both
- // sessions in its provenance.
+ edges: [{ from: "q1", to: "q2", type: "follows", messages: ["u1"], evidence: ["e"] }],
+ attribution: { quiz: {} },
+ dropped: [],
+ };
+ stats = mergeFindings(chat2.id, verdict2, { chatStore, graphStore: ledgers });
+
+ // No new nodes; ONE new assertion line (per-session honest provenance).
assert.deepEqual(stats, { nodes: 0, edges: 1, events: 0 }, "no new nodes, one re-assertion line");
- assert.equal(ledgers.counts.nodes, 2);
+ assert.equal(ledgers.counts.nodes, 4, "2 defining lines + 2 link-appends for the linked propositions");
const lines = ledgers.readEdges();
assert.equal(lines.length, 2, "two assertion lines for the same edge");
assert.equal(lines[0]!.from, lines[1]!.from);
assert.equal(lines[0]!.to, lines[1]!.to);
- assert.deepEqual(lines.map((l) => l.provenance), [[chat1.id], [chat2.id]]);
+ assert.deepEqual(lines.map((l) => l.provenance), [
+ [{ chatId: chat1.id, messageId: "u1" }],
+ [{ chatId: chat2.id, messageId: "u1" }],
+ ]);
+
+ // Read-time merging: one derived edge, both sessions in its provenance.
+ const derived = getGraph(ledgers).edges;
+ assert.equal(derived.length, 1);
+ assert.deepEqual(derived[0]!.provenance, [
+ { chatId: chat1.id, messageId: "u1" },
+ { chatId: chat2.id, messageId: "u1" },
+ ]);
});
});
diff --git a/tests/graph.test.ts b/tests/graph.test.ts
index 7759530..567d144 100644
--- a/tests/graph.test.ts
+++ b/tests/graph.test.ts
@@ -1,9 +1,9 @@
/**
- * Graph merger + trigger + derivation tests (vision §1; contracts.ts
- * "extraction pipeline"). The merger (core/graph.ts mergeFindings) is the
- * pipeline's deterministic apply stage: findings in, ledger appends out —
- * identity resolution, retroactive probe attribution, event rebuild, and
- * the idempotency guarantees.
+ * Graph merger + trigger + derivation tests (vision §1, 2026-09-02 redesign).
+ * The merger (core/graph.ts mergeFindings) is the pipeline's deterministic
+ * apply stage: AUDITED VERDICTS in, ledger appends out — link-or-create
+ * identity (the auditor's linkTo, never string matching), deterministic quiz
+ * events, understanding observations, and the idempotency guarantees.
*
* Ledger isolation: PATHS is fixed at import time, so the five data/graph
* ledgers are backed up and cleared before the suite and restored after
@@ -18,9 +18,8 @@ import assert from "node:assert/strict";
import * as fs from "node:fs";
import { after, describe, it } from "node:test";
import type { UIMessage } from "ai";
-import type { PlanTree, PlanToolResult, SessionFindings } from "../src/core/contracts.ts";
+import type { AuditResult } from "../src/core/contracts.ts";
import type { QuestionAnswerData } from "../src/core/question-core.ts";
-import { createTree, markDone } from "../src/core/plan-tree.ts";
import { getGraph, graphSnapshot, mergeFindings, pendingQuizAnswers, type GraphDeps } from "../src/core/graph.ts";
import { graphStore } from "../src/store/graph.ts";
import { chatStore } from "../src/store/chats.ts";
@@ -80,16 +79,6 @@ function quizPart(
return { type: "tool-quiz", toolCallId, state: "output-available", input, output };
}
-function planPart(toolCallId: string, name: string, input: Record, tree: PlanTree): unknown {
- return {
- type: `tool-${name}`,
- toolCallId,
- state: "output-available",
- input,
- output: { ok: true, tree } satisfies PlanToolResult,
- };
-}
-
function answer(over: Partial): QuestionAnswerData {
return {
questionId: "q",
@@ -102,59 +91,17 @@ function answer(over: Partial): QuestionAnswerData {
};
}
-/** Create a chat in the real store with a tree and messages; returns the record. */
-function fixtureChat(title: string, tree: PlanTree | null, messages: UIMessage[]) {
+/** Create a chat in the real store with messages; returns the record.
+ * Tree-blind world: the plan tree is irrelevant to the merger — no tree. */
+function fixtureChat(title: string, messages: UIMessage[]) {
const chat = chatStore.createChat(title);
createdChats.push(chat.id);
- if (tree) chatStore.saveTree(chat.id, tree);
chatStore.saveMessages(chat.id, messages);
return chatStore.getChat(chat.id)!;
}
-/** The fixture tree: alpha → beta, gamma free-floating, alpha done. */
-function fixtureTree(): PlanTree {
- return markDone(
- createTree(
- [
- { id: "alpha", title: "Alpha intuition" },
- { id: "beta", title: "Beta rule" },
- { id: "gamma", title: "Gamma application" },
- ],
- [{ from: "alpha", to: "beta" }],
- ),
- "alpha",
- );
-}
-
-/**
- * Findings shaped exactly like what the extractor would produce for a
- * session's plan tree: one proposition per tree node (statement = title,
- * treeNodeId set), one `requires` edge per tree edge.
- */
-function treeFindings(tree: PlanTree): SessionFindings {
- const idByNodeId = new Map(tree.nodes.map((n, i) => [n.id, `p${i + 1}`]));
- return {
- propositions: tree.nodes.map((n, i) => ({
- id: `p${i + 1}`,
- name: n.title,
- statement: n.title,
- aliases: [],
- kind: "truth" as const,
- treeNodeId: n.id,
- evidence: ["fixture evidence"],
- })),
- edges: tree.edges.map((e) => ({
- from: idByNodeId.get(e.from)!,
- to: idByNodeId.get(e.to)!,
- type: "requires" as const,
- evidence: ["fixture evidence"],
- })),
- attribution: { quiz: {} },
- };
-}
-
-/** The full fixture transcript: probes → plan → checks → mark_done → noise. */
-function fixtureMessages(tree: PlanTree): UIMessage[] {
+/** The full fixture transcript: five graded quizzes + a dismissed one + an ask. */
+function fixtureMessages(): UIMessage[] {
return [
user("u1", "teach me the fixture topic"),
assistant("a1", [
@@ -171,32 +118,30 @@ function fixtureMessages(tree: PlanTree): UIMessage[] {
answer({ questionId: "call-2", questionText: "Q2 text", dontKnow: true, correct: false, answeredAt: "2026-08-22T01:01:00.000Z" }),
),
]),
- assistant("a3", [planPart("p1", "plan_init", { nodes: [], edges: [] }, tree)]),
- assistant("a4", [
+ assistant("a3", [
+ // input.nodeId "alpha" is deliberately present and IGNORED —
+ // attribution is the only quiz→node path in the tree-blind world.
quizPart(
"call-3",
{ question: "Q3", nodeId: "alpha", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
answer({ questionId: "call-3", questionText: "Q3 text", correct: true, note: "recalled the diagram", answeredAt: "2026-08-22T02:00:00.000Z" }),
),
]),
- assistant("a5", [
+ assistant("a4", [
quizPart(
"call-4",
- { question: "Q4", nodeId: "beta", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
+ { question: "Q4", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
answer({ questionId: "call-4", questionText: "Q4 text", correct: false, answeredAt: "2026-08-22T02:01:00.000Z" }),
),
]),
- assistant("a6", [
- // nodeId present but NOT in the tree → tagged-but-unmapped: the
- // attribution can rescue it, or it lands null + question text.
+ assistant("a5", [
quizPart(
"call-5",
- { question: "Q5", nodeId: "ghost", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
+ { question: "Q5", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
answer({ questionId: "call-5", questionText: "Q5 text", dontKnow: true, correct: false, answeredAt: "2026-08-22T02:02:00.000Z" }),
),
]),
- assistant("a7", [planPart("p2", "plan_mark_done", { id: "alpha" }, tree)]),
- assistant("a8", [
+ assistant("a6", [
// dismissed quiz + an answered ask — neither becomes an event
quizPart(
"call-6",
@@ -214,25 +159,58 @@ function fixtureMessages(tree: PlanTree): UIMessage[] {
];
}
-/**
- * The extractor's findings for the fixture chat: the tree as propositions +
- * the probe attribution. The audit stage is a no-op pass-through in these
- * merger tests — its behavior is tested with the pipeline (extraction.test).
- */
-function fixtureFindings(): SessionFindings {
- const findings = treeFindings(fixtureTree());
- // Untagged probe quizzes: call-1 attributed retroactively to Alpha (the
- // vision's open question — resolved by the extractor); call-2 genuinely
- // unattributable; call-5's ghost nodeId is rescued onto Gamma.
- findings.attribution.quiz = { "call-1": "p1", "call-2": null, "call-5": "p3" };
- return findings;
+/** The auditor's verdict for the fixture chat: three verified propositions
+ * (no linkTo — the graph starts empty), one follows edge, all quizzes
+ * attributed, one understanding note. */
+function fixtureAudit(): AuditResult {
+ return {
+ propositions: [
+ {
+ id: "p1",
+ statement: "Alpha intuition",
+ description: "Alpha intuition, elaborated: what alpha means and how it behaves in practice.",
+ aliases: ["alpha feel"],
+ messages: ["a1", "a3"],
+ evidence: ["fixture evidence"],
+ understanding: "solid after the check",
+ },
+ {
+ id: "p2",
+ statement: "Beta rule",
+ description: "The beta rule and when it applies.",
+ aliases: [],
+ messages: ["a4"],
+ evidence: ["fixture evidence"],
+ },
+ {
+ id: "p3",
+ statement: "Gamma application",
+ description: "Applying the whole framework to gamma cases.",
+ aliases: [],
+ messages: ["a5"],
+ evidence: ["fixture evidence"],
+ },
+ ],
+ edges: [
+ {
+ from: "p1",
+ to: "p2",
+ type: "follows",
+ messages: ["a4"],
+ evidence: ["fixture evidence"],
+ },
+ ],
+ attribution: { quiz: { "call-1": "p1", "call-2": null, "call-3": "p1", "call-4": "p2", "call-5": "p3" } },
+ dropped: [],
+ };
}
/** The chat every merger test folds (shared, so ordering is intentional). */
-const chat1 = fixtureChat("Graph fixture", fixtureTree(), fixtureMessages(fixtureTree()));
+const chat1 = fixtureChat("Graph fixture", fixtureMessages());
-/** Set by the cross-session test; the snapshot test (which runs after it) reads it. */
+/** Set by the cross-session tests; the snapshot test (which runs after) reads them. */
let chat2Id = "";
+let chat3Id = "";
// ─── the merger ──────────────────────────────────────────────────────────────
@@ -248,71 +226,83 @@ function scopedGraph(chatIds: string[]) {
);
return {
nodes: graph.nodes.filter((n) => nodeIds.has(n.id)),
- edges: graph.edges.filter((e) => e.provenance.some((c) => chatIds.includes(c))),
+ edges: graph.edges.filter((e) => e.provenance.some((pt) => chatIds.includes(pt.chatId))),
events: graph.events.filter((e) => chatIds.includes(e.chatId)),
};
}
describe("the merger (mergeFindings)", () => {
- it("merges propositions/edges by identity and rebuilds events with retroactive probe attribution", () => {
- const stats = mergeFindings(chat1.id, fixtureFindings(), graphDeps);
+ it("creates nodes from the verdicts, rebuilds quiz events deterministically, records observations", () => {
+ const stats = mergeFindings(chat1.id, fixtureAudit(), graphDeps);
assert.deepEqual(stats, { nodes: 3, edges: 1, events: 6 });
const graph = scopedGraph([chat1.id]);
assert.equal(graph.nodes.length, 3);
const idByStatement = new Map(graph.nodes.map((n) => [n.statement, n.id]));
for (const node of graph.nodes) {
- assert.equal(node.kind, "truth");
- assert.deepEqual(node.aliases, []);
assert.match(node.id, /^n_/);
+ assert.ok(node.description.length > 0, "description is required");
+ assert.equal("kind" in node, false, "no kind field");
+ assert.ok(Array.isArray(node.provenance), "message-granular provenance");
}
+ // Provenance is message-granular: p1 came up in a1 and a3.
+ const alpha = graph.nodes.find((n) => n.statement === "Alpha intuition")!;
+ assert.deepEqual(alpha.provenance, [
+ { chatId: chat1.id, messageId: "a1" },
+ { chatId: chat1.id, messageId: "a3" },
+ ]);
- // The tree edge alpha→beta becomes a `requires` edge on the mapped nodes.
+ // The follows edge, provenance attached.
assert.equal(graph.edges.length, 1);
const edge = graph.edges[0]!;
assert.equal(edge.from, idByStatement.get("Alpha intuition"));
assert.equal(edge.to, idByStatement.get("Beta rule"));
- assert.equal(edge.type, "requires");
- assert.deepEqual(edge.provenance, [chat1.id]);
+ assert.equal(edge.type, "follows");
+ assert.deepEqual(edge.provenance, [{ chatId: chat1.id, messageId: "a4" }]);
+
+ // Mapping lines: one per (chat, sourceId).
+ assert.deepEqual(
+ graphStore.readMappings().map((m) => m.sourceId),
+ ["p1", "p2", "p3"],
+ );
// Events — kinds, refs, notes, and the attribution paths.
assert.equal(graph.events.length, 6);
const byRef = new Map(graph.events.map((e) => [e.ref, e]));
- const probeMiss = byRef.get("call-1")!;
- assert.equal(probeMiss.kind, "probe-miss"); // quiz BEFORE plan_init
- assert.equal(probeMiss.nodeId, idByStatement.get("Alpha intuition"), "retroactive probe attribution");
- assert.equal(probeMiss.question, undefined, "attributed → no question text needed");
+ const miss = byRef.get("call-1")!;
+ assert.equal(miss.kind, "miss");
+ assert.equal(miss.nodeId, idByStatement.get("Alpha intuition"), "attributed quiz");
- const probeDontKnow = byRef.get("call-2")!;
- assert.equal(probeDontKnow.kind, "dont-know"); // same kind, both phases
- assert.equal(probeDontKnow.nodeId, null, "unattributable probe → null");
- assert.equal(probeDontKnow.question, "Q2 text");
+ const dontKnow = byRef.get("call-2")!;
+ assert.equal(dontKnow.kind, "dont-know");
+ assert.equal(dontKnow.nodeId, null, "unattributable → null");
+ assert.equal(dontKnow.question, "Q2 text");
- const check = byRef.get("call-3")!;
- assert.equal(check.kind, "check-correct"); // quiz AFTER plan_init
- assert.equal(check.nodeId, idByStatement.get("Alpha intuition"), "tagged quiz → deterministic");
- assert.equal(check.note, "recalled the diagram");
+ const correct = byRef.get("call-3")!;
+ assert.equal(correct.kind, "correct");
+ assert.equal(correct.nodeId, idByStatement.get("Alpha intuition"), "attribution wins over the ignored input.nodeId");
+ assert.equal(correct.note, "recalled the diagram");
- const checkMiss = byRef.get("call-4")!;
- assert.equal(checkMiss.kind, "check-incorrect");
- assert.equal(checkMiss.nodeId, idByStatement.get("Beta rule"));
+ const miss2 = byRef.get("call-4")!;
+ assert.equal(miss2.kind, "miss");
+ assert.equal(miss2.nodeId, idByStatement.get("Beta rule"));
- const ghost = byRef.get("call-5")!;
- assert.equal(ghost.kind, "dont-know");
- assert.equal(ghost.nodeId, idByStatement.get("Gamma application"), "tagged-but-unmapped rescued by attribution");
- assert.equal(ghost.question, undefined);
+ const dontKnow2 = byRef.get("call-5")!;
+ assert.equal(dontKnow2.kind, "dont-know");
+ assert.equal(dontKnow2.nodeId, idByStatement.get("Gamma application"));
- const taught = byRef.get("taught:alpha")!;
- assert.equal(taught.kind, "taught");
- assert.equal(taught.nodeId, idByStatement.get("Alpha intuition"), "mark_done → tree-node mapping");
+ const observation = byRef.get("obs:p1")!;
+ assert.equal(observation.kind, "observation");
+ assert.equal(observation.nodeId, idByStatement.get("Alpha intuition"));
+ assert.equal(observation.note, "solid after the check");
// The dismissed quiz and the ask question became nothing.
assert.equal([...byRef.keys()].filter((r) => r === "call-6" || r === "call-7").length, 0);
});
- it("is idempotent — re-merging the same findings appends nothing", () => {
- const stats = mergeFindings(chat1.id, fixtureFindings(), graphDeps);
+ it("is idempotent — re-merging the same verdicts appends nothing", () => {
+ const stats = mergeFindings(chat1.id, fixtureAudit(), graphDeps);
assert.deepEqual(stats, { nodes: 0, edges: 0, events: 0 });
const graph = scopedGraph([chat1.id]);
assert.equal(graph.nodes.length, 3);
@@ -322,38 +312,63 @@ describe("the merger (mergeFindings)", () => {
assert.equal(graphStore.readCheckpoints().filter((c) => c.chatId === chat1.id).length, 2);
});
- it("merges a second session onto the same nodes and accumulates edge provenance", () => {
- // chat2 re-teaches the same two propositions (identical statements) —
- // the conservative identity resolution recognizes them.
- const tree2 = createTree(
- [
- { id: "a", title: "Alpha intuition" },
- { id: "b", title: "Beta rule" },
- ],
- [{ from: "a", to: "b" }],
- );
- const chat2 = fixtureChat("Graph fixture 2", tree2, [
+ it("applies a linkTo verdict: phrasings join aliases, provenance unions, no duplicate node", () => {
+ const alphaId = scopedGraph([chat1.id]).nodes.find((n) => n.statement === "Alpha intuition")!.id;
+ const chat2 = fixtureChat("Graph fixture 2", [
user("u1", "again"),
- assistant("a1", [planPart("p1", "plan_init", { nodes: [], edges: [] }, tree2)]),
+ assistant("a1", [
+ quizPart(
+ "call-9",
+ { question: "Q9", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
+ answer({ questionId: "call-9", questionText: "Q9 text", correct: true, answeredAt: "2026-08-22T03:00:00.000Z" }),
+ ),
+ ]),
]);
-
- const stats = mergeFindings(chat2.id, treeFindings(tree2), graphDeps);
- assert.deepEqual(stats, { nodes: 0, edges: 1, events: 0 });
chat2Id = chat2.id;
+ // The auditor's identity verdict: this proposition IS the alpha node.
+ const stats = mergeFindings(
+ chat2.id,
+ {
+ propositions: [
+ {
+ id: "q1",
+ statement: "Alpha feel", // a DIFFERENT phrasing — identity is the verdict, not strings
+ description: "The intuitive grasp of alpha.",
+ aliases: ["alpha intuition"],
+ messages: ["a1"],
+ evidence: ["fixture evidence"],
+ linkTo: alphaId,
+ },
+ ],
+ edges: [],
+ attribution: { quiz: { "call-9": "q1" } },
+ dropped: [],
+ },
+ graphDeps,
+ );
+ assert.deepEqual(stats, { nodes: 0, edges: 0, events: 1 });
+
const after = scopedGraph([chat1.id, chat2.id]);
- assert.equal(after.nodes.length, 3); // no duplicate nodes
- assert.equal(after.edges.length, 1); // one merged edge…
- assert.deepEqual(after.edges[0]!.provenance, [chat1.id, chat2.id]); // …two sessions
+ assert.equal(after.nodes.length, 3, "no duplicate node");
+ const alpha = getGraph(graphStore).nodes.find((n) => n.id === alphaId)!;
+ assert.ok(alpha.aliases.includes("Alpha feel"), "the session's phrasing joins the aliases");
+ assert.ok(alpha.aliases.includes("alpha intuition"));
+ assert.ok(
+ alpha.provenance.some((pt) => pt.chatId === chat2.id && pt.messageId === "a1"),
+ "provenance unions",
+ );
+ // The mapping records the verdict: (chat2, q1) → the linked node.
+ const m = graphStore.readMappings().find((x) => x.chatId === chat2.id)!;
+ assert.deepEqual(m, { chatId: chat2.id, sourceId: "q1", graphNodeId: alphaId });
});
- it("derives node states, sessions, and node provenance in graphSnapshot", () => {
+ it("derives node states and sessions in graphSnapshot", () => {
const snap = graphSnapshot(graphDeps);
const idByStatement = new Map(snap.nodes.map((n) => [n.statement, n.id]));
- assert.equal(snap.nodeStates[idByStatement.get("Alpha intuition")!], "taught");
+ // Alpha's latest quiz is call-9 (chat2, correct) → known.
+ assert.equal(snap.nodeStates[idByStatement.get("Alpha intuition")!], "known");
assert.equal(snap.nodeStates[idByStatement.get("Beta rule")!], "missed");
- // Gamma: no taught/checks — but the rescued dont-know probe (call-5)
- // makes its latest event a dont-know.
assert.equal(snap.nodeStates[idByStatement.get("Gamma application")!], "dont-know");
assert.equal(snap.sessions[chat1.id], "Graph fixture");
@@ -366,8 +381,35 @@ describe("the merger (mergeFindings)", () => {
assert.deepEqual(snap.nodeSessions[idByStatement.get("Gamma application")!], [chat1.id]);
});
+ it("never string-matches without a verdict — an identical statement without linkTo is a NEW node", () => {
+ const chat3 = fixtureChat("Graph fixture 3", [user("u1", "third")]);
+ chat3Id = chat3.id;
+ const stats = mergeFindings(
+ chat3.id,
+ {
+ propositions: [
+ {
+ id: "r1",
+ statement: "Alpha intuition", // IDENTICAL to the existing node's statement
+ description: "Same words, no verdict.",
+ aliases: [],
+ messages: ["u1"],
+ evidence: ["fixture evidence"],
+ // no linkTo — identity is the auditor's call alone
+ },
+ ],
+ edges: [],
+ attribution: { quiz: {} },
+ dropped: [],
+ },
+ graphDeps,
+ );
+ assert.equal(stats.nodes, 1, "created — the merger does not match strings");
+ assert.equal(getGraph(graphStore).nodes.filter((n) => n.statement === "Alpha intuition").length, 2);
+ });
+
it("returns zeros for unknown chats", () => {
- assert.deepEqual(mergeFindings("chat_does_not_exist", treeFindings(fixtureTree()), graphDeps), {
+ assert.deepEqual(mergeFindings("chat_does_not_exist", fixtureAudit(), graphDeps), {
nodes: 0,
edges: 0,
events: 0,
@@ -379,14 +421,12 @@ describe("the merger (mergeFindings)", () => {
describe("the trigger helper (pendingQuizAnswers)", () => {
it("counts answered quizzes not yet in the ledger — dismissed/ask never count", () => {
- const tree = createTree([{ id: "solo", title: "Solo node" }], []);
- const chat = fixtureChat("Trigger fixture", tree, [
+ const chat = fixtureChat("Trigger fixture", [
user("u1", "go"),
- assistant("a1", [planPart("p1", "plan_init", { nodes: [], edges: [] }, tree)]),
- assistant("a2", [
+ assistant("a1", [
quizPart(
"call-s",
- { question: "QS", nodeId: "solo", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
+ { question: "QS", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
answer({ questionId: "call-s", questionText: "QS text", correct: true, answeredAt: "2026-08-22T03:00:00.000Z" }),
),
quizPart(
@@ -407,7 +447,9 @@ describe("the trigger helper (pendingQuizAnswers)", () => {
assert.equal(pendingQuizAnswers(chat.id, graphDeps), 1, "one answered quiz pending");
assert.equal(pendingQuizAnswers("chat_does_not_exist", graphDeps), 0);
- mergeFindings(chat.id, treeFindings(tree), graphDeps);
+ mergeFindings(chat.id, fixtureAudit(), graphDeps);
+ // call-s is unattributed here (fixtureAudit has no call-s attribution)
+ // — but it still becomes an event, so nothing stays pending.
assert.equal(pendingQuizAnswers(chat.id, graphDeps), 0, "extracted → nothing pending");
// A NEW answered quiz raises the count again (the re-trigger path).
@@ -416,7 +458,7 @@ describe("the trigger helper (pendingQuizAnswers)", () => {
assistant("a3", [
quizPart(
"call-s4",
- { question: "QS4", nodeId: "solo", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
+ { question: "QS4", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
answer({ questionId: "call-s4", questionText: "QS4 text", correct: false, answeredAt: "2026-08-22T03:01:00.000Z" }),
),
]),
@@ -429,29 +471,43 @@ describe("the trigger helper (pendingQuizAnswers)", () => {
describe("graph ledger store", () => {
it("round-trips lines and skips corrupt / foreign-shaped lines (strict reads)", () => {
- const node = { id: "n_storetest", name: "Store test", statement: "Store test", aliases: [], kind: "truth" as const, createdAt: "2026-08-22T00:00:00.000Z" };
+ const node = {
+ id: "n_storetest",
+ statement: "Store test",
+ description: "Store test, elaborated.",
+ aliases: [] as string[],
+ createdAt: "2026-08-22T00:00:00.000Z",
+ provenance: [{ chatId: "chat_c", messageId: "m1" }],
+ };
graphStore.appendNode(node);
- // A torn append and a legacy V0-shaped node (different schema) — both skipped.
+ // A torn append and a legacy pre-redesign node (old schema) — both skipped.
fs.appendFileSync(
LEDGER_FILES.nodes,
- `{not json}\n${JSON.stringify({ id: "legacy", statement: "Legacy shape", domain: "x", state: "known" })}\n`,
+ `{not json}\n${JSON.stringify({ id: "legacy", statement: "Legacy shape", kind: "truth", name: "Legacy", createdAt: "2026-08-22T00:00:00.000Z" })}\n`,
"utf-8",
);
const nodes = graphStore.readNodes().filter((n) => n.id === "n_storetest" || n.id === "legacy");
assert.equal(nodes.length, 1);
assert.equal(nodes[0]!.statement, "Store test");
- assert.equal(nodes[0]!.name, "Store test");
+ assert.deepEqual(nodes[0]!.provenance, [{ chatId: "chat_c", messageId: "m1" }]);
});
- it("derives a name for pre-name ledger lines (legacy reads)", () => {
- // A full, valid node line written before `name` existed.
- fs.appendFileSync(
- LEDGER_FILES.nodes,
- `${JSON.stringify({ id: "n_prename", statement: "Derivatives measure instantaneous change; they are limits.", aliases: [], kind: "truth", createdAt: "2026-08-22T00:00:00.000Z" })}\n`,
- "utf-8",
- );
- const node = graphStore.readNodes().find((n) => n.id === "n_prename");
- assert.ok(node);
- assert.equal(node.name, "Derivatives measure instantaneous change");
+ it("returns RAW node lines — repeated ids (link-appends) are each their own line; getGraph merges", () => {
+ const line = (msgId: string) =>
+ JSON.stringify({
+ id: "n_linktest",
+ statement: `Phrasing ${msgId}`,
+ description: "Elaboration.",
+ aliases: [],
+ createdAt: "2026-08-22T00:00:00.000Z",
+ provenance: [{ chatId: "chat_c", messageId: msgId }],
+ });
+ fs.appendFileSync(LEDGER_FILES.nodes, `${line("m1")}\n${line("m2")}\n`, "utf-8");
+ const lines = graphStore.readNodes().filter((n) => n.id === "n_linktest");
+ assert.equal(lines.length, 2, "the store returns raw lines, one per append");
+ const merged = getGraph(graphStore).nodes.find((n) => n.id === "n_linktest")!;
+ assert.equal(merged.statement, "Phrasing m1", "the FIRST line defines the node");
+ assert.ok(merged.aliases.includes("Phrasing m2"), "later phrasings join the aliases");
+ assert.deepEqual(merged.provenance.map((p) => p.messageId), ["m1", "m2"]);
});
});