diff --git a/docs/vision-context-injection.md b/docs/vision-context-injection.md new file mode 100644 index 0000000..b644bfc --- /dev/null +++ b/docs/vision-context-injection.md @@ -0,0 +1,54 @@ +# Vision — Graph context injection + +Ideation settled 2026-09-14. Not yet planned, not yet assigned to a wave. This document is the decision record; `vision.md` § "The graph as session context" settles its open *selection problem* here (V1 only). + +One line: a new session's first message is its **topic**; the topic is embedded, the semantically closest propositions from the global knowledge graph are selected under a token budget, and delivered to the session as a **steer** — *"what this learner already knows."* This is the embeddings investment paying off: the same vectors that power clustering now personalize teaching. + +--- + +## The mechanism, end to end + +**Trigger — one-shot, at session start.** The injection job fires when the session's first user message exists (the topic). One shot per chat, idempotent, deterministic steer id — same discipline as every other job/steer. No mid-session re-injection (parked), no retroactive injection for sessions already in flight. + +**The topic is a query; the propositions are documents (DECIDED 2026-09-14).** EmbeddingGemma's model card prescribes *role-asymmetric* input formats: retrieval queries use `task: search result | query: `, retrieval documents use `title: none | text: `. The ledger already embeds propositions in the document format — it stays untouched. The topic embeds through the **same warm embedder singleton** with the **query format**. The requirement is not "same format" — it is **same model, same 512-dim MRL truncation** (one vector space), with each side in the format the model was trained for. The job asserts the ledger's model+dim stamp before scanning; there is never a parallel embedding path. + +- *Why formats matter at all:* semantics dominate — raw text would still roughly work — but the benchmark quality this model was picked for is measured with the trained templates, and using them costs nothing. The embedder's own header already warns that mixing prefixed and unprefixed inputs degrades quality. +- *Rejected:* re-embedding the whole corpus as plain `title\ntext`. It voids every vector (a minutes-long full re-embed) for a format the model wasn't trained on. If retrieval quality ever looks weak on real data, the graph stream can benchmark plain-vs-card formats against the 2k-node harness — an experiment, not the default. + +**Search — brute-force cosine, exact, no index.** Scan the in-memory embeddings ledger for top-K similarity. ~1M multiply-adds at 2k propositions (sub-millisecond); ~25ms at 50k. No ANN, no vector DB — not until ~10⁶ scale, which is far away. + +**Latency is a non-issue.** The topic embed takes tens to ~200 ms warm (extraction keeps the pipeline singleton hot); the whole job — embed, scan, build block, queue steer — is comfortably under a second warm. Cold start is seconds (weights load once), and harmless because delivery is async anyway. + +**Selection — pure token budget; similarity only ranks (DECIDED 2026-09-14).** The user-facing control is a **token budget** — there is deliberately **no similarity threshold**. Irrelevant-but-cheap content rarely hurts; the only real hazard is *too much context*, and token count is what governs behavior. Fill greedily in similarity-descending order, stop before exceeding the budget, plus a hard count cap (tuned at build time) as belt-and-braces. + +**Token estimate: chars / 4.** Heuristic, no server-side tokenizer — good enough for a budget dial and for the observability card. + +**Annotation — UnderstandingLevel, folded most-recent (DECIDED 2026-09-14).** Levels live **per extraction instance**, tied to the moments a proposition appeared in sessions; a node can carry several across its life. The injected block reports the level from the **most recent instance** that touched the node: latest evidence is the best estimate of current understanding, because learners progress — an optimistic fold (highest ever) risks planning on stale claims, a pessimistic fold (lowest) wastes ground actually covered. Recency also keeps `resisted` visible exactly when it is current. The block's gloss: **solid** = build on it, **shaky** = target it, **resisted** = teach around it, **not yet engaged** = no evidence yet, **accepted** = took it in. + +**Delivery — a steer through the existing wake pipeline, gate-for-gate identical to sub-agent results.** The prober typically opens with a run of quiz/ask calls; the steer queues behind the pending question and rides the turn after the learner answers. The agent never "waits up" — delivery lands at the natural seam mid-probing, well before plan_init, which is where prior knowledge matters most. Cost: one visible prober turn acknowledging the context. Accepted as a *feature*: the learner watches the teacher recognize what they already know — the product's premise, enacted. The transcript keeps the audit trail: exactly what was injected, and when. + +**V1 keeps it raw (DECIDED 2026-09-14).** Plain similarity-ordered top-K. No dedup or cluster-representative substitution, no `follows`-links among the injected set, no frontier/diagnostic ranking — see how raw retrieval works and feels first. All of those stay parked. + +--- + +## The setting + +One field on **ChatSettings** (rides the same pre-chat `PUT /api/chats/[id]/settings` as model + thinking, so the draft view gets it nearly free): the graph-context token budget. + +**Default 1024, range 0–4096, step 128, 0 = off (DECIDED 2026-09-14).** + +**Placement (DECIDED): the blank session-data section of the graph view, for unstarted sessions.** The symmetry is the point: the panel that will later show what the session contributed to the graph starts by asking what the graph will contribute to the session — input before output. + +**The topic reframe is presentational, not structural.** Message 1 is functionally already the topic (it is the title, the opener, the embedding source). Parked refinements: the draft composer framed as a topic ask (matching home's "What do you want to learn?"), and message 1 rendered as a quiet topic header rather than a chat bubble. The record contract is unchanged — zero adapter churn; the embedder reads message 1's text. + +--- + +## Observability + +- The job appears in the **background-processes dock** (a new JobKind in the existing pattern) from send-of-first-message to steer delivery. +- The delivered steer renders as a **quiet marker card in the transcript** carrying the numbers: token estimate and proposition count (e.g. *"knowledge context — ~1.1k tokens · 23 propositions"*). Same marker-part pattern as verdicts and handoffs; the model-facing block text is adapter/canonical-owned like all steer text. +- A small prompt line (agents lane, at build time) tells prober/teacher to treat the block as reliable prior knowledge. + +## Lane split — decided at planning time, not now + +Graph owns the embedder service + ledger scan; agents own the JobKind, the steer, and the prompt line; UI owns the slider; master owns the ChatSettings contract and the seam. Nothing here is assigned or scheduled. \ No newline at end of file diff --git a/docs/vision-edge-glyph.md b/docs/vision-edge-glyph.md new file mode 100644 index 0000000..9609345 --- /dev/null +++ b/docs/vision-edge-glyph.md @@ -0,0 +1,35 @@ +# Vision — The edge glyph: follows vs supports + +Ideation settled 2026-09-14. Not yet planned, not yet assigned. Companion: `vision.md` § "Edges" (the relation types) and § "Rendering". + +One line: the two relation types become visually distinct — **`follows` renders as a double-shaft arrow** (two parallel tails side by side, the implication glyph ⟹), **`supports` stays a single line**. Arrows keep their existing direction: prerequisite → dependent. + +--- + +## Why double-shaft (DECIDED 2026-09-14) + +- **Semantically exact.** `follows` means prerequisite ⇒ successor — derivability. The implication glyph *is* the relation; nothing encodes "derivable from" more literally. +- **Weight where the structure is.** The DAG backbone (`follows`) gets the heavy glyph; the majority relation (`supports` is over half of all edges) stays light. The structural skeleton reads first. +- **Shape survives everything color can't.** Cluster regions already own the color channel; a shape code collides with nothing and survives colorblindness. +- **Scale-stable.** Pure geometry degrades nowhere — see the rejected alternative below. + +## Why not dashed (rejected 2026-09-14) + +Dash patterns break under our **constant screen-space stroke width**: the dash array is re-projected across zoom levels, so the pattern pops and crawls instead of scaling cleanly (the same class of artifact as the non-integer blur radii). And with `supports` the *majority* relation, dashes would dominate the canvas rather than mark the exception. Dashed = wrong twice. + +## Geometry + +The renderer is the custom 2D canvas; edges draw in the edge layer with shafts + arrowheads from `arrowGeometry` (sigma-style head, prerequisite → dependent). For `follows`: + +- Both shafts offset along the edge normal by a **fixed screen-space gap**, converging on a **shared single arrowhead**. +- The gap is proportional to the constant stroke width — a constant visual ratio at every zoom level. +- **No LOD collapse (withdrawn 2026-09-14).** The earlier idea of collapsing to a single shaft below a stroke-width threshold assumed width shrinks with zoom — it doesn't; stroke width is constant on screen, so the double shaft renders identically at every zoom level. Simpler than proposed: nothing to gate. + +## Legend and scope + +- Add a legend entry to the existing graph legend: `⟹ follows · → supports`. +- Main graph view only. The plan-block renderer (dagre) is untouched — the plan DAG is single-relation, so there is nothing to distinguish there. + +## Lane + +lh-graph's renderer files (`graph-canvas.ts`, `graph-view.tsx`). Not assigned, not scheduled. \ No newline at end of file diff --git a/docs/vision.md b/docs/vision.md index 965c606..0bb92bd 100644 --- a/docs/vision.md +++ b/docs/vision.md @@ -109,6 +109,8 @@ When the graph fills up, the canvas needs a readable at-a-glance structure: **cl ### The graph as session context (DECIDED 2026-08-22) +*Selection problem V1 settled 2026-09-14 — see `vision-context-injection.md`: semantic-closeness top-K under a per-session token budget, delivered as a steer. The structural-neighborhood and diagnostic-hotspot strategies below stay parked.* + Every session starts with the learner's knowledge graph as base context for the teacher. This is what makes probing *precise* — probe questions can target subtleties and nuances at the boundary of what the learner actually holds, instead of re-discovering it from scratch each session. Personalization compounds across sessions. **The selection problem (open):** the whole graph may eventually be too much context, and pedagogical relevance ≠ semantic closeness. What matters for a new topic: (a) its **structural neighborhood** — the `follows`-edge subtree the topic stands on, regardless of surface similarity; (b) **diagnostic hotspots** — nodes the ledger marks as shaky / misunderstood / misconceived, anywhere in the graph (a live misconception — events showing the learner holds a node wrong — is always relevant context); (c) the frontier strip near the topic. Candidate strategies, to tune on real data: whole graph while it's small; k-hop structural neighborhood around the topic's nodes; ledger-informed slices (weak/misconceived/floating first); a cheap routing pass that picks subtrees given the session topic.