Root cause: sigma's getNodeAtPosition reads the picking framebuffer,
which is rendered at HALF the device resolution
(pickingDownSizingRatio = 2 * pixelRatio). Small nodes' hit circles
are a couple of device pixels across in that buffer with ragged,
quantized edges — near a node boundary, tiny cursor movements flip
the hover state on/off, and each flip makes sigma clear the hover
layer immediately. That state churn is the flicker; it survived every
animation change because it was never in the animation.
Mitigations:
- patchNodePicking() replaces Sigma.prototype.getNodeAtPosition with
a smooth distance test (rendered radius + HIT_PAD_PX = 6px) built on
public renderer APIs only (getGraph, getNodeDisplayData,
framedGraphToViewport, scaleSize). No ragged edges, and small
candidate nodes get roughly double the hoverable area.
- Leave-debounce (90ms): when hover drops, hold repainting the
captured hover paint at the current fade (sigma would otherwise
clear the layer the instant hoveredNode flips to null), and only
fade out if the cursor hasn't bounced back within the window. A
re-enter cancels the fade, so boundary oscillation never visibly
dims anything.
The instant muted->bright color swap felt abrupt next to the fading
halo. The hover name is again drawn at partial alpha over the muted
resting label, using the SAME transition factor as the halo — text
and glow fade in/out in sync. Because every enter/leave interpolates
from the current fade value, rapid hover toggles at a node's hit
boundary glide instead of snapping (the flicker-damping fix).
- The halo no longer grows during the fade: radius is constant at
size + 9px; only its opacity fades in/out over ~180ms.
- The hover name is drawn opaque in the bright color instead of
cross-fading over the muted resting label. Partial-alpha text
oscillated whenever hover toggles rapidly at a small node's hit
boundary (sigma fires enterNode/leaveNode on every state change) —
an un-animated color can't flicker.
- The glow no longer pulses: it's a static halo (9px beyond the disc,
0.45 alpha at the edge fading outward) in the node's state color.
- Hover enter/leave now cross-fades over ~180ms instead of appearing
instantly: entering schedules sigma renders so drawHover repaints at
the interpolated fade; leaving draws through a repainter captured on
the last hover paint (sigma stops calling drawHover once unhovered).
The text cross-fades too — the brighter hover name is drawn at
partial alpha over the muted resting label on the layer beneath.
- The halo starts flush at the node disc edge (was size+1, leaving a
dark gap ring that read as a gray border between node and glow).
- Both fade directions cancel each other and stop on completion, so
there is no continuous loop — idle CPU stays near zero.
- defaultDrawNodeHover no longer paints a background box: the hovered
node now pulses a soft radial-gradient halo ring in its own state
color (evenodd punch-out keeps the node disc untouched), phase driven
by performance.now() inside the drawer.
- The hovered label text brightens (muted-foreground -> foreground) at
identical geometry — drawName is shared by both drawers so nothing
shifts when hover begins/ends.
- The pulse is animated by a rAF loop that only runs while a node is
hovered (enterNode starts, leaveNode/leaveStage stop, cleanup
cancels) — idle CPU stays near zero, preserving run-then-freeze.
- roundedRect removed with the pill (its only remaining consumer).
- roundedRect typo: the first arcTo's second control point used y+width
as the x-coordinate (y + width instead of x + width). Since arcTo
tangent math is in absolute coordinates, the pill outline sprouted a
stray wedge/notch whose size scaled with the pill's on-screen
position (diagnosed live by the tester subagent: ~1000px left spike
on a top-right node, right-side bites near the left edge).
- Labels now fade instead of popping: fully visible at fit zoom or
closer, then the labels canvas layer's opacity is interpolated across
a 0.5 camera-ratio zoom span before renderLabels switches off, with
the setSetting call guarded so pans don't spam refreshes.
labelRenderedSizeThreshold gated each node's label by its own rendered
size, so big (taught) nodes showed names several zoom stops before
small (candidate) ones. Zero the per-node threshold and instead toggle
renderLabels from the camera ratio: every name appears at fit zoom or
closer and they all vanish together when zooming further out. Hover
still reveals any node's name regardless of zoom.
- Custom sigma label drawers replace the defaults: every label renders
the node name word-wrapped under the node (max 170px line, overlong
words hard-broken, never truncated) instead of one long strip to the
right of the node.
- Hover swaps sigma's hardcoded white box for a dark rounded pill using
the app's popover tokens; hover always shows the name, even below the
zoom threshold.
- labelRenderedSizeThreshold 12 -> 6 (names are short now) and labels
use the app's body font at weight 500.
- Click card anchors just below the clicked node's viewport position
(flipping above near the bottom edge, clamped horizontally, measured
in a layout effect so the flip never flashes); any pan/zoom dismisses
it via the camera 'updated' event.
Schema: GraphNode + ExtractedProposition carry 'name' — a short 2-6 word
header, the rendering handle. Node theory made explicit: a node is an
objective world-facing fact; the learner's relationship to it lives
entirely in the event ledger (state labels are derived, never stored).
Extractor + auditor instructions enforce it — misconceptions state the
wrong idea itself as a claim, never 'the learner believes Y'. Legacy
ledger lines without a name derive one from the statement on read.
graph-view: labels use the name; the detail panel shows name as header
and statement as body.
Compared the port against the original pi setup
(vault/π/learn/.pi: quiz.ts, ask-user-question.ts, teach SKILL.md) —
the pi version never produced invalid correctAnswer values. Root cause
of the port's validation failures: the pi options schema's 'Give each
option a stable value; you reference the correct one by that value in
correctAnswer' sentence was dropped in the port. Without it the model
omits values and must copy a long natural-language label verbatim into
correctAnswer — and instead slugifies it ('cultural-diagnosis' for
'A cultural diagnosis') every turn.
Now matched to pi:
- quiz schema: pi's field order (question, details, options,
multiSelect, correctAnswer, explanation) and descriptions — options
carry the stable-value instruction, correctAnswer is phrased as a
value reference ('the value field of the option you intend'), not a
copy-exact-or-fail threat. nodeId (lh-only, graph) stays.
- ask schema: dropped the redundant 'mode' param (pi has none; the dock
already infers from options presence), pi's descriptions.
- quiz tool description: pi's full text (ask contrast, note field,
dontKnow semantics) minus terminal-only details, plus lh's turn
protocol sentence.
- system-prompt: fixed the stale 'exactly two tools' line (plan/research/
demo/visual tools have their own sections).
Not matched, deliberately: tool results stay JSON (pi sends curated
text; formatAnswerMessage exists if we want it later), shuffle param
(lh always shuffles seeded), pi's ask '(Recommended)' label hint.
In Next dev (Turbopack), each route compiles as its own entry — a
module-level Map in turn-liveness.ts was a DIFFERENT instance in the page
render, GET /api/chats/[id], and POST /api/chat. The chat route held the
live turn (hence the 409), but the page render read an empty registry:
initiallyInFlight=false → mount continuation fired a send → 409 → the
'Something went wrong' banner on returning mid-turn. The registry now
lives on globalThis (the standard Next singleton channel) — one Map per
process regardless of module instances. Semantics unchanged (TTL leak
guard, begin/end round trip).
Browser-verified end-to-end (headless tester): answer quiz → navigate
away → return mid-turn → graded card (no re-prompt), 'Thinking…' shimmer,
composer gated, no error banner; teacher response auto-delivered when the
background turn finished. Polling GET /api/chats/[id] observed working.
The bug: answering a quiz and navigating away mid-turn, then returning,
re-prompted the same question. Root causes: (1) the answer was durable only
at onFinish — every mid-turn view read the stale pre-answer record;
(2) the server had no notion of a turn in flight; (3) remount treated the
stale record as current, with no re-attach path. Worse, the stale dock let
the quiz be answered again, racing two turns into a last-writer-wins
saveMessages clobber.
Fix (four pieces):
- Write-ahead persist in chat-handler: the request body IS the client's
canonical state — saved to disk BEFORE the stream starts. The answer is
durable the instant the request arrives; crashes recover via the existing
mount continuation.
- Turn liveness registry (src/server/turn-liveness.ts): in-memory chatId →
in-flight map with a TTL leak guard; beginTurn rejects concurrent turns
(409) so one turn ever owns a chat. endTurn runs in onFinish AFTER
persistTurn, so pollers that see not-in-flight always read a finished
record; a turn-start failure clears the registry (no lockout).
- GET /api/chats/[id] reports turnInFlight; the chat page passes
initiallyInFlight to the client.
- Client re-attach (components/chat.tsx): while in flight, busy shimmer +
composer gated + mount continuation suppressed; poll /api/chats/[id] until
the turn finishes, then syncFromServer adopts the completed transcript.
Stop stays local-stream only (a server turn has no client request to
abort).
Verified by scripted-model tests: concurrent turn 409 + registry lifecycle,
write-ahead durability mid-turn, failed-turn lockout guard.
MODELS in config.ts is now THE single tuning table: model id + OpenRouter
reasoning effort for teacher/researcher/demoBuilder/visualMaker/extractor/
auditor. .env.local is secrets-only (LH_* model vars removed; all sub-agents
previously had NO reasoning setting -> provider-default unbounded thinking,
the extraction token burn). One factory (src/agent/models.ts modelFor) wires
all six call sites; test injection unchanged. EXTRACTION.everyNQuizAnswers
moved to a config literal too. Plus the §5 one-line-per-seam logging in the
extraction pipeline: pipeline start, extractor/auditor start + per-step,
audit kept, merged — previously silent until completion, which made live
runs look like they never fired.
quizInputSchema now enforces (via pure quizInputProblems + superRefine) that
correctAnswer resolves against option values, options are non-empty and
unique, and multi-answer requires multiSelect — schema-valid now implies
gradeable. The AI SDK already returns invalid tool input to the model as a
tool-error part (verified in ai@7 run loop), so the teacher retries in the
same turn. chat-handler onError passes tool-input validation messages
through to the client (masked 'An error occurred.' for everything else).
Covers the Gate 3 GLM failure: correctAnswer slug matching no option.
- Commands: add npm run lint (with the import-law note)
- Architecture: one-sentence orientation before the mermaid; split the
turn composition (src/server: chat-handler) out of the teacher-agent
subgraph — the diagram now matches the layers
- Layout: add the server route entry points + the server barrel
- Header: quiz outcomes reach the graph through the extraction pipeline
Replaces the pre-pivot v0 scaffold doc (preserved in git history) with
the as-built spec: the five layers and their dependency directions, the
machine-enforced import law (four zones + the two documented-only
rules), ports/adapters and the two composition roots, the extraction
pipeline as sole graph writer (division of trust, merger invariants,
probe-attribution resolution, the v1 trigger seam), the pure read
channel, the turn seams, minimal observability, the testing strata, and
the deferred-with-links list. The doc states it describes the system,
not an intention — it changes with the commit that changes the system.
README reference updated (was: 'pre-pivot v0 scaffold — history only').
The knowledge graph's write channel is now the pivot's target
architecture (vision §1 construction):
- src/agent/subagents/extraction.ts — the pipeline factory
(createExtractionPipeline({ chatStore, graphStore, extractor?,
auditor? }); models env LH_EXTRACT_MODEL ?? LH_MODEL):
* transcriptView(): the compact view both LLM stages see — final plan
tree with node ids, conversation verbatim, QUIZ lines with outcomes,
mark_done lines; job tool parts are one-liners, plan ops fold into
the tree line.
* Extractor: transcript → SessionFindings (propositions incl.
misconceptions and prose-only supports edges, attribution for
UNTAGGED probes, all with verbatim evidence) via one zod-validated
submit_findings call.
* Auditor: transcript + findings → the verified subset + drops (the
faithfulness net) via submit_review.
- src/core/graph.ts — emitChat/foldStaleChats/replayFinalTree retire;
mergeFindings() replaces them as the pipeline's deterministic apply
stage: exact/alias identity resolution (one mapping line per (chat,
graph node)), edge dedupe, and event rebuild FROM THE TRANSCRIPT's tool
parts — kinds/timestamps/refs/notes deterministic, the LLM's only event
say being WHICH proposition an untagged probe tested (retroactive —
the vision's open question #3, resolved structurally: tagged events
resolve deterministically; attribution rescues untagged and
tagged-but-unmapped ones; unattributable land null + question text).
pendingQuizAnswers() is the trigger helper.
- Trigger (turn-service.ts triggerExtraction, replacing the old
foldTurnIntoGraph body): once a chat accumulates
EXTRACTION.everyNQuizAnswers (LH_EXTRACT_N, default 3) new answered
quizzes, fire the pipeline async — single-flight per chat, failure
checkpoints nothing and re-runs on the next trigger. Deliberately a v1
placeholder isolated in ONE seam, trigger-agnostic pipeline.
- GET /api/graph is now a pure read (foldStaleChats gone).
- NodeMapping.treeNodeId → sourceId (the mapping key is now the writer's
local proposition id — tree node when tree-derived); store strict-read
schema updated. Local ledger data regenerates through the pipeline.
- Tests: graph.test.ts rewritten around mergeFindings (identity,
attribution paths incl. ghost-rescue, idempotency, cross-session
provenance, trigger counting), graph-context.test.ts merge section
converted to findings-driven, NEW extraction.test.ts — transcriptView
units + the full pipeline with mock models (audit drops, failure
checkpoints nothing, single-flight). 195/195, tsc/eslint/build clean.
Probe attribution (vision open question): probes attach retroactively by
the extractor against the whole transcript; a pre-tree probe maps to a
proposition the tree only articulates later. Fuzzy/embedding merge
adjudication stays deferred (conservative exact/alias only).
The chat turn is now composed, not implemented, by chat-handler.ts:
- src/server/turn-service.ts: one named, error-contained seam per turn
responsibility — buildTeacherContext (W8 read channel), persistTurn
(canonical state write), spawnTurnJobs (async protocol kick-off),
foldTurnIntoGraph (session→graph consolidation; the extraction-pipeline
stage swaps this body behind the same seam). Each takes its deps
explicitly; failures log one [chat <id>] line and never touch the
response.
- chat-handler.ts wires: storage ports (stores), the job runner, the
teacher tools (ctx = chatId + injected ChatStore), and the services.
It no longer contains any logic beyond request validation.
Agent layer now receives dependencies instead of importing them — a new
lint zone enforces it (src/agent never imports src/server or src/store;
verified to fire both ways):
- runner.ts: createJobRunner({ chatStore, jobStore }) factory returning
the spawn hook + the three job runners; the model-override test seam
stays module-level. Bodies otherwise verbatim.
- plan tools: persist through the ctx-injected ChatStore
(TeacherToolContext = { chatId, chatStore }).
- teacher-agent's dependency-free default registry is now the true
client-side set: questionTools + jobTools (jobTools was ctx-parameter-
shaped but never used its ctx — now honestly context-free). Production
always passes createTeacherTools({ chatId, chatStore }); tests that
drive a bare mock model get the client-side set unchanged.
Zero behavior change: tsc clean, 190/190 tests, eslint clean, production
build clean. Tests inject the same fs ports the chat-handler does.
Per-entity semantic ports (src/core/ports.ts), one adapter family
(src/store/*, fs JSON/JSONL), exactly one composition point
(src/server/stores.ts). Dependency direction is now machine-checked:
adapters implement core interfaces; core never imports an adapter.
The inversion:
- core/graph.ts (emitChat/getGraph/graphSnapshot/foldStaleChats) is now
pure — it takes the storage ports as explicit parameters (GraphDeps).
No store imports; newGraphNodeId is local; node id format unchanged.
- core/graph-context.ts buildGraphContext() no longer defaults to the
live ledgers: the chat-handler hands in getGraph(graphStore) per turn.
- store adapters export exactly one object each implementing their port
(chatStore/graphStore/jobStore); bodies otherwise verbatim.
NodeMapping + SessionCheckpoint moved store/graph -> core/contracts
(they ARE contract shapes); CreateJobInit/JobPatch live in ports.
- titleFromPrompt moved store/chats -> core/contracts (pure helper,
planNodeStatus precedent). Store modules now contain persistence only.
- server/stores.ts is the composition root; server/{chats,graph,jobs}.ts
and chat-handler wire from it; agent (plan tools, sub-agent runner)
call the port objects. The shell surface (@/src/server exports) is
unchanged.
Lint law extended: the whole of src/core is now a purity zone (was a
file glob with a graph/graph-context exemption). Verified it fires on
core->store imports; tsc clean, 190/190 tests, eslint clean, prod build
clean (prerender 6/6). Tests pass deps explicitly — graph.test uses the
same fs-adapter pair stores.ts wires in production; graph-context.test
keeps its in-memory ledgers as a GraphStore implementation.
The shell now imports through exactly two barrels, enforced by
import/no-restricted-paths (npm run lint):
components/** -> "@/src/core" (pure core API, browser-safe)
app/** -> "@/src/core" + "@/src/server" (server entries)
Zone violations are lint errors with architecture-pointer messages; the
pure-core modules (contracts, question-core, plan-tree, phases, pending,
math-delims, ui-message) may never import agent/server/store.
Supporting moves (direction: IO layers -> core):
- core/index.ts: the core public API barrel
- core/ui-message.ts: moved from src/agent (it was shell-facing all along)
- quiz/ask input zod schemas moved agent/tools -> core/question-core
(the question contract, alongside the grading logic)
- ChatRecord/ChatSummary moved store/chats -> core/contracts (single schema)
- GraphSnapshot/NodeStateClass moved core/graph -> core/contracts (the
/api/graph payload IS a contract; the pure renderer needs only these)
- src/server/{chats,graph,jobs}.ts + index.ts: thin server entries for the
transport layer; GET /api/graph's fold side-effect now lives behind
graphView() instead of inline in the route
Gates: tsc clean, 190/190 tests, eslint clean (law verified to fire on
store/agent/subpath violations), production build clean.
(Repo repair en route: a git object (app/chat/[id] tree) was missing from
the object store — reconstructed byte-exactly via mktree from its blob;
the broken node_modules that stalled the build was cleanly reinstalled.)
Delete src/extract/ (pi-session normalize/extract/merge stubs), src/ledger.ts,
src/schema.ts, src/derive.ts, src/queries/, src/cli.ts (npm run ingest/graph/
what-next), data/legacy-graph/, data/agent/ (pi agentDir vestige), and their
config.ts/package.json references.
The retroactive-bootstrap goal is dropped: the live chat store (AI SDK
UIMessages) is the only session format now, and the old pi JSONLs in
data/sessions/ stay on disk, inert. contracts.ts is the single schema.
The old README documented the pre-pivot pi/Fastify/Vite scaffold (npm run
teach, doctor scripts, Fastify+WS server, extensions/) — all removed in the
pivot to standalone Next.js + Vercel AI SDK. New README covers the current
commands, env vars, embedded architecture diagram, key mechanisms
(client-side question tools, plan tree, async sub-agent protocol, live
graph emission), the dormant legacy scaffold, testing, and lineage.
Also commits the pending next-env.d.ts regeneration.
Gate 1 found the teacher staying silent after a research brief landed.
The SDK only evaluates sendAutomaticallyWhen inside addToolOutput — an
external setMessages (our syncFromServer) updates the UI but never
triggers the continuation. Fix: syncFromServer checks
lastAssistantMessageIsCompleteWithToolCalls and calls sendMessage()
(itself a valid continuation trigger); plus a mount-continuation effect
for the reload case (job fulfilled while no client was open).
- src/core/math-delims.ts: pure preprocessing for the \(...\)/\[...\] math
contract — balanced delimiters rewrite to the two-dollar forms remark-math
parses ($ is never math). Skips code fences/spans, escaped delimiters, and
leaves unbalanced (mid-stream) delimiters untouched. Display math renders
as a block: mid-paragraph gets blank-line padding; indented (list items)
degrades to inline so list continuations don't half-parse.
- message.tsx: transform in the render path (children-memoized); math plugin
now singleDollarTextMath: false — the currency hazard dies structurally.
remark-breaks via the cjk remarkPluginsAfter slot (the bare remarkPlugins
prop would replace the [remark-gfm, codeMeta] defaults).
- mermaid: dark theme + sans font via Streamdown's mermaid prop; dark-styled
error fallback (components/mermaid-error.tsx) with error, source in
<details>, and retry.
- globals.css (appended): @source for streamdown dist — its internal UI
chrome (code blocks, mermaid containers, tables) was entirely unstyled
because Tailwind v4 never scans node_modules; --color-sidebar → card; wide
mermaid SVGs capped to the column; .katex-display scrolls.
- tests/render-markdown.test.tsx: 26 tests (transform unit + render). The
math describe block in render-cards.test.tsx is updated to the new
contract ($...$ now renders as literal text — pinned).
- src/store/jobs.ts: per-job JSON store in data/jobs (create/get/update/list,
dedup by toolCallId)
- src/agent/tools/jobs.ts: research tool — client-side (no execute), turn ends
at the call; input per contracts ResearchToolInput
- src/agent/tools/web.ts: server-side web_search (Google CSE) + web_fetch
(Readability/Turndown/linkedom, Jina fallback) — errors return as strings,
never thrown into the agent loop. Deps: @mozilla/readability, linkedom,
turndown (+@types/turndown dev)
- src/agent/subagents/researcher.ts: isolated ToolLoopAgent (web tools only,
12-step cap), researcher prompt ported from pi
- src/agent/subagents/runner.ts: spawnPendingJobs hook + runResearchJob —
fire-and-forget, activity line per tool call, fulfills the part in the
LATEST chat record (output-available) before flipping job status; failure
still fulfills the part so the teacher learns the outcome
- src/server/chat-handler.ts: onFinish spawn hook after saveMessages
- app/api/jobs/route.ts: GET ?chatId= polling endpoint
- components/artifacts/job-card.tsx + registry entry: polling activity card
(2s) → syncFromServer on finish; research-brief card (markdown + sources)
- src/agent/prompts/20-jobs.md: research doctrine for the teacher
- tests/jobs.test.ts: store CRUD, spawn hook via scripted mock teacher,
runner completion/failure with fake model + stubbed fetch
- src/core/plan-tree.ts: pure tree ops (createTree/addNode/updateNode/removeNode/
addEdge/removeEdge/setCurrent/markDone), DAG invariants (unique ids, existing
endpoints, no self-loops/duplicates/cycles), cascade remove, version bumps on
state changes only (true no-ops for idempotent setCurrent/markDone).
- src/agent/tools/plan.ts: plan_init (refuses re-init), plan_add_node (optional
parents → requires edges), plan_update_node, plan_remove_node, plan_add_edge,
plan_remove_edge, plan_set_current, plan_mark_done. Execute = getChat → op →
saveTree → PlanToolResult with the full updated tree; PlanTreeError and missing
chats become {ok:false, error, tree}.
- src/agent/prompts/10-plan-tree.md: tree-authoring doctrine (when: post-probe;
grain: one reasoning step per node; discipline: set_current on start,
mark_done on demonstrated mastery; freedom: mid-session restructure).
- tests/plan-tree.test.ts: every validation error path, version semantics,
cascade remove, no-mutation, tool round-trips against the real chat store.
- Every session starts with the learner's knowledge graph as base
context (precision probing); selection problem noted: pedagogical
relevance != semantic closeness (structural neighborhood, diagnostic
hotspots, frontier strip); strategies to tune on data.
- Sub-agent architecture: async background workers with live activity
indicators; demo-builder TESTS code headlessly; visual-maker verifies
by looking (port of vault svg-maker); mermaid needs no sub-agent;
researcher tool for fact verification (port of pi web-search/web-fetch
+ interactive-subagents researcher prompt).
- Not-a-chat presentation principle marked LOW PRIORITY per user.
amosblomqvist