With a steer coalesced into a quiz-answer continuation, the steer sits AFTER
the assistant message being continued, so the SDK's onFinish list carries
the input copy AND the extended response copy as two entries with the SAME
id — persistTurn saved both, the jobs poller's syncFromServer imported the
dup, and React keyed a single message twice (the user's live test hit it).
withMessageIds now dedupes by id keeping the LAST occurrence — the SDK's
final merge state (a superset of the earlier copy) — which also yields the
right semantic shape: [user, steer, assistant(extended)].
Out-of-lane surgical touch (flagged for master review): turn-service.ts.
Squashed the live session's already-corrupted record by hand (same dedupe).
The bug (seen live): a teacher that ends every turn with a new pending quiz
keeps the wake pipeline's idle gates permanently closed — the steer sits in
'delivering result' forever while the learner answers check after check.
- claimSteers (server/jobs.ts): the chat-handler claims every queued,
undelivered steer the instant a client turn wins the turn registry, and
appends it after the learner's answer — conversion-safe, the answered
quiz carries its output. Every answer flushes pending steers immediately.
- The pump keeps its post-idle duplicate-guard (re-check steerDelivered
before firing), so a claim racing the pump converges to exactly one
delivery either way.
- The stale-job sweep now skips chats with a turn in flight (a healthy but
slow wake no longer gets falsely re-queued as 'recovering').
- Out-of-lane surgical touch (flagged for master review): the 3-line
handleChatRequest seam (claim after beginTurn, persist+stream 'outgoing').
- Verified live end-to-end on the stuck session: answering the pending quiz
fired the steer + wake turn within a second; job flipped done/delivered.
- components/chat-transport.ts (NEW): createChatTransport(chatId) — the
chat.tsx seam is now a one-line swap. Its fetch turns a 409 on POST
/api/chat (a learner send racing the wake pipeline's ~1s adoption window)
into: wait out the in-flight turn (poll turnInFlight, 30s cap), refetch
the record, MERGE the outgoing messages onto the fresh server list, re-POST.
A bare retry would clobber the steer + wake messages via the write-ahead
persist (steerDelivered makes that loss permanent) — merging is the whole
point. Unit-tested: pass-through, wait+merge+reorder, dedupe by id,
repeated-409 loop, bounded give-up.
- src/core/pending.ts: findPendingJobCall deleted (dead since wave 1 —
job tools resolve server-side with an ack, nothing pends client-side).
JOB_TOOLS kept (runner.ts still uses it). Header doc rewritten.
- src/server/jobs.ts: GRACE_MS 1500 → 750, with the evidence in the
comment: the race window is only the learner's answer→POST flight
(sendAutomaticallyWhen dispatches synchronously; ~10ms localhost,
<300ms realistic RTT — 750 is 2.5× worst case) and misses self-heal
via the pump's gate recheck, the 409 backoff ramp, and the new
transport merge-retry. POLL_MS=400 kept (sampling granularity only).
- tests/jobs.test.ts: prompt-size guard — every fragment ≤8k chars,
composed prompt ≤60k — the README-leak class (815 bytes ≈ 200 tokens
per teacher turn before wave 1's fix) can't silently recur.
Gates: typecheck, 279/279 tests, lint, build — all green.
The pi-style ack protocol (previous commit) left the client describing the
old world: artifact cards read the ack string as a result ("could not be
displayed"), the only job poller rendered on input-available parts that no
longer exist (so wake turns landed invisibly until re-navigation), steers
rendered as learner bubbles with no SVG anywhere, and server-persisted
assistant messages carried empty SDK ids (duplicate React keys).
- persistTurn (turn-service): fill empty message ids at the canonical write
— server-only messages (last turn's, wake turns') now key uniquely;
client-authored ids untouched.
- chat.tsx: ONE central /api/jobs poll (fast while jobs run, slow discovery
cadence, off for job-less sessions) drives every job surface, re-syncs
once when a job settles (the record already holds the steer + wake
response at flip time), and adopts server-side wake turns via the same
turnInFlight machinery as returning mid-turn. Steers branch to the
artifact card — never a bubble. Composer no longer gates on jobs.
- /api/jobs now reports turnInFlight alongside the job list.
- dispatch-card: the tool-call box — live activity by toolCallId while
running, quiet "delivered"/"failed" one-liner after; the ack is visible
observability, the artifact lives in the steer.
- steer-card: renders the delivered artifact by kind (research brief /
sanitized SVG / sandboxed demo iframe) from the data-subagent-artifact
part; result renderers extracted and exported from the artifact cards.
- sub-agents-panel: right-sidebar section above the plan tree listing
running sub-agents (kind, elapsed, live activity) — vanishes when idle.
- tests: dispatch/steer/panel render tests re-target the new protocol;
persistTurn id-hygiene suite. 229/229 green, build green.
Out-of-lane touches for master review: chat.tsx + components/** (cleared
this wave), app/api/jobs/route.ts, src/server/turn-service.ts, contracts
protocol doc §2. Live-verified on :3003 (full loop, SSR DOM checked).
The job tools (research/demo/visual) switch from client-side turn-breaking
calls to pi's model: execute resolves IMMEDIATELY with a fire-and-forget
ack, the teacher's turn continues, and the result arrives later as an
injected user-role steer message that wakes the teacher in a new
server-side turn.
- contracts.ts: THE ASYNC PROTOCOL rewritten (ack now, steer later); steer
types (SubagentSteerMessage, metadata.subagentResult + data-subagent-
artifact part), pure buildSubagentSteer/steerDelivered/isSubagentFailure
- tools/jobs.ts: jobTools(ctx) factory — server-side execute creates the
job (deduped by toolCallId), fires spawn with the dispatch-time input
(the part isn't persisted yet at execute — a transcript scan would race)
- runner.ts: runJob(jobId, input?) builds the steer and hands it to the
injected deliver seam; pipeline owns the done/error flip (a polling
client re-syncs once, at the end); failures still steer
- server/jobs.ts (new pipeline): per-chat serialized queues on globalThis,
idle gates (turn liveness + pending question, 1.5s grace), 409 retry
with backoff, idempotent per jobId, stale-job recovery sweep (re-deliver
or re-spawn); wake turns run through the same handleChatRequest
composition via setTurnRunner injection
- chat-handler.ts: wires runner+deliver+tools ctx; drops spawnTurnJobs
from onFinish; teacher default registry = question tools only
- plan.ts: PlanToolContext split out (plan tools need no job wiring)
- visual-maker: edit_svg added (pi parity — exact-match single
replacement, fuzzy context on a miss); researcher: pi's kept/dropped
source accounting restored
- prompts 20-jobs/40-demos: async discipline rewritten (announce and keep
teaching, no waiting/polling/fabricating, parallel dispatch, teach with
the artifact when it arrives)
- tests: jobs.test.ts rewritten (dispatch ack mid-turn, runner steers,
wake pipeline: deferral/409/idempotency/failure), demos.test.tsx runner
tests ported to runJob + fake deliver; 222 green
FLAGGED surgical touches outside the stream's owned files, for master
review at merge: src/core/contracts.ts (protocol doc + steer types),
src/core/ui-message.ts (job tool + data-part types), src/server/
chat-handler.ts (wiring), src/server/turn-service.ts (spawnTurnJobs
retirement), src/agent/teacher-agent.ts (default tools), src/agent/tools/
plan.ts (context type split).
Verified live on PORT=3003 with real models: teacher dispatched research
mid-turn, researcher ran real searches/fetches, steer injected on quiz
answer, wake turn wove the brief in citing its sources, job flipped done;
a client/wake race correctly 409'd.
In Next dev (Turbopack), each route compiles as its own entry — a
module-level Map in turn-liveness.ts was a DIFFERENT instance in the page
render, GET /api/chats/[id], and POST /api/chat. The chat route held the
live turn (hence the 409), but the page render read an empty registry:
initiallyInFlight=false → mount continuation fired a send → 409 → the
'Something went wrong' banner on returning mid-turn. The registry now
lives on globalThis (the standard Next singleton channel) — one Map per
process regardless of module instances. Semantics unchanged (TTL leak
guard, begin/end round trip).
Browser-verified end-to-end (headless tester): answer quiz → navigate
away → return mid-turn → graded card (no re-prompt), 'Thinking…' shimmer,
composer gated, no error banner; teacher response auto-delivered when the
background turn finished. Polling GET /api/chats/[id] observed working.
The bug: answering a quiz and navigating away mid-turn, then returning,
re-prompted the same question. Root causes: (1) the answer was durable only
at onFinish — every mid-turn view read the stale pre-answer record;
(2) the server had no notion of a turn in flight; (3) remount treated the
stale record as current, with no re-attach path. Worse, the stale dock let
the quiz be answered again, racing two turns into a last-writer-wins
saveMessages clobber.
Fix (four pieces):
- Write-ahead persist in chat-handler: the request body IS the client's
canonical state — saved to disk BEFORE the stream starts. The answer is
durable the instant the request arrives; crashes recover via the existing
mount continuation.
- Turn liveness registry (src/server/turn-liveness.ts): in-memory chatId →
in-flight map with a TTL leak guard; beginTurn rejects concurrent turns
(409) so one turn ever owns a chat. endTurn runs in onFinish AFTER
persistTurn, so pollers that see not-in-flight always read a finished
record; a turn-start failure clears the registry (no lockout).
- GET /api/chats/[id] reports turnInFlight; the chat page passes
initiallyInFlight to the client.
- Client re-attach (components/chat.tsx): while in flight, busy shimmer +
composer gated + mount continuation suppressed; poll /api/chats/[id] until
the turn finishes, then syncFromServer adopts the completed transcript.
Stop stays local-stream only (a server turn has no client request to
abort).
Verified by scripted-model tests: concurrent turn 409 + registry lifecycle,
write-ahead durability mid-turn, failed-turn lockout guard.
quizInputSchema now enforces (via pure quizInputProblems + superRefine) that
correctAnswer resolves against option values, options are non-empty and
unique, and multi-answer requires multiSelect — schema-valid now implies
gradeable. The AI SDK already returns invalid tool input to the model as a
tool-error part (verified in ai@7 run loop), so the teacher retries in the
same turn. chat-handler onError passes tool-input validation messages
through to the client (masked 'An error occurred.' for everything else).
Covers the Gate 3 GLM failure: correctAnswer slug matching no option.
The knowledge graph's write channel is now the pivot's target
architecture (vision §1 construction):
- src/agent/subagents/extraction.ts — the pipeline factory
(createExtractionPipeline({ chatStore, graphStore, extractor?,
auditor? }); models env LH_EXTRACT_MODEL ?? LH_MODEL):
* transcriptView(): the compact view both LLM stages see — final plan
tree with node ids, conversation verbatim, QUIZ lines with outcomes,
mark_done lines; job tool parts are one-liners, plan ops fold into
the tree line.
* Extractor: transcript → SessionFindings (propositions incl.
misconceptions and prose-only supports edges, attribution for
UNTAGGED probes, all with verbatim evidence) via one zod-validated
submit_findings call.
* Auditor: transcript + findings → the verified subset + drops (the
faithfulness net) via submit_review.
- src/core/graph.ts — emitChat/foldStaleChats/replayFinalTree retire;
mergeFindings() replaces them as the pipeline's deterministic apply
stage: exact/alias identity resolution (one mapping line per (chat,
graph node)), edge dedupe, and event rebuild FROM THE TRANSCRIPT's tool
parts — kinds/timestamps/refs/notes deterministic, the LLM's only event
say being WHICH proposition an untagged probe tested (retroactive —
the vision's open question #3, resolved structurally: tagged events
resolve deterministically; attribution rescues untagged and
tagged-but-unmapped ones; unattributable land null + question text).
pendingQuizAnswers() is the trigger helper.
- Trigger (turn-service.ts triggerExtraction, replacing the old
foldTurnIntoGraph body): once a chat accumulates
EXTRACTION.everyNQuizAnswers (LH_EXTRACT_N, default 3) new answered
quizzes, fire the pipeline async — single-flight per chat, failure
checkpoints nothing and re-runs on the next trigger. Deliberately a v1
placeholder isolated in ONE seam, trigger-agnostic pipeline.
- GET /api/graph is now a pure read (foldStaleChats gone).
- NodeMapping.treeNodeId → sourceId (the mapping key is now the writer's
local proposition id — tree node when tree-derived); store strict-read
schema updated. Local ledger data regenerates through the pipeline.
- Tests: graph.test.ts rewritten around mergeFindings (identity,
attribution paths incl. ghost-rescue, idempotency, cross-session
provenance, trigger counting), graph-context.test.ts merge section
converted to findings-driven, NEW extraction.test.ts — transcriptView
units + the full pipeline with mock models (audit drops, failure
checkpoints nothing, single-flight). 195/195, tsc/eslint/build clean.
Probe attribution (vision open question): probes attach retroactively by
the extractor against the whole transcript; a pre-tree probe maps to a
proposition the tree only articulates later. Fuzzy/embedding merge
adjudication stays deferred (conservative exact/alias only).
The chat turn is now composed, not implemented, by chat-handler.ts:
- src/server/turn-service.ts: one named, error-contained seam per turn
responsibility — buildTeacherContext (W8 read channel), persistTurn
(canonical state write), spawnTurnJobs (async protocol kick-off),
foldTurnIntoGraph (session→graph consolidation; the extraction-pipeline
stage swaps this body behind the same seam). Each takes its deps
explicitly; failures log one [chat <id>] line and never touch the
response.
- chat-handler.ts wires: storage ports (stores), the job runner, the
teacher tools (ctx = chatId + injected ChatStore), and the services.
It no longer contains any logic beyond request validation.
Agent layer now receives dependencies instead of importing them — a new
lint zone enforces it (src/agent never imports src/server or src/store;
verified to fire both ways):
- runner.ts: createJobRunner({ chatStore, jobStore }) factory returning
the spawn hook + the three job runners; the model-override test seam
stays module-level. Bodies otherwise verbatim.
- plan tools: persist through the ctx-injected ChatStore
(TeacherToolContext = { chatId, chatStore }).
- teacher-agent's dependency-free default registry is now the true
client-side set: questionTools + jobTools (jobTools was ctx-parameter-
shaped but never used its ctx — now honestly context-free). Production
always passes createTeacherTools({ chatId, chatStore }); tests that
drive a bare mock model get the client-side set unchanged.
Zero behavior change: tsc clean, 190/190 tests, eslint clean, production
build clean. Tests inject the same fs ports the chat-handler does.
Per-entity semantic ports (src/core/ports.ts), one adapter family
(src/store/*, fs JSON/JSONL), exactly one composition point
(src/server/stores.ts). Dependency direction is now machine-checked:
adapters implement core interfaces; core never imports an adapter.
The inversion:
- core/graph.ts (emitChat/getGraph/graphSnapshot/foldStaleChats) is now
pure — it takes the storage ports as explicit parameters (GraphDeps).
No store imports; newGraphNodeId is local; node id format unchanged.
- core/graph-context.ts buildGraphContext() no longer defaults to the
live ledgers: the chat-handler hands in getGraph(graphStore) per turn.
- store adapters export exactly one object each implementing their port
(chatStore/graphStore/jobStore); bodies otherwise verbatim.
NodeMapping + SessionCheckpoint moved store/graph -> core/contracts
(they ARE contract shapes); CreateJobInit/JobPatch live in ports.
- titleFromPrompt moved store/chats -> core/contracts (pure helper,
planNodeStatus precedent). Store modules now contain persistence only.
- server/stores.ts is the composition root; server/{chats,graph,jobs}.ts
and chat-handler wire from it; agent (plan tools, sub-agent runner)
call the port objects. The shell surface (@/src/server exports) is
unchanged.
Lint law extended: the whole of src/core is now a purity zone (was a
file glob with a graph/graph-context exemption). Verified it fires on
core->store imports; tsc clean, 190/190 tests, eslint clean, prod build
clean (prerender 6/6). Tests pass deps explicitly — graph.test uses the
same fs-adapter pair stores.ts wires in production; graph-context.test
keeps its in-memory ledgers as a GraphStore implementation.
The shell now imports through exactly two barrels, enforced by
import/no-restricted-paths (npm run lint):
components/** -> "@/src/core" (pure core API, browser-safe)
app/** -> "@/src/core" + "@/src/server" (server entries)
Zone violations are lint errors with architecture-pointer messages; the
pure-core modules (contracts, question-core, plan-tree, phases, pending,
math-delims, ui-message) may never import agent/server/store.
Supporting moves (direction: IO layers -> core):
- core/index.ts: the core public API barrel
- core/ui-message.ts: moved from src/agent (it was shell-facing all along)
- quiz/ask input zod schemas moved agent/tools -> core/question-core
(the question contract, alongside the grading logic)
- ChatRecord/ChatSummary moved store/chats -> core/contracts (single schema)
- GraphSnapshot/NodeStateClass moved core/graph -> core/contracts (the
/api/graph payload IS a contract; the pure renderer needs only these)
- src/server/{chats,graph,jobs}.ts + index.ts: thin server entries for the
transport layer; GET /api/graph's fold side-effect now lives behind
graphView() instead of inline in the route
Gates: tsc clean, 190/190 tests, eslint clean (law verified to fire on
store/agent/subpath violations), production build clean.
(Repo repair en route: a git object (app/chat/[id] tree) was missing from
the object store — reconstructed byte-exactly via mktree from its blob;
the broken node_modules that stalled the build was cleanly reinstalled.)
- src/store/jobs.ts: per-job JSON store in data/jobs (create/get/update/list,
dedup by toolCallId)
- src/agent/tools/jobs.ts: research tool — client-side (no execute), turn ends
at the call; input per contracts ResearchToolInput
- src/agent/tools/web.ts: server-side web_search (Google CSE) + web_fetch
(Readability/Turndown/linkedom, Jina fallback) — errors return as strings,
never thrown into the agent loop. Deps: @mozilla/readability, linkedom,
turndown (+@types/turndown dev)
- src/agent/subagents/researcher.ts: isolated ToolLoopAgent (web tools only,
12-step cap), researcher prompt ported from pi
- src/agent/subagents/runner.ts: spawnPendingJobs hook + runResearchJob —
fire-and-forget, activity line per tool call, fulfills the part in the
LATEST chat record (output-available) before flipping job status; failure
still fulfills the part so the teacher learns the outcome
- src/server/chat-handler.ts: onFinish spawn hook after saveMessages
- app/api/jobs/route.ts: GET ?chatId= polling endpoint
- components/artifacts/job-card.tsx + registry entry: polling activity card
(2s) → syncFromServer on finish; research-brief card (markdown + sources)
- src/agent/prompts/20-jobs.md: research doctrine for the teacher
- tests/jobs.test.ts: store CRUD, spawn hook via scripted mock teacher,
runner completion/failure with fake model + stubbed fetch
The teaching agent no longer runs on pi. Stack:
- ToolLoopAgent (ai@7) + OpenRouter (OPENROUTER_API_KEY), model via LH_MODEL,
default anthropic/claude-sonnet-4.5 with reasoning tokens
- quiz/ask_user_question = client-side tools (no server execute): the call
lands as a typed tool part in input-available state; the card grades via
src/core/question-core.ts; addToolOutput sends the verdict; the teacher's
reaction continues the same assistant message
- resume-mid-quiz is free: the pending tool part persists in the transcript
- persistence: one JSON per chat (data/chats/), full UIMessage transcripts,
saved onFinish (also on abort)
- UI: AI Elements (owned copy-paste components, pruned to the five in use)
+ button-based QuizCard/AnsweredCard (ask answers never graded)
- pi fully removed: TUI, extensions, Fastify+WS server, bridge, custom
protocol all deleted (recoverable from git history)
- tests: question-core units carry over; chat-flow integration drives
handleChatRequest with MockLanguageModelV4 (48 tests, zero tokens);
render smoke tests guard the cards (renderToStaticMarkup)
Carried over unchanged: teacher system prompt, question-core, schema/
ledger/derive/extract pipeline, data model.
Root causes of the v1 browser bugs (all fixed + covered by deterministic tests):
- duplicated messages: per-socket snapshot + global broadcast marking raced;
replaced with a single append-only entry log (exactly-once per socket,
structurally) — covered by tests/server.test.ts
- delayed/missing echo of own messages: optimistic append with sending/failed
status, reconciled by text when the real entry arrives
- no thinking shown: thinking deltas streamed live (kind: thinking) and
persisted in assistant entries so they survive reloads
- ask answers graded as wrong: AnsweredCard no longer applies quiz grading to
ask answers (no ✓/✗, no verdict)
- broken option selection: native radio/checkbox inputs fighting React state
replaced with button-based options (single source of truth, one-click select)
- no URLs: History API routing — / home, /s/<file> per session (deep links,
refresh, back button); session view resumes its file on mount
- lost connection = dead UI: exponential-backoff reconnect with full resync
- stick-to-bottom transcript with jump-to-latest pill
Server: startServer() with injectable runtime factory (scripted model in
tests); createSession returns file for routing; scripted model gained thinking
turns. 51 deterministic tests green.
amosblomqvist