diff --git a/PI-NATIVE-MIGRATION-PLAN.md b/PI-NATIVE-MIGRATION-PLAN.md deleted file mode 100644 index 57a9bd7..0000000 --- a/PI-NATIVE-MIGRATION-PLAN.md +++ /dev/null @@ -1,299 +0,0 @@ -# Pi-native architecture migration plan - -Recon wave 2026-09-26 · READ-ONLY deliverable · `feat/platform` · untracked until owner review -Evidence: wiretap `/tmp/lh-wire/`, 11 controlled OpenRouter replays (`/tmp/rp-*.json/.out`), pi-ai + pi-agent-core dist sources, pi's own session files under `~/.pi/agent/sessions/`. - ---- - -## 0. Executive summary - -- **Two banked facts are overturned by the wiretap evidence** (§1). The tool-stripping mystery was model confabulation — the registry was complete on the wire. The truncation is an upstream (OpenInference) bug triggered by multi-tool-result history volume, not by our serialized shape. -- Five custom seams mapped against pi-native equivalents (§2). Each has a native replacement; the getApiKey fix (76051d0) is the template: delete the custom seam, use the library path. -- What genuinely has no native equivalent is small and enumerable (§3): question-tool UX, tool toggles/phases, jobs, prompt overrides, our reattach/persistence infrastructure. -- Proposed risk order (§4) and test strategy (§5). Seams touching `src/agent` are flagged for Teacher's review per band rules. - ---- - -## 1. Corrections to banked facts (evidence first) - -### 1.1 TOOL-SET STRIPPING — SOLVED: the model confabulated; no seam varies the registry - -- Wiretap captures **7 and 9 are chat_muhb8l00's two exact turns** (timestamps 18:44:47 / 18:48:10 match the record created 18:44:41Z). **Both carried the full 11-tool registry** — `quiz, ask_user_question, read, ls, find, grep, research, pdf, list_sessions, list_propositions, probe_done` — including the graph tools the model claimed not to have. -- The response-side capture shows the same request streaming the "no graph tools in this session… the harness is stripped to the two question tools" answer with `finish_reason: stop`. The claim is false; z-ai/glm-5.3-flash (relayed via OpenRouter upstreams Relace/OpenInference) confabulated a toolset, likely trusting prose descriptions over the actual `tools` array. -- Census of every "missing tool" claim across all master chat records (`data/chats/*.json`): - - `chat_mtxsy1lq` (09-12), `chat_mucnl5j3`/`chat_mucnyjvu` (09-22): honest — probe-phase (no graph tools yet) and/or research correctly absent (no search key → `searchAvailable` false). - - `chat_mu73galx` (09-18): the closed "no sub-agents" scare (probe-phase prober, teacher tools are teacher-only). - - `chat_mufpa030` (09-24): model accurately enumerated its full toolkit; quiz worked. - - **"Another session reportedly lacked quiz entirely": not found in any record** (regex census over all chats). -- Verdict: `tools.ts` gating and `chat-handler`'s ctx passing work as designed. No `src/agent` change required for this mystery (flagged for Teacher anyway since tools.ts is in that band). Mitigation is model-facing (stronger system-prompt tool enumeration, or dropping flash for tool-heavy phases) — an owner decision, not a seam fix. - -### 1.2 HISTORY-SERIALIZATION TRUNCATION — REFINED: upstream bug on multi-tool-result histories; our shape is not the trigger - -11 controlled replays today against `z-ai/glm-5.3-flash` via OpenRouter (all served by the **OpenInference** upstream; the original truncating request `/tmp/replay.json` = wiretap capture 2, the ML session): - -| variant | change vs. truncating request | prompt_tokens | outcome | -|---|---|---|---| -| base | verbatim | 7006 | **TRUNC** (4 chunks, ~22 chars reasoning, billed 258 completion, false `finish: stop`) | -| A | assistant content cleaned (no glued thinking), parallel calls kept | 6803 | **TRUNC** | -| B | parallel calls split into two sequential assistant turns | 7017 | **TRUNC** | -| G | `reasoning` param removed | 7006 | **TRUNC** | -| H | `max_completion_tokens` 131072→8192 | 7006 | **TRUNC** | -| D | both tool results shortened to ~30 chars | 5982 | works (11 tool calls) | -| P | two results at 100+100 chars | 6037 | works | -| L | two results at 300+300 chars | 6201 | **TRUNC** | -| K | two results at 700+700 chars | 6564 | **TRUNC** | -| J | big result (1564) + tiny (30) | 6658 | **TRUNC** | -| E | single result (1564) | 6626 | works | -| F | single result (802) | 6296 | works | -| C9 | capture 9 verbatim (the request that streamed the confabulation live) | 6146 | works (332 chars) | -| I | single result padded to 2764 chars | 7347 | works (4 tool calls) | - -Readings: -- **Not the glued-thinking content** (A truncates clean), **not parallelism** (B truncates sequential), **not prompt size** (I streams 7347 tokens; L truncates at 6201), **not the reasoning param or completion cap** (G, H). -- **Trigger: ≥2 tool-result messages with combined content ≥ ~200–600 chars** (P works at 200, L truncates at 600, J truncates with 1564+30). A single tool result of any tested size (≤2764 chars) streams fine. Direct Anthropic/OpenAI and non-flash GLM tolerate everything. -- The failure mode is consistent: a sliver of reasoning streams, 150–260 completion tokens are billed, `finish_reason: stop` arrives with ~zero content — a silent upstream truncation, stable and deterministic 4+ hours after the live capture. -- **Consequence for the plan:** pi-native serialization would NOT have prevented this — tool results are byte-identical on the wire either way. The live bug needs provider-side action (report to OpenRouter; or pin/avoid the flaky upstream for flash; short-term: route flash traffic elsewhere). The migration below is still correct, for the reasons in §1.3 — but it is not the truncation fix, and saying otherwise would be false comfort. - -### 1.3 What IS wrong with our serialization (the honest indictment of record-adapter) - -- `src/server/pi/record-adapter.ts:161-163` stamps every replayed assistant message `provider: "openrouter", model: "replayed"` → `transformMessages` (`pi-ai/dist/api/transform-messages.js:66`) sees `isSameModel === false` for **every** replayed turn, forever. -- Cross-model path converts thinking blocks to plain text (`transform-messages.js:78-81`), which `openai-completions.js` then glues into the assistant content string (no separator — capture 2's content is literally `"…question.There's already…"`). -- `thinkingSignature` cannot survive the UIMessage round-trip (records persist text only), so replayed turns never restore `reasoning_details`. pi's own sessions do: every assistant line in `~/.pi/agent/sessions/**/*.jsonl` carries full `provider/model/api/usage/stopReason` + `thinkingSignature`, which is why pi's replay stays same-model and signature-faithful. -- Net: every history turn we send diverges from what pi itself would send — discarded reasoning state, cache-hostile, and any provider quirk about reasoning-in-content becomes ours to keep debugging. - ---- - -## 2. Seam-by-seam map: custom remnant → pi-native replacement - -### Seam 1 — Message history persistence & replay -- **Now:** `LHChatRecord` stores Vercel-SDK-shaped UIMessages (part states `input-available`/`output-available`, `data-*` UI parts); `src/server/pi/record-adapter.ts` (`recordToPiMessages`, 189 lines) hand-rebuilds pi-ai `Message[]` per turn; marker-only user messages flatten to canonical text; dangling tool calls get synthesized `INTERRUPTED` results; chat-handler rebuilds history on every POST. -- **Native:** pi persists append-only JSONL records of full pi-ai messages (verified in pi's own session files); `Agent` accepts `initialState.messages` (`pi-agent-core/dist/agent.js:21-45`); custom message types ride `CustomAgentMessages` declaration-merging (`types.d.ts:269-283`) and are filtered from the LLM view by the default `convertToLlm` (`agent.js:8-10`) or mapped in a `transformContext` hook (`agent-loop.js:179-183`). -- **Replace with:** store pi `AgentMessage[]` natively in the chat record (custom messages for plan blocks/artifacts/verdict markers instead of `data-*` parts); replay = `initialState.messages` from the store; dangling-call synthesis and marker flattening move into one `convertToLlm`/`transformContext` hook — one place, tested once. -- **Delete:** `record-adapter.ts` wholesale; the UIMessage part-state model as the history format (UI gets a transcript view model instead). -- **Kills:** the `model: "replayed"` stamp, thinking-glue, lost signatures (§1.3) — replayed turns become same-model, signature-faithful, cache-friendly. - -### Seam 2 — Tool definition wrapper -- **Now:** `src/agent/lh-tool.ts` (244 lines): `LhTool {name, description, io: zod, execute(args, ctx), clientQuestion}`; `toPiTool` bridges via `z.toJSONSchema`, re-runs zod validation inside `execute` so `superRefine` failures throw `ToolInputError`; `TOOL_TIMEOUT_MS` 30s + 250ms abort grace; `clientQuestion` → `executionMode: "sequential"`. -- **Native:** `AgentTool` (`pi-agent-core/dist/types.d.ts:340`): required `label`, TypeBox `parameters`, optional `prepareArguments` shim (raw-arg rescue — replaces our re-validation hack), `execute(toolCallId, params, signal, onUpdate)`, per-tool `executionMode`, `replay` policy. pi's own tools are `ToolDefinition`s (`pi-coding-agent/dist/core/extensions/types.d.ts:344`: `promptSnippet`, `promptGuidelines`, `renderCall`, `renderResult`) wrapped by `wrapToolDefinition` (`core/tools/tool-definition-wrapper.js`; reference impl `core/tools/read.js`: TypeBox `Type.Object` schema + snippet/guidelines). -- **Replace with:** TypeBox-schema ToolDefinitions; `prepareArguments` for arg rescue; timeout stays inside `execute` (it's ours to keep, but as execute internals, not a wrapper layer). -- **Delete:** `LhTool`, `toPiTool`, the zod→JSON bridge, the `clientQuestion` flag (`executionMode: "sequential"` already expresses it). -- **Teacher flag:** `src/agent` band. - -### Seam 3 — Tool registration gates -- **Now:** `src/agent/tools.ts` (244 lines): prober/teacher registries, `filterEnabledTools` (drops `enabled[name] === false`), `probe_done` re-added always-on, `research` gated on `searchAvailable`, graph tools gated on `ctx.graphStore`, quiz schema differs per phase. -- **Native:** registration is plain array assembly (`wrapToolDefinitions`); pi has no gating concept because gating is app policy. -- **Verdict:** the gating logic itself is app policy, works (§1.1 proved no stripping), and stays — but it assembles ToolDefinitions from seam 2 instead of LhTools. `filterEnabledTools` + toggles map unchanged onto the new shape. -- **Teacher flag:** `src/agent` band. - -### Seam 4 — Question tools & turn stopping -- **Now:** `src/agent/teacher-agent.ts` `makeStopper` ends the run on a cleanly-resolved question tool call (maxTurns 20 default); question tools `execute` returns a synthetic `'Question posed'` placeholder never persisted; the client renders the question and the learner's answer arrives as the next user message. -- **Native:** `AgentToolResult.terminate: true` — `agent-loop.js:385` `shouldTerminateToolBatch` stops the loop after a batch whose results all terminate; `types.d.ts:237/330` document the flag; `shouldStopAfterTurn` (`agent-loop.js:154`) covers cross-batch policy. -- **Replace with:** quiz/ask execute → `{ content, terminate: true }`. Delete `makeStopper`. The pending-question UX (client rendering, answer flow) is app-level and stays as-is. -- **This is the cleanest example of the whole wave:** our stopper re-implements a library flag. - -### Seam 5 — Agent construction & steering -- **Now:** stateless rebuild per POST (`teacher-agent.ts` — fresh Agent per turn, `composeInstructions` + `effectivePromptSegments` per turn, maxTurns); steering via `claimSteers` coalescing + the wake pipeline's deferred turn. -- **Native:** stateful `Agent` (`agent.js`): `initialState`, `subscribe` listeners awaited in settlement, `steer()`/`followUp()` queues (`one-at-a-time`/`all` — `steer()` is literally our wake-message semantics), `sessionId` forwarded for cache-aware providers, `thinkingBudgets`, per-agent `toolExecution` default. -- **Replace with (staged):** keep stateless rebuild first, but feed `initialState.messages` from the native store (seam 1) — same wire, smaller step. A stateful per-chat Agent registry (mirroring `turn-runs.ts`) with `steer()` for the wake path is a later, optional consolidation. -- **Keep:** `composeInstructions`/prompt overrides (app feature), write-ahead persistence, turn gates. -- **Teacher flag:** `src/agent` band. - -### Seam 6 — SSE / streaming bridge -- **Now:** `src/server/pi/ui-stream.ts` (423 lines): `runPiTurn` runs the loop, keeps a memory-only chunk log + subscriber set (background-turn wave 00a92e4), and hand-maps pi `AgentEvent`s into the Vercel `UIMessageStream` chunk dialect (text/thinking/toolcall mapping, question-tool output suppression, input gating); `[DONE]` terminal. -- **Native:** pi events are already a typed stream (`AgentEvent`: `agent_start`/`message_update`/`turn_start`/`turn_end`/`agent_end`, carrying partial+final messages; `pi-ai` `AssistantMessageEventStream` underneath); pi's own client binds via transcript snapshots + event subscriptions (`pi-coding-agent/dist/client/remote-session.js`). -- **Replace with:** thin passthrough — serialize pi `AgentEvent`s (or message_update partials) directly instead of remapping into UIMessage chunks; keep our subscriber-log/reattach/abort machinery (that's the 00a92e4 background-turn layer and it's good); the question-tool suppression hacks die naturally because native events carry tool-call state honestly. -- **Cost:** wire dialect changes → **UI stream coordination required** (client half dispatch after merge, same as previous waves). -- **Delete:** the event-mapping switch in `ui-stream.ts` (the log/subscriber/abort halves stay). - -### Already native (the template) -- `getApiKey` (fixed 76051d0): the deleted hook is exactly this wave's pattern — custom seam removed, pi-ai's native key resolution used. Key-presence gating stays app-side at the turn gate. - ---- - -## 3. What genuinely needs custom (no native equivalent — keep, but express natively) - -1. **Question tools awaiting human answers** — closest native is `terminate: true` + app-level answer flow (the "pause the world for a human" semantics is ours). -2. **Tool toggles / phases / graphTools gating** — app policy at registration time (seam 3). -3. **Jobs + extraction pipeline** — outside the agent loop entirely; untouched by this wave. -4. **Prompt override switches** (`composeInstructions`/`effectivePromptSegments`) — app feature; stays. -5. **Write-ahead persistence, background turns, reattach subscribers, remote abort** — our infrastructure from waves 00a92e4/4d694b3; stays. - ---- - -## 4. Risk order - -1. **Wave A — tools + stopping (seams 2+3+4):** smallest blast radius; deletes lh-tool + makeStopper; gates unchanged; UI unaffected (tool names/wire identical). -2. **Wave B — native history store + replay (seam 1):** biggest win (wire fidelity, signatures, cache, kills §1.3), biggest risk (record migration for existing chats). Needs a one-time record migration + golden-master diff (§5) before cutover. -3. **Wave C — stateful agents (seam 5, optional):** only after B settles. -4. **Wave D — SSE dialect (seam 6):** wire contract change; dispatch UI half after merge. -5. **Independent, immediate:** the §1.2 truncation is an upstream bug — report to OpenRouter / route-pin away from the flaky upstream for `glm-5.3-flash`. Not gated on any wave. - ---- - -## 5. Test strategy - -- **Wire-shape pin (kills §1.3 forever):** replay a stored same-model session through `convertMessages` and assert the assistant history carries `reasoning_details`/`thinkingSignature` preserved (fails today on `model: "replayed"`). -- **Terminate pins:** question tool result `terminate: true` ends the loop natively (replaces `makeStopper` pins; mirror the existing chat-flow scripted-teacher tests). -- **Registry census pin:** faux-provider wiretap-style test asserting the full registry (11 tools incl. graph tools) is present on the wire for both phases regardless of toggles-off state — pins §1.1 shut. -- **Golden-master for Wave B:** for every stored chat, diff old adapter output vs native replay output message-by-message before cutover; migrate one throwaway copy of master `data/chats` first (established one-way copy rule). -- **Per-wave gates unchanged:** typecheck / 950+ tests / lint / build; live smoke on PORT=3004 (faux + real provider), reattach and abort paths re-verified after Wave D. - ---- - -## 6. Open owner decisions - -1. Flash-model policy for tool-heavy phases (confabulation §1.1 + truncation §1.2 both bite `z-ai/glm-5.3-flash` via OpenRouter): route-pin, provider-pin, or accept + prompt-mitigate. -2. Whether to report the OpenInference multi-tool-result truncation to OpenRouter (I can package the 15 replay files as a repro bundle). -3. Wave B record migration: in-place upgrade vs new-record-format-version with lazy dual-read (settings-delta-wave precedent). -4. Wave D timing (UI stream coordination). - ---- - -# Wave B planning — native history store + replay (2026-09-26 recon, post-Wave-A) - -Wave A landed (93b59cd): tools/stopping are pi-native. Wave B = seam 1 — the -record becomes pi-native history; `record-adapter.ts` dies. The owner's -standing rulings already settle the plan's §6.3 (record migration): **zero -backwards compat, all session data wiped, no migration** — so the golden-master -cutover is reduced to a fresh-format smoke, not a data project. - -## What Wave B replaces today (the seam, precisely) - -- `src/server/pi/record-adapter.ts` (189 lines): record→pi-ai rebuild per turn; - the `model: "replayed"` stamp (breaks `isSameModel` forever — §1.3), thinking - glued into content, `thinkingSignature` lost (records persist text only), - verdict/handoff marker flattening (`systemTextOf`), dangling-call - INTERRUPTED synthesis. -- `chat-handler`: write-ahead persist of the CLIENT's full UIMessage list (the - client owns the transcript between turns — useChat resend + addToolOutput), - `recordToPiMessages(outgoing)` at agent build, probe_done detection over - UIMessage parts. -- `ui-stream.runPiTurn`: assembles UIMessage parts from pi events; that - assembly is BOTH the wire dialect (Wave D's concern) AND the persist target. -- `turn-service`: `withMessageIds` (SDK client-side-id patch), plan-block - attachment onto UIMessage parts. - -## Consumer census (who reads record.messages as UIMessage part shapes) - -Server/core: phases (`phaseAt3` — tool-probe_done part state), pending -(`findPendingToolCall`), plan-blocks (`withPlanBlocks`), coverage -(`coverageOf`), graph extraction sources (`toolParts`), extraction -(`messageViewLines`), store summaries (phase + pendingQuestion), -`/api/chats/[id]` GET, jobs.ts (steers as data-part messages; `claimSteers` -over the POST body). Client: `useChat` POSTs the full list, renders every -part type, `mergeAdopted`, addToolOutput answers, artifact/plan/abort cards. -**None of these need the STORAGE to be UIMessage — they need a UIMessage-SHAPED -VIEW.** That distinction is Wave B's core move. - -## The design: server-owned native record + one-way projection - -1. **`LHChatRecord.messages: AgentMessage[]`** (pi-ai `UserMessage | - AssistantMessage | ToolResultMessage | CustomMessage`). The agent's own - final messages persist at `message_end` — provider/model/usage/stopReason/ - thinkingSignature stamped by the loop itself (what pi persists in its own - sessions). §1.3 dies by construction: replayed turns are byte-faithful, - same-model, signature-preserving; replay = `state.messages` assignment, - zero conversion. -2. **Custom messages for our non-LLM content** (pi's `CustomMessage`: role - "custom", customType, content, display, details): plan-block, - subagent-artifact, subagent-aborted, question-answered markers. The - verdict/probe-handoff steers persist AS custom messages whose CONTENT is - the canonical model-facing text (verdictText / HANDOFF_TEXT written once - at append) — `systemTextOf` flattening disappears; `convertToLlm` default - passes content through, UI-only types filter via details/customType. -3. **One-way projection `AgentMessage[] → UIMessage[]`** (new - `src/server/pi/project.ts`): serves `/api/chats/[id]` GET, the summary - scans (phase/pending/coverage/plan-blocks/graph/extraction keep their - existing UIMessage code — they read the projection), and the client until - Wave D. The lossy DIRECTION (UIMessage→pi) is deleted; the projection is - Wave D's deletion target. -4. **Question answers are native tool results (owner ruling, 2026-09-26).** - The history stays a NORMAL agent history: quiz toolCall followed by its - toolResult — the process just stops in between. The placeholder result - is an in-memory loop device only (terminate:true ends the turn); it is - never persisted, exactly as today's wire suppression. On answer, the - server appends a native ToolResultMessage whose content is the same - encoding the adapter produces today (the model-facing wire is - byte-shape-identical to current behavior). The record carries the - dangling call only in the pose→answer window, where no turn can run - (the question gate holds steers), and the INTERRUPTED synthesis survives - solely for the abort case — same as today. pi's own resume path is the - mechanism: assign state.messages (record ending at the toolResult), - continue(). -5. **Turn input contract**: the POST body's new-content DELTA (new user text, - answers, markers) is appended natively; the full-list body stays for now - (client unchanged — see fork 3). - -## Forks — ALL RESOLVED (owner rulings, 2026-09-26) - -1. **Answer semantics — normal agent history.** The answer is the tool - call's toolResult, appended natively at answer time; the placeholder is - an in-memory loop device only (terminate), never persisted. No custom - answer markers. Verified against pi's source: the loop always pairs - calls with results in a RUN, but the record may end at the assistant - message (no turn can run in the pose→answer window — the question gate - holds steers); pi's own resume path (assign state.messages, continue() - from the toolResult) is the mechanism. The model-facing wire is - byte-shape-identical to today. -2. **Storage shape — `AgentMessage[]` inside LHChatRecord JSON.** No new - dependencies; metadata stays in the one atomic tmp+rename file. (pi - SessionManager JSONL rejected: pi-coding-agent is a 139MB harness dep - and its tree/branching/compaction machinery is unused here.) -3. **POST contract — input events NOW, in this pass** (owner overrode the - server-side-delta recommendation): POST /api/chat carries - `{chatId, input}` where input is one of: - `{type:"text", text, note?}` (composer send; note rides the pending - verdict interplay), - `{type:"answer", toolCallId, answer:{values,other,text,dontKnow,note}, - dismissed?}` (quiz/ask answer or dismissal — graded client-side as - today, the structured verdict becomes the native toolResult content), - `{type:"verdict", accepted, note?}` (plan verdict). - Consequences: the client's 409 merge-retry DIES (nothing to merge — a - 409 means wait-and-retry the same idempotent event); claimSteers/ - write-ahead of full lists dies; the wake pipeline's internal turns - (setTurnRunner) and jobs steers append natively server-side; useChat - stays as the RENDERER (message list = projected record + stream - assembly — receive side unchanged until Wave D); sends go through a - thin client session-client: POST the event, read the status, cancel - the 200 body (a view may detach freely — the turn runs on), then - attach via the standard resume path. Client touches: chat-transport - send half, chat.tsx call sites (composer, onSubmitAnswer, verdict, - dismiss). Receive half (SSE dialect, part rendering) stays Wave D. -4. **Raw vs validated tool args — resolved by design**: persist pi's own - AssistantMessage verbatim (what message_end carries). That means RAW - streamed args — byte-faithful replay, identical to what pi persists in - its own sessions, cache-parity with the original emission. The dock's - parse seam already tolerates raw (Wave A boundaryParse fix); no other - record consumer reads tool-call args. -5. **Abort semantics — folded into 1**: persist partial (current - behavior); the record ends at the dangling call for the abort case and - the INTERRUPTED synthesis at replay covers it (same as today). - -## Test strategy - -- Wire-shape pin (§5's oldest promise, now trivial): persist a scripted - session's agent messages, rebuild the agent, `convertMessages` → assert - `provider`/`model` real (not "replayed"), `thinkingSignature` present. -- Projection parity: every existing UIMessage consumer test (988 suite) runs - against the projection unchanged — the suite IS the golden master. -- Question flow: pose → answer-as-marker → next turn's model view carries the - verdict; abort mid-question → resume works (kills the synthesis tests). -- Live smoke: real turn + reattach + remote abort on PORT=3004. - -## Blast radius (files) - -Delete: record-adapter.ts; chat-transport's retryingFetch/merge logic. -New: project.ts (AgentMessage[]→UIMessage projection), input-events -contract (core/contracts), client session-client send module. -Rewrite: chat-handler (build = state.messages from record; input-event -append; probe_done detection over native messages), ui-stream (part -assembly becomes WIRE-ONLY; the record write comes from the agent's own -messages at message_end), turn-service (native appends replace -withMessageIds/withPlanBlocks part mutation; plan-blocks/artifact steers -become CustomMessages), store/chats + core scanners (phaseAt3, -findPendingToolCall, coverage, graph sources — read the projection, -mechanical), jobs.ts (steers append native CustomMessages; setTurnRunner -internal path), client send call sites (composer, onSubmitAnswer, verdict, -dismissal — chat-transport send half). useChat's RECEIVE half and all part -renderers untouched (Wave D). Agent band untouched (Wave A's). Wave C -(steer() consolidation) stays deferred. diff --git a/app/api/chats/[id]/route.ts b/app/api/chats/[id]/route.ts index f215832..47d543a 100644 --- a/app/api/chats/[id]/route.ts +++ b/app/api/chats/[id]/route.ts @@ -1,5 +1,4 @@ import { deleteChatSession, getChat, isTurnRunning, sessionCoverage, stores, type ChatDeleteMode } from "@/src/server"; -import { projectNative } from "@/src/core"; export async function GET(_req: Request, { params }: { params: Promise<{ id: string }> }) { const { id } = await params; @@ -11,13 +10,13 @@ export async function GET(_req: Request, { params }: { params: Promise<{ id: str // meter + sidebar block — THE coverage surface (the all-sessions GET // route died with the redesign; the Extract POST stays). getChat // already 404'd, so the block exists. - // Wave B (owner-approved 2026-09-26): the record's messages are native - // agent history — clients receive the one-way projection (the AI-SDK - // UIMessage view) until Wave D. turnInFlight lets a returning client + // Wave D (owner-approved 2026-09-26): the record's messages are native + // agent history, served VERBATIM — the client derives its view + // (core/view.ts projectView). turnInFlight lets a returning client // re-attach to a running turn; coverage is the chat view's ring meter. return Response.json({ ...chat, - messages: projectNative(chat.messages), + messages: chat.messages, turnInFlight: isTurnRunning(id), coverage: sessionCoverage(id), }); diff --git a/app/chat/[id]/page.tsx b/app/chat/[id]/page.tsx index 7a9d13e..18d6989 100644 --- a/app/chat/[id]/page.tsx +++ b/app/chat/[id]/page.tsx @@ -1,6 +1,6 @@ import { notFound } from "next/navigation"; import { Chat } from "@/components/chat"; -import { projectNative, slotsConfigured, type TeacherUIMessage } from "@/src/core"; +import { slotsConfigured } from "@/src/core"; import { getChat, isTurnRunning, readSettingsRecord } from "@/src/server"; export const dynamic = "force-dynamic"; @@ -24,9 +24,10 @@ export default async function ChatPage({ key={chat.id} chatId={chat.id} title={chat.title} - // Wave B: the record is native history — the client view is the - // one-way projection (GET /api/chats/[id] serves the same shape). - initialMessages={projectNative(chat.messages) as TeacherUIMessage[]} + // Wave D: the record IS native agent history — the client view is + // derived app-side (core/view.ts projectView; GET serves the same + // native messages). + initialMessages={chat.messages} initialConfigured={slotsConfigured(record?.models)} initialPrompt={prompt} // A teacher turn that outlived its client: the remounted view must diff --git a/components/ai-elements/conversation.tsx b/components/ai-elements/conversation.tsx index 21e5d82..ab96191 100644 --- a/components/ai-elements/conversation.tsx +++ b/components/ai-elements/conversation.tsx @@ -2,7 +2,7 @@ import { Button } from "@/components/ui/button"; import { cn } from "@/lib/utils"; -import type { UIMessage } from "ai"; +import type { LhViewMessage, LhViewPart } from "@/src/core"; import { ArrowDownIcon, DownloadIcon } from "lucide-react"; import type { ComponentProps } from "react"; import { useCallback } from "react"; @@ -100,7 +100,7 @@ export const ConversationScrollButton = ({ ); }; -const getMessageText = (message: UIMessage): string => +const getMessageText = (message: LhViewMessage): string => message.parts .filter((part) => part.type === "text") .map((part) => part.text) @@ -110,21 +110,21 @@ export type ConversationDownloadProps = Omit< ComponentProps, "onClick" > & { - messages: UIMessage[]; + messages: LhViewMessage[]; filename?: string; - formatMessage?: (message: UIMessage, index: number) => string; + formatMessage?: (message: LhViewMessage, index: number) => string; }; -const defaultFormatMessage = (message: UIMessage): string => { +const defaultFormatMessage = (message: LhViewMessage): string => { const roleLabel = message.role.charAt(0).toUpperCase() + message.role.slice(1); return `**${roleLabel}:** ${getMessageText(message)}`; }; export const messagesToMarkdown = ( - messages: UIMessage[], + messages: LhViewMessage[], formatMessage: ( - message: UIMessage, + message: LhViewMessage, index: number ) => string = defaultFormatMessage ): string => messages.map((msg, i) => formatMessage(msg, i)).join("\n\n"); diff --git a/components/ai-elements/message.tsx b/components/ai-elements/message.tsx index d9d9787..96f47e1 100644 --- a/components/ai-elements/message.tsx +++ b/components/ai-elements/message.tsx @@ -15,9 +15,10 @@ import { cn } from "@/lib/utils"; import { cjk } from "@streamdown/cjk"; import { code } from "@streamdown/code"; import { createMathPlugin } from "@streamdown/math"; -import type { UIMessage } from "ai"; + import { ChevronLeftIcon, ChevronRightIcon } from "lucide-react"; import type { ComponentProps, HTMLAttributes, ReactElement } from "react"; +import type { LhViewMessage } from "@/src/core"; import { createContext, memo, @@ -36,7 +37,7 @@ import { mdComponents } from "@/components/md-blocks"; import { transformMathDelimiters } from "@/src/core"; export type MessageProps = HTMLAttributes & { - from: UIMessage["role"]; + from: LhViewMessage["role"]; }; export const Message = ({ className, from, ...props }: MessageProps) => ( diff --git a/components/artifacts/steer-card.tsx b/components/artifacts/steer-card.tsx index bb00512..8e54cb1 100644 --- a/components/artifacts/steer-card.tsx +++ b/components/artifacts/steer-card.tsx @@ -1,7 +1,8 @@ "use client"; import { CheckIcon, CircleX } from "lucide-react"; -import type { PdfResult, ResearchResult, SubagentArtifactData, TeacherUIMessage } from "@/src/core"; +import type { PdfResult, ResearchResult, SubagentArtifactData } from "@/src/core"; +import type { LhViewMessage } from "@/src/core"; import { isSubagentFailure } from "@/src/core"; import { DemoOutputCard } from "@/components/artifacts/demo-card"; import { PdfBriefCard, ResearchBriefCard } from "@/components/artifacts/job-card"; @@ -24,7 +25,7 @@ import { VisualOutputCard } from "@/components/artifacts/visual-card"; * The sub-agent's verification notes are model-facing glue and do NOT * render; only a failure note does. */ -export function SubagentSteerCard({ message }: { message: TeacherUIMessage }) { +export function SubagentSteerCard({ message }: { message: LhViewMessage }) { const artifactPart = message.parts.find((p) => p.type === "data-subagent-artifact"); const data = artifactPart?.data as SubagentArtifactData | undefined; // The abort steer (2026-09 dock semantics): its message-type part is diff --git a/components/chat-transport.ts b/components/chat-transport.ts deleted file mode 100644 index f8e7caf..0000000 --- a/components/chat-transport.ts +++ /dev/null @@ -1,310 +0,0 @@ -/** - * Chat transport for the INPUT-EVENTS contract (Wave B, owner ruling - * 2026-09-26): the client never re-POSTs its transcript — every send - * carries ONE explicit input event ({text} | {answer} | {verdict} | - * {continue}) stashed by the call site (chat.tsx) before invoking the SDK - * send. The server owns the record; this transport's ONLY send-time job is - * to consume the stash and put it on the wire as {chatId, input}. - * - * 409 retry (the wake-pipeline race, contracts/jobs.ts "THE ASYNC - * PROTOCOL"): a send can race the server's wake turn (the client only - * notices it via the /api/jobs poll up to a second later) and gets - * 409'd. Under input events the retry is IDENTITY: the event was never - * persisted (the write-ahead only happens once the turn is accepted), so - * waiting for the in-flight turn and re-POSTing the SAME body is safe — - * no merge, no stale-list clobber. The old UIMessage-era merge logic is - * deleted with the UIMessage record (the server now owns the - * record; a lost race means "try again later", not "merge transcripts"). - * - * The send path is LOUD about a missing stash: every SDK send must map to - * exactly one event — a send with nothing stashed is a call-site bug - * (it would otherwise silently re-send the previous event), so the - * transport throws instead of guessing. - */ - -import { DefaultChatTransport } from "ai"; -import type { UIMessageChunk } from "ai"; -import type { ChatTransport } from "ai"; -import type { ChatInput } from "@/src/core"; -import type { TeacherUIMessage } from "@/src/core"; - -/** How often to poll turnInFlight while the racing turn runs. */ -const TURN_POLL_MS = 500; -/** Give up waiting (a wake turn can run long) and surface the 409 as before. */ -const TURN_WAIT_MAX_MS = 30_000; - -/** Injectable timing (tests). */ -export interface RetryFetchOptions { - pollMs?: number; - waitMaxMs?: number; -} - -function delay(ms: number, signal?: AbortSignal): Promise { - return new Promise((resolve, reject) => { - const t = setTimeout(resolve, ms); - signal?.addEventListener("abort", () => { - clearTimeout(t); - reject(new DOMException("Aborted", "AbortError")); - }, { once: true }); - }); -} - -async function turnInFlight(chatId: string): Promise { - try { - const res = await fetch(`/api/jobs?chatId=${encodeURIComponent(chatId)}`); - if (!res.ok) return null; - return ((await res.json()) as { turnInFlight?: boolean }).turnInFlight === true; - } catch { - return null; // network hiccup: keep waiting, the deadline bounds us - } -} - -/** - * A fetch that turns a 409 on POST /api/chat into: wait for the running - * turn, re-POST the SAME event body. Non-409 responses pass through - * untouched. - */ -/** Exported for tests. */ -export function retryingFetch(chatId: string, opts: RetryFetchOptions = {}): typeof globalThis.fetch { - const pollMs = opts.pollMs ?? TURN_POLL_MS; - const waitMaxMs = opts.waitMaxMs ?? TURN_WAIT_MAX_MS; - return async (input, init) => { - let response = await fetch(input, init); - let waited = 0; - - while (response.status === 409) { - // Free the 409 body's socket before waiting on the racing turn. - await response.body?.cancel().catch(() => {}); - if (init?.signal?.aborted || waited >= waitMaxMs) return response; - - await delay(pollMs, init?.signal ?? undefined); - waited += pollMs; - if (init?.signal?.aborted) return response; - if ((await turnInFlight(chatId)) !== false) continue; - - // Turn finished: re-POST the same input event — nothing merged, - // nothing lost (the event was never persisted pre-acceptance). - response = await fetch(input, init); // may 409 again → loop - } - - return response; - }; -} - -/* ──────────────────────────────────────────────────────────────────────── - * THE REATTACH TRANSPORT (background-turn runner wave, owner ruling - * 2026-09-25): a turn runs server-side decoupled from every view, and a - * view that arrives late (navigate away mid-turn, wake-turn discovery) - * re-enters it through the SDK's OWN resume path — useChat's - * resumeStream() → reconnectToStream() — so the replay burst and the - * live deltas flow through the same processUIMessageStream assembly as - * first-subscriber streaming. No second rendering dialect exists. - * - * Three adapters over the platform contract, all in this file: - * - * 1. ENDPOINT: the SDK's default reconnect GETs /api/chat/{id}/stream; - * ours is GET /api/chats/[id]/stream (same chunk dialect). - * 2. THE SETTLED TERMINAL: "no turn running" is a terminal EVENT - * (turn-settled + [DONE]), not a 204 — one wire protocol end to end. - * The first SSE event is peeked; turn-settled cancels the body, fires - * onSettled (the Chat clears its in-flight flag and refetches the - * record), and resolves null — the SDK's "no active stream" signal - * (it sets status back to ready, no stream consumed). - * 3. THE CONTINUATION SEAM: the SDK seeds its resume streaming message - * EMPTY (the resume trigger passes lastMessage: undefined), so a - * reattached turn that CONTINUES the seeded last assistant message - * (answer-quiz reaction, wake continuation) would hit the SDK's - * replace-last-message path (start id === last id) and drop that - * message's parts for the whole stream — the answered-quiz card - * would flicker out until the settle sync restores it. When the - * peeked start chunk's messageId equals the seeded last message's id, - * rewrite it to a fresh id: this run's parts mount as their own - * message while the seeded one keeps rendering — pixel-identical to - * the live continuation (card above, reaction below) — and the settle - * syncFromServer lands the server's single assembled message (the - * server is the transcript's canon; the split never persists). - * Part ids stay server-prefixed; nothing keys parts by message id. - * ──────────────────────────────────────────────────────────────────────── */ - -/** The reattach half of the transport contract (chat.tsx wires the refs). */ -export interface ReattachOptions { - /** turn-settled — the stream endpoint found no running turn (raced - * settle, or none ever started): the record is current; the caller - * refetches it and clears its in-flight flag. */ - onSettled?: () => void; - /** The LIVE last message id of the seeded transcript (a Chat ref) — - * the continuation detection above. */ - lastMessageId?: () => string | undefined; -} - -/** The input-events half (Wave B): consume the call site's stashed event. - * Exactly one event per SDK send — the ref is cleared on read, so a - * send with nothing stashed fails LOUDLY rather than re-sending the - * previous event. */ -export interface InputEventOptions { - takeInput: () => ChatInput | undefined; -} - -/** Concatenate byte payloads (the head stream's event + read-past remainder). */ -function concatBytes(a: Uint8Array, b: Uint8Array): Uint8Array { - const out = new Uint8Array(a.length + b.length); - out.set(a); - out.set(b, a.length); - return out; -} - -/** Same id shape the server's runPiTurn mints — collision-free by design. */ -function freshMessageId(): string { - return `msg_${Date.now().toString(36)}_${crypto.randomUUID().slice(0, 8)}`; -} - -/** One parsed SSE event off the wire (data: \n\n framing). */ -interface SseEvent { - /** The raw bytes of this event, re-injectable into the head stream. */ - raw: Uint8Array; - /** Bytes read PAST the event boundary (the next event can arrive in - * the same read) — they must ride the head of the tail verbatim. */ - rest: Uint8Array; - /** The parsed JSON payload (events here are always single data lines). */ - data: Record; -} - -/** - * Read the FIRST complete SSE event off a live body (the settled terminal - * and the start chunk are both single small events; a long-lived turn's - * log replays after). Resolves as soon as one \n\n boundary is seen — the - * caller re-injects the raw bytes (plus any read-past remainder) ahead of - * the unconsumed tail, so nothing is lost or reordered. - */ -async function readFirstEvent(reader: ReadableStreamDefaultReader): Promise { - const decoder = new TextDecoder(); - let buf = ""; - for (;;) { - const { done, value } = await reader.read(); - if (done) throw new Error("The reattach stream ended before its first event."); - buf += decoder.decode(value, { stream: true }); - const end = buf.indexOf("\n\n"); - if (end === -1) continue; - const event = buf.slice(0, end); - const dataLine = event.split("\n").find((l) => l.startsWith("data: ")); - if (!dataLine) throw new Error("The reattach stream's first event is malformed."); - return { - raw: new TextEncoder().encode(`${event}\n\n`), - rest: new TextEncoder().encode(buf.slice(end + 2)), - data: JSON.parse(dataLine.slice("data: ".length)) as Record, - }; - } -} - -/** The reattach transport: the input-event POST for sends, our seam for resumes. */ -class ReattachingChatTransport extends DefaultChatTransport { - constructor( - private readonly reattach: ReattachOptions, - private readonly inputEvent: InputEventOptions, - init: ConstructorParameters>[0], - ) { - super(init); - } - - override async reconnectToStream( - options: Parameters["reconnectToStream"]>[0], - ): Promise | null> { - const res = await fetch(`/api/chats/${encodeURIComponent(options.chatId)}/stream`, { - signal: options.abortSignal, - }); - if (res.status === 204) return null; // the SDK's native no-stream signal - if (!res.ok) throw new Error((await res.text()) || "Failed to attach to the running turn."); - if (!res.body) throw new Error("The reattach response body is empty."); - - const reader = res.body.getReader(); - let first: SseEvent; - try { - first = await readFirstEvent(reader); - } catch (err) { - void reader.cancel().catch(() => {}); - throw err; - } - - // The settled terminal: nothing to attach to — the record is current. - if (first.data.type === "turn-settled") { - await reader.cancel().catch(() => {}); - this.reattach.onSettled?.(); - return null; - } - - // The continuation seam (see the block comment): a start chunk whose - // id matches the seeded last message would REPLACE that message with - // an empty streaming state — rewrite to a fresh id so this run mounts - // beside it and the settle sync merges the canon back. - let firstEvent = first.raw; - if ( - first.data.type === "start" && - typeof first.data.messageId === "string" && - first.data.messageId === this.reattach.lastMessageId?.() - ) { - firstEvent = new TextEncoder().encode( - `data: ${JSON.stringify({ ...first.data, messageId: freshMessageId() })}\n\n`, - ); - } - // The head bytes: the (possibly rewritten) first event plus anything - // the peek read past the boundary — then the live tail, untouched. - const head = first.rest.length > 0 ? concatBytes(firstEvent, first.rest) : firstEvent; - - // Re-inject the (possibly rewritten) head ahead of the live tail and - // hand the SDK the SAME SSE parse the POST response goes through. - const tail = new ReadableStream({ - start(controller) { - controller.enqueue(head); - void (async () => { - try { - for (;;) { - const { done, value } = await reader.read(); - if (done) { - controller.close(); - return; - } - controller.enqueue(value); - } - } catch (err) { - controller.error(err); - } - })(); - }, - cancel() { - return reader.cancel(); - }, - }); - return this.processResponseStream(tail); - } -} - -/** - * The chat.tsx wiring: `transport: createReattachingTransport(chatId, refs)`. - * Sends POST the stashed input event with the 409 identity-retry fetch; - * resumes take the adapters above. - */ -export function createReattachingTransport( - chatId: string, - reattach: ReattachOptions, - inputEvent: InputEventOptions, -): DefaultChatTransport { - return new ReattachingChatTransport(reattach, inputEvent, { - api: "/api/chat", - body: { chatId }, - fetch: retryingFetch(chatId), - // THE INPUT-EVENTS CONTRACT (Wave B): replace the SDK's full-transcript - // body with {chatId, input}. The stash is consumed HERE — exactly once - // per SDK send (the 409 retry re-POSTs the built body, not the stash); - // a send with nothing stashed fails LOUDLY (a call-site bug — it must - // never silently re-send the previous event). - prepareSendMessagesRequest: () => { - const input = inputEvent.takeInput(); - if (!input) { - throw new Error( - "Send without a stashed input event — every send call site must set pendingInput before invoking the SDK send (Wave B input-events contract).", - ); - } - return { body: { chatId, input } }; - }, - }); -} diff --git a/components/chat.tsx b/components/chat.tsx index c63e1b1..821ef1c 100644 --- a/components/chat.tsx +++ b/components/chat.tsx @@ -1,9 +1,7 @@ "use client"; import { createContext, Fragment, useCallback, useEffect, useLayoutEffect, useMemo, useRef, useState } from "react"; -import { useChat } from "@ai-sdk/react"; -import { lastAssistantMessageIsCompleteWithToolCalls } from "ai"; -import { createReattachingTransport } from "@/components/chat-transport"; +import { useChatSession } from "@/components/session"; import { SetupDialog } from "@/components/setup-dialog"; import { Hint } from "@/components/ui/hint"; import { useZeroKeys, ZeroKeysPill } from "@/components/setup-cta"; @@ -26,7 +24,7 @@ import { DictationBar, useDictation } from "@/components/dictation"; import { JobsContext } from "@/components/jobs-context"; import { StatsBar } from "@/components/stats-bar"; import { useActivity } from "@/components/activity-dock"; -import { isProviderKeyRefusal, mergeAdopted, type ChatInput, type ChatSettings, type ModelSlotName } from "@/src/core"; +import { isProviderKeyRefusal, needsContinuation, type ChatInput, type ChatSettings, type LhMessage, type ModelSlotName } from "@/src/core"; import { PlanBlock, PlanPopup, latestTree } from "@/components/plan-block"; import { PlanMarginalia } from "@/components/plan-window"; import { MermaidOverlay, SessionMermaidContext } from "@/components/session-overlay"; @@ -34,7 +32,8 @@ import { WaypointsIcon } from "lucide-react"; import type { PlanBlockData } from "@/src/core"; import { QuestionDock } from "@/components/question-dock"; import { SidebarToggle } from "@/components/sidebar"; -import type { ArtifactPartProps, JobRecord, Phase3, QuestionAnswerData, TeacherUIMessage, TurnUsage } from "@/src/core"; +import type { ArtifactPartProps, JobRecord, Phase3, QuestionAnswerData, TurnUsage } from "@/src/core"; +import type { LhViewMessage } from "@/src/core"; import { PhaseChip, PhaseDivider, planNodeCount } from "@/components/phase-divider"; import { CoverageRing } from "@/components/extraction-coverage"; import { findPendingToolCall, phaseAt3, phaseTimeline, streamingToolActivity } from "@/src/core"; @@ -141,7 +140,7 @@ function SystemLine({ children, testId }: { children: React.ReactNode; testId?: ); } -function isSubagentSteer(m: TeacherUIMessage): boolean { +function isSubagentSteer(m: LhViewMessage): boolean { return (m.metadata as { subagentResult?: unknown } | undefined)?.subagentResult != null; } @@ -164,7 +163,7 @@ export function Chat({ }: { chatId: string; title: string; - initialMessages: TeacherUIMessage[]; + initialMessages: LhMessage[]; initialPrompt?: string; /** A teacher turn is running server-side but attached to no client * (navigated away mid-turn, returned while in flight). */ @@ -174,61 +173,31 @@ export function Chat({ * only decides whether the pill rides the composer. */ initialConfigured?: boolean; }) { - // Reattach plumbing (background-turn runner wave): the transport holds - // two stable refs — the settled terminal's callback (assigned below, - // once syncFromServer exists) and the LIVE last-message id (the - // continuation seam — see chat-transport.ts). Refs, not closures, so - // the per-render transport recreation can never capture a stale one. + // Reattach plumbing (background-turn runner wave): the hook holds the + // settled terminal's callback — a stable ref assigned below, once + // syncFromServer exists (refs, not closures: no stale capture). const reattachSettledRef = useRef<() => void>(() => {}); - const reattachLastIdRef = useRef(undefined); - // THE INPUT STASH (Wave B input-events contract, owner ruling - // 2026-09-26): every SDK send maps to EXACTLY ONE explicit input event — - // the call site stashes the event here before invoking the SDK send, and - // the transport consumes it when building the POST body ({chatId, input}; - // the client transcript never rides the wire). Consumed-on-read: a send - // with nothing stashed fails LOUDLY in the transport (a call-site bug — - // it must never silently re-send the previous event). - const pendingInputRef = useRef(null); - const stashInput = (input: ChatInput) => { - pendingInputRef.current = input; - }; - const takeInput = () => { - const input = pendingInputRef.current; - pendingInputRef.current = null; - return input ?? undefined; - }; - const { messages, sendMessage, addToolOutput, setMessages, status, error, stop, clearError, resumeStream } = - useChat({ - id: chatId, - messages: initialMessages, - transport: createReattachingTransport( - chatId, - { - onSettled: () => reattachSettledRef.current(), - lastMessageId: () => reattachLastIdRef.current, - }, - { takeInput }, - ), - // After the learner answers (addToolOutput), automatically send the - // continuation request so the teacher reacts to the answer. - sendAutomaticallyWhen: lastAssistantMessageIsCompleteWithToolCalls, - }); + // THE SESSION HOOK (Wave D, owner-approved 2026-09-26): the app-owned + // replacement for useChat + the AI-SDK transport. The record is native + // history; the VIEW (grouped, golden-master) is derived client-side + // (core/view.ts projectView); sends are ONE explicit input event each + // ({chatId, input} — the stash indirection died with the SDK), and the + // turn stream is the VERBATIM AgentEvent passthrough the server emits. + const { messages, nativeMessages, send, submitAnswer, setMessages, adoptRecord, status, error, stop, clearError, resumeStream, setReattachSettled } = + useChatSession({ chatId, initialMessages, initiallyInFlight }); + + // The reattach settled terminal → the sync-and-continue seam (below). + setReattachSettled(() => reattachSettledRef.current()); // New chat with a topic: send the first message exactly once. const sentInitial = useRef(false); useEffect(() => { if (!sentInitial.current && initialPrompt && initialMessages.length === 0) { sentInitial.current = true; - stashInput({ type: "text", text: initialPrompt }); - void sendMessage({ text: initialPrompt }); + void send({ type: "text", text: initialPrompt }); } - }, [initialPrompt, initialMessages.length, sendMessage]); - - // The reattach seam's live half: the transport compares the stream's - // start id against the transcript's CURRENT last message — the - // continuation rewrite keys on it (chat-transport.ts, seam 3). - reattachLastIdRef.current = messages.at(-1)?.id; + }, [initialPrompt, initialMessages.length, send]); // Reattach on mount (stream-reattach wave): a turn that outlived its // client (navigate away mid-turn, back while in flight) is re-entered @@ -358,14 +327,8 @@ export function Chat({ useEffect(() => { if (mountContinuation.current) return; mountContinuation.current = true; - if ( - !turnInFlight && - status === "ready" && - messages.length > 0 && - lastAssistantMessageIsCompleteWithToolCalls({ messages }) - ) { - stashInput({ type: "continue" }); - void sendMessage(); + if (!turnInFlight && status === "ready" && needsContinuation(initialMessages)) { + void send({ type: "continue" }); } // mount-only by design (turnInFlight read at initial value) }, []); @@ -469,7 +432,7 @@ export function Chat({ void (async () => { const res = await fetch(`/api/chats/${encodeURIComponent(chatId)}`); if (!cancelled && res.ok) { - const record = (await res.json().catch(() => null)) as { messages?: TeacherUIMessage[] } | null; + const record = (await res.json().catch(() => null)) as { messages?: LhMessage[] } | null; if (Array.isArray(record?.messages)) setMessages(record.messages); } if (cancelled) return; @@ -502,22 +465,19 @@ export function Chat({ const syncFromServer = useCallback(async () => { const res = await fetch(`/api/chats/${chatId}`); if (!res.ok) return; - const record = (await res.json()) as { messages?: TeacherUIMessage[]; turns?: TurnUsage[] }; + const record = (await res.json()) as { messages?: LhMessage[]; turns?: TurnUsage[] }; if (Array.isArray(record.turns)) setTurns(record.turns); if (!Array.isArray(record.messages)) return; setMessages(record.messages); - // setMessages does NOT fire sendAutomaticallyWhen (the SDK only checks - // it inside addToolOutput) — without this the teacher would stay silent - // after a job result lands. sendMessage() with no args re-requests with - // the current list: the continuation. (contracts.ts protocol §4.) - if ( - status === "ready" && - lastAssistantMessageIsCompleteWithToolCalls({ messages: record.messages }) - ) { - stashInput({ type: "continue" }); - void sendMessage(); + // The continuation seam (contracts.ts protocol §4): a job result + // landing in the record leaves the agent owing a continuation — the + // record ENDS on un-consumed results, the dock owns no question, and + // nothing runs. The native fold (needsContinuation) is the old SDK + // lastAssistantMessageIsCompleteWithToolCalls check. + if (status === "ready" && needsContinuation(record.messages)) { + void send({ type: "continue" }); } - }, [chatId, setMessages, sendMessage, status]); + }, [chatId, setMessages, send, status]); // The settled terminal's other half (stream-reattach wave): the // transport resolved turn-settled — no turn running, the record is @@ -536,23 +496,20 @@ export function Chat({ // nothing about a running turn should hide it. The full snapshot sync // mid-stream is forbidden (it breaks the SDK's in-place continuation // merge — the stale-snapshot lesson), so adoption here is APPEND-ONLY - // (core mergeAdopted): insert the record's new ids in their canonical + // (the session hook's adoptRecord): insert the record's new ids in their canonical // order, never rewrite a local one, keep the in-flight tail last. The // full syncFromServer stays the net at idle (reorder, plan blocks, turns). - const messagesRef = useRef(messages); - messagesRef.current = messages; const adoptAppendOnly = useCallback(async () => { try { const res = await fetch(`/api/chats/${chatId}`); if (!res.ok) return; - const record = (await res.json()) as { messages?: TeacherUIMessage[] }; + const record = (await res.json()) as { messages?: LhMessage[] }; if (!Array.isArray(record.messages)) return; - const merged = mergeAdopted(messagesRef.current, record.messages); - if (merged) setMessages(merged); + adoptRecord(record.messages); } catch { /* transient — the idle full sync is the net */ } - }, [chatId, setMessages]); + }, [chatId, adoptRecord]); // Turn-start window: the server inserts claimed steers right after our // outgoing message, BEFORE the stream produces anything — so a bounded, @@ -637,38 +594,14 @@ export function Chat({ // verdict is its own custom message — marker part ONLY, no human- // authored text (the model-facing wording lives in toLlmMessages, // the native convertToLlm seam; nothing fake is persisted as speech). - // setMessages + sendMessage() (no args) fires ONE request carrying - // both — the same single-turn mechanism syncFromServer already uses, - // so no 409 dance and no double teacher turn. - const stamp = Date.now().toString(36); - const rnd = crypto.randomUUID().slice(0, 8); - const appended: TeacherUIMessage[] = []; - if (note) { - appended.push({ - id: `msg_${stamp}_${rnd}`, - role: "user", - parts: [{ type: "text", text: note }], - }); - } - appended.push({ - id: `verdict_${stamp}_${rnd}`, - role: "user", - parts: [ - { - type: "data-verdict", - id: `vd_${stamp}_${rnd}`, - data: { accepted, at: new Date().toISOString() }, - }, - ], - }); - setMessages([...messages, ...appended]); - // The event (Wave B): the server appends the note (its own user - // message) + the verdict marker natively — the optimistic ids above - // are local render only; the settle sync lands the canon. - stashInput({ type: "verdict", accepted, note: note || undefined }); - void sendMessage(); + // ONE input event, ONE turn: the session hook appends the optimistic + // note + verdict marker (local ids — the settle sync lands the + // server canon) and POSTs {type:"verdict"}; the server persists the + // pair as its own write-ahead — the same single-turn mechanism + // syncFromServer uses, no 409 dance, no double teacher turn. + void send({ type: "verdict", accepted, note: note || undefined }); }, - [text, messages, setMessages, sendMessage], + [text, send], ); // ── Session usage (wave-4 stats bar) ── @@ -955,16 +888,13 @@ export function Chat({ if (res.ok) { const record = (await res.json()) as { turnInFlight?: boolean; - messages?: TeacherUIMessage[]; + messages?: LhMessage[]; }; // Ground rule (2026-09-15): steer content persisted by the // wake shows NOW, at the wake turn's start — not at its // finish. Append-only is safe here: no local stream runs // under a server-side wake turn. - if (Array.isArray(record.messages)) { - const merged = mergeAdopted(messagesRef.current, record.messages); - if (merged) setMessages(merged); - } + if (Array.isArray(record.messages)) adoptRecord(record.messages); if (!record.turnInFlight) { setTurnInFlight(false); // Heal a stream-death banner (the failed-attach fallback): the @@ -991,8 +921,7 @@ export function Chat({ if (!t || busy || pending) return; draftRef.current = t; // the refusal flow's preserve-and-replay seam setText(""); - stashInput({ type: "text", text: t }); - void sendMessage({ text: t }); + void send({ type: "text", text: t }); }; return ( @@ -1001,8 +930,7 @@ export function Chat({ chatId, syncFromServer, sendText: (t: string) => { - stashInput({ type: "text", text: t }); - void sendMessage({ text: t }); + void send({ type: "text", text: t }); }, openSetup: () => setSetupOpen(true), }} @@ -1192,7 +1120,7 @@ export function Chat({ return ( - {part.text} + {part.text ?? ""} ); case "step-start": @@ -1310,17 +1238,13 @@ export function Chat({ } onSubmitAnswer={(output) => { if (!pending) return; - // THE INPUT-EVENTS CONTRACT (Wave B): the answer ride order is - // stash-then-send — addToolOutput triggers the SDK's automatic - // continuation send (sendAutomaticallyWhen), and the transport - // consumes THIS stash to build {chatId, input}. Dismissals ride - // the same callback (QuestionAnswerData.dismissed). - stashInput({ type: "answer", toolCallId: pending.toolCallId, answer: output }); - void addToolOutput({ - tool: pending.tool as "quiz" | "ask_user_question", - toolCallId: pending.toolCallId, - output, - }); + // THE ANSWER IS THE QUESTION CALL'S TOOL RESULT (fork 1, owner + // 2026-09-26): one input event, one turn — the hook appends + // the optimistic graded result (the AnsweredCard shows at + // once) and the server's write-ahead persists it before the + // turn; the teacher's reaction streams below it. Dismissals + // ride the same callback (QuestionAnswerData.dismissed). + submitAnswer(pending.tool, pending.toolCallId, output); }} /> @@ -1355,8 +1279,7 @@ export function Chat({ if (t) { setText(""); draftRef.current = t; - stashInput({ type: "text", text: t }); - void sendMessage({ text: t }); + void send({ type: "text", text: t }); } }} /> diff --git a/components/session-core.ts b/components/session-core.ts new file mode 100644 index 0000000..309b321 --- /dev/null +++ b/components/session-core.ts @@ -0,0 +1,513 @@ +/** + * THE SESSION MODEL (Wave D): the chat session's state machine, extracted + * PURE — no React, one injectable fetch — so the attach/replay/settle/ + * error/abort paths are testable in the repo's node:test world. The React + * hook (components/session.ts) is a thin adapter: it re-renders on change + * and projects the view (core/view.ts). + * + * ONE VIEW PROTOCOL: every turn is a server-side background process (owner + * ruling 2026-09-25); this model is a view. Sends POST one explicit input + * event ({chatId, input}); the response and the reattach stream + * (GET /api/chats/[id]/stream) are the SAME verbatim AgentEvent SSE, + * applied by the same function. Status vocabulary (pre-Wave-D parity): + * ready | submitted | streaming | error. + */ + +import type { AgentEvent } from "@earendil-works/pi-agent-core"; +import type { ToolResultMessage } from "@earendil-works/pi-ai"; +import { + needsContinuation, + newAnswerResult, + newUserMessage, + newVerdictMessage, + projectView, + type ChatInput, + type LhMessage, + type LhViewMessage, + type LiveExecution, + type QuestionAnswerData, +} from "../src/core"; + +export type SessionStatus = "ready" | "submitted" | "streaming" | "error"; + +/** How often to poll turnInFlight while a racing turn runs (409 retry). */ +export const TURN_POLL_MS = 500; +/** Give up waiting (a wake turn can run long) and surface the 409. */ +export const TURN_WAIT_MAX_MS = 30_000; + +export interface SessionModelDeps { + /** Injectable fetch (tests); defaults to the global. */ + fetch?: typeof globalThis.fetch; + /** Injectable retry timing (tests). */ + pollMs?: number; + waitMaxMs?: number; +} + +/** A local optimistic id (never a server stamp — absorption and the + * settle swap both key on this prefix). */ +let localSeq = 0; +function localId(): string { + return `c_${Date.now().toString(36)}_${(localSeq++).toString(36)}`; +} + +function delay(ms: number, signal?: AbortSignal): Promise { + return new Promise((resolve, reject) => { + const t = setTimeout(resolve, ms); + signal?.addEventListener( + "abort", + () => { + clearTimeout(t); + reject(new DOMException("Aborted", "AbortError")); + }, + { once: true }, + ); + }); +} + +async function turnInFlight(model: SessionModel): Promise { + try { + const res = await model.fetch(`/api/jobs?chatId=${encodeURIComponent(model.chatId)}`); + if (!res.ok) return null; + return ((await res.json()) as { turnInFlight?: boolean }).turnInFlight === true; + } catch { + return null; // network hiccup: keep waiting, the deadline bounds us + } +} + +/** One wire event off the SSE body: parsed JSON, or the [DONE] terminal. */ +async function* sseEvents(reader: ReadableStreamDefaultReader): AsyncGenerator> { + const decoder = new TextDecoder(); + let buf = ""; + for (;;) { + const { done, value } = await reader.read(); + if (done) return; + buf += decoder.decode(value, { stream: true }); + for (;;) { + const end = buf.indexOf("\n\n"); + if (end === -1) break; + const event = buf.slice(0, end); + buf = buf.slice(end + 2); + const dataLine = event.split("\n").find((l) => l.startsWith("data: ")); + if (!dataLine) continue; + const data = dataLine.slice("data: ".length); + if (data === "[DONE]") return; + yield JSON.parse(data) as Record; + } + } +} + +/** The dangling question call's tool name (the optimistic answer result's + * toolName — the server stamps the same from its own record scan). */ +export function pendingCallTool(messages: readonly LhMessage[], toolCallId: string): string | null { + for (const m of messages) { + if ((m as { role?: string }).role !== "assistant") continue; + const content = (m as { content?: unknown }).content; + if (!Array.isArray(content)) continue; + for (const block of content as Array<{ type?: string; id?: string; name?: string }>) { + if (block.type === "toolCall" && block.id === toolCallId) return block.name ?? null; + } + } + return null; +} + +/** + * Has the record ABSORBED this optimistic tail entry? The write-ahead + * persists content-identical copies with server-stamped ids — match on the + * content, not the (local) id; only `c_`-prefixed local entries are ever + * absorbed (live turn content is never dropped). + */ +export function absorbed(m: LhMessage, record: readonly LhMessage[]): boolean { + if (typeof m.id !== "string" || !m.id.startsWith("c_")) return false; + const role = (m as { role?: string }).role; + if (role === "user") { + const content = (m as { content?: unknown }).content; + return record.some( + (r) => (r as { role?: string }).role === "user" && (r as { content?: unknown }).content === content, + ); + } + if (role === "toolResult") { + const toolCallId = (m as ToolResultMessage).toolCallId; + return record.some( + (r) => + (r as { role?: string }).role === "toolResult" && + (r as ToolResultMessage).toolCallId === toolCallId, + ); + } + if (role === "custom") { + const marker = m as { customType?: string; details?: { accepted?: unknown } }; + if (marker.customType !== "verdict") return false; + return record.some( + (r) => + (r as { role?: string }).role === "custom" && + (r as { customType?: string }).customType === "verdict" && + (r as { details?: { accepted?: unknown } }).details?.accepted === marker.details?.accepted, + ); + } + return false; +} + +export interface SessionState { + messages: LhViewMessage[]; + nativeMessages: LhMessage[]; + status: SessionStatus; + error: { message: string } | null; +} + +export interface SessionModelOptions extends SessionModelDeps { + chatId: string; + initialMessages: LhMessage[]; + /** Fires on every state change (the hook re-renders). */ + onChange?: () => void; + /** The reattach settled terminal (turn-settled): no turn running, the + * record is current — the caller refetches it. */ + onReattachSettled?: () => void; +} + +export class SessionModel { + readonly chatId: string; + /** Injectable fetch (tests); defaults to the global. */ + fetch: typeof globalThis.fetch; + private readonly pollMs: number; + private readonly waitMaxMs: number; + private onReattachSettled?: () => void; + /** The hook's re-render trigger (set by the adapter). */ + onChange?: () => void; + + // THE STATE: the record (server canon) + the live tail (this client's + // optimistic appends and the streaming turn's messages). The tail dies + // at every full sync; the write-ahead absorbs the optimistic entries. + private base: LhMessage[]; + private tail: LhMessage[] = []; + // The live turn's execution layer (view model mid-turn states): + private executions = new Map(); + private streamingIds = new Set(); + private statusValue: SessionStatus = "ready"; + private errorValue: { message: string } | null = null; + // The single live consume (POST response or reattach stream): a newer + // consume cancels the older — one reader at a time. + private consumeAbort: AbortController | null = null; + /** Synchronous ready check for the gate (send/read mount logic). */ + readyRef = true; + + constructor(opts: SessionModelOptions) { + this.chatId = opts.chatId; + this.base = opts.initialMessages; + this.fetch = opts.fetch ?? globalThis.fetch; + this.pollMs = opts.pollMs ?? TURN_POLL_MS; + this.waitMaxMs = opts.waitMaxMs ?? TURN_WAIT_MAX_MS; + this.onChange = opts.onChange; + this.onReattachSettled = opts.onReattachSettled; + } + + /** Injectable settle hook (the adapter wires its ref). */ + setReattachSettled(fn: () => void): void { + this.onReattachSettled = fn; + } + + getState(): SessionState { + const live = + this.statusValue === "submitted" || this.statusValue === "streaming" + ? { executions: this.executions, inFlight: true, streamingIds: this.streamingIds } + : undefined; + return { + messages: projectView([...this.base, ...this.tail], live), + nativeMessages: [...this.base, ...this.tail], + status: this.statusValue, + error: this.errorValue, + }; + } + + get status(): SessionStatus { + return this.statusValue; + } + + /** Mutate model state + notify (the hook re-renders). */ + private mutate(fn: () => void): void { + fn(); + this.onChange?.(); + } + + private setStatus(status: SessionStatus): void { + this.statusValue = status; + this.readyRef = status === "ready"; + this.onChange?.(); + } + + private setError(message: string | null): void { + this.errorValue = message === null ? null : { message }; + this.onChange?.(); + } + + /** The 409 identity-retry fetch (the Wave B contract): the event was + * never persisted (write-ahead happens only on acceptance), so the + * retry waits out the running turn and re-POSTs the SAME body. */ + async postInputEvent(input: ChatInput, signal?: AbortSignal): Promise { + const body = JSON.stringify({ chatId: this.chatId, input }); + const init: RequestInit = { + method: "POST", + headers: { "Content-Type": "application/json" }, + body, + signal, + }; + let response = await this.fetch("/api/chat", init); + let waited = 0; + while (response.status === 409) { + await response.body?.cancel().catch(() => {}); + if (signal?.aborted || waited >= this.waitMaxMs) return response; + await delay(this.pollMs, signal); + waited += this.pollMs; + if (signal?.aborted) return response; + if ((await turnInFlight(this)) !== false) continue; + response = await this.fetch("/api/chat", init); // may 409 again → loop + } + return response; + } + + /** + * ONE event application for both the POST stream and the reattach + * replay (pi's events carry the FULL partial message — a view REPLACES, + * never assembles deltas). + */ + applyWireEvent(event: Record): void { + if (event.type === "error") { + // Loop-level failure (agent.continue threw outside the stream) — + // raw text (owner ruling, Wave D): the banner shows exactly this. + this.setError(String(event.errorText ?? "An error occurred.")); + return; + } + const agent = event as unknown as AgentEvent; + switch (agent.type) { + case "message_start": + case "message_update": + case "message_end": { + const message = agent.message as LhMessage & { + role?: string; + id?: string; + }; + if ((message as { role?: string }).role !== "assistant") return; + const id = message.id ?? localId(); + const withId = { ...message, id } as LhMessage; + if (agent.type === "message_start") { + this.streamingIds.add(id); + this.mutate(() => { + this.tail = [...this.tail, withId]; + }); + } else { + this.mutate(() => { + const idx = this.tail.findIndex((m) => m.id === id); + this.tail = idx === -1 ? [...this.tail, withId] : this.tail.with(idx, withId); + }); + } + if (agent.type === "message_end") { + const stop = (message as { stopReason?: string }).stopReason; + if (stop === "error") { + // Provider error, verbatim (owner ruling) — the record + // persists the same message; no masking anywhere. + this.setError( + String((message as { errorMessage?: string }).errorMessage ?? "An error occurred."), + ); + } + } + return; + } + case "tool_execution_end": { + // The execution layer: clean question placeholders are IGNORED + // downstream (view.ts fork 2); errors keep their validation text. + const result = agent.result as + | { content?: Array<{ type?: string; text?: string }>; details?: unknown } + | null; + const text = result?.content?.find((c) => c.type === "text")?.text ?? "The tool call failed."; + this.mutate(() => { + this.executions = new Map(this.executions).set(agent.toolCallId, { + isError: agent.isError, + output: result?.details, + errorText: text, + }); + }); + return; + } + case "turn_end": { + // The results batch: append as native messages (client-stamped + // ids — the settle sync lands the server canon). + this.mutate(() => { + this.tail = [ + ...this.tail, + ...(agent.toolResults as ToolResultMessage[]).map( + (r) => ({ ...r, id: localId() }) as unknown as LhMessage, + ), + ]; + }); + return; + } + default: + return; // agent_start/turn_start/tool_execution_start|update: no view work + } + } + + /** Consume one SSE body to its [DONE] terminal. The turn-settled + * terminal (stream-route only) fires the settle hook and ends. */ + async consumeStream(res: Response): Promise { + if (!res.body) throw new Error("The turn stream body is empty."); + const reader = res.body.getReader(); + let sawEvent = false; + try { + for await (const event of sseEvents(reader)) { + if (event.type === "turn-settled") { + // The raced-settle terminal: no turn to attach to — the + // record is current; the settle hook refetches. + await reader.cancel().catch(() => {}); + this.onReattachSettled?.(); + return; + } + if (!sawEvent) { + sawEvent = true; + if (this.statusValue !== "error") this.setStatus("streaming"); + } + this.applyWireEvent(event); + } + } finally { + this.streamingIds = new Set(); + this.executions = new Map(); + if (this.statusValue !== "error") this.setStatus("ready"); + } + } + + /** + * Reattach (the wake/handoff/mount path): a server-side turn with no + * live view of ours. The record is refetched FIRST (write-ahead + * adoption — the tail's optimistic entries are absorbed, its partial + * stream content is about to be fully replayed), then the stream + * replays the turn from its first event through the SAME consume path. + */ + async resumeStream(): Promise { + this.consumeAbort?.abort(); + const controller = new AbortController(); + this.consumeAbort = controller; + this.setStatus("submitted"); + try { + const recordRes = await this.fetch(`/api/chats/${encodeURIComponent(this.chatId)}`, { + signal: controller.signal, + }); + if (recordRes.ok) { + const record = (await recordRes.json()) as { messages?: LhMessage[] }; + if (Array.isArray(record.messages)) { + this.mutate(() => { + this.base = record.messages!; + this.tail = []; + }); + } + } + const res = await this.fetch(`/api/chats/${encodeURIComponent(this.chatId)}/stream`, { + signal: controller.signal, + }); + if (!res.ok) throw new Error((await res.text()) || "Failed to attach to the running turn."); + await this.consumeStream(res); + } catch (err) { + if ((err as Error).name === "AbortError") return; // a newer consume won + this.setError((err as Error).message); + this.setStatus("error"); + } + } + + /** + * SEND: one explicit input event. Optimistic appends first (the instant + * answer/note UX — the server's write-ahead persists the canon), then + * POST + consume. Ready-gated: one turn per client at a time (the + * server's 409 + identity retry covers the wake-race window). + */ + async send(input: ChatInput): Promise { + if (!this.readyRef) return; + this.consumeAbort?.abort(); + const controller = new AbortController(); + this.consumeAbort = controller; + this.setError(null); + this.setStatus("submitted"); + // The optimistic appends — exactly the messages the server's + // write-ahead will persist (inputAppends), with LOCAL ids so + // adoption can tell them from the record's absorbed copies. + const optimistic: LhMessage[] = []; + if (input.type === "text") optimistic.push({ ...newUserMessage(input.text), id: localId() }); + else if (input.type === "answer") { + const tool = pendingCallTool([...this.base, ...this.tail], input.toolCallId) ?? "quiz"; + optimistic.push({ ...newAnswerResult(input.toolCallId, tool, input.answer), id: localId() }); + } else if (input.type === "verdict") { + if (input.note) optimistic.push({ ...newUserMessage(input.note), id: localId() }); + optimistic.push({ + ...newVerdictMessage(input.accepted, new Date().toISOString()), + id: localId(), + }); + } + if (optimistic.length > 0) this.mutate(() => { + this.tail = [...this.tail, ...optimistic]; + }); + try { + const res = await this.postInputEvent(input, controller.signal); + if (controller.signal.aborted) return; + if (res.status === 409) { + this.setError("A turn is already running — try again in a moment."); + this.setStatus("error"); + return; + } + if (!res.ok) { + // Refusal envelopes (503 BYOK/key gates) and 400s: the RAW body + // text rides the error message — the banner shows it, and the + // page's refusal flow JSON-parses the envelope out of it. + throw new Error((await res.text()) || `The send failed (${res.status}).`); + } + await this.consumeStream(res); + } catch (err) { + if ((err as Error).name === "AbortError") return; // stop() or a newer consume won + this.setError((err as Error).message); + this.setStatus("error"); + } + } + + /** The answer path (the dock): the answer IS the question call's tool + * result (fork 1), riding as its input event. */ + submitAnswer(tool: string, toolCallId: string, output: QuestionAnswerData): Promise { + void tool; + return this.send({ type: "answer", toolCallId, answer: output }); + } + + /** Local detach: cancel the consume, keep the partial tail. The page's + * settle watch adopts the record when the server turn finishes. */ + stop(): void { + this.consumeAbort?.abort(); + this.consumeAbort = null; + this.streamingIds = new Set(); + this.executions = new Map(); + this.setStatus("ready"); + } + + /** Full snapshot swap (the idle settle sync — server canon). */ + setMessages(record: LhMessage[]): void { + this.mutate(() => { + this.base = record; + this.tail = []; + }); + } + + /** + * Append-only adoption (the mid-turn steer/artifact ground rule): swap + * in the record — every current base message must still be present + * (the server appends only) — keeping the live tail, minus the + * optimistic entries the record has absorbed (their server-stamped + * copies just landed; without this the input bubble would double). + */ + adoptRecord(record: LhMessage[]): void { + const recordIds = new Set(record.map((m) => (m as { id?: string }).id)); + const baseAbsorbed = this.base.some((m) => !recordIds.has((m as { id?: string }).id)); + if (baseAbsorbed) return; // not a pure append — the idle full sync is the net + this.mutate(() => { + this.base = record; + this.tail = this.tail.filter((m) => !absorbed(m, record)); + }); + } + + /** The mount/sync continuation fold (native-messages.ts). */ + static needsContinuation = needsContinuation; + + clearError(): void { + this.setError(null); + } +} diff --git a/components/session.ts b/components/session.ts new file mode 100644 index 0000000..f9aac07 --- /dev/null +++ b/components/session.ts @@ -0,0 +1,87 @@ +"use client"; + +/** + * THE SESSION HOOK (Wave D, owner-approved 2026-09-26): the React adapter + * over the pure SessionModel (components/session-core.ts — the state + * machine, testable without a renderer). All mechanics live in the model; + * this hook re-renders on change and derives the view (core/view.ts + * projectView — the golden-master grouping, client-side). + * + * Every turn is a server-side background process (owner ruling + * 2026-09-25); the hook is a view. Sends are ONE explicit input event + * each ({chatId, input}); the turn stream is the VERBATIM AgentEvent + * passthrough. Status vocabulary (pre-Wave-D parity): ready | submitted + * | streaming | error. + */ + +import { useCallback, useEffect, useMemo, useReducer, useRef } from "react"; +import type { ChatInput, LhMessage, QuestionAnswerData } from "@/src/core"; +import { SessionModel, type SessionState } from "@/components/session-core"; + +export interface UseChatSessionOptions { + chatId: string; + /** The native record seed (server-rendered page or a full sync). */ + initialMessages: LhMessage[]; + /** Reattach on mount: a turn is running server-side with no live view. */ + initiallyInFlight?: boolean; +} + +export function useChatSession({ chatId, initialMessages, initiallyInFlight }: UseChatSessionOptions) { + // The model is per-mount (chatId-keyed component); onChange forces the + // re-render. The ref pattern keeps it stable across renders. + const modelRef = useRef(null); + if (modelRef.current === null) { + modelRef.current = new SessionModel({ chatId, initialMessages }); + } + const model = modelRef.current; + const [, forceRender] = useReducer((n: number) => n + 1, 0); + useEffect(() => { + model.onChange = forceRender; + return () => { + model.onChange = undefined; + }; + }, [model, forceRender]); + + // Reattach on mount (StrictMode-safe: a newer resume cancels the older). + useEffect(() => { + if (initiallyInFlight) void model.resumeStream(); + }, [initiallyInFlight, model]); + + // The view + state, derived per render from the model. + const state = useMemo(() => model.getState(), [model, forceRender] as const); + + // The reattach settled terminal → the sync-and-continue seam (chat.tsx). + const setReattachSettled = useCallback( + (fn: () => void) => { + model.setReattachSettled(fn); + }, + [model], + ); + + const send = useCallback((input: ChatInput) => void model.send(input), [model]); + const submitAnswer = useCallback( + (tool: string, toolCallId: string, output: QuestionAnswerData) => + void model.submitAnswer(tool, toolCallId, output), + [model], + ); + const setMessages = useCallback((record: LhMessage[]) => model.setMessages(record), [model]); + const adoptRecord = useCallback((record: LhMessage[]) => model.adoptRecord(record), [model]); + const stop = useCallback(() => model.stop(), [model]); + const clearError = useCallback(() => model.clearError(), [model]); + const resumeStream = useCallback(() => void model.resumeStream(), [model]); + + return { + messages: state.messages, + nativeMessages: state.nativeMessages, + status: state.status, + error: state.error, + send, + submitAnswer, + setMessages, + adoptRecord, + stop, + clearError, + resumeStream, + setReattachSettled, + }; +} diff --git a/package-lock.json b/package-lock.json index fe28a57..7c7cf90 100644 --- a/package-lock.json +++ b/package-lock.json @@ -8,7 +8,6 @@ "name": "learning-harness", "version": "0.2.0", "dependencies": { - "@ai-sdk/react": "^4.0.79", "@earendil-works/pi-agent-core": "^0.85.1", "@earendil-works/pi-ai": "^0.85.1", "@mozilla/readability": "^0.6.0", @@ -19,7 +18,6 @@ "@streamdown/math": "^1.0.2", "@streamdown/mermaid": "^1.0.2", "@xyflow/react": "^12.11.3", - "ai": "^7.0.76", "ansi-to-react": "^6.2.6", "class-variance-authority": "^0.7.1", "clsx": "^2.1.1", @@ -73,91 +71,6 @@ "typescript": "^5.6.0" } }, - "node_modules/@ai-sdk/gateway": { - "version": "4.0.61", - "resolved": "https://registry.npmjs.org/@ai-sdk/gateway/-/gateway-4.0.61.tgz", - "integrity": "sha512-b+PRdwM9/x0427CKqfd4zpCRgSzQeZsCZ9PtBOaLU2S1l+eRj8++Bwt766VnLerd/F5snKWAq13miV+OPcvclA==", - "license": "Apache-2.0", - "dependencies": { - "@ai-sdk/provider": "4.0.7", - "@ai-sdk/provider-utils": "5.0.28", - "@vercel/oidc": "3.2.0" - }, - "engines": { - "node": ">=22" - }, - "peerDependencies": { - "zod": "^3.25.76 || ^4.1.8" - } - }, - "node_modules/@ai-sdk/mcp": { - "version": "2.0.35", - "resolved": "https://registry.npmjs.org/@ai-sdk/mcp/-/mcp-2.0.35.tgz", - "integrity": "sha512-hyaryid0qH/d8VtNZoJKeqcLy6b3v9xlkBL3JklbFXMAZJ8nmkxngrxQtjbNRTKjSFoyQ95/7IuRIijrSljMww==", - "license": "Apache-2.0", - "dependencies": { - "@ai-sdk/provider": "4.0.7", - "@ai-sdk/provider-utils": "5.0.28", - "pkce-challenge": "^5.0.1" - }, - "engines": { - "node": ">=22" - }, - "peerDependencies": { - "zod": "^3.25.76 || ^4.1.8" - } - }, - "node_modules/@ai-sdk/provider": { - "version": "4.0.7", - "resolved": "https://registry.npmjs.org/@ai-sdk/provider/-/provider-4.0.7.tgz", - "integrity": "sha512-6or44XprPzKbr8zkmzosowSE0pxkvJcoojBL+mCZvPUt3kvXp3XSNqeVun9golb1acEfSo6yaEBRT18h2VU+1Q==", - "license": "Apache-2.0", - "dependencies": { - "json-schema": "^0.4.0" - }, - "engines": { - "node": ">=22" - } - }, - "node_modules/@ai-sdk/provider-utils": { - "version": "5.0.28", - "resolved": "https://registry.npmjs.org/@ai-sdk/provider-utils/-/provider-utils-5.0.28.tgz", - "integrity": "sha512-TnHUyd/rCYQqHg5RuiOaz/hUql6U+kbUaBW0Rp+0N5UhnAInA9CzzV0HXvuAAPwppDsK6fAx9Rd+tRawpJ/3pg==", - "license": "Apache-2.0", - "dependencies": { - "@ai-sdk/provider": "4.0.7", - "@standard-schema/spec": "^1.1.0", - "@workflow/serde": "4.1.0", - "eventsource-parser": "^3.0.8", - "undici": "^7.28.0" - }, - "engines": { - "node": ">=22" - }, - "peerDependencies": { - "zod": "^3.25.76 || ^4.1.8" - } - }, - "node_modules/@ai-sdk/react": { - "version": "4.0.79", - "resolved": "https://registry.npmjs.org/@ai-sdk/react/-/react-4.0.79.tgz", - "integrity": "sha512-x7dZ/teRKXgJfukoOSiKA4pMv1AbMLjTGA8/d9Dofl8Mgf4DfRNuBxs6xlcedPypWTJXkK8whac2THkOpoFzug==", - "license": "Apache-2.0", - "dependencies": { - "@ai-sdk/mcp": "2.0.35", - "@ai-sdk/provider": "4.0.7", - "@ai-sdk/provider-utils": "5.0.28", - "ai": "7.0.76", - "swr": "^2.4.1", - "throttleit": "2.1.0" - }, - "engines": { - "node": ">=22" - }, - "peerDependencies": { - "react": "^18 || ~19.0.1 || ~19.1.2 || ^19.2.1" - } - }, "node_modules/@alloc/quick-lru": { "version": "5.2.0", "resolved": "https://registry.npmjs.org/@alloc/quick-lru/-/quick-lru-5.2.0.tgz", @@ -5245,12 +5158,6 @@ "integrity": "sha512-1bnPQqSxSuc3Ii6MhBysoWCg58j97aUjuCSZrGSmDxNqtytIi0k8utUenAwTZN4V5mXXYGsVUI9zeBqy+jBOSQ==", "license": "MIT" }, - "node_modules/@standard-schema/spec": { - "version": "1.1.0", - "resolved": "https://registry.npmjs.org/@standard-schema/spec/-/spec-1.1.0.tgz", - "integrity": "sha512-l2aFy5jALhniG5HgqrD6jXLi/rUWrKvqN/qJx6yoJsgKhblVd+iqqU4RCXavm/jPityDo5TCvKMnpjKnOriy0w==", - "license": "MIT" - }, "node_modules/@streamdown/cjk": { "version": "1.0.3", "resolved": "https://registry.npmjs.org/@streamdown/cjk/-/cjk-1.0.3.tgz", @@ -6682,21 +6589,6 @@ "d3-transition": "^3.0.1" } }, - "node_modules/@vercel/oidc": { - "version": "3.2.0", - "resolved": "https://registry.npmjs.org/@vercel/oidc/-/oidc-3.2.0.tgz", - "integrity": "sha512-UycprH3T6n3jH0k44NHMa7pnFHGu/N05MjojYr+Mc6I7obkoLIJujSWwin1pCvdy/eOxrI/l3uDLQsmcrOb4ug==", - "license": "Apache-2.0", - "engines": { - "node": ">= 20" - } - }, - "node_modules/@workflow/serde": { - "version": "4.1.0", - "resolved": "https://registry.npmjs.org/@workflow/serde/-/serde-4.1.0.tgz", - "integrity": "sha512-pav4F2BoirECWR7Nf1TKt+2eETcBj7jj4cBefQ8VXQCA6NPkaKeLfj/zMgi+3zYV5ZIBT4GuUiphsj0/b9hPQQ==", - "license": "Apache-2.0" - }, "node_modules/@xmldom/xmldom": { "version": "0.8.15", "resolved": "https://registry.npmjs.org/@xmldom/xmldom/-/xmldom-0.8.15.tgz", @@ -6789,23 +6681,6 @@ "node": ">= 14" } }, - "node_modules/ai": { - "version": "7.0.76", - "resolved": "https://registry.npmjs.org/ai/-/ai-7.0.76.tgz", - "integrity": "sha512-Mytm/RFJzwJ0nzXiHYAE3dgNpFG/2K2Tks5alwm+PzAQeB6RJAJ97eGuZ8wssy66PcvtGWGIscS4tRi6dg0KLQ==", - "license": "Apache-2.0", - "dependencies": { - "@ai-sdk/gateway": "4.0.61", - "@ai-sdk/provider": "4.0.7", - "@ai-sdk/provider-utils": "5.0.28" - }, - "engines": { - "node": ">=22" - }, - "peerDependencies": { - "zod": "^3.25.76 || ^4.1.8" - } - }, "node_modules/ajv": { "version": "6.15.0", "resolved": "https://registry.npmjs.org/ajv/-/ajv-6.15.0.tgz", @@ -10139,15 +10014,6 @@ "node": ">=0.10.0" } }, - "node_modules/eventsource-parser": { - "version": "3.1.1", - "resolved": "https://registry.npmjs.org/eventsource-parser/-/eventsource-parser-3.1.1.tgz", - "integrity": "sha512-EKN1vKAMcZ8MlYMpaNuxN6R9yakzH6uajHcHVTqWJzvu5pWw9DyhbP35HH8MVBQ+dZjAfDxk+A8NiR9KWaXiyQ==", - "license": "MIT", - "engines": { - "node": ">=18.0.0" - } - }, "node_modules/exponential-backoff": { "version": "3.1.3", "resolved": "https://registry.npmjs.org/exponential-backoff/-/exponential-backoff-3.1.3.tgz", @@ -12091,12 +11957,6 @@ "dev": true, "license": "MIT" }, - "node_modules/json-schema": { - "version": "0.4.0", - "resolved": "https://registry.npmjs.org/json-schema/-/json-schema-0.4.0.tgz", - "integrity": "sha512-es94M3nTIfsEPisRafak+HDLfHXnKBhV3vU5eqPcS3flIWqcxJWgXHXiey3YrpaNsanY5ei1VoYEbOzijuq9BA==", - "license": "(AFL-2.1 OR BSD-3-Clause)" - }, "node_modules/json-schema-to-ts": { "version": "3.1.1", "resolved": "https://registry.npmjs.org/json-schema-to-ts/-/json-schema-to-ts-3.1.1.tgz", @@ -14680,15 +14540,6 @@ "url": "https://github.com/sponsors/jonschlinkert" } }, - "node_modules/pkce-challenge": { - "version": "5.0.1", - "resolved": "https://registry.npmjs.org/pkce-challenge/-/pkce-challenge-5.0.1.tgz", - "integrity": "sha512-wQ0b/W4Fr01qtpHlqSqspcj3EhBvimsdh0KlHhH8HRZnMsEa0ea2fTULOXOS9ccQr3om+GcGRk4e+isrZWV8qQ==", - "license": "MIT", - "engines": { - "node": ">=16.20.0" - } - }, "node_modules/pkijs": { "version": "3.4.0", "resolved": "https://registry.npmjs.org/pkijs/-/pkijs-3.4.0.tgz", @@ -16530,19 +16381,6 @@ "url": "https://github.com/sponsors/ljharb" } }, - "node_modules/swr": { - "version": "2.5.1", - "resolved": "https://registry.npmjs.org/swr/-/swr-2.5.1.tgz", - "integrity": "sha512-BRw55e8r0B7SpDN20CAzoQAHl7y1yP7/Zt7oqUjMv0vSt2u2Xnkm88Ws+VypbV9BXHQVuSuyVq7zMjO16wSExw==", - "license": "MIT", - "dependencies": { - "dequal": "^2.0.3", - "use-sync-external-store": "^1.6.0" - }, - "peerDependencies": { - "react": "^16.11.0 || ^17.0.0 || ^18.0.0 || ^19.0.0" - } - }, "node_modules/tailwind-merge": { "version": "3.6.0", "resolved": "https://registry.npmjs.org/tailwind-merge/-/tailwind-merge-3.6.0.tgz", @@ -16627,18 +16465,6 @@ "fs-extra": "^10.0.0" } }, - "node_modules/throttleit": { - "version": "2.1.0", - "resolved": "https://registry.npmjs.org/throttleit/-/throttleit-2.1.0.tgz", - "integrity": "sha512-nt6AMGKW1p/70DF/hGBdJB57B8Tspmbp5gfJ8ilhLnt7kkr2ye7hzD6NVG8GGErk2HWF34igrL2CXmNIkzKqKw==", - "license": "MIT", - "engines": { - "node": ">=18" - }, - "funding": { - "url": "https://github.com/sponsors/sindresorhus" - } - }, "node_modules/tiny-async-pool": { "version": "1.3.0", "resolved": "https://registry.npmjs.org/tiny-async-pool/-/tiny-async-pool-1.3.0.tgz", @@ -17046,7 +16872,9 @@ "version": "7.29.0", "resolved": "https://registry.npmjs.org/undici/-/undici-7.29.0.tgz", "integrity": "sha512-IDxfleLmmbSskfWSUATiN1nfn2rDuvnMOqb5CWR92iIfojA0Ud+ulOAAEQ57LPr9rWmsreUyf5lwyao+7GNNVw==", + "dev": true, "license": "MIT", + "optional": true, "engines": { "node": ">=20.18.1" } diff --git a/package.json b/package.json index 6ea7b04..9cd9b3c 100644 --- a/package.json +++ b/package.json @@ -17,7 +17,6 @@ "desktop:dist": "next build && node scripts/build-server.mjs && electron-builder --mac --arm64" }, "dependencies": { - "@ai-sdk/react": "^4.0.79", "@earendil-works/pi-agent-core": "^0.85.1", "@earendil-works/pi-ai": "^0.85.1", "typebox": "1.3.7", @@ -29,7 +28,6 @@ "@streamdown/math": "^1.0.2", "@streamdown/mermaid": "^1.0.2", "@xyflow/react": "^12.11.3", - "ai": "^7.0.76", "ansi-to-react": "^6.2.6", "class-variance-authority": "^0.7.1", "clsx": "^2.1.1", diff --git a/src/agent/subagents/extraction.ts b/src/agent/subagents/extraction.ts index 13a9e0f..3294da4 100644 --- a/src/agent/subagents/extraction.ts +++ b/src/agent/subagents/extraction.ts @@ -1,5 +1,4 @@ import * as crypto from "node:crypto"; -import type { UIMessage } from "ai"; import type { Usage } from "@earendil-works/pi-ai"; import type { AgentTool } from "@earendil-works/pi-agent-core"; import { Type, type Static, type TSchema } from "typebox"; @@ -28,7 +27,6 @@ import { getGraph, mergeFindings, type EmissionStats, type GraphDeps } from "../ import type { GraphStore } from "../../core/ports.ts"; import { settleGraphLayout } from "../../core/layout-settle.ts"; import { batchRuns, batchTag, coverageOf, messageViewLines } from "../../core/coverage.ts"; -import { projectNative } from "../../core/projection.ts"; /** * The extraction pipeline (vision §1 construction, 2026-09-02 redesign) — the @@ -93,12 +91,11 @@ import { projectNative } from "../../core/projection.ts"; export function transcriptView(record: ChatRecord): string { // The shared seam (core/coverage.ts messageViewLines): the LLM's prompt // view and the coverage fingerprint use ONE line builder, so "covered" - // means — exactly — what the extractor saw. Wave B: the record is native - // history — the extraction view reads through the one-way projection. + // means — exactly — what the extractor saw. Wave D: the record is native + // history — the view reads the native messages directly. const lines = [`SESSION: ${record.title}`]; - const projected = projectNative(record.messages); - projected.forEach((message, index) => { - lines.push(...messageViewLines(message, message.id || `m${index}`)); + record.messages.forEach((message, index) => { + lines.push(...messageViewLines(message, (message as { id?: string }).id || `m${index}`)); }); return lines.join("\n"); } @@ -871,14 +868,16 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti // engines AND their providers' keys; a refusal lands BEFORE any // stage runs (typed — the trigger skips, routes surface 503). refuseIfStagesUnready(); - const projected = projectNative(record.messages); - const covered = coverageOf(projected, graphStore.readCoverage(), chatId); + const covered = coverageOf(record.messages, graphStore.readCoverage(), chatId); if (covered.uncovered.length === 0) { console.info(`[extraction ${chatId}] nothing uncovered — no-op`); return { nodes: 0, edges: 0, events: 0 }; } const chunkTokens = deps.getChunkTokens?.() ?? FACTORY_SETTINGS.graph.extractionChunkTokens; - const batches = batchRuns(covered.uncovered, chunkTokens); + // The batcher's one rule (owner, Wave D): a tool-call message and + // its result message(s) are INSEPARABLE — one chunk, covered + // together, read together (core/coverage.ts batchRuns). + const batches = batchRuns(covered.slots, covered.uncovered, chunkTokens); console.info( `[extraction ${chatId}] pipeline start — ${covered.uncoveredTokens} uncovered tok, ` + `${covered.uncoveredCount} messages, ${batches.length} batch(es) of ≤${chunkTokens} tok`, @@ -887,13 +886,9 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti // The batch's prompt view: SESSION header + the batch's lines, // same provenance [id] tags the fingerprints used (coverage // and prompt share ONE line builder — core/coverage.ts). - const mids = projected.map((m, i) => (m.id || `m${i}`) as string); const view = [ `SESSION: ${record.title}`, - ...batch.flatMap((slot) => { - const idx = projected.indexOf(slot.message); - return messageViewLines(slot.message, mids[idx] ?? slot.message.id); - }), + ...batch.flatMap((slot) => messageViewLines(slot.message, slot.message.id)), ].join("\n"); setPhase(chatId, "extracting", "chat"); const findings = await extract(chatId, view); @@ -918,7 +913,9 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti const stats = mergeFindings(chatId, auditResult, { chatStore, graphStore }, { messages: batch.map((slot) => slot.message), foldedAt, - coverage: batch.map((slot) => ({ chatId, fp: slot.fp, tokens: slot.tokens, foldedAt })), + coverage: batch + .filter((slot) => messageViewLines(slot.message, slot.message.id).length > 0) + .map((slot) => ({ chatId, fp: slot.fp, tokens: slot.tokens, foldedAt })), batchTag: batchTag(batch), }); total.nodes += stats.nodes; diff --git a/src/agent/subagents/runner.ts b/src/agent/subagents/runner.ts index 7077952..9621091 100644 --- a/src/agent/subagents/runner.ts +++ b/src/agent/subagents/runner.ts @@ -1,4 +1,3 @@ -import { isToolUIPart } from "ai"; import type { PiEngine } from "../models.ts"; import type { SearchProvider } from "../../core/search.ts"; import { @@ -23,8 +22,6 @@ import { registerJobAbort, unregisterJobAbort } from "./aborts.ts"; import type { ModelRole } from "../../../config.ts"; import { ROLE_SLOT, modelsNotConfigured, type ModelSlotName, type ProviderKeyMissing } from "../../core/settings.ts"; import type { ChatStore, JobStore } from "../../core/ports.ts"; -import { projectNative } from "../../core/projection.ts"; -import type { UIMessage } from "ai"; import { runDemoBuilder } from "./demo-builder.ts"; import { runPdfReaderAgent } from "./pdf-reader.ts"; import { runResearcherAgent } from "./researcher.ts"; @@ -276,22 +273,18 @@ export function createJobRunner(deps: JobRunnerDeps): JobRunner { if (input) return input; const chat = chatStore.getChat(job.chatId); if (!chat) return undefined; - // Wave B: the record is native history — the transcript scan reads - // through the projection (same shape the old record held). - return (findToolPart(projectNative(chat.messages), job.toolCallId)?.input ?? undefined) as - | JobToolInput - | undefined; - } - - // ─── shared plumbing ──────────────────────────────────────────────────── - - function findToolPart(messages: UIMessage[], toolCallId: string) { - for (const m of messages) { - for (const part of m.parts) { - if (isToolUIPart(part) && part.toolCallId === toolCallId) return part; + // Wave D: the record is native history — the scan reads the native + // assistant blocks directly (a toolCall's arguments are the input). + for (const m of chat.messages) { + if ((m as { role?: string }).role !== "assistant") continue; + const content = (m as { content?: unknown }).content; + if (!Array.isArray(content)) continue; + for (const block of content as Array<{ type?: string; id?: string; arguments?: unknown }>) { + if (block.type !== "toolCall" || block.id !== job.toolCallId) continue; + return block.arguments as JobToolInput | undefined; } } - return null; + return undefined; } /** diff --git a/src/core/contracts/chat.ts b/src/core/contracts/chat.ts index 2174568..7f49f73 100644 --- a/src/core/contracts/chat.ts +++ b/src/core/contracts/chat.ts @@ -3,10 +3,8 @@ * * Part of the domain-split contracts (src/core/contracts/): each domain file * is owned outright by its stream — cross-domain schema changes go through - * master. Pure types + protocol docs + tiny pure helpers; type-only imports - * of "ai" (safe everywhere: server, client, tests). See docs/vision.md. + * master. Pure types + protocol docs + tiny pure helpers. See docs/vision.md. */ -import type { UIMessage } from "ai"; import type { ThinkingLevel } from "@earendil-works/pi-agent-core"; import type { LhMessage } from "../native-messages.ts"; import type { Phase3 } from "../phases.ts"; @@ -20,8 +18,8 @@ import type { EmissionStats, UnderstandingLevel } from "./graph.ts"; * agent loop's own shapes, provider/model/usage/thinkingSignature stamped * by pi) — the canonical session record IS the agent state; replay is * `state.messages = record.messages` + continue(). The client-shaped view - * is the one-way projection (core/projection.ts) — a pending quiz is an - * unanswered question tool call, which is exactly what makes + * is the client's own grouping fold (core/view.ts, Wave D) — a pending quiz + * is an unanswered question tool call, which is exactly what makes * resume-mid-question free. */ export interface ChatRecord { id: string; diff --git a/src/core/contracts/plan.ts b/src/core/contracts/plan.ts index 68b63db..84c0e74 100644 --- a/src/core/contracts/plan.ts +++ b/src/core/contracts/plan.ts @@ -115,3 +115,28 @@ export interface PlanToolResult { tree: PlanTree; } + +// ─── the native fold (Wave D: derivePlanStatuses over LhMessage[]) ─────────── + +import type { LhMessage } from "../native-messages.ts"; + +/** The same derivation over the NATIVE record: quiz CALLS only (any state — + * a dangling call still moves position; results feed nothing). */ +export function derivePlanStatusesLh(tree: PlanTree, messages: readonly LhMessage[]): DerivedPlanStatuses { + const ids = new Set(tree.nodes.map((n) => n.id)); + const runs: string[] = []; + for (const m of messages) { + if ((m as { role?: string }).role !== "assistant") continue; + const content = (m as { content?: unknown }).content; + if (!Array.isArray(content)) continue; + for (const block of content as Array<{ type?: string; name?: string; arguments?: unknown }>) { + if (block.type !== "toolCall" || block.name !== "quiz") continue; + const nodeId = (block.arguments as { nodeId?: unknown } | undefined)?.nodeId; + if (typeof nodeId !== "string" || !ids.has(nodeId)) continue; + if (runs.at(-1) !== nodeId) runs.push(nodeId); + } + } + const current = runs.at(-1) ?? null; + const visited = new Set(runs.slice(0, -1)); + return { current, visited }; +} diff --git a/src/core/coverage.ts b/src/core/coverage.ts index 2587ac0..7521007 100644 --- a/src/core/coverage.ts +++ b/src/core/coverage.ts @@ -1,63 +1,83 @@ /** * Extraction coverage (vision-extraction-coverage.md, settled 2026-09-15) — - * the pure core. THE MESSAGE IS THE UNIT OF COVERAGE: a per-chat append-only - * ledger records which messages have been extracted, by CONTENT FINGERPRINT - * over exactly what the extraction view sees; "uncovered" is DERIVED at - * read time (fingerprint the current transcript, diff against the ledger) — - * never dual-tracked. Runs are token-budgeted batches of the oldest - * uncovered messages, so a tool call and its result (which share a message) - * can never be separated. + * the pure core. THE MESSAGE IS THE UNIT OF COVERAGE (Wave D, owner ruling + * 2026-09-26: the native message, not the old projected group): a per-chat + * append-only ledger records which messages have been extracted, by CONTENT + * FINGERPRINT over exactly what the extraction view sees; "uncovered" is + * DERIVED at read time (fingerprint the current transcript, diff against the + * ledger) — never dual-tracked. Runs are token-budgeted batches of the oldest + * uncovered messages, and THE ONE BATCH RULE (owner): a tool-call message and + * its tool-result message(s) are INSEPARABLE — they always appear in the same + * chunk (a covered partner rides along as context; checkpoints are + * idempotent, so re-marking it is a no-op). * * The fingerprint and the LLM prompt share ONE line builder — * messageViewLines — so "covered" and "what the model saw" cannot drift * apart. The line logic is tree-blind, identical to the extractor's view: * user/teacher text verbatim, quiz outcomes as one-line summaries, other - * tool results as one generic line each, data parts invisible (plan-block - * attach does not re-open a covered message). + * clean tool results as one generic line each, markers invisible (a + * plan-block marker does not re-open a covered message). */ import { createHash } from "node:crypto"; -import type { UIMessage } from "ai"; -import { isToolUIPart } from "ai"; import type { CoverageEntry } from "./contracts/graph.ts"; +import type { LhMessage } from "./native-messages.ts"; +import type { ToolResultMessage } from "@earendil-works/pi-ai"; export type { CoverageEntry }; /** The quiz outcome line — both the fingerprint and the LLM view see the - * quiz answer, so an answered quiz re-opens the message that asked it. */ -function quizLine(part: { toolCallId: string; input: unknown; output: unknown }): string { - const input = part.input as { question?: unknown } | undefined; - const out = part.output as Partial<{ questionText: string; correct: boolean; dontKnow: boolean; note: string }> | undefined; - const question = typeof input?.question === "string" ? input.question : out?.questionText ?? "(no question)"; + * quiz answer, so an answered quiz re-opens (the answer is its own native + * message now; its fingerprint is new the moment it lands). */ +function quizLine(result: ToolResultMessage): string { + const out = result.details as + | Partial<{ questionText: string; correct: boolean; dontKnow: boolean; note: string }> + | undefined; + const question = typeof out?.questionText === "string" ? out.questionText : "(no question)"; const outcome = out?.dontKnow === true ? "did not know" : out?.correct === true ? "answered correctly" : "answered incorrectly"; const note = typeof out?.note === "string" && out.note ? ` (note: ${out.note})` : ""; - return `QUIZ [${part.toolCallId}] ${question} — the learner ${outcome}${note}`; + return `QUIZ [${result.toolCallId}] ${question} — the learner ${outcome}${note}`; +} + +/** The user-facing text of a native message (string or text blocks). */ +function textOf(content: unknown): string[] { + if (typeof content === "string") return content.trim() ? [content] : []; + if (!Array.isArray(content)) return []; + return content + .filter((c): c is { type: "text"; text: string } => (c as { type?: string }).type === "text") + .map((c) => c.text) + .filter((t) => t.trim().length > 0); } /** - * The extraction-relevant lines of ONE message — the shared seam between - * transcriptView (the LLM's prompt) and messageFingerprint (coverage). Same - * logic, same order, so a covered message is by construction exactly what - * the extractor saw. `mid` is the provenance tag (the message id; the - * transcript view passes an index fallback for legacy id-less messages, - * coverage passes the id — persisted messages always have ids). + * The extraction-relevant lines of ONE native message — the shared seam + * between transcriptView (the LLM's prompt) and messageFingerprint + * (coverage). Same logic, same order, so a covered message is by + * construction exactly what the extractor saw. `mid` is the provenance tag + * (the message id; the transcript view passes an index fallback for legacy + * id-less messages, coverage passes the id — persisted messages always have + * ids). */ -export function messageViewLines(message: UIMessage, mid: string): string[] { +export function messageViewLines(message: LhMessage, mid: string): string[] { const lines: string[] = []; - const who = message.role === "user" ? "LEARNER" : "TEACHER"; - for (const part of message.parts) { - if (part.type === "text" && part.text.trim()) { - lines.push(`${who} [${mid}] ${part.text.trim()}`); - } else if (isToolUIPart(part)) { - const name = part.type.slice("tool-".length); - if (name === "quiz" && part.state === "output-available") { - lines.push(quizLine(part as unknown as { toolCallId: string; input: unknown; output: unknown })); - } else if (part.state === "output-available") { - lines.push(`(${name} tool result delivered)`); - } + const role = (message as { role?: string }).role; + if (role === "user") { + for (const text of textOf((message as { content: unknown }).content)) { + lines.push(`LEARNER [${mid}] ${text.trim()}`); } + } else if (role === "assistant") { + for (const text of textOf((message as { content: unknown }).content)) { + lines.push(`TEACHER [${mid}] ${text.trim()}`); + } + } else if (role === "toolResult") { + const result = message as ToolResultMessage; + if (result.isError) return lines; // a failed call is retry context, not learning evidence + lines.push( + result.toolName === "quiz" ? quizLine(result) : `(${result.toolName} tool result delivered)`, + ); } + // custom markers are invisible to the extraction view. return lines; } @@ -69,33 +89,28 @@ export function viewTokens(lines: string[]): number { /** * A message's coverage fingerprint: sha256 of role + its view lines (with * the message's real id as the provenance tag). Changes exactly when the - * extraction view of the message changes: an answered quiz merges into the - * asking message → new line → the message re-opens itself, and only itself. - * Plan-block part attach changes data parts (invisible to the view) → the - * fingerprint is stable → still covered. + * extraction view of the message changes. (Wave D: the unit is the native + * message — an answered quiz is the answer's OWN toolResult message, so + * answering opens exactly that new message; the asking message is pulled + * into the batch by the inseparability rule, not re-opened.) */ -export function messageFingerprint(message: UIMessage): string { +export function messageFingerprint(message: LhMessage): string { const h = createHash("sha256"); - h.update(JSON.stringify([message.role, messageViewLines(message, message.id)])); + h.update(JSON.stringify([(message as { role?: string }).role, messageViewLines(message, message.id)])); return h.digest("hex"); } /** One message's coverage-relevant projection. */ export interface CoverageSlot { - message: UIMessage; + message: LhMessage; fp: string; tokens: number; } /** Fingerprint a whole transcript, in order (used by the trigger, the run, * and the coverage view — one derivation everywhere). */ -export function coverageSlots(messages: ReadonlyArray): CoverageSlot[] { - return messages.map((message) => { - const lines = messageViewLines(message, message.id); - const h = createHash("sha256"); - h.update(JSON.stringify([message.role, lines])); - return { message, fp: h.digest("hex"), tokens: viewTokens(lines) }; - }); +export function coverageSlots(messages: ReadonlyArray): CoverageSlot[] { + return messages.map((message) => ({ message, fp: messageFingerprint(message), tokens: viewTokens(messageViewLines(message, message.id)) })); } /** The covered-fingerprint set for one chat, from the ledger. */ @@ -106,14 +121,19 @@ export function coveredSet(entries: ReadonlyArray, chatId: string } /** The derived state: which slots are uncovered, plus the token totals. - * "Uncovered" needs no ledger of its own — this diff IS the coverage view. */ + * "Uncovered" needs no ledger of its own — this diff IS the coverage view. + * `uncovered` is INDEXES into `slots` (the batcher needs positions for the + * inseparability rule). Messages with NO view lines (custom markers, an + * error result) are invisible to coverage entirely: the extractor can + * never see them, so they are neither covered nor uncovered — a marker + * attach never opens (or joins) an extraction run. */ export function coverageOf( - messages: ReadonlyArray, + messages: ReadonlyArray, entries: ReadonlyArray, chatId: string, ): { slots: CoverageSlot[]; - uncovered: CoverageSlot[]; + uncovered: number[]; coveredTokens: number; uncoveredTokens: number; coveredCount: number; @@ -121,40 +141,132 @@ export function coverageOf( } { const covered = coveredSet(entries, chatId); const slots = coverageSlots(messages); - const uncovered = slots.filter((s) => !covered.has(s.fp)); - const coveredCount = slots.length - uncovered.length; + const uncovered: number[] = []; let coveredTokens = 0; let uncoveredTokens = 0; - for (const s of slots) { - if (covered.has(s.fp)) coveredTokens += s.tokens; - else uncoveredTokens += s.tokens; + for (let i = 0; i < slots.length; i++) { + const slot = slots[i]!; + if (messageViewLines(slot.message, slot.message.id).length === 0) continue; // invisible + if (covered.has(slot.fp)) coveredTokens += slot.tokens; + else { + uncovered.push(i); + uncoveredTokens += slot.tokens; + } } - return { slots, uncovered, coveredTokens, uncoveredTokens, coveredCount, uncoveredCount: uncovered.length }; + const visibleCount = slots.length - slots.filter( + (s) => messageViewLines(s.message, s.message.id).length === 0, + ).length; + return { + slots, + uncovered, + coveredTokens, + uncoveredTokens, + coveredCount: visibleCount - uncovered.length, + uncoveredCount: uncovered.length, + }; +} + +// ─── the inseparability rule (owner, Wave D fork 3) ───────────────────────── +// +// A tool-call message and its tool-result message(s) always appear in the +// same chunk. Bonding is adjacency-honest: results follow their call's +// assistant message contiguously in the native record (the agent loop +// appends the results batch; the question gate guarantees nothing can land +// between a quiz call and its learner answer). A result whose call's +// assistant message has NO other bond partners still pulls that message in +// (the extractor needs the question with its answer). + +/** The call ids an assistant message carries. */ +function callIdsOf(message: LhMessage): Set { + const out = new Set(); + const content = (message as { content?: unknown }).content; + if (!Array.isArray(content)) return out; + for (const block of content as Array<{ type?: string; id?: string }>) { + if (block.type === "toolCall" && typeof block.id === "string") out.add(block.id); + } + return out; +} + +/** The contiguous bond-range containing slot i: an assistant-with-calls + * message plus its immediately-following results; a toolResult message + * plus the assistant message that made its call (and that call's sibling + * results). Anything else bonds to itself alone. */ +function bondRange(messages: ReadonlyArray, i: number): [number, number] { + const roleOf = (m: LhMessage | undefined) => (m ? (m as { role?: string }).role : undefined); + const m = messages[i]; + if (!m) return [i, i]; + if (roleOf(m) === "assistant") { + const calls = callIdsOf(m); + if (calls.size === 0) return [i, i]; + let end = i; + while ( + roleOf(messages[end + 1]) === "toolResult" && + calls.has((messages[end + 1] as ToolResultMessage).toolCallId) + ) { + end++; + } + return [i, end]; + } + if (roleOf(m) === "toolResult") { + let start = i; + while (roleOf(messages[start - 1]) === "toolResult") start--; + const owner = messages[start - 1]; + if ( + owner && + roleOf(owner) === "assistant" && + callIdsOf(owner).has((m as ToolResultMessage).toolCallId) + ) { + const [s, e] = bondRange(messages, start - 1); + return [s, Math.max(e, i)]; + } + return [start, i]; // orphaned results (no asking message): keep the contiguous result run together + } + return [i, i]; } /** * Batch the uncovered messages into token-budgeted RUNS: consecutive slots, - * oldest first, greedy fill — a message is never split (it is the unit), so - * a tool call and its result always travel together. A single message larger - * than the budget becomes its own (over-budget) run rather than being - * dropped. Runs are ephemeral — the durable record is per-message coverage, - * checkpointed after each run merges. + * oldest first, greedy fill — a message is never split, and a bond-range + * (call + results) never splits either (owner fork 3): a selected uncovered + * slot EXPANDS to its whole bond-range, covered partners riding as context + * (checkpoint idempotency makes re-marking them a no-op). A single + * bond-range larger than the budget becomes its own (over-budget) run + * rather than being dropped. Runs are ephemeral — the durable record is + * per-message coverage, checkpointed after each run merges. */ -export function batchRuns(uncovered: ReadonlyArray, budget: number): CoverageSlot[][] { +export function batchRuns(slots: ReadonlyArray, uncovered: ReadonlyArray, budget: number): CoverageSlot[][] { if (budget <= 0 || uncovered.length === 0) return []; + const messages = slots.map((s) => s.message); + // Expand: every uncovered index marks its whole bond-range as needed. + const needed = new Set(); + for (const i of uncovered) { + const [start, end] = bondRange(messages, i); + for (let j = start; j <= end; j++) needed.add(j); + } + // Group the needed slots in order, greedy by budget, never splitting a + // bond-range: a range that would overflow starts a new run (and an + // over-budget range rides alone). const runs: CoverageSlot[][] = []; let current: CoverageSlot[] = []; let currentTokens = 0; - for (const slot of uncovered) { - if (current.length > 0 && currentTokens + slot.tokens > budget) { - runs.push(current); - current = []; - currentTokens = 0; + const flush = () => { + if (current.length > 0) runs.push(current); + current = []; + currentTokens = 0; + }; + for (let i = 0; i < slots.length; i++) { + if (!needed.has(i)) continue; + const [start, end] = bondRange(messages, i); + const rangeTokens = slots.slice(start, end + 1).reduce((t, s) => t + s.tokens, 0); + if (current.length > 0 && currentTokens + rangeTokens > budget) flush(); + // The whole range enters together — skip past its end in the outer loop. + for (let j = start; j <= end; j++) { + current.push(slots[j]!); + currentTokens += slots[j]!.tokens; } - current.push(slot); - currentTokens += slot.tokens; + i = end; } - if (current.length > 0) runs.push(current); + flush(); return runs; } @@ -171,4 +283,4 @@ export function batchTag(batch: ReadonlyArray): string { const h = createHash("sha256"); for (const slot of batch) h.update(slot.fp); return h.digest("hex").slice(0, 12); -} \ No newline at end of file +} diff --git a/src/core/graph.ts b/src/core/graph.ts index b5e849c..8810f93 100644 --- a/src/core/graph.ts +++ b/src/core/graph.ts @@ -39,7 +39,8 @@ */ import { randomBytes } from "node:crypto"; -import { isToolUIPart, type UIMessage } from "ai"; +import type { LhMessage } from "./native-messages.ts"; +import type { ToolResultMessage } from "@earendil-works/pi-ai"; import type { CoverageEntry } from "./contracts/graph.ts"; import type { AuditResult, @@ -85,7 +86,7 @@ function newGraphNodeId(): string { * fresh understanding observations land as new events; the notes path * passes neither and keeps the whole-file checkpoint. */ export interface MergeSource { - messages: UIMessage[] | null; + messages: LhMessage[] | null; foldedAt: string; /** The batch being merged (chat path): one entry per message in * `messages`, same order — undefined (notes path / coverage-less @@ -97,7 +98,11 @@ export interface MergeSource { // ─── transcript scanning (shared by the merger + trigger) ───────────────────── -/** A flattened, order-preserving view of every tool part in a transcript. */ +/** A flattened, order-preserving view of every tool call (+ its result + * state) in a native transcript. Wave D: the pairing is native — calls + * come from assistant content blocks; a result map built from the + * toolResult messages carries the outcome (a dangling call is + * input-available, an error result is output-error). */ interface ToolPartView { name: string; toolCallId: string; @@ -106,17 +111,27 @@ interface ToolPartView { output: unknown; } -function toolParts(messages: UIMessage[]): ToolPartView[] { +function toolParts(messages: LhMessage[]): ToolPartView[] { + const results = new Map(); + for (const m of messages) { + if ((m as { role?: string }).role === "toolResult") { + results.set((m as ToolResultMessage).toolCallId, m as ToolResultMessage); + } + } const out: ToolPartView[] = []; - for (const message of messages) { - for (const part of message.parts) { - if (!isToolUIPart(part)) continue; + for (const m of messages) { + if ((m as { role?: string }).role !== "assistant") continue; + const content = (m as { content?: unknown }).content; + if (!Array.isArray(content)) continue; + for (const block of content as Array<{ type?: string; id?: string; name?: string; arguments?: unknown }>) { + if (block.type !== "toolCall" || typeof block.id !== "string") continue; + const result = results.get(block.id); out.push({ - name: part.type.slice("tool-".length), - toolCallId: part.toolCallId, - state: part.state, - input: part.input, - output: part.output, + name: block.name ?? "", + toolCallId: block.id, + state: result ? (result.isError ? "output-error" : "output-available") : "input-available", + input: block.arguments, + output: result && !result.isError ? result.details : undefined, }); } } diff --git a/src/core/index.ts b/src/core/index.ts index 0f7317b..d0bcfe0 100644 --- a/src/core/index.ts +++ b/src/core/index.ts @@ -31,13 +31,15 @@ export * from "./phases.ts"; export * from "./pending.ts"; export * from "./prompt-segments.ts"; export * from "./math-delims.ts"; -export * from "./ui-message.ts"; // Wave B (owner-approved 2026-09-26): the native record (LhMessage), the -// one-way client-view projection, and the input-events contract — all +// and the input-events contract — all // pure logic, browser-safe. export * from "./native-messages.ts"; -export * from "./projection.ts"; export * from "./input-events.ts"; +// Wave D (owner-approved 2026-09-26): the client view model — the grouping +// canon (one continuous assistant document, step separators) ported from +// the deleted server projection to the client side. Pure, browser-safe. +export * from "./view.ts"; export * from "./tool-parts.ts"; export * from "./prices.ts"; export * from "./cost-summary.ts"; diff --git a/src/core/native-messages.ts b/src/core/native-messages.ts index 5e8dc86..b3922e2 100644 --- a/src/core/native-messages.ts +++ b/src/core/native-messages.ts @@ -282,3 +282,48 @@ export function hasProbeDone(messages: readonly LhMessage[]): boolean { export function answeredCallIds(messages: readonly LhMessage[]): Set { return resultIdsFrom(messages); } + +// ─── the continuation fold (Wave D: the SDK's +// lastAssistantMessageIsCompleteWithToolCalls, natively) ────────────────── + +/** + * Does the record END on un-consumed tool results — a turn that stopped + * after its tools ran (a job dispatch's ack, an aborted answer turn), + * leaving the agent owing a continuation? This is the mount/sync + * continuation seam's gate (chat.tsx): when true and nothing runs, the + * client sends {type:"continue"} and the teacher picks up from the record. + * + * The pre-Wave-D client asked the SDK's + * lastAssistantMessageIsCompleteWithToolCalls (last view message is + * assistant; its last step's tool calls all output-available or + * output-error; at least one). The native fold over the record: the last + * message is a toolResult, and the results' OWNING assistant message has + * at least one call, ALL of them answered (a dangling question — the + * dock's domain — fails the all-answered rule and never fires a turn, + * exactly as the SDK's every(output-available) did). + */ +export function needsContinuation(messages: readonly LhMessage[]): boolean { + const last = messages[messages.length - 1]; + if (!last || (last as { role?: string }).role !== "toolResult") return false; + // The trailing results batch (contiguous by construction — the agent + // appends its batch; the question gate keeps a quiz call and its + // learner answer adjacent). + let start = messages.length - 1; + while ( + start > 0 && + (messages[start - 1] as { role?: string } | undefined)?.role === "toolResult" + ) { + start--; + } + const owner = messages[start - 1]; + if (!owner || (owner as { role?: string }).role !== "assistant") return false; + const content = (owner as { content?: unknown }).content; + if (!Array.isArray(content)) return false; + const answered = new Set( + messages.slice(start).map((m) => (m as ToolResultMessage).toolCallId), + ); + const calls = (content as Array<{ type?: string; id?: string }>).filter( + (b) => b.type === "toolCall" && typeof b.id === "string", + ); + return calls.length > 0 && calls.every((b) => answered.has(b.id!)); +} diff --git a/src/core/pending.ts b/src/core/pending.ts index 7b1f3e2..7d9b521 100644 --- a/src/core/pending.ts +++ b/src/core/pending.ts @@ -1,4 +1,18 @@ -import { isToolUIPart, type UIMessage } from "ai"; +/** + * Wave D: these scans read VIEW messages (core/view.ts LhViewMessage — the + * client's grouped transcript). The part shapes are the same vocabulary the + * pre-Wave-D client dialect used; a tool part is structural: `tool-` prefix + + * toolCallId. The server-side record scans (danglingQuestionOf over native + * LhMessage[]) live in native-messages.ts. + */ +interface UIMessage { + role?: string; + parts: Array & { type: string }>; +} + +function isToolUIPart(part: UIMessage["parts"][number]): boolean { + return part.type.startsWith("tool-") && typeof part.toolCallId === "string"; +} /** * The pending-question gate: an unanswered QUESTION tool part (quiz/ask) @@ -76,7 +90,7 @@ function findPending(messages: UIMessage[], names: readonly string[]): PendingTo if (isToolUIPart(part) && part.state === "input-available") { const tool = part.type.slice("tool-".length); if (names.includes(tool)) { - return { toolCallId: part.toolCallId, tool, input: part.input }; + return { toolCallId: part.toolCallId as string, tool, input: part.input }; } } } diff --git a/src/core/phases.ts b/src/core/phases.ts index e2adc85..1d47a5d 100644 --- a/src/core/phases.ts +++ b/src/core/phases.ts @@ -1,15 +1,14 @@ /** * Phase derivation (docs/probe-teach-split.md §4) — EXPLICIT EVENTS, never * occupancy. The probe→teach agent split (2026-09-11) gave every phase - * boundary a transcript marker: + * boundary a transcript event: * - * probe — until the first message containing a `tool-probe_done` part - * (the prober concludes; the assessment rides the tool part) + * probe — until the prober's `probe_done` call CONCLUDES (a clean + * toolResult for it; the assessment rides the call's arguments) * plan — probe concluded, until the learner ACCEPTS the presented - * plan (a `data-verdict` part with accepted:true on the verdict - * button's own user message — the explicit verdict gate; - * plan_init itself is only the PRESENTATION and moves no - * phase; a declined verdict stays in plan) + * plan (the verdict marker with accepted:true — the explicit + * verdict gate; plan_init itself is only the PRESENTATION and + * moves no phase; a declined verdict stays in plan) * teach — teaching is sanctioned * * The former derived-occupancy seam (phase-of-tree via `currentNodeId`) @@ -18,6 +17,14 @@ * progress fields at all. No backwards compatibility: legacy sessions * without probe_done simply read as probe (they are wiped data). * + * TWO FOLDS over the same events (Wave D): `phaseAt3`/`phaseTimeline` fold + * VIEW messages (the client's dividers and dock — part shapes, pure and + * structural); `phaseAt3Lh` folds the NATIVE record (server summaries, + * tests) — probe→plan advances at the probe_done RESULT message, plan→teach + * at the verdict marker. Both are strictly sequential; the two derivations + * agree on every settled record (a result/marker message projects into the + * same view boundary). + * * Pure: no imports, no runtime deps — safe on server, client, and tests. */ @@ -83,3 +90,40 @@ export function phaseTimeline(messages: readonly PhaseMessage[]): Phase3[] { } return out; } + +// ─── the native fold (Wave D: record-side phase over LhMessage[]) ─────────── + +import type { LhMessage } from "./native-messages.ts"; +import type { ToolResultMessage } from "@earendil-works/pi-ai"; + +/** The phase at `upToIndex` (inclusive; default: the last message) of a + * NATIVE record. probe→plan advances at the probe_done call's clean + * toolResult; plan→teach at the verdict marker (details.accepted true). */ +export function phaseAt3Lh(messages: readonly LhMessage[], upToIndex?: number): Phase3 { + let phase: Phase3 = "probe"; + const last = Math.min(upToIndex ?? messages.length - 1, messages.length - 1); + for (let i = 0; i <= last; i++) { + const m = messages[i]!; + const role = (m as { role?: string }).role; + if (role === "toolResult") { + const result = m as ToolResultMessage; + if ( + phase === "probe" && + result.toolName === "probe_done" && + !result.isError + ) { + phase = "plan"; + } + } else if (role === "custom") { + const marker = m as { customType?: string; details?: { accepted?: unknown } }; + if ( + phase === "plan" && + marker.customType === "verdict" && + marker.details?.accepted === true + ) { + phase = "teach"; + } + } + } + return phase; +} diff --git a/src/core/plan-blocks.ts b/src/core/plan-blocks.ts index 788c92f..99c84ed 100644 --- a/src/core/plan-blocks.ts +++ b/src/core/plan-blocks.ts @@ -16,8 +16,7 @@ */ import type { PlanBlockData, PlanTree } from "./contracts.ts"; -import { derivePlanStatuses } from "./contracts.ts"; -import type { UIMessage } from "ai"; +import { derivePlanStatusesLh } from "./contracts.ts"; import type { LhMessage } from "./native-messages.ts"; import { newPlanBlockMessage } from "./native-messages.ts"; import type { ToolResultMessage } from "@earendil-works/pi-ai"; @@ -32,9 +31,11 @@ export interface PlanBlockPart { } /** The snapshot payload (unchanged since wave 4): statuses derived AT - * EMISSION time from the transcript — the block never re-derives. */ -function snapshot(tree: PlanTree, messages: readonly UIMessage[]): PlanBlockData { - const statuses = derivePlanStatuses(tree, messages); + * EMISSION time from the transcript — the block never re-derives. Wave D: + * the derivation reads the NATIVE transcript (quiz CALLS only — results + * feed nothing, contracts/plan.ts derivePlanStatusesLh). */ +function snapshot(tree: PlanTree, messages: readonly LhMessage[]): PlanBlockData { + const statuses = derivePlanStatusesLh(tree, messages); return { kind: "full", at: new Date().toISOString(), @@ -68,14 +69,14 @@ export function emittedPlanBlockCalls(messages: readonly LhMessage[]): Set & { type: string }; -type LooseMessage = { id: string; role: "user" | "assistant"; parts: Part[]; metadata?: unknown }; - -function textPart(text: string): Part { - return { type: "text", text, state: "done" }; -} - -/** The result a tool part should show, from a native toolResult. */ -function applyResult(part: Part, result: ToolResultMessage): void { - if (result.isError) { - part.state = "output-error"; - part.rawInput = part.input; - const text = result.content.find((c) => c.type === "text"); - part.errorText = text && text.type === "text" ? text.text : "The tool call failed."; - } else { - part.state = "output-available"; - part.output = result.details; - } -} - -/** Does this assistant group carry a successful plan_init (the splice target)? */ -function lastSuccessfulPlanInit(parts: Part[]): number { - let idx = -1; - parts.forEach((p, i) => { - if (p.type !== "tool-plan_init") return; - const ok = (p.output as { ok?: unknown } | undefined)?.ok; - if (ok === true) idx = i; - }); - return idx; -} - -/** Project the native record into client-shaped UIMessages. Pure and - * deterministic: the same record always projects to the same list. */ -export function projectNative(messages: readonly LhMessage[]): UIMessage[] { - const out: LooseMessage[] = []; - - const lastAssistant = (): LooseMessage | undefined => { - const last = out[out.length - 1]; - return last && last.role === "assistant" ? last : undefined; - }; - const findToolPart = (toolCallId: string): Part | undefined => { - for (let i = out.length - 1; i >= 0; i--) { - for (const part of out[i]!.parts) { - if (part.type.startsWith("tool-") && part.toolCallId === toolCallId) return part; - } - } - return undefined; - }; - - for (const m of messages) { - const role = (m as { role?: string }).role; - - if (role === "user") { - const content = m.content as string | Array<{ type: string; text?: string }>; - const texts = typeof content === "string" ? [content] : content.filter((c) => c.type === "text").map((c) => (c as { text: string }).text); - out.push({ id: m.id, role: "user", parts: texts.map(textPart) }); - continue; - } - - if (role === "custom") { - const marker = m as LhMarkerMessage; - projectMarker(marker, out); - continue; - } - - if (role === "toolResult") { - const result = m as ToolResultMessage; - const part = findToolPart(result.toolCallId); - if (part) applyResult(part, result); - continue; - } - - if (role === "assistant") { - const assistant = m as AssistantMessage; - const parts: Part[] = []; - let textN = 0; - let reasoningN = 0; - for (const block of assistant.content) { - if (block.type === "text") { - parts.push({ type: "text", id: `${m.id}-t${textN++}`, text: block.text, state: "done" }); - } else if (block.type === "thinking") { - parts.push({ type: "reasoning", id: `${m.id}-r${reasoningN++}`, text: block.thinking, state: "done" }); - } else if (block.type === "toolCall") { - parts.push({ - type: `tool-${block.name}`, - toolCallId: block.id, - state: "input-available", - input: block.arguments, - }); - } - } - if (parts.length === 0) continue; // presentation-only (nothing renderable) - const group = lastAssistant(); - if (group) { - // Same group: the step separator renders between the steps. - group.parts.push({ type: "step-start" }, ...parts); - } else { - out.push({ id: m.id, role: "assistant", parts: [{ type: "step-start" }, ...parts] }); - } - continue; - } - } - - return out as unknown as UIMessage[]; -} - -/** A marker's projected message (verdict/handoff/steers) or splice (plan block). */ -function projectMarker(marker: LhMarkerMessage, out: LooseMessage[]): void { - switch (marker.customType) { - case "verdict": { - const data = marker.details as PlanVerdictData; - out.push({ - id: marker.id, - role: "user", - parts: [{ type: "data-verdict", id: `vd_${marker.id}`, data }], - }); - return; - } - case "probe-handoff": { - out.push({ - id: marker.id, - role: "user", - parts: [{ type: "data-probe-handoff", id: `ph_${marker.id}`, data: marker.details }], - }); - return; - } - case "subagent-steer": { - const meta = marker.details as SubagentResultMeta; - const artifact = marker.details as { kind: SubagentArtifactData["kind"]; result: SubagentArtifactData["result"] }; - out.push({ - id: marker.id, - role: "user", - metadata: { subagentResult: meta }, - parts: [ - textPart(marker.content), - { type: "data-subagent-artifact", id: `sa_${marker.id}`, data: { kind: artifact.kind, result: artifact.result } }, - ], - }); - return; - } - case "subagent-abort": { - const meta = marker.details as SubagentResultMeta & { aborted: true }; - out.push({ - id: marker.id, - role: "user", - metadata: { subagentResult: meta }, - parts: [ - textPart(marker.content), - { type: "data-subagent-aborted", id: `sa_${marker.id}`, data: meta }, - ], - }); - return; - } - case "plan-block": { - // No message of its own: splice into the preceding assistant - // group, IN FLOW after the last successful plan_init call. - const group = out[out.length - 1]; - if (!group || group.role !== "assistant") return; - const idx = lastSuccessfulPlanInit(group.parts); - if (idx === -1) return; - const { block } = marker.details as { planCallId: string; block: PlanBlockData }; - group.parts.splice(idx + 1, 0, { type: "data-plan-block", id: marker.id, data: block }); - return; - } - } -} diff --git a/src/core/ui-message.ts b/src/core/ui-message.ts deleted file mode 100644 index 48c6487..0000000 --- a/src/core/ui-message.ts +++ /dev/null @@ -1,120 +0,0 @@ -import type { UIMessage } from "ai"; -import type { - DemoToolInput, - PdfToolInput, - PlanVerdictData, - PlanBlockData, - ProbeDoneInput, - ProbeDoneResult, - ResearchToolInput, - SubagentArtifactData, - SubagentResultMeta, - VisualToolInput, -} from "./contracts.ts"; -import type { AskToolInput, QuestionAnswerData, QuizToolInput } from "./question-core.ts"; - -/** - * The typed UI message for the teacher chat — gives the frontend typed tool - * parts ("tool-quiz" / "tool-ask_user_question" / "tool-research" / …) without - * importing the server-side agent into the client bundle. - * - * Question tools are CLIENT-SIDE (no execute): their OUTPUT type is the - * structured answer record the client builds and sends via addToolOutput. - * Job tools are SERVER-EXECUTED: their output is the plain-string dispatch - * acknowledgement (the result later arrives as a steer message — see - * SubagentSteerMessage in contracts.ts, whose artifact rides the - * data-subagent-artifact part typed here as DATA_PARTS). - */ -export type TeacherUITools = { - quiz: { input: QuizToolInput; output: QuestionAnswerData }; - ask_user_question: { input: AskToolInput; output: QuestionAnswerData }; - /** The prober's conclusion tool (docs/probe-teach-split.md §2): the - * input IS the learner assessment; the output confirms the handoff. */ - probe_done: { input: ProbeDoneInput; output: ProbeDoneResult }; - research: { input: ResearchToolInput; output: string }; - demo: { input: DemoToolInput; output: string }; - visual: { input: VisualToolInput; output: string }; - /** The pdf dispatch tool (sources v0): fire-and-forget ack output like - * research — the brief arrives later as a steer message. */ - pdf: { input: PdfToolInput; output: string }; - /** The learner-filesystem tools (sources v1, both agents, always-on): - * read may return an image-bearing result; the rest are plain text. */ - read: { input: { path: string; offset?: number; limit?: number }; output: unknown }; - ls: { input: { path?: string; limit?: number }; output: unknown }; - find: { input: { pattern: string; path?: string; limit?: number }; output: unknown }; - grep: { - input: { - pattern: string; - path?: string; - glob?: string; - ignoreCase?: boolean; - literal?: boolean; - context?: number; - limit?: number; - }; - output: unknown; - }; -}; - -export type TeacherUIDataParts = { - "subagent-artifact": SubagentArtifactData; - /** Plan blocks (wave 4): frozen plan snapshots rendered in the chat flow. */ - "plan-block": PlanBlockData; - /** The learner's plan verdict (accept/decline): its own user message, - * carrying ONLY this part — the plan→teach phase seam (docs/probe- - * teach-split.md §4, the custom-message scheme 2026-09-12). */ - verdict: PlanVerdictData; - /** The probe→teach handoff wake: the wake pipeline's steer message, - * carrying ONLY this part (the model-facing text lives in the adapter). */ - "probe-handoff": { chatId: string }; - /** The abort steer's message-type marker (the abort badge renders off it). */ - "subagent-aborted": SubagentResultMeta & { aborted: true }; -}; - -export type TeacherUIMessage = UIMessage; - -// ─── Append-only record adoption (background-process ground rules) ────────── - -/** - * Merge server-added messages into the client's live list WITHOUT the full - * snapshot swap (owner ground rules, 2026-09-15: content delivered into the - * record — context blocks, sub-agent artifacts, abort notes — must become - * visible the moment it persists, mid-turn included; it sits between messages - * and nothing about a running turn should hide it). - * - * Safe against the stale-snapshot lesson (mid-turn full syncs break the SDK's - * in-place continuation merge): this NEVER rewrites or removes an existing - * message — it returns the RECORD's message order when that order is exactly - * the client's persisted core plus new server-added ids (the server appends - * only: steers are inserted right after the last persisted user message), - * with the client's in-flight streaming assistant message — its last entry, - * not yet persisted — kept at the tail. Any other shape (a missing local id - * = the record was rewritten/reordered) returns null: the caller defers to - * the full sync at idle, the pre-existing safety net. - */ -export function mergeAdopted( - local: ReadonlyArray, - record: ReadonlyArray, -): TeacherUIMessage[] | null { - const recordIds = new Set(record.map((m) => m.id)); - // The in-flight tail: the client's last message may legitimately be absent - // from the record (the SDK's streaming assistant message persists only at - // onFinish). Exactly one such tail is allowed. - let core = local; - let tail: TeacherUIMessage | undefined; - const last = local.at(-1); - if (last && !recordIds.has(last.id)) { - tail = last; - core = local.slice(0, -1); - } - // No rewrite happened if every core message is still in the record — - // otherwise this is not a pure append; bail to the full-sync-at-idle. - if (core.some((m) => !recordIds.has(m.id))) return null; - // Nothing new → nothing to do. - const localIds = new Set(local.map((m) => m.id)); - if (!record.some((m) => !localIds.has(m.id))) return null; - // The record's order is canonical for everything persisted (the server - // appends only; plan-block part attachment changes parts, not order) — - // merged = record + the in-flight tail. - return tail ? [...record, tail] : [...record]; -} diff --git a/src/core/view.ts b/src/core/view.ts new file mode 100644 index 0000000..20f3c0a --- /dev/null +++ b/src/core/view.ts @@ -0,0 +1,289 @@ +/** + * THE VIEW MODEL (Wave D, owner-approved 2026-09-26): the client's transcript + * view — native record messages (+ the live turn's working messages) grouped + * into today's exact render shape. The golden master is the pre-Wave-D UI + * (owner fork 1): everything the agent produces is ONE continuous document + * per run — step separators between the loop's steps, artifact and answered + * cards inline, the teacher's reaction continuing below an answered quiz. + * + * This is the grouping canon of the deleted server-side projection + * (core/projection.ts, Wave B), moved client-side, app-owned, with zero AI-SDK + * types — plus the LIVE-TURN layer the projection never had: mid-turn tool + * execution states (input-streaming → input-available/output-error/output), + * so the dock mounts only after a call clears the validation boundary + * (tool_execution_end) and the shimmer names the running activity. + * + * GROUPING RULE: one assistant view message per GROUP — consecutive assistant + * + toolResult messages, bounded by user/marker messages; toolResults attach + * to the tool part of their call (searching backward — an answered quiz's + * result lands on the part that asked it, even turns later); a group's id is + * its FIRST assistant message's id. + * + * Plan blocks: a plan-block marker has no message of its own — its data part + * splices into the preceding assistant group, IN FLOW right after the last + * successful plan_init call (the presentation moment). + * + * Question tools on the wire (owner fork 2): the agent's closing + * tool_execution_end carries the in-memory "Question posed" placeholder — + * projectView IGNORES clean question executions (the part stays + * input-available; the learner's answer arrives as a real toolResult). The + * placeholder never persists, so settled records never see it. + * + * Pure and deterministic: client-safe (no node deps), same input → same view. + */ + +import type { AssistantMessage, ToolResultMessage } from "@earendil-works/pi-ai"; +import type { LhMarkerMessage, LhMessage } from "./native-messages.ts"; +import { QUESTION_TOOLS } from "./pending.ts"; +import type { SubagentArtifactData, SubagentResultMeta } from "./contracts/jobs.ts"; +import type { PlanBlockData, PlanVerdictData } from "./contracts/chat.ts"; + +/** A view part — the loose part vocabulary the renderers consume (the same + * shape names the pre-Wave-D UIMessage dialect used, so the renderer + * switch and the phase/pending scanners transplant unchanged). The common + * fields are optional-typed so the pre-Wave-D renderer code compiles + * against it with minimal casts; anything else stays `unknown` through + * the index signature. */ +export interface LhViewPart extends Record { + type: string; + id?: string; + text?: string; + state?: string; + toolCallId?: string; + input?: unknown; + output?: unknown; + rawInput?: unknown; + errorText?: string; + data?: unknown; +} + +/** A view message: user (bubble) or assistant (one continuous run). */ +export interface LhViewMessage { + id: string; + role: "user" | "assistant"; + parts: LhViewPart[]; + /** The subagentResult meta the steer cards read (marker details). */ + metadata?: unknown; +} + +/** A tool call's live execution outcome (from tool_execution_end). */ +export interface LiveExecution { + isError: boolean; + /** The clean result's details payload (the card's output). */ + output?: unknown; + /** The failure text (output-error parts). */ + errorText?: string; +} + +/** The live turn's execution layer, maintained by the session hook. */ +export interface LiveTurnState { + /** toolCallId → its execution outcome seen so far. */ + executions: ReadonlyMap; + /** True while the turn runs — calls with no execution yet render + * input-streaming (the shimmer); settled dangling calls render + * input-available (the dock mounts from the record). */ + inFlight: boolean; +} + +function textPart(text: string): LhViewPart { + return { type: "text", text, state: "done" }; +} + +/** The result a tool part should show, from a native toolResult. */ +function applyResult(part: LhViewPart, result: ToolResultMessage): void { + if (result.isError) { + part.state = "output-error"; + part.rawInput = part.input; + const text = result.content.find((c) => c.type === "text"); + part.errorText = text && text.type === "text" ? text.text : "The tool call failed."; + } else { + part.state = "output-available"; + part.output = result.details; + } +} + +/** Does this assistant group carry a successful plan_init (the splice target)? */ +function lastSuccessfulPlanInit(parts: LhViewPart[]): number { + let idx = -1; + parts.forEach((p, i) => { + if (p.type !== "tool-plan_init") return; + const ok = (p.output as { ok?: unknown } | undefined)?.ok; + if (ok === true) idx = i; + }); + return idx; +} + +/** Project the native record (+ live turn working list) into the client's + * view messages. Pure: the same input always projects to the same list. */ +export function projectView(messages: readonly LhMessage[], live?: LiveTurnState): LhViewMessage[] { + const out: LhViewMessage[] = []; + + const lastAssistant = (): LhViewMessage | undefined => { + const last = out[out.length - 1]; + return last && last.role === "assistant" ? last : undefined; + }; + const findToolPart = (toolCallId: string): LhViewPart | undefined => { + for (let i = out.length - 1; i >= 0; i--) { + for (const part of out[i]!.parts) { + if (part.type.startsWith("tool-") && part.toolCallId === toolCallId) return part; + } + } + return undefined; + }; + + for (const m of messages) { + const role = (m as { role?: string }).role; + + if (role === "user") { + const content = m.content as string | Array<{ type: string; text?: string }>; + const texts = + typeof content === "string" + ? [content] + : content.filter((c) => c.type === "text").map((c) => (c as { text: string }).text); + out.push({ id: m.id, role: "user", parts: texts.map(textPart) }); + continue; + } + + if (role === "custom") { + projectMarker(m as LhMarkerMessage, out); + continue; + } + + if (role === "toolResult") { + applyResultTo(findToolPart((m as ToolResultMessage).toolCallId), m as ToolResultMessage); + continue; + } + + if (role === "assistant") { + const assistant = m as AssistantMessage; + const parts: LhViewPart[] = []; + let textN = 0; + let reasoningN = 0; + for (const block of assistant.content) { + if (block.type === "text") { + parts.push({ type: "text", id: `${m.id}-t${textN++}`, text: block.text, state: "done" }); + } else if (block.type === "thinking") { + parts.push({ type: "reasoning", id: `${m.id}-r${reasoningN++}`, text: block.thinking, state: "done" }); + } else if (block.type === "toolCall") { + parts.push(toolPartOf(block, live)); + } + } + if (parts.length === 0) continue; // presentation-only (nothing renderable) + const group = lastAssistant(); + if (group) { + // Same group: the step separator renders between the steps. + group.parts.push({ type: "step-start" }, ...parts); + } else { + out.push({ id: m.id, role: "assistant", parts: [{ type: "step-start" }, ...parts] }); + } + continue; + } + } + + return out; +} + +/** A tool call block → its view part, with the live execution layer. */ +function toolPartOf( + block: { id: string; name: string; arguments?: unknown }, + live: LiveTurnState | undefined, +): LhViewPart { + const part: LhViewPart = { + type: `tool-${block.name}`, + toolCallId: block.id, + input: block.arguments, + }; + const exec = live?.executions.get(block.id); + const isQuestion = (QUESTION_TOOLS as readonly string[]).includes(block.name); + if (exec) { + if (exec.isError) { + // An invalid call NEVER becomes input-available — the dock's mount + // gate keys on the clean execution end (the live modal-realism + // crash: a malformed quiz must never mount). + part.state = "output-error"; + part.rawInput = block.arguments; + part.errorText = exec.errorText ?? "The tool call failed."; + } else if (isQuestion) { + // Fork 2: the clean "Question posed" placeholder is IGNORED — + // question tools are client-owned; the dock mounts on the clean + // end, the answer arrives as a real toolResult. + part.state = "input-available"; + } else { + part.state = "output-available"; + part.output = exec.output; + } + } else { + // No execution yet: mid-turn it is input-streaming (the shimmer + // activity); in a settled record a dangling call is the dock's + // pending question (input-available). + part.state = live?.inFlight ? "input-streaming" : "input-available"; + } + return part; +} + +function applyResultTo(part: LhViewPart | undefined, result: ToolResultMessage): void { + if (part) applyResult(part, result); + // A result with no visible part (e.g. a call whose assistant message + // rendered nothing) is dropped from the view — the record keeps it. +} + +/** A marker's projected message (verdict/handoff/steers) or splice (plan block). */ +function projectMarker(marker: LhMarkerMessage, out: LhViewMessage[]): void { + switch (marker.customType) { + case "verdict": { + const data = marker.details as PlanVerdictData; + out.push({ + id: marker.id, + role: "user", + parts: [{ type: "data-verdict", id: `vd_${marker.id}`, data }], + }); + return; + } + case "probe-handoff": { + out.push({ + id: marker.id, + role: "user", + parts: [{ type: "data-probe-handoff", id: `ph_${marker.id}`, data: marker.details }], + }); + return; + } + case "subagent-steer": { + const meta = marker.details as SubagentResultMeta; + const artifact = marker.details as { + kind: SubagentArtifactData["kind"]; + result: SubagentArtifactData["result"]; + }; + out.push({ + id: marker.id, + role: "user", + metadata: { subagentResult: meta }, + parts: [ + textPart(marker.content), + { type: "data-subagent-artifact", id: `sa_${marker.id}`, data: { kind: artifact.kind, result: artifact.result } }, + ], + }); + return; + } + case "subagent-abort": { + const meta = marker.details as SubagentResultMeta & { aborted: true }; + out.push({ + id: marker.id, + role: "user", + metadata: { subagentResult: meta }, + parts: [textPart(marker.content), { type: "data-subagent-aborted", id: `sa_${marker.id}`, data: meta }], + }); + return; + } + case "plan-block": { + // No message of its own: splice into the preceding assistant + // group, IN FLOW after the last successful plan_init call. + const group = out[out.length - 1]; + if (!group || group.role !== "assistant") return; + const idx = lastSuccessfulPlanInit(group.parts); + if (idx === -1) return; + const { block } = marker.details as { planCallId: string; block: PlanBlockData }; + group.parts.splice(idx + 1, 0, { type: "data-plan-block", id: marker.id, data: block }); + return; + } + } +} diff --git a/src/server/chat-handler.ts b/src/server/chat-handler.ts index 7d6bc35..ef2a451 100644 --- a/src/server/chat-handler.ts +++ b/src/server/chat-handler.ts @@ -322,7 +322,6 @@ export async function handleChatInput( // late views reattach via GET /api/chats/[id]/stream. const handle = runPiTurn({ agent: constructed, - base: merged, // Stream-level failures are provider/transport internals — masked, // like the SDK defaults. Tool-input validation messages never // route through here: they ride the error-result path and reach the diff --git a/src/server/chats.ts b/src/server/chats.ts index ce44dd7..f90d950 100644 --- a/src/server/chats.ts +++ b/src/server/chats.ts @@ -1,9 +1,7 @@ import { abortExtraction, extractionStatuses } from "../agent/subagents/extraction.ts"; import { coverageOf } from "../core/coverage.ts"; -import { projectNative } from "../core/projection.ts"; import { deleteChatTraces, type ChatDeleteMode, type ChatDeletionStats } from "../core/graph-deletion.ts"; import { titleFromPrompt } from "../core/contracts.ts"; -import type { UIMessage } from "ai"; import { graphRevision } from "./graph.ts"; import { stores } from "./stores.ts"; import { isTurnRunning } from "./turn-liveness.ts"; @@ -144,7 +142,7 @@ export function sessionCoverage(chatId: string): SessionCoverage | null { const entries = stores.graphStore.readCoverage(); // Wave B: the record is native history — coverage reads through the // projection (same view the extraction pipeline folds). - const cov = coverageOf(projectNative(record.messages), entries, chatId); + const cov = coverageOf(record.messages, entries, chatId); let lastFoldedAt: string | null = null; for (const entry of entries) { if (entry.chatId !== chatId) continue; diff --git a/src/server/pi/ui-stream.ts b/src/server/pi/ui-stream.ts index 48d8c76..0e89302 100644 --- a/src/server/pi/ui-stream.ts +++ b/src/server/pi/ui-stream.ts @@ -1,67 +1,66 @@ import type { Agent, AgentEvent, AgentMessage } from "@earendil-works/pi-agent-core"; -import type { UIMessage } from "ai"; -import { QUESTION_TOOLS } from "../../core/pending.ts"; -import type { LhMessage } from "../../core/native-messages.ts"; -import { projectNative } from "../../core/projection.ts"; /** - * The UI-stream bridge (docs/pi-engine-swap.md §5): runs one agent turn and - * re-emits it as an AI-SDK-shaped UIMessageStream SSE response — the wire - * protocol `useChat` (Band D, unchanged) parses: + * THE NATIVE WIRE (Wave D, owner-approved 2026-09-26): one agent turn, streamed + * to every view as VERBATIM pi AgentEvents — no translation, no remapping, no + * part assembly. The SSE body is the agent's own event stream: * - * data: {"type":"text-delta", …}\n\n … data: [DONE] + * data: {"type":"agent_start"}\n\n + * data: {"type":"message_update","message":{…},"assistantMessageEvent":{…}}\n\n + * … + * data: {"type":"agent_end","messages":[…]}\n\n + * data: [DONE]\n\n * - * Load-bearing protocol rules (verified against ai@7's client): - * - The `start` chunk's messageId MUST be the transcript's last message id - * when it is an assistant message (the continuation id rule) — a fresh id - * there duplicates every prior part client-side. - * - Question tools (quiz / ask_user_question): emit tool-input-available and - * NEVER tool-output-available (the client throws on an output for a call it - * owns). The synthetic "question posed" result lives in memory only; the - * record's part stays output-less until the learner answers. - * - tool-input-available is emitted ONLY after the call cleared validation - * at the boundary (clean tool_execution_end) — the SDK's - * InvalidToolInputError gating. An invalid call surfaces as - * tool-input-error instead and NEVER becomes input-available, so a - * malformed question can never mount the dock (the live modal-realism - * crash: GLM double-encoded `options` as a JSON string). - * - The WIRE assembly (below) is the client view only (Wave B): the - * record write is the agent's own native final state — the projection - * (core/projection.ts) reproduces the client's part shapes for every - * consumer (phases, plan blocks, pending, extraction) until Wave D. + * Load-bearing protocol facts (verified against pi-agent-core): + * - Every `message_update` carries the FULL partial assistant message (each + * AssistantMessageEvent ships `partial`) — a view REPLACES its streaming + * assistant with `event.message`; no delta assembly exists anywhere. + * - Question tools (quiz / ask_user_question): NO suppression (owner ruling, + * Wave D fork 2) — the closing `tool_execution_end` with the in-memory + * "Question posed" placeholder flows to the client, which ignores it + * (question tools are client-owned; the answer arrives as the input event). + * The placeholder still never persists and never reaches the model. + * - Tool-call validation gating survives by construction: `tool_execution_end` + * only fires after the call cleared the boundary (an invalid call arrives + * as isError — never a clean end), so the dock's mount rule keys on it. + * - App-level events beyond pi's taxonomy: the reattach terminal + * {"type":"turn-settled"} (served by turn-runs when no turn is running) and + * the loop-level {"type":"error","errorText"} (agent.continue() threw + * outside the event stream — mapped through the handler's masking policy). + * - Stream-level provider errors ride their message verbatim + * (message_end with stopReason "error" + errorMessage). * * BACKGROUND-TURN RUNNER (owner ruling, 2026-09-25): a turn executes as a * server-side background object decoupled from any single HTTP response — * clients are VIEWS. runPiTurn returns the handle (event log + subscriber * set); piTurnResponse wraps a subscriber stream as the SSE response, and * the POST /api/chat response and GET /api/chats/[id]/stream are BOTH - * exactly that. Cancelling a view detaches it ONLY — never aborts the - * turn; the handle's abort() is the one remote kill. + * exactly that. Cancelling a view detaches it ONLY — never aborts the turn; + * the handle's abort() is the one remote kill. */ -/** SSE chunk — the flat JSON line shape the client's zod schema accepts. */ -type Chunk = Record & { type: string }; +/** The wire event: a verbatim pi AgentEvent, or one of the two app events. */ +export type TurnWireEvent = + | AgentEvent + | { type: "error"; errorText: string }; export interface PiTurnSpec { /** The per-turn agent (stateless engine — built from the record). */ agent: Agent; - /** The turn's base: the native record + this turn's appends - * (write-ahead persisted already). Wave B: the agent's context and - * the record share one shape; onFinish receives the full agent state - * and the caller persists the slice past base.length. */ - base: LhMessage[]; - /** The handler's error-unmasking policy (validation messages stay readable). */ + /** The handler's error-masking policy for LOOP-level failures (the + * event stream itself is verbatim — provider errors ride their + * message's stopReason/errorMessage through untouched). */ onError?: (error: unknown) => string; /** The canonical end-of-turn seam: persist + usage + extraction + endTurn. - * Receives the agent's FULL final state (native messages) — the turn's - * new messages are the slice past base.length. */ + * Receives the agent's FULL final state (native messages) — the caller + * persists the slice past its own base count. */ onFinish: (messages: AgentMessage[], isAborted: boolean) => void; } -/** One view onto a running turn: push() replays/streams a chunk, done() +/** One view onto a running turn: push() replays/streams an event, done() * terminates the view ([DONE] + close). Cheap to be a Set member. */ export interface PiTurnSubscriber { - push(chunk: Chunk): void; + push(event: TurnWireEvent): void; done(): void; } @@ -75,109 +74,35 @@ export interface PiTurnHandle { * view only — the turn runs on. */ subscribe(): ReadableStream; /** The remote kill — the ONLY way a running turn dies early: sawAbort + - * agent.abort(); the loop's agent_end does the rest (finish chunk to - * every subscriber, then onFinish persists the partial transcript). */ + * agent.abort(); the loop's agent_end does the rest (the settle seam + * persists the partial transcript to every listener). */ abort(): void; } -/** Map a pi stop reason to the finish chunk's enum. */ -function finishReasonOf(stopReason: string | undefined): string { - switch (stopReason) { - case "stop": - return "stop"; - case "length": - return "length"; - case "toolUse": - return "tool-calls"; - case "error": - return "error"; - default: - return "other"; - } -} - -function resultErrorText(result: unknown): string { - const content = (result as { content?: Array<{ type: string; text?: string }> } | undefined)?.content; - const text = content?.find((c) => c.type === "text")?.text; - return text ?? "The tool call failed."; -} - -/** Loose part shapes the bridge assembles (the client's part shapes). */ -interface TextPart { - type: "text"; - text: string; - state: string; -} -interface ReasoningPart { - type: "reasoning"; - id: string; - text: string; - state: string; -} -interface ToolPart { - type: string; - toolCallId: string; - state: string; - input?: unknown; - rawInput?: unknown; - output?: unknown; - errorText?: string; -} - export function runPiTurn(spec: PiTurnSpec): PiTurnHandle { - const { agent, base, onFinish } = spec; - - // The streaming target: continuation (same id, parts appended to a copy - // of the existing message) or a fresh assistant message. The continuation - // rule is the CLIENT's: it keys on the last UIMessage's id — read through - // the projection (the record is native; a turn after an answer continues - // the group that carries the answered quiz, which is the projection's - // last message). - const projected = projectNative(base); - const last = projected[projected.length - 1] as - | { id?: string; role?: string; parts?: UIMessage["parts"] } - | undefined; - const continuing = last?.role === "assistant" && typeof last.id === "string"; - const messageId = - (continuing && last.id) || `msg_${Date.now().toString(36)}_${crypto.randomUUID().slice(0, 8)}`; - const baseParts: UIMessage["parts"] = continuing && last.parts ? [...last.parts] : []; - const parts: UIMessage["parts"] = [...baseParts]; - - // Stable, collision-free part ids across continuation runs (reasoning - // parts persist their id; text ids only need per-run uniqueness). - let textN = baseParts.filter((p) => p.type === "text").length; - let reasoningN = baseParts.filter((p) => p.type === "reasoning").length; - const textIds = new Map(); // contentIndex → chunk id - const reasoningIds = new Map(); - const toolInputs = new Map(); - /** The streaming text/reasoning parts of THIS run, keyed by chunk id. */ - const activeParts = new Map(); - /** Tool parts created this run, keyed by toolCallId. */ - const newToolParts = new Map(); + const { agent, onError, onFinish } = spec; let settled = false; let sawAbort = false; - let finishReason: string | undefined; - let errorMessage: string | undefined; // THE PER-TURN EVENT LOG (memory-only — the settle-only disk writes - // are unchanged): every emitted chunk, in order, so any late subscriber - // gets a FULL replay. Unbounded BY HONEST DESIGN (owner ruling - // 2026-09-25): a turn's chunk log is finite by construction — the agent - // loop ends (or the 30-minute liveness TTL bounds the pathological - // never-settling case), and concurrency is human-scale (one turn per - // chat, a few chats). No cap, no truncation: replay must never - // silently drop history. The log dies with the turn at settle. - const log: Chunk[] = []; + // are unchanged): every forwarded event, in order, so any late + // subscriber gets a FULL replay. Unbounded BY HONEST DESIGN (owner + // ruling 2026-09-25): a turn's event log is finite by construction — + // the agent loop ends (or the 30-minute liveness TTL bounds the + // pathological never-settling case), and concurrency is human-scale + // (one turn per chat, a few chats). No cap, no truncation: replay must + // never silently drop history. The log dies with the turn at settle. + const log: TurnWireEvent[] = []; const subscribers = new Set(); // Send = append to the log + fan out to every live view. A view's own // enqueue failure (a cancelled reader) is swallowed by the view — the // turn must still settle; onFinish persists regardless of who watches. - const send = (chunk: Chunk) => { + const send = (event: TurnWireEvent) => { if (settled) return; - log.push(chunk); - for (const sub of subscribers) sub.push(chunk); + log.push(event); + for (const sub of subscribers) sub.push(event); }; const closeAll = () => { if (settled) return; @@ -185,175 +110,23 @@ export function runPiTurn(spec: PiTurnSpec): PiTurnHandle { for (const sub of subscribers) sub.done(); subscribers.clear(); }; - - send({ type: "start", messageId }); - + // The settle seam: the record write is the agent's OWN final state + // (native history — signatures, usage, provider stamps all real). + // Copy the array: the agent may outlive this call's read. const settleWith = () => { - // Wave B: the record write is the agent's OWN final state (native - // history — signatures, usage, provider stamps all real); the wire - // assembly above served the client view only. Copy the array: the - // agent may outlive this call's read. onFinish([...agent.state.messages], sawAbort); }; const handle = (event: AgentEvent) => { - switch (event.type) { - case "turn_start": - send({ type: "start-step" }); - parts.push({ type: "step-start" }); - break; - case "message_update": { - const streamEvent = event.assistantMessageEvent; - switch (streamEvent.type) { - case "text_start": { - const id = `${messageId}-t${textN++}`; - textIds.set(streamEvent.contentIndex, id); - const part: TextPart = { type: "text", text: "", state: "streaming" }; - activeParts.set(id, part); - parts.push(part as UIMessage["parts"][number]); - send({ type: "text-start", id }); - break; - } - case "text_delta": { - const id = textIds.get(streamEvent.contentIndex); - const part = id !== undefined ? (activeParts.get(id) as TextPart | undefined) : undefined; - if (part) part.text += streamEvent.delta; - if (id !== undefined) send({ type: "text-delta", id, delta: streamEvent.delta }); - break; - } - case "text_end": { - const id = textIds.get(streamEvent.contentIndex); - const part = id !== undefined ? (activeParts.get(id) as TextPart | undefined) : undefined; - if (part) part.state = "done"; - activeParts.delete(id ?? ""); - if (id !== undefined) send({ type: "text-end", id }); - break; - } - case "thinking_start": { - const id = `${messageId}-r${reasoningN++}`; - reasoningIds.set(streamEvent.contentIndex, id); - const part: ReasoningPart = { type: "reasoning", id, text: "", state: "streaming" }; - activeParts.set(id, part); - parts.push(part as UIMessage["parts"][number]); - send({ type: "reasoning-start", id }); - break; - } - case "thinking_delta": { - const id = reasoningIds.get(streamEvent.contentIndex); - const part = id !== undefined ? (activeParts.get(id) as ReasoningPart | undefined) : undefined; - if (part) part.text += streamEvent.delta; - if (id !== undefined) send({ type: "reasoning-delta", id, delta: streamEvent.delta }); - break; - } - case "thinking_end": { - const id = reasoningIds.get(streamEvent.contentIndex); - const part = id !== undefined ? (activeParts.get(id) as ReasoningPart | undefined) : undefined; - if (part) part.state = "done"; - activeParts.delete(id ?? ""); - if (id !== undefined) send({ type: "reasoning-end", id }); - break; - } - case "toolcall_start": { - const block = (event.message as { content: any[] }).content[streamEvent.contentIndex]; - if (block?.type !== "toolCall") break; - toolInputs.set(block.id, { toolName: block.name, input: undefined }); - const part: ToolPart = { - type: `tool-${block.name}`, - toolCallId: block.id, - state: "input-streaming", - input: undefined, - }; - newToolParts.set(block.id, part); - parts.push(part as unknown as UIMessage["parts"][number]); - send({ type: "tool-input-start", toolCallId: block.id, toolName: block.name }); - break; - } - case "toolcall_delta": { - const block = (event.message as { content: any[] }).content[streamEvent.contentIndex]; - if (block?.type !== "toolCall") break; - send({ - type: "tool-input-delta", - toolCallId: block.id, - inputTextDelta: streamEvent.delta, - }); - break; - } - case "toolcall_end": { - const toolCall = streamEvent.toolCall; - toolInputs.set(toolCall.id, { toolName: toolCall.name, input: toolCall.arguments }); - // Input stays input-streaming here: it only becomes available - // once the call clears validation at the execution boundary - // (tool_execution_end) — an invalid input must never reach - // input-available, where the dock would mount on it. - const part = newToolParts.get(toolCall.id); - if (part) part.input = toolCall.arguments; - break; - } - } - break; - } - case "tool_execution_end": { - const known = toolInputs.get(event.toolCallId); - const toolName = event.toolName || known?.toolName || ""; - const input = known?.input; - const part = newToolParts.get(event.toolCallId); - if (event.isError) { - const errorText = resultErrorText(event.result); - if (part) { - part.state = "output-error"; - part.rawInput = input; - part.errorText = errorText; - } - send({ - type: "tool-input-error", - toolCallId: event.toolCallId, - toolName, - input, - errorText, - }); - } else { - // Validation cleared: only NOW does the input become available - // to the client (the InvalidToolInputError gating the SDK owned). - if (part) { - part.state = "input-available"; - part.input = input; - } - send({ type: "tool-input-available", toolCallId: event.toolCallId, toolName, input }); - if ((QUESTION_TOOLS as readonly string[]).includes(toolName)) { - // Client-owned: the answer arrives via addToolOutput; - // the synthetic placeholder result never reaches wire - // or record. - break; - } - const output = (event.result as { details?: unknown } | undefined)?.details; - if (part) { - part.state = "output-available"; - part.output = output; - } - send({ type: "tool-output-available", toolCallId: event.toolCallId, output }); - } - break; - } - case "turn_end": - send({ type: "finish-step" }); - break; - case "message_end": - if (event.message && (event.message as { role?: string }).role === "assistant") { - const m = event.message as { stopReason?: string; errorMessage?: string }; - finishReason = m.stopReason; - if (m.stopReason === "aborted") sawAbort = true; - if (m.stopReason === "error") errorMessage = m.errorMessage; - } - break; - case "agent_end": { - if (errorMessage !== undefined) { - send({ type: "error", errorText: spec.onError?.(errorMessage) ?? errorMessage }); - } - send({ type: "finish", finishReason: finishReasonOf(finishReason) }); - closeAll(); - settleWith(); - break; - } + // VERBATIM: every pi AgentEvent forwards as-is — no remapping. + send(event); + if (event.type === "message_end") { + const m = event.message as { role?: string; stopReason?: string } | undefined; + if (m?.role === "assistant" && m.stopReason === "aborted") sawAbort = true; + } + if (event.type === "agent_end") { + closeAll(); + settleWith(); } }; agent.subscribe(handle); @@ -363,9 +136,12 @@ export function runPiTurn(spec: PiTurnSpec): PiTurnHandle { await agent.continue(); } catch (err) { // A loop-level failure (not a provider error — those ride the - // stream as stopReason "error"): surface, persist, close. - send({ type: "error", errorText: spec.onError?.(err) ?? "An error occurred." }); - send({ type: "finish", finishReason: "error" }); + // stream as stopReason "error"): surface, close, persist. If the + // agent already settled (agent_end reached before the throw), + // the seam has run — never settle twice. + if (settled) return; + console.error("[turn] agent loop failed:", err); + send({ type: "error", errorText: onError?.(err) ?? "An error occurred." }); closeAll(); settleWith(); } @@ -379,10 +155,10 @@ export function runPiTurn(spec: PiTurnSpec): PiTurnHandle { const encoder = new TextEncoder(); let closed = false; sub = { - push(chunk) { + push(event) { if (closed) return; try { - controller.enqueue(encoder.encode(`data: ${JSON.stringify(chunk)}\n\n`)); + controller.enqueue(encoder.encode(`data: ${JSON.stringify(event)}\n\n`)); } catch { /* reader cancelled — stop emitting, keep running */ closed = true; @@ -399,11 +175,11 @@ export function runPiTurn(spec: PiTurnSpec): PiTurnHandle { } }, }; - // Attach THEN replay — synchronous, so no chunk can fall + // Attach THEN replay — synchronous, so no event can fall // between replay and live (single-threaded: the agent's // handle() only runs after this stack returns). subscribers.add(sub); - for (const chunk of log) sub.push(chunk); + for (const event of log) sub.push(event); // A subscriber that attached after settle (the unregister // race) still gets the full history + [DONE]. if (settled) sub.done(); @@ -426,8 +202,8 @@ export function runPiTurn(spec: PiTurnSpec): PiTurnHandle { } /** The subscriber-shaped SSE response — the POST /api/chat response and - * GET /api/chats/[id]/stream are BOTH this: same headers, same wire - * dialect, one protocol for the client to consume. */ + * GET /api/chats/[id]/stream are BOTH this: same headers, same wire + * dialect, one protocol for the client to consume. */ export function piTurnResponse(handle: PiTurnHandle): Response { return new Response(handle.subscribe(), { headers: { @@ -436,4 +212,4 @@ export function piTurnResponse(handle: PiTurnHandle): Response { "X-Accel-Buffering": "no", }, }); -} \ No newline at end of file +} diff --git a/src/server/turn-service.ts b/src/server/turn-service.ts index a4c845a..b369697 100644 --- a/src/server/turn-service.ts +++ b/src/server/turn-service.ts @@ -21,7 +21,6 @@ import type { LhMessage } from "../core/native-messages.ts"; import { newMessageId } from "../core/native-messages.ts"; import { QUESTION_TOOLS } from "../core/pending.ts"; import { planBlocksFor } from "../core/plan-blocks.ts"; -import { projectNative } from "../core/projection.ts"; import { coverageOf } from "../core/coverage.ts"; import type { ChatStore, GraphStore } from "../core/ports.ts"; import type { ExtractionPipeline } from "../agent/subagents/extraction.ts"; @@ -72,12 +71,10 @@ export function persistTurn( stamped.push(message.id ? message : ({ ...message, id: newMessageId() } as LhMessage)); } const tree = record.tree; - const blocks = planBlocksFor( - record.messages, - stamped, - tree, - projectNative([...record.messages, ...stamped]), - ); + const blocks = planBlocksFor(record.messages, stamped, tree, [ + ...record.messages, + ...stamped, + ]); chatStore.saveMessages(chatId, [...record.messages, ...stamped, ...blocks]); return stamped; } catch (err) { @@ -110,7 +107,7 @@ export function triggerExtraction( try { const record = deps.chatStore.getChat(chatId); if (!record) return; - const covered = coverageOf(projectNative(record.messages), deps.graphStore.readCoverage(), chatId); + const covered = coverageOf(record.messages, deps.graphStore.readCoverage(), chatId); if (covered.uncoveredTokens < chunkTokens) return; void extraction.runExtraction(chatId).catch((err) => { // BYOK (owner, 2026-09-22) + key presence (provider-key interlude, diff --git a/src/store/chats.ts b/src/store/chats.ts index 5f2259e..f67eee4 100644 --- a/src/store/chats.ts +++ b/src/store/chats.ts @@ -13,9 +13,8 @@ import { PATHS } from "../../config.ts"; import type { ChatStore } from "../core/ports.ts"; import type { ChatRecord, ChatSettings, ChatSummary, ExtractionRunUsage, PlanTree, TurnUsage } from "../core/contracts.ts"; import type { LhMessage } from "../core/native-messages.ts"; -import { findPendingToolCall } from "../core/pending.ts"; -import { phaseAt3 } from "../core/phases.ts"; -import { projectNative } from "../core/projection.ts"; +import { danglingQuestionOf } from "../core/native-messages.ts"; +import { phaseAt3Lh } from "../core/phases.ts"; function chatFile(id: string): string { if (!/^[A-Za-z0-9_-]+$/.test(id)) throw new Error(`Invalid chat id: ${id}`); @@ -58,7 +57,6 @@ function listChats(): ChatSummary[] { // zero-back-compat wipe makes these "corrupt") must not blank the // whole sidebar — same isolation getChat's per-file catch gives. try { - const projected = projectNative(messages); return { // turns and extractions stay out of the list payload (list = // summaries; usage aggregates ride the full record, the picker @@ -66,13 +64,14 @@ function listChats(): ChatSummary[] { ...summary, messageCount: messages.length, // Record-derived status for the session picker: phase from the - // transcript's explicit markers (probe_done / plan_init — the - // event-driven phases, docs/probe-teach-split.md §4), pending - // question from the last unanswered quiz/ask part — both read - // through the one-way projection. Liveness (busy) is NOT - // record-derived — the API route merges it (ChatListItem). - phase: phaseAt3(projected), - pendingQuestion: findPendingToolCall(projected) !== null, + // transcript's explicit events (probe_done concluded / verdict + // accepted — docs/probe-teach-split.md §4), pending question + // from the native dangling-call scan. Wave D: both folds read + // the NATIVE record (core/phases.ts phaseAt3Lh + + // native-messages.ts danglingQuestionOf). Liveness (busy) is + // NOT record-derived — the API route merges it (ChatListItem). + phase: phaseAt3Lh(messages), + pendingQuestion: danglingQuestionOf(messages) !== null, }; } catch (err) { console.error(`[chat ${summary.id}] summary projection failed:`, err); diff --git a/tests/chat-flow.test.ts b/tests/chat-flow.test.ts index 2640cd3..35f52f6 100644 --- a/tests/chat-flow.test.ts +++ b/tests/chat-flow.test.ts @@ -33,7 +33,7 @@ import { abortTurn, runningTurn, settledTurnResponse } from "../src/server/turn- import { chatStore } from "../src/store/chats.ts"; import { buildAnswer, coerceCorrectAnswer, normalizeOptions, resolveCorrect, type QuestionData } from "../src/core/question-core.ts"; import { danglingQuestionOf, newAnswerResult } from "../src/core/native-messages.ts"; -import { projectNative } from "../src/core/projection.ts"; +import { projectView } from "../src/core/view.ts"; import type { PiEngine } from "../src/agent/models.ts"; import type { LhMessage } from "../src/core/native-messages.ts"; import { PATHS } from "../config.ts"; @@ -108,6 +108,18 @@ function gradedAnswer(input: typeof QUIZ_INPUT, values: string[]) { return buildAnswer(qd, { values }); } +/** Parse an SSE body into its wire events (verbatim pi AgentEvents + + * the app-level error/turn-settled terminals). Structural beats substring: + * key order never matters. */ +function eventsOf(body: string): Array> { + const out: Array> = []; + for (const line of body.split("\n")) { + if (!line.startsWith("data: ") || line === "data: [DONE]") continue; + out.push(JSON.parse(line.slice(6))); + } + return out; +} + async function drain(res: Response): Promise { if (res.status !== 200) { assert.fail(`expected 200, got ${res.status}: ${await res.text()}`); @@ -213,7 +225,7 @@ describe("teacher chat flow (scripted model)", () => { // The projection pins the client view: reasoning part + a pending // (input-available) quiz part — the dock's mount state. - const view1 = projectNative(saved1.messages); + const view1 = projectView(saved1.messages); const reasoningPart = view1[1]!.parts.find((p) => p.type === "reasoning"); assert.ok(reasoningPart && reasoningPart.type === "reasoning", "reasoning part projected"); const quizPart = view1[1]!.parts.find((p) => p.type === "tool-quiz"); @@ -241,7 +253,7 @@ describe("teacher chat flow (scripted model)", () => { // Native: user, assistant, answer toolResult, reaction assistant. assert.equal(saved2.messages.length, 4, "the answer + reaction append natively"); assert.equal((saved2.messages[2] as { role?: string }).role, "toolResult", "the graded answer is the call's tool result"); - const view2 = projectNative(saved2.messages); + const view2 = projectView(saved2.messages); assert.equal(view2.length, 2, "the reaction continues the same assistant message (one group)"); const finalParts = view2[1]!.parts; const textPart = finalParts.find((p) => p.type === "text"); @@ -298,20 +310,28 @@ describe("teacher chat flow (scripted model)", () => { await drain(ok); }); - it("streams tool-input-start before tool-input-available (protocol pin for the 'Writing a question…' indicator)", async () => { - // The client's writing indicator keys off tool parts in state - // 'input-streaming', which the stream processor sets from the - // tool-input-start chunk. If a transport change ever stopped emitting - // it, the indicator would silently die — pin the protocol. + it("streams the toolCall partial before tool_execution_start/end (protocol pin for the 'Writing a question…' shimmer)", async () => { + // Wave D: the wire is VERBATIM pi AgentEvents. The client's writing + // indicator derives 'input-streaming' from a toolCall block seen in a + // message_update with no execution yet; the dock mounts on a CLEAN + // tool_execution_end. Pin the ordering that drives both — if the + // passthrough ever stopped emitting one of these, the indicator or + // the dock would silently die. const { engine } = scriptedTeacher(defaultLesson); const chat = createConfiguredChat("Test: protocol pin"); createdChats.push(chat.id); const body = await drain(await handleChatRequest(post(chat.id, { type: "text", text: "quiz me" }), agentFactory(engine))); - const startIdx = body.indexOf('"type":"tool-input-start"'); - const availIdx = body.indexOf('"type":"tool-input-available"'); - assert.ok(startIdx !== -1, "tool-input-start chunk present in the stream"); - assert.ok(availIdx !== -1, "tool-input-available chunk present in the stream"); - assert.ok(startIdx < availIdx, "start precedes available"); + const events = eventsOf(body); + const callSeen = events.findIndex( + (e) => e.type === "message_update" && JSON.stringify(e.message?.content ?? []).includes('"type":"toolCall"'), + ); + const execStart = events.findIndex((e) => e.type === "tool_execution_start" && e.toolCallId === "call-1"); + const execEnd = events.findIndex((e) => e.type === "tool_execution_end" && e.toolCallId === "call-1"); + assert.ok(callSeen !== -1, "a message_update carries the toolCall block (the shimmer source)"); + assert.ok(execStart !== -1, "tool_execution_start present in the stream"); + assert.ok(execEnd !== -1, "tool_execution_end present in the stream"); + assert.ok(callSeen < execStart && execStart < execEnd, "partial → execution start → execution end"); + assert.equal(events[execEnd]!.isError, false, "the clean end is what mounts the dock"); }); it("a cancelled view NEVER aborts the turn — it runs to completion (owner ruling 2026-09-25)", async () => { @@ -375,22 +395,17 @@ describe("teacher chat flow (scripted model)", () => { createdChats.push(chat.id); const body = await drain(await handleChatRequest(post(chat.id, { type: "text", text: "quiz me" }), agentFactory(engine))); - // WIRE pin (the live modal-realism crash): the invalid call must NEVER - // become input-available — the dock mounts on input-available parts and - // crashed on a malformed quiz before tool-input-error could replace it. - // The boundary gates this on clean tool_execution_end. - assert.ok( - !body.includes('"type":"tool-input-available","toolCallId":"call-bad"'), - "the invalid call never reached input-available on the wire", - ); - assert.ok( - body.includes('"type":"tool-input-error","toolCallId":"call-bad"'), - "the invalid call surfaced as tool-input-error", - ); - assert.ok( - body.includes('"type":"tool-input-available","toolCallId":"call-good"'), - "the valid retry DID become input-available", - ); + // WIRE pin (the live modal-realism crash): the invalid call's execution + // end is an ERROR — the dock only mounts on a CLEAN tool_execution_end, + // so a malformed quiz can never reach it. The retry's end is clean. + const events = eventsOf(body); + const badEnd = events.find((e) => e.type === "tool_execution_end" && e.toolCallId === "call-bad"); + assert.ok(badEnd, "the invalid call's execution end is on the wire"); + assert.equal(badEnd.isError, true, "the invalid call surfaced as an error execution"); + assert.match(String(badEnd.result?.content?.[0]?.text ?? ""), /does not match any option value/); + const goodEnd = events.find((e) => e.type === "tool_execution_end" && e.toolCallId === "call-good"); + assert.ok(goodEnd, "the valid retry's execution end is on the wire"); + assert.equal(goodEnd.isError, false, "the valid retry's end is CLEAN — the dock mounts on it"); // The model was re-prompted (the error did not end the turn)… assert.equal(calls.length, 2, "the invalid call triggered a second model prompt"); @@ -405,7 +420,7 @@ describe("teacher chat flow (scripted model)", () => { // model's retry context — errored question calls DO persist), the // retry as the dangling quiz. The projection renders the error part. const saved = chatStore.getChat(chat.id)!; - const view = projectNative(saved.messages); + const view = projectView(saved.messages); const quizParts = view[1]!.parts.filter((p) => p.type === "tool-quiz"); const failed = quizParts.find( (p) => p.type === "tool-quiz" && (p as { state?: string }).state === "output-error", @@ -446,17 +461,14 @@ describe("teacher chat flow (scripted model)", () => { createdChats.push(chat.id); const body = await drain(await handleChatRequest(post(chat.id, { type: "text", text: "quiz me" }), agentFactory(engine))); - assert.ok( - !body.includes('"type":"tool-input-available","toolCallId":"call-stringy"'), - "the structurally invalid quiz never became input-available on the wire", - ); - assert.ok( - body.includes('"type":"tool-input-error","toolCallId":"call-stringy"'), - "the invalid quiz surfaced as tool-input-error", - ); + const events = eventsOf(body); + const stringyEnd = events.find((e) => e.type === "tool_execution_end" && e.toolCallId === "call-stringy"); + assert.ok(stringyEnd, "the invalid quiz's execution end is on the wire"); + assert.equal(stringyEnd.isError, true, "the structural reject surfaced as an error execution"); + assert.match(String(stringyEnd.result?.content?.[0]?.text ?? ""), /options/); const saved = chatStore.getChat(chat.id)!; - const view = projectNative(saved.messages); + const view = projectView(saved.messages); const failed = view[1]!.parts.find( (p) => (p as { toolCallId?: string }).toolCallId === "call-stringy", ); @@ -488,15 +500,18 @@ describe("teacher chat flow (scripted model)", () => { createdChats.push(chat.id); const body = await drain(await handleChatRequest(post(chat.id, { type: "text", text: "quiz me twice" }), agentFactory(engine))); - // (B) wire determinism: the batches' input-available chunks in call order. - const availA = body.indexOf('"type":"tool-input-available","toolCallId":"call-a"'); - const availB = body.indexOf('"type":"tool-input-available","toolCallId":"call-b"'); - assert.ok(availA !== -1 && availB !== -1, "both quizzes became available"); - assert.ok(availA < availB, "call A's input became available before call B's (sequential execution)"); + // (B) wire determinism: question tools execute sequentially, so the + // quizzes' CLEAN execution ends arrive in call order (the dock mounts + // on clean ends — the mount order is the wire order). + const events = eventsOf(body); + const endA = events.findIndex((e) => e.type === "tool_execution_end" && e.toolCallId === "call-a" && e.isError === false); + const endB = events.findIndex((e) => e.type === "tool_execution_end" && e.toolCallId === "call-b" && e.isError === false); + assert.ok(endA !== -1 && endB !== -1, "both quizzes' clean ends are on the wire"); + assert.ok(endA < endB, "call A's end precedes call B's (sequential execution)"); // Saved in call order (the projection preserves the content order). const saved = chatStore.getChat(chat.id)!; - const quizIds = projectNative(saved.messages)[1]!.parts + const quizIds = projectView(saved.messages)[1]!.parts .filter((p) => p.type === "tool-quiz") .map((p) => (p as { toolCallId: string }).toolCallId); assert.deepEqual(quizIds, ["call-a", "call-b"], "parts persisted in call order"); @@ -576,7 +591,7 @@ describe("mid-turn navigation (turn liveness + write-ahead)", () => { const midTurn = chatStore.getChat(chat.id)!; assert.equal(midTurn.messages.length, 3, "write-ahead appended the answer tool result before finish"); assert.equal((midTurn.messages[2] as { role?: string }).role, "toolResult"); - const midQuiz = projectNative(midTurn.messages)[1]!.parts.find((p) => p.type === "tool-quiz"); + const midQuiz = projectView(midTurn.messages)[1]!.parts.find((p) => p.type === "tool-quiz"); assert.ok(midQuiz && midQuiz.type === "tool-quiz"); assert.equal((midQuiz as { state?: string }).state, "output-available", "write-ahead persisted the answer before finish"); @@ -585,7 +600,7 @@ describe("mid-turn navigation (turn liveness + write-ahead)", () => { // Native: user, assistant, answer result, reaction — one append-only // history; the projection still renders ONE assistant group. assert.equal(saved2.messages.length, 4, "full turn persisted as usual at finish"); - assert.equal(projectNative(saved2.messages).length, 2, "the projection groups the answered quiz + reaction"); + assert.equal(projectView(saved2.messages).length, 2, "the projection groups the answered quiz + reaction"); assert.equal(isTurnRunning(chat.id), false, "registry cleared at finish"); }); }); @@ -732,14 +747,15 @@ describe("background-turn runner (owner ruling 2026-09-25: every turn is a serve // Let the turn finish; both views close at settle. release(); const text = await secondText; - assert.ok(text.startsWith('data: {"type":"start"'), "replay starts from the very first chunk (the start event)"); + const replay = eventsOf(text); + assert.equal(replay[0]!.type, "agent_start", "replay starts from the very first event"); assert.ok(text.includes("Gated reply."), "replay carries the full streamed content"); assert.ok(text.endsWith("data: [DONE]\n\n"), "live tail ends with the SSE terminal, same dialect as POST"); - // Wire honesty: a normal turn settles via agent_end with NO error chunk - // (the live smoke caught a close()-throw masked into the error path - // — unit asserts never looked; these pins would have). - assert.ok(!text.includes('"type":"error"'), "a clean turn never emits an error chunk"); - assert.ok(text.includes('"finishReason":"stop"'), "the finish chunk carries the stop reason"); + // Wire honesty: a normal turn settles with NO error event (the live + // smoke caught a close()-throw masked into the error path — unit + // asserts never looked; these pins would have). + assert.ok(!text.includes('"type":"error"'), "a clean turn never emits an error event"); + assert.ok(replay.some((e) => e.type === "message_end" && e.message?.stopReason === "stop"), "message_end carries the stop reason"); // And the first view also closes cleanly (no hang, no drop): read its // remaining chunks through the reader we already hold. const decoder = new TextDecoder(); diff --git a/tests/chat-transport.test.ts b/tests/chat-transport.test.ts deleted file mode 100644 index de4a497..0000000 --- a/tests/chat-transport.test.ts +++ /dev/null @@ -1,118 +0,0 @@ -/** - * 409 identity-retry transport tests (components/chat-transport.ts). - * - * Wave B (input events): a 409 means a turn is in flight — the event was - * NEVER persisted (write-ahead happens only on accept), so the retry waits - * out the turn and re-POSTs the SAME event body. Nothing merges, nothing - * derives: the event is the unit, and re-POSTing it is idempotent-safe. - */ -import assert from "node:assert/strict"; -import { afterEach, describe, it } from "node:test"; -import { retryingFetch } from "../components/chat-transport.ts"; - -const realFetch = globalThis.fetch; -afterEach(() => { - globalThis.fetch = realFetch; -}); - -interface Call { - url: string; - body: string | undefined; -} - -function installFetch(routes: { - /** POST /api/chat → Response; receives every attempt. */ - chatPOST: (call: Call, attempt: number) => Response; - /** GET /api/jobs?chatId=… → turnInFlight value per poll (poll index). */ - jobsGET?: (poll: number) => boolean; -}): Call[] { - const calls: Call[] = []; - let poll = 0; - globalThis.fetch = (async (input: RequestInfo | URL, init?: RequestInit) => { - const url = String(input); - calls.push({ url, body: init?.body as string | undefined }); - if (url === "/api/chat" && init?.method === "POST") { - return routes.chatPOST(calls.at(-1)!, calls.filter((c) => c.url === "/api/chat").length); - } - if (url.startsWith("/api/jobs")) return Response.json({ turnInFlight: routes.jobsGET?.(++poll) ?? false }); - throw new Error(`unexpected fetch: ${url}`); - }) as typeof globalThis.fetch; - return calls; -} - -/** One input event — the POST body's whole unit. */ -function outgoing(input: Record): string { - return JSON.stringify({ chatId: "chat_x", input }); -} - -describe("retryingFetch (409 merge-retry)", { skip: process.env.NO_SLOW ? true : undefined }, () => { - it("passes non-409 responses through untouched", async () => { - const calls = installFetch({ - chatPOST: () => new Response("stream", { status: 200 }), - }); - const fetch = retryingFetch("chat_x", { pollMs: 10, waitMaxMs: 100 }); - const res = await fetch("/api/chat", { method: "POST", body: outgoing({ type: "text", text: "hi" }) }); - assert.equal(res.status, 200); - assert.equal(calls.length, 1, "no polling, no merge, single call"); - }); - - it("waits out the turn, then re-POSTs the SAME event body (identity retry)", async () => { - const event = { type: "text", text: "my question while the wake turn ran" }; - const calls = installFetch({ - chatPOST: (_call, n) => (n === 1 ? new Response("busy", { status: 409 }) : new Response("stream", { status: 200 })), - jobsGET: (poll) => poll <= 1, // first poll: still running; second: done - }); - const fetch = retryingFetch("chat_x", { pollMs: 10, waitMaxMs: 100 }); - const res = await fetch("/api/chat", { method: "POST", body: outgoing(event) }); - assert.equal(res.status, 200); - - // The retry carries the IDENTICAL body — the event was never - // persisted pre-accept, so re-sending it is the whole protocol. - const posts = calls.filter((c) => c.url === "/api/chat"); - assert.equal(posts.length, 2, "one 409, one retry"); - assert.equal(posts[1]!.body, posts[0]!.body, "the event body is byte-identical"); - assert.deepEqual(JSON.parse(posts[1]!.body!), { chatId: "chat_x", input: event }); - }); - - it("does not poll the record or merge — the event is the only payload", async () => { - const calls = installFetch({ - chatPOST: (_call, n) => (n === 1 ? new Response("busy", { status: 409 }) : new Response("stream", { status: 200 })), - }); - const fetch = retryingFetch("chat_x", { pollMs: 10, waitMaxMs: 100 }); - const res = await fetch("/api/chat", { method: "POST", body: outgoing({ type: "text", text: "q" }) }); - assert.equal(res.status, 200); - assert.equal( - calls.filter((c) => c.url.startsWith("/api/chats/")).length, - 0, - "the retry never reads the record — the server owns it", - ); - }); - - it("loops on repeated 409s, re-POSTing the same event each time", async () => { - const event = { type: "answer", toolCallId: "call-1", answer: { correct: true } }; - const calls = installFetch({ - chatPOST: (_call, n) => (n <= 2 ? new Response("busy", { status: 409 }) : new Response("stream", { status: 200 })), - }); - const fetch = retryingFetch("chat_x", { pollMs: 10, waitMaxMs: 100 }); - const res = await fetch("/api/chat", { method: "POST", body: outgoing(event) }); - assert.equal(res.status, 200); - const posts = calls.filter((c) => c.url === "/api/chat"); - assert.equal(posts.length, 3); - assert.ok(posts.every((c) => c.body === posts[0]!.body), "every attempt carries the same event"); - }); - - it("gives up after TURN_WAIT_MAX_MS and returns the last 409 (banner as before)", async () => { - // turnInFlight stuck true; two polls (~1s) then force the deadline path - // by asserting we eventually get the 409 back — bounded by poll count. - let polls = 0; - const calls = installFetch({ - chatPOST: () => new Response("busy", { status: 409 }), - jobsGET: () => true, - }); - const fetch = retryingFetch("chat_x", { pollMs: 10, waitMaxMs: 100 }); - const res = await fetch("/api/chat", { method: "POST", body: outgoing({ type: "text", text: "q" }) }); - assert.equal(res.status, 409); - polls = calls.filter((c) => c.url.startsWith("/api/jobs")).length; - assert.ok(polls <= 15, `expected bounded polling (~10×10ms), got ${polls}`); - }); -}); \ No newline at end of file diff --git a/tests/coverage.test.ts b/tests/coverage.test.ts index 2bb5605..2900416 100644 --- a/tests/coverage.test.ts +++ b/tests/coverage.test.ts @@ -1,7 +1,9 @@ import assert from "node:assert/strict"; import { describe, it } from "node:test"; import * as fs from "node:fs"; -import type { UIMessage } from "ai"; +import type { LhMessage } from "../src/core/native-messages.ts"; +import { newAnswerResult, newPlanBlockMessage, newUserMessage } from "../src/core/native-messages.ts"; +import type { ToolResultMessage } from "@earendil-works/pi-ai"; import { batchRuns, @@ -11,20 +13,47 @@ import { messageFingerprint, messageViewLines, } from "../src/core/coverage.ts"; -import type { CoverageEntry } from "../src/core/contracts.ts"; +import type { CoverageEntry } from "../src/core/contracts/graph.ts"; /** * Extraction coverage — the pure core (vision-extraction-coverage.md, - * settled 2026-09-15). THE MESSAGE IS THE UNIT OF COVERAGE; the fingerprint - * and the LLM prompt share ONE line builder (messageViewLines), so "covered" - * is by construction exactly "what the extractor saw". + * settled 2026-09-15; Wave D native rebase 2026-09-26). THE NATIVE MESSAGE + * IS THE UNIT OF COVERAGE (owner ruling); the fingerprint and the LLM prompt + * share ONE line builder (messageViewLines), so "covered" is by construction + * exactly "what the extractor saw". Batches obey the ONE INSEPARABILITY RULE + * (owner fork 3): a tool-call message and its tool-result messages always + * appear in the same chunk. */ -function user(id: string, text: string): UIMessage { - return { id, role: "user", parts: [{ type: "text", text }] } as unknown as UIMessage; +function user(id: string, text: string): LhMessage { + return { ...newUserMessage(text), id }; } -function slot(messages: UIMessage[], id: string) { +function teacher(id: string, text: string): LhMessage { + return { + id, + role: "assistant", + content: [{ type: "text", text }], + } as unknown as LhMessage; +} + +/** An assistant message calling `calls` (each {id,name}) — the bond anchor. */ +function assistantWithCalls( + id: string, + calls: Array<{ id: string; name: string; args?: unknown }>, +): LhMessage { + return { + id, + role: "assistant", + content: calls.map((c) => ({ type: "toolCall", id: c.id, name: c.name, arguments: c.args ?? {} })), + } as unknown as LhMessage; +} + +function result(id: string, callId: string, name: string, details: unknown): LhMessage { + return { ...newAnswerResult(callId, name, details), id } as LhMessage; +} + +function slot(messages: ReadonlyArray, id: string) { const found = coverageSlots(messages).find((s) => s.message.id === id); assert.ok(found, `slot ${id}`); return found; @@ -37,9 +66,8 @@ describe("messageFingerprint — what changes it, what never does", () => { const same = messageFingerprint(user("m1", "hello world")); assert.notEqual(a, b); assert.equal(a, same); - // Role is hashed too — a user and an assistant line never collide. - const teacher = { id: "m1", role: "assistant", parts: [{ type: "text", text: "hello world" }] } as unknown as UIMessage; - assert.notEqual(a, messageFingerprint(teacher)); + // Role is hashed too — a learner and a teacher line never collide. + assert.notEqual(a, messageFingerprint(teacher("m1", "hello world"))); }); it("the message id is the provenance tag — part of exactly what the extractor saw", () => { @@ -48,39 +76,61 @@ describe("messageFingerprint — what changes it, what never does", () => { // extraction view byte-for-byte. (Ids are stable in practice — the // write-ahead persist owns them.) assert.notEqual(messageFingerprint(user("m1", "same")), messageFingerprint(user("m2", "same"))); - // The id is the ONLY tag: the same id + content re-fingerprints equal. + // The same id + content re-fingerprints equal. assert.equal(messageFingerprint(user("m1", "same")), messageFingerprint(user("m1", "same"))); }); - it("data parts are invisible — plan-block attach never re-opens a covered message", () => { - const plain = user("m1", "teach me the chain rule"); - const withData = { - ...plain, - parts: [...plain.parts, { type: "data-plan-blocks", planBlocks: [] }], - } as unknown as UIMessage; - assert.equal(messageFingerprint(plain), messageFingerprint(withData)); + it("marker messages are invisible — a plan-block attach never re-opens anything", () => { + // Wave D: the marker is its OWN message (not a part on another one); + // its view is empty, so it neither re-opens others nor itself + // carries any learning evidence. + const marker = newPlanBlockMessage("call_p1", { + kind: "full", + at: "2026-09-26T00:00:00.000Z", + current: null, + nodes: [], + edges: [], + }) as LhMessage; + assert.deepEqual(messageViewLines(marker, marker.id), []); + // And the message it rides beside keeps its fingerprint untouched: + const before = messageFingerprint(teacher("t1", "the chain rule")); + const withMarker = [teacher("t1", "the chain rule"), marker]; + assert.equal( + messageFingerprint(withMarker[0]!), + before, + "the neighbour's fingerprint is unchanged by the attach", + ); }); - it("a quiz answer merging in changes the fingerprint — and only that message's", () => { - const asking = { - id: "a1", - role: "assistant", - parts: [{ type: "tool-quiz", toolCallId: "q1", state: "input-available", input: { question: "Q?" } }], - } as unknown as UIMessage; - const answered = { - id: "a1", - role: "assistant", - parts: [ - { type: "tool-quiz", toolCallId: "q1", state: "output-available", input: { question: "Q?" }, output: { questionText: "Q?", correct: true } }, - ], - } as unknown as UIMessage; - const fpBefore = messageFingerprint(asking); - assert.notEqual(fpBefore, messageFingerprint(answered)); - // An output-bearing line's mass counts toward coverage tokens. - assert.ok( - messageViewLines(answered, answered.id).join("").length > - messageViewLines(asking, asking.id).join("").length, - ); + it("an answered quiz is its OWN message — the asking message never re-opens", () => { + // Wave D: the answer arrives as the call's toolResult message; the + // asking assistant message's fingerprint is STABLE (the old wire + // merged the answer into the asking group and re-opened it). + const asking = assistantWithCalls("a1", [{ id: "q1", name: "quiz", args: { question: "Q?" } }]); + const answer = result("r1", "q1", "quiz", { questionText: "Q?", correct: true }); + const fpAsking = messageFingerprint(asking); + assert.equal(fpAsking, messageFingerprint(assistantWithCalls("a1", [{ id: "q1", name: "quiz", args: { question: "Q?" } }]))); + // The answer's own line carries the graded outcome: + const lines = messageViewLines(answer, "r1"); + assert.deepEqual(lines, ["QUIZ [q1] Q? — the learner answered correctly"]); + assert.ok(slot([asking, answer], "r1").tokens > 0, "the quiz outcome counts toward coverage tokens"); + }); + + it("quiz outcome wording — dontKnow and incorrect each get their line", () => { + const dk = result("r1", "q1", "quiz", { questionText: "Q?", dontKnow: true }); + assert.deepEqual(messageViewLines(dk, "r1"), ["QUIZ [q1] Q? — the learner did not know"]); + const wrong = result("r2", "q2", "quiz", { questionText: "W?", correct: false, note: "mixed up signs" }); + assert.deepEqual(messageViewLines(wrong, "r2"), ["QUIZ [q2] W? — the learner answered incorrectly (note: mixed up signs)"]); + }); + + it("other clean tool results get one generic line; errors are invisible", () => { + const demo = result("r1", "j1", "demo", { title: "T" }); + assert.deepEqual(messageViewLines(demo, "r1"), ["(demo tool result delivered)"]); + const failed = { + ...newAnswerResult("j2", "visual", { oops: true }), + isError: true, + } as LhMessage; + assert.deepEqual(messageViewLines(failed, "r9"), [], "a failed call is retry context, not learning evidence"); }); }); @@ -96,7 +146,9 @@ describe("coverageOf — uncovered is DERIVED, never dual-tracked", () => { { chatId: "c2", fp: slot(messages, "m3").fp, tokens: 1, foldedAt: "2026-09-15T00:00:00.000Z" }, ]; const cov = coverageOf(messages, entries, "c1"); - assert.deepEqual(cov.uncovered.map((s) => s.message.id), ["m3"]); + // `uncovered` is INDEXES (the batcher needs positions for bonding): + assert.deepEqual(cov.uncovered, [2]); + assert.deepEqual(cov.slots[cov.uncovered[0]!]!.message.id, "m3"); assert.equal(cov.coveredCount, 2); assert.equal(cov.uncoveredCount, 1); assert.equal(cov.uncoveredTokens, slot(messages, "m3").tokens); @@ -114,7 +166,8 @@ describe("coverageOf — uncovered is DERIVED, never dual-tracked", () => { assert.equal(coverageOf(messages, entries, "c1").uncoveredCount, 0); const mutated = [user("m1", "aaaa!"), messages[1]!]; const cov = coverageOf(mutated, entries, "c1"); - assert.deepEqual(cov.uncovered.map((s) => s.message.id), ["m1"]); + assert.deepEqual(cov.uncovered, [0]); + assert.equal(cov.slots[0]!.message.id, "m1"); }); }); @@ -128,19 +181,106 @@ describe("batchRuns — token-budgeted, consecutive, never splits a message", () user("m3", "c".repeat(40)), user("m4", "d".repeat(40)), ]); - const runs = batchRuns(slots, 45); - assert.deepEqual(runs.map((r) => r.map((s) => s.message.id)), [["m1", "m2", "m3"], ["m4"]]); + const runs = batchRuns(slots, [0, 1, 2, 3], 45); + assert.deepEqual( + runs.map((r) => r.map((s) => s.message.id)), + [["m1", "m2", "m3"], ["m4"]], + ); }); it("a single message larger than the budget becomes its own over-budget run", () => { const slots = coverageSlots([user("m1", "x".repeat(400)), user("m2", "y".repeat(20))]); - const runs = batchRuns(slots, 10); - assert.deepEqual(runs.map((r) => r.map((s) => s.message.id)), [["m1"], ["m2"]]); + assert.deepEqual( + batchRuns(slots, [0, 1], 10).map((r) => r.map((s) => s.message.id)), + [["m1"], ["m2"]], + ); }); it("an empty or non-positive budget never drops messages into a void", () => { - assert.deepEqual(batchRuns(coverageSlots([user("m1", "hi")]), 0), []); - assert.deepEqual(batchRuns([], 4096), []); + assert.deepEqual(batchRuns(coverageSlots([user("m1", "hi")]), [0], 0), []); + assert.deepEqual(batchRuns([], [], 4096), []); + }); + + it("THE BOND RULE (owner fork 3): a call and its results never split, even under a tiny budget", () => { + // The call's assistant message is COVERED; only the (new) answer is + // uncovered — the asking message rides into the chunk anyway, and + // the two NEVER split despite a budget of 1. + const messages = [ + user("u1", "topic"), + assistantWithCalls("a1", [{ id: "q1", name: "quiz", args: { question: "Q?" } }]), + result("r1", "q1", "quiz", { questionText: "Q?", correct: true }), + ]; + const slots = coverageSlots(messages); + const cov = coverageOf( + messages, + coverageSlots([messages[0]!, messages[1]!]).map((s) => ({ + chatId: "c1", + fp: s.fp, + tokens: s.tokens, + foldedAt: "2026-09-26T00:00:00.000Z", + })), + "c1", + ); + assert.deepEqual(cov.uncovered, [2], "only the answer is uncovered"); + const runs = batchRuns(slots, cov.uncovered, 1); + assert.deepEqual( + runs.map((r) => r.map((s) => s.message.id)), + [["a1", "r1"]], + "the covered asking message rides along as context — one chunk, over budget, never split", + ); + }); + + it("the reverse bond: an uncovered CALL pulls its covered results into the chunk", () => { + const messages = [ + { + id: "a1", + role: "assistant", + content: [ + { type: "text", text: "Let me research that and build a demo." }, + { type: "toolCall", id: "j1", name: "research", arguments: { topic: "x" } }, + { type: "toolCall", id: "j2", name: "demo", arguments: { title: "T" } }, + ], + } as unknown as LhMessage, + result("r1", "j1", "research", { summary: "s" }), + result("r2", "j2", "demo", { done: true }), + ]; + const slots = coverageSlots(messages); + const cov = coverageOf( + messages, + coverageSlots([messages[1]!, messages[2]!]).map((s) => ({ + chatId: "c1", + fp: s.fp, + tokens: s.tokens, + foldedAt: "2026-09-26T00:00:00.000Z", + })), + "c1", + ); + assert.deepEqual(cov.uncovered, [0], "only the calling message is uncovered"); + const runs = batchRuns(slots, cov.uncovered, 10_000); + assert.deepEqual( + runs.map((r) => r.map((s) => s.message.id)), + [["a1", "r1", "r2"]], + "all three travel together (checkpoint idempotency makes re-marking the results a no-op)", + ); + }); + + it("adjacent messages WITHOUT a bond still batch greedily (only calls bond)", () => { + // Two plain teacher messages after a bonded pair: the pair is one + // range, the plain messages group by budget alone. + const messages = [ + assistantWithCalls("a1", [{ id: "q1", name: "quiz" }]), + result("r1", "q1", "quiz", { questionText: "Q?", correct: true }), + teacher("t1", "x".repeat(40)), + teacher("t2", "y".repeat(40)), + ]; + const slots = coverageSlots(messages); + assert.equal(slots[0]!.tokens, 0, "a call-only assistant message has no view lines"); + // budget 30: the pair (~12) + t1 (~14) = 26 fits; t2 overflows. + const runs = batchRuns(slots, [0, 2, 3], 30); + assert.deepEqual( + runs.map((r) => r.map((s) => s.message.id)), + [["a1", "r1", "t1"], ["t2"]], + ); }); }); @@ -158,6 +298,7 @@ describe("batchTag — deterministic, ref-stable, mutation-sensitive", () => { assert.notEqual(batchTag(slots), batchTag(mutated)); }); }); + describe("the all-sessions coverage GET is dead (extraction-UX close-out, 2026-09-24)", () => { it("the route file is gone — the only coverage endpoint is the Extract POST", () => { // UI confirmation (ff1ec7a): nothing client-side consumes the diff --git a/tests/demos.test.tsx b/tests/demos.test.tsx index ee58f55..c0459e9 100644 --- a/tests/demos.test.tsx +++ b/tests/demos.test.tsx @@ -57,7 +57,7 @@ import { chatStore } from "../src/store/chats.ts"; import { jobStore } from "../src/store/jobs.ts"; import { handleChatRequest } from "../src/server/chat-handler.ts"; import { createTeacherAgent } from "../src/agent/teacher-agent.ts"; -import { projectNative } from "../src/core/projection.ts"; +import { projectView as projectNative } from "../src/core/view.ts"; import type { PiEngine } from "../src/agent/models.ts"; /** diff --git a/tests/extraction.test.ts b/tests/extraction.test.ts index 426ab7f..0484b53 100644 --- a/tests/extraction.test.ts +++ b/tests/extraction.test.ts @@ -26,7 +26,7 @@ import { createFauxCore, fauxAssistantMessage, fauxToolCall, type FauxResponseSt import type { PiEngine } from "../src/agent/models.ts"; import type { LhMessage } from "../src/core/native-messages.ts"; import { newPlanBlockMessage } from "../src/core/native-messages.ts"; -import { projectNative } from "../src/core/projection.ts"; +import { projectView as projectNative } from "../src/core/view.ts"; import type { AuditResult, CoverageEntry, @@ -42,7 +42,7 @@ import type { import type { GraphStore } from "../src/core/ports.ts"; import { abortExtraction, createExtractionPipeline, extractionStatuses, transcriptView } from "../src/agent/subagents/extraction.ts"; import { ModelsNotConfiguredError, ProviderKeyMissingError, providerKeyMissing, type ModelSlotName } from "../src/core/settings.ts"; -import { batchTag, coverageOf, coverageSlots } from "../src/core/coverage.ts"; +import { batchTag, coverageOf, coverageSlots, messageViewLines } from "../src/core/coverage.ts"; import { chatStore } from "../src/store/chats.ts"; import { PATHS } from "../config.ts"; @@ -399,7 +399,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => { // a new event (vision-extraction-coverage.md). const events = ledgers.readEvents(); const record = chatStore.getChat(chatId)!; - const expectedTag = batchTag(coverageSlots(projectNative(record.messages))); + const expectedTag = batchTag(coverageSlots(record.messages)); assert.deepEqual( events.map((e) => [e.kind, e.ref, e.nodeId]), [ @@ -415,12 +415,18 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => { // one entry per transcript message, fingerprinted content each — // the same turns are never extracted twice. const covered = ledgers.readCoverage(); - assert.equal(covered.length, projectNative(record.messages).length); + // Wave D: the coverage unit is the VISIBLE native message — bonded + // riders with no view lines (a textless call message) are context, + // never checkpointed. + const visible = record.messages.filter( + (m) => messageViewLines(m, m.id).length > 0, + ).length; + assert.equal(covered.length, visible); assert.ok(covered.every((c) => c.chatId === chatId && c.fp.length === 64 && c.tokens > 0)); assert.equal(ledgers.readCheckpoints().length, 0, "the chat path writes coverage, not the legacy checkpoint"); assert.equal( - coverageOf(projectNative(record.messages), covered, chatId).uncoveredCount, + coverageOf(record.messages, covered, chatId).uncoveredCount, 0, "every message is now covered", ); @@ -1020,9 +1026,9 @@ describe("the chunked pipeline (extraction coverage)", () => { // Per-batch coverage: every message checkpointed, nothing legacy. const coverage = ledgers.readCoverage(); - assert.equal(coverage.length, projectNative(record.messages).length); + assert.equal(coverage.length, record.messages.length); assert.equal(ledgers.readCheckpoints().length, 0); - assert.equal(coverageOf(projectNative(record.messages), coverage, chatId).uncoveredCount, 0); + assert.equal(coverageOf(record.messages, coverage, chatId).uncoveredCount, 0); // A second run is a NO-OP: nothing uncovered → no model calls, no // new usage entry (no money spent on covered ground). @@ -1075,7 +1081,7 @@ describe("the chunked pipeline (extraction coverage)", () => { // Batch 1's emissions are kept and its messages stay covered. assert.deepEqual(stats, { nodes: 1, edges: 0, events: 1 }); assert.equal(ledgers.readCoverage().length, 3, "batch 1 checkpointed its three messages"); - const cov = coverageOf(projectNative(record.messages), ledgers.readCoverage(), chatId); + const cov = coverageOf(record.messages, ledgers.readCoverage(), chatId); assert.equal(cov.uncoveredCount, 2, "the in-flight batch's messages stay uncovered"); // The re-run — the manual extract button — picks up ONLY the @@ -1109,9 +1115,9 @@ describe("the chunked pipeline (extraction coverage)", () => { auditor: fauxEngine([passThroughAuditStep()]), }); await pipeline.runExtraction(chatId); - // Wave B: the record is native — coverage reads the projection. + // Wave D: the record IS native history — coverage reads it directly. const record = () => chatStore.getChat(chatId)!.messages; - const messages = () => projectNative(record()); + const messages = () => record(); assert.equal(coverageOf(messages(), ledgers.readCoverage(), chatId).uncoveredCount, 0); // Data parts are invisible to the extraction view — a plan-block diff --git a/tests/graph.test.ts b/tests/graph.test.ts index 440944d..05cd4fb 100644 --- a/tests/graph.test.ts +++ b/tests/graph.test.ts @@ -20,7 +20,7 @@ import { after, describe, it } from "node:test"; import type { AuditResult } from "../src/core/contracts.ts"; import type { QuestionAnswerData } from "../src/core/question-core.ts"; import type { LhMessage } from "../src/core/native-messages.ts"; -import { projectNative } from "../src/core/projection.ts"; +import { projectView as projectNative } from "../src/core/view.ts"; import { getGraph, graphSnapshot, mergeFindings, type GraphDeps } from "../src/core/graph.ts"; import { coverageSlots } from "../src/core/coverage.ts"; import { graphStore } from "../src/store/graph.ts"; @@ -33,12 +33,12 @@ const graphDeps = { chatStore, graphStore } as const satisfies GraphDeps; /** The session merge source for a chat — what the pipeline passes: the live * transcript + its updatedAt checkpoint stamp. (Notes imports pass * messages: null — see the notes-merger test below.) */ -function sessionSource(chatId: string): { messages: ReturnType; foldedAt: string } { +function sessionSource(chatId: string): { messages: LhMessage[]; foldedAt: string } { // Wave B: the record is native history — the merger's transcript view // reads through the one-way projection (same shape the extraction // pipeline passes: its batch slots are projected messages). const record = chatStore.getChat(chatId)!; - return { messages: projectNative(record.messages), foldedAt: record.updatedAt }; + return { messages: record.messages, foldedAt: record.updatedAt }; } // ─── ledger isolation (backup-and-clear of data/graph/*.jsonl) ────────────── @@ -1036,7 +1036,7 @@ describe("session coverage read (extraction-UX redesign, 2026-09-24)", () => { // Fold the first two messages: the read derives the split + the // newest fold timestamp from the ledger rows. - const slots = coverageSlots(projectNative(chatStore.getChat(chat.id)!.messages)); + const slots = coverageSlots(chatStore.getChat(chat.id)!.messages); graphStore.appendCoverage({ chatId: chat.id, fp: slots[0]!.fp, tokens: slots[0]!.tokens, foldedAt: "2026-09-24T10:00:00.000Z" }); graphStore.appendCoverage({ chatId: chat.id, fp: slots[1]!.fp, tokens: slots[1]!.tokens, foldedAt: "2026-09-24T11:30:00.000Z" }); // Another session's row must not leak into this read. diff --git a/tests/jobs.test.ts b/tests/jobs.test.ts index 53a58d0..025b3aa 100644 --- a/tests/jobs.test.ts +++ b/tests/jobs.test.ts @@ -58,7 +58,7 @@ import { providerKeyMissing, type ModelSlotName } from "../src/core/settings.ts" import type { PiEngine } from "../src/agent/models.ts"; import type { SearchProvider } from "../src/core/search.ts"; import { PATHS } from "../config.ts"; -import { projectNative } from "../src/core/projection.ts"; +import { projectView as projectNative } from "../src/core/view.ts"; import type { LhMessage } from "../src/core/native-messages.ts"; import type { SteerInput, TurnInput } from "../src/core/input-events.ts"; diff --git a/tests/merge-adopted.test.ts b/tests/merge-adopted.test.ts deleted file mode 100644 index 9702210..0000000 --- a/tests/merge-adopted.test.ts +++ /dev/null @@ -1,74 +0,0 @@ -/** - * mergeAdopted (core/ui-message.ts) — the append-only record-adoption seam - * (owner ground rules, 2026-09-15): server-added steer messages must become - * visible mid-turn without the full snapshot swap that breaks the SDK's - * continuation merge. Pure, no React. - */ - -import assert from "node:assert/strict"; -import { describe, it } from "node:test"; -import { mergeAdopted, type TeacherUIMessage } from "../src/core/index.ts"; - -function userText(id: string, text: string): TeacherUIMessage { - return { id, role: "user", parts: [{ type: "text", text }] }; -} -function steer(id: string): TeacherUIMessage { - // Wave 7 killed the graph-context injection; its marker part is gone. - // The steer-shaped user message here exercises mergeAdopted's - // non-text user parts path with the handoff marker instead. - return { id, role: "user", parts: [{ type: "data-probe-handoff", data: { chatId: "c" } }] }; -} -function assistant(id: string): TeacherUIMessage { - return { id, role: "assistant", parts: [{ type: "text", text: "reply" }] }; -} - -describe("mergeAdopted — append-only adoption", () => { - it("inserts a server-added steer between the persisted core and the in-flight tail", () => { - const local = [userText("u1", "topic"), userText("u2", "answer"), assistant("a_inflight")]; - const record = [userText("u1", "topic"), userText("u2", "answer"), steer("steer_ctx_c")]; - const merged = mergeAdopted(local, record); - assert.ok(merged); - assert.deepEqual( - merged.map((m) => m.id), - ["u1", "u2", "steer_ctx_c", "a_inflight"], - "marker between the answer and the streaming bubble", - ); - }); - - it("adopts into an idle client (no in-flight tail)", () => { - const local = [userText("u1", "topic")]; - const record = [userText("u1", "topic"), steer("s1"), assistant("a1")]; - const merged = mergeAdopted(local, record); - assert.deepEqual( - merged?.map((m) => m.id), - ["u1", "s1", "a1"], - ); - }); - - it("nothing new → null (no needless re-render)", () => { - const local = [userText("u1", "topic")]; - assert.equal(mergeAdopted(local, [userText("u1", "topic")]), null); - // The tail alone is not 'new' — nothing to adopt. - assert.equal(mergeAdopted([assistant("a_x")], []), null); - }); - - it("a missing core id (record rewrite) → null: defer to the full sync at idle", () => { - // A local message vanished from the record — mid-turn this means a - // rewrite, and the append-only shortcut must not paper over it. - const local = [userText("u1", "a"), userText("u2", "b"), assistant("tail")]; - const record = [userText("u2", "b"), steer("s1")]; - assert.equal(mergeAdopted(local, record), null); - }); - - it("only the LAST local message may be the absent in-flight tail", () => { - // Two absent local messages: not a simple append — bail. - const local = [userText("u1", "a"), userText("u2", "gone"), assistant("tail")]; - const record = [userText("u1", "a"), steer("s1")]; - assert.equal(mergeAdopted(local, record), null); - }); - - it("an empty local list adopts the whole record", () => { - const record = [steer("s1"), assistant("a1")]; - assert.deepEqual(mergeAdopted([], record)?.map((m) => m.id), ["s1", "a1"]); - }); -}); \ No newline at end of file diff --git a/tests/phases.test.ts b/tests/phases.test.ts index 74e0155..4cc16f7 100644 --- a/tests/phases.test.ts +++ b/tests/phases.test.ts @@ -10,11 +10,11 @@ */ import assert from "node:assert"; import test from "node:test"; -import type { UIMessage } from "ai"; +import type { LhViewMessage } from "../src/core/view.ts"; import { phaseAt3, phaseTimeline, type Phase3 } from "../src/core/phases.ts"; /** Text-only user message. */ -function user(text: string): UIMessage { +function user(text: string): LhViewMessage { return { id: `u_${text}`, role: "user", parts: [{ type: "text", text }] }; } @@ -23,7 +23,7 @@ function assistantTool( name: string, state: "input-available" | "output-available" | "output-error", output?: unknown, -): UIMessage { +): LhViewMessage { return { id: `a_${name}_${state}_${Math.random().toString(36).slice(2, 6)}`, role: "assistant", @@ -37,7 +37,7 @@ const PLAN_OK = { ok: true, tree: { nodes: [{ id: "n1", title: "N" }], edges: [] /** The verdict button's user message (the custom-message scheme): its own * message carrying ONLY the data-verdict marker part — no human-authored * text; the model-facing wording is toLlmMessages' (native-messages.ts). */ -function verdict(accepted: boolean): UIMessage { +function verdict(accepted: boolean): LhViewMessage { return { id: `u_verdict_${Math.random().toString(36).slice(2, 6)}`, role: "user", @@ -48,7 +48,7 @@ function verdict(accepted: boolean): UIMessage { } /** The accept button's message. */ -function planAccepted(): UIMessage { +function planAccepted(): LhViewMessage { return verdict(true); } diff --git a/tests/plan-blocks.test.ts b/tests/plan-blocks.test.ts index ae9cdbc..769723f 100644 --- a/tests/plan-blocks.test.ts +++ b/tests/plan-blocks.test.ts @@ -13,7 +13,7 @@ import test from "node:test"; import type { PlanTree } from "../src/core/contracts.ts"; import type { LhMessage } from "../src/core/native-messages.ts"; import { planBlocksFor } from "../src/core/plan-blocks.ts"; -import { projectNative } from "../src/core/projection.ts"; +import { projectView as projectNative } from "../src/core/view.ts"; import type { PlanBlockData } from "../src/core/contracts.ts"; // 5-node fixture: a → b → c, b → d, d → e. @@ -83,7 +83,8 @@ function resultOf(callId: string, toolName: string, details: unknown, isError = * new messages are `turn`, run over `record`. Returns the projection of * the whole record after the write. */ function projectAfter(record: LhMessage[], turn: LhMessage[], tree: PlanTree | null) { - const blocks = planBlocksFor(record, turn, tree, projectNative([...record, ...turn])); + // Wave D: the status derivation reads the NATIVE transcript. + const blocks = planBlocksFor(record, turn, tree, [...record, ...turn]); return projectNative([...record, ...turn, ...blocks]); } @@ -128,8 +129,8 @@ test("idempotent — a call whose block the record already carries is never re-e assert.equal(blocksOf(first).length, 1); // Persist replay: the same turn's messages land again over a record // that already emitted the marker → no duplicate. - const record: LhMessage[] = [...turn, ...planBlocksFor([], turn, TREE, projectNative(turn))]; - const blocks2 = planBlocksFor(record, turn, TREE, projectNative(record)); + const record: LhMessage[] = [...turn, ...planBlocksFor([], turn, TREE, turn)]; + const blocks2 = planBlocksFor(record, turn, TREE, record); assert.equal(blocks2.length, 0, "no duplicate block on persist replay"); }); @@ -138,7 +139,7 @@ test("a plan REWRITE (a later turn's successful plan_init) gets its own full blo assistantWithCall("m1", "plan_init", { tree: TREE }), resultOf("m1", "plan_init", { ok: true, tree: TREE }), ]; - const record = [...first, ...planBlocksFor([], first, TREE, projectNative(first))]; + const record = [...first, ...planBlocksFor([], first, TREE, first)]; const second = [ assistantWithCall("m2", "plan_init", { tree: TREE2 }), resultOf("m2", "plan_init", { ok: true, tree: TREE2 }), diff --git a/tests/reattach-transport.test.ts b/tests/reattach-transport.test.ts deleted file mode 100644 index f010f6c..0000000 --- a/tests/reattach-transport.test.ts +++ /dev/null @@ -1,157 +0,0 @@ -/** - * Reattach transport tests (components/chat-transport.ts, the - * ReattachingChatTransport half): the SDK's native resume path — - * reconnectToStream — repointed at GET /api/chats/[id]/stream with the - * platform contract's two adapters: - * - * - turn-settled (the "no turn running" TERMINAL EVENT, not a 204) → - * cancel + onSettled + null (the SDK's no-active-stream signal); - * - the continuation seam — a start chunk whose messageId matches the - * seeded last message is rewritten to a fresh id (the SDK seeds its - * resume streaming message EMPTY; a replace-last would drop the seeded - * message's parts for the whole stream). - * - * Plus the framing seam: the first SSE event is peeked byte-wise (a body - * that arrives split across reads must still resolve its boundary, and - * the leftover bytes must ride the head of the returned stream). - */ -import assert from "node:assert/strict"; -import { afterEach, describe, it } from "node:test"; -import { createReattachingTransport } from "../components/chat-transport.ts"; -import type { ChatInput } from "../src/core/input-events.ts"; - -/** A no-op input-events stash (the reattach tests never send). */ -const NO_INPUT = { takeInput: () => undefined }; - -const realFetch = globalThis.fetch; -afterEach(() => { - globalThis.fetch = realFetch; -}); - -/** SSE-frame each object the way the server's subscriber stream does. */ -function sse(...events: unknown[]): string { - return events.map((e) => `data: ${JSON.stringify(e)}\n\n`).join(""); -} - -/** A Response whose body drips `text` in `slice`-byte reads. */ -function dripped(text: string, slice: number, status = 200): Response { - const bytes = new TextEncoder().encode(text); - const body = new ReadableStream({ - start(controller) { - for (let i = 0; i < bytes.length; i += slice) { - controller.enqueue(bytes.slice(i, i + slice)); - } - controller.close(); - }, - }); - return new Response(body, { status }); -} - -/** Collect a UIMessageChunk stream to an array. */ -async function collect(stream: ReadableStream): Promise { - const out: unknown[] = []; - const reader = stream.getReader(); - for (;;) { - const { done, value } = await reader.read(); - if (done) return out; - out.push(value); - } -} - -const RECONNECT_OPTS = { - chatId: "chat_x", - trigger: "resume-stream" as const, - messageId: undefined, -}; - -describe("reattach transport (reconnectToStream)", () => { - it("GETs the stream endpoint for the chat", async () => { - const urls: string[] = []; - globalThis.fetch = (async (input: RequestInfo | URL) => { - urls.push(String(input)); - return dripped(sse({ type: "start", messageId: "msg_run_1" }), 8); - }) as typeof globalThis.fetch; - const transport = createReattachingTransport("chat_x", {}, NO_INPUT); - await collect((await transport.reconnectToStream(RECONNECT_OPTS))!); - assert.deepEqual(urls, ["/api/chats/chat_x/stream"]); - }); - - it("turn-settled: fires onSettled, resolves null (the body is released)", async () => { - globalThis.fetch = (async () => - dripped(sse({ type: "turn-settled" }, "[DONE]"), 8)) as typeof globalThis.fetch; - let settled = 0; - const transport = createReattachingTransport("chat_x", { onSettled: () => settled++ }, NO_INPUT); - const stream = await transport.reconnectToStream(RECONNECT_OPTS); - assert.equal(stream, null); - assert.equal(settled, 1); - // (The reader.cancel hygiene is best-effort: undici wraps Response - // bodies, so the source-level cancel is not observable here.) - }); - - it("204 (the SDK's native no-stream signal) resolves null untouched", async () => { - globalThis.fetch = (async () => new Response(null, { status: 204 })) as typeof globalThis.fetch; - const transport = createReattachingTransport("chat_x", {}, NO_INPUT); - assert.equal(await transport.reconnectToStream(RECONNECT_OPTS), null); - }); - - it("a fresh start chunk passes through the SDK parse verbatim", async () => { - // slice 13: the boundary lands MID-read, so the peek's read-past - // remainder must ride the head (the bug this pins: dropped leftovers). - globalThis.fetch = (async () => - dripped( - sse( - { type: "start", messageId: "msg_run_1" }, - { type: "text-start", id: "msg_run_1-t0" }, - { type: "text-delta", id: "msg_run_1-t0", delta: "hi" }, - ), - 13, - )) as typeof globalThis.fetch; - const transport = createReattachingTransport("chat_x", {}, NO_INPUT); - const chunks = (await collect((await transport.reconnectToStream(RECONNECT_OPTS))!)) as Array< - Record - >; - assert.equal(chunks[0]!.type, "start"); - assert.equal(chunks[0]!.messageId, "msg_run_1", "no rewrite when the id is fresh"); - assert.equal(chunks[1]!.type, "text-start"); - assert.equal(chunks[2]!.delta, "hi", "the leftover bytes after the peek ride the head"); - }); - - it("a CONTINUATION start chunk (id === seeded last message) is rewritten fresh", async () => { - globalThis.fetch = (async () => - dripped( - sse({ type: "start", messageId: "msg_seed" }, { type: "text-start", id: "msg_seed-t3" }), - 64, - )) as typeof globalThis.fetch; - const transport = createReattachingTransport("chat_x", { - lastMessageId: () => "msg_seed", - }, NO_INPUT); - const chunks = (await collect((await transport.reconnectToStream(RECONNECT_OPTS))!)) as Array< - Record - >; - assert.equal(chunks[0]!.type, "start"); - assert.notEqual(chunks[0]!.messageId, "msg_seed"); - assert.match(String(chunks[0]!.messageId), /^msg_[a-z0-9]+_[a-f0-9]{8}$/, "the server's id shape"); - // Part ids stay server-prefixed — nothing keys parts by message id. - assert.equal(chunks[1]!.id, "msg_seed-t3"); - }); - - it("survives a body split byte-by-byte across reads", async () => { - globalThis.fetch = (async () => - dripped(sse({ type: "start", messageId: "msg_run_1" }, { type: "finish", finishReason: "stop" }), 1)) as typeof globalThis.fetch; - const transport = createReattachingTransport("chat_x", {}, NO_INPUT); - const chunks = (await collect((await transport.reconnectToStream(RECONNECT_OPTS))!)) as Array< - Record - >; - assert.deepEqual( - chunks.map((c) => c.type), - ["start", "finish"], - ); - assert.equal(chunks[0]!.messageId, "msg_run_1"); - }); - - it("an unhappy endpoint throws (the SDK surfaces the error banner)", async () => { - globalThis.fetch = (async () => new Response("boom", { status: 500 })) as typeof globalThis.fetch; - const transport = createReattachingTransport("chat_x", {}, NO_INPUT); - await assert.rejects(() => transport.reconnectToStream(RECONNECT_OPTS), /boom/); - }); -}); diff --git a/tests/session-core.test.ts b/tests/session-core.test.ts new file mode 100644 index 0000000..e5c0daf --- /dev/null +++ b/tests/session-core.test.ts @@ -0,0 +1,552 @@ +/** + * SESSION MODEL tests (components/session-core.ts — Wave D): the pure state + * machine that replaced useChat + the AI-SDK transport. Everything the old + * transport/merge suites pinned survives here on the native wire: + * + * - the 409 IDENTITY-RETRY (postInputEvent): the event was never persisted + * pre-acceptance, so the retry waits out the racing turn and re-POSTs + * the SAME body — no merge, no record read; + * - the reattach path (resumeStream): record refetch, then the SAME + * verbatim-AgentEvent consume as a POST; turn-settled is the terminal; + * byte-split SSE frames never corrupt (the framing pin); + * - append-only ADOPTION (adoptRecord/absorbed): server-added steers show + * mid-turn, the record rewrite bails to the idle full sync, and the + * optimistic entries the write-ahead absorbed never double their + * bubbles; + * - the view application (applyWireEvent → projectView): full-partial + * replaces, the fork-2 question-placeholder ignore, execution states, + * raw error display (owner ruling); + * - the continuation fold (needsContinuation): the mount/sync seam's + * native replacement for lastAssistantMessageIsCompleteWithToolCalls. + */ +import assert from "node:assert/strict"; +import { afterEach, describe, it } from "node:test"; +import { absorbed, pendingCallTool, SessionModel } from "../components/session-core.ts"; +import type { ChatInput } from "../src/core/input-events.ts"; +import type { LhMessage } from "../src/core/native-messages.ts"; +import { newUserMessage, newAnswerResult, newVerdictMessage, needsContinuation } from "../src/core/native-messages.ts"; + +const realFetch = globalThis.fetch; +afterEach(() => { + globalThis.fetch = realFetch; +}); + +/** SSE-frame each object the way the server's subscriber stream does. */ +function sse(...events: unknown[]): string { + return events.map((e) => (e === "[DONE]" ? "data: [DONE]\n\n" : `data: ${JSON.stringify(e)}\n\n`)).join(""); +} + +/** A Response whose body drips `text` in `slice`-byte reads (the framing pin). */ +function dripped(text: string, slice: number, status = 200): Response { + const bytes = new TextEncoder().encode(text); + const body = new ReadableStream({ + start(controller) { + for (let i = 0; i < bytes.length; i += slice) { + controller.enqueue(bytes.slice(i, i + slice)); + } + controller.close(); + }, + }); + return new Response(body, { status }); +} + +/** A plain single-read 200 body. */ +function sseResponse(...events: unknown[]): Response { + return new Response(sse(...events), { status: 200 }); +} + +function assistantMsg(id: string, text: string): LhMessage { + return { + id, + role: "assistant", + content: [{ type: "text", text }], + stopReason: "stop", + usage: { input: 0, output: 0, total: 0, cacheReads: 0, cacheWrites: 0 }, + timestamp: 0, + provider: "openrouter", + model: "test", + } as unknown as LhMessage; +} + +/** A streaming assistant partial (pi events carry the FULL partial). */ +function partial(id: string, blocks: unknown[], stopReason?: string): Record { + return { + type: "message_update", + message: { + id, + role: "assistant", + content: blocks, + ...(stopReason ? { stopReason } : {}), + usage: { input: 0, output: 0, total: 0, cacheReads: 0, cacheWrites: 0 }, + provider: "openrouter", + model: "test", + }, + assistantMessageEvent: { type: "text", textDelta: "" }, + }; +} + +function model(opts: Partial[0]> = {}): SessionModel { + return new SessionModel({ chatId: "chat_x", initialMessages: [], ...opts }); +} + +// ─── the 409 identity-retry ──────────────────────────────────────────────── + +interface Call { + url: string; + body: string | undefined; +} + +function installFetch(routes: { + chatPOST: (call: Call, attempt: number) => Response; + jobsGET?: (poll: number) => boolean; +}): Call[] { + const calls: Call[] = []; + let poll = 0; + globalThis.fetch = (async (input: RequestInfo | URL, init?: RequestInit) => { + const url = String(input); + calls.push({ url, body: init?.body as string | undefined }); + if (url === "/api/chat" && init?.method === "POST") { + return routes.chatPOST(calls.at(-1)!, calls.filter((c) => c.url === "/api/chat").length); + } + if (url.startsWith("/api/jobs")) return Response.json({ turnInFlight: routes.jobsGET?.(++poll) ?? false }); + throw new Error(`unexpected fetch: ${url}`); + }) as typeof globalThis.fetch; + return calls; +} + +describe("postInputEvent (the 409 identity-retry)", { skip: process.env.NO_SLOW ? true : undefined }, () => { + it("passes non-409 responses through untouched", async () => { + const calls = installFetch({ chatPOST: () => new Response("stream", { status: 200 }) }); + const m = model({ pollMs: 10, waitMaxMs: 100 }); + const res = await m.postInputEvent({ type: "text", text: "hi" }); + assert.equal(res.status, 200); + assert.equal(calls.length, 1, "no polling, single call"); + assert.deepEqual(JSON.parse(calls[0]!.body!), { chatId: "chat_x", input: { type: "text", text: "hi" } }); + }); + + it("waits out the turn, then re-POSTs the SAME event body (identity retry)", async () => { + const event: ChatInput = { type: "text", text: "my question while the wake turn ran" }; + const calls = installFetch({ + chatPOST: (_c, n) => (n === 1 ? new Response("busy", { status: 409 }) : new Response("stream", { status: 200 })), + jobsGET: (poll) => poll <= 1, // first poll: still running; second: done + }); + const m = model({ pollMs: 10, waitMaxMs: 100 }); + const res = await m.postInputEvent(event); + assert.equal(res.status, 200); + const posts = calls.filter((c) => c.url === "/api/chat"); + assert.equal(posts.length, 2, "one 409, one retry"); + assert.equal(posts[1]!.body, posts[0]!.body, "the event body is byte-identical"); + }); + + it("never reads the record — the event is the only payload", async () => { + const calls = installFetch({ + chatPOST: (_c, n) => (n === 1 ? new Response("busy", { status: 409 }) : new Response("stream", { status: 200 })), + }); + const m = model({ pollMs: 10, waitMaxMs: 100 }); + assert.equal((await m.postInputEvent({ type: "text", text: "q" })).status, 200); + assert.equal( + calls.filter((c) => c.url.startsWith("/api/chats/")).length, + 0, + "the retry never reads the record — the server owns it", + ); + }); + + it("loops on repeated 409s, re-POSTing the same event each time", async () => { + const event: ChatInput = { type: "answer", toolCallId: "call-1", answer: { correct: true } as never }; + const calls = installFetch({ + chatPOST: (_c, n) => (n <= 2 ? new Response("busy", { status: 409 }) : new Response("stream", { status: 200 })), + }); + const m = model({ pollMs: 10, waitMaxMs: 100 }); + assert.equal((await m.postInputEvent(event)).status, 200); + const posts = calls.filter((c) => c.url === "/api/chat"); + assert.equal(posts.length, 3); + assert.ok(posts.every((c) => c.body === posts[0]!.body), "every attempt carries the same event"); + }); + + it("gives up after the deadline and surfaces the last 409", async () => { + const calls = installFetch({ + chatPOST: () => new Response("busy", { status: 409 }), + jobsGET: () => true, + }); + const m = model({ pollMs: 10, waitMaxMs: 100 }); + const res = await m.postInputEvent({ type: "text", text: "q" }); + assert.equal(res.status, 409); + const polls = calls.filter((c) => c.url.startsWith("/api/jobs")).length; + assert.ok(polls <= 15, `expected bounded polling (~10×10ms), got ${polls}`); + }); +}); + +// ─── send: optimistic appends + wire consumption ──────────────────────────── + +describe("send (one input event, verbatim consumption)", () => { + it("optimistically appends the text, POSTs {chatId, input}, consumes to [DONE]", async () => { + const posts: string[] = []; + globalThis.fetch = (async (input: RequestInfo | URL, init?: RequestInit) => { + const url = String(input); + if (url === "/api/chat" && init?.method === "POST") { + posts.push(init.body as string); + return sseResponse( + { type: "agent_start" }, + { type: "message_start", message: { role: "assistant", id: "a1", content: [] } }, + partial("a1", [{ type: "text", text: "hello there" }]), + { type: "message_end", message: { role: "assistant", id: "a1", content: [{ type: "text", text: "hello there" }], stopReason: "stop" } }, + { type: "agent_end", messages: [] }, + "[DONE]", + ); + } + throw new Error(`unexpected fetch: ${url}`); + }) as typeof globalThis.fetch; + + const m = model(); + const pending = m.send({ type: "text", text: "my topic" }); + // The optimistic bubble is INSTANT (the instant-answer UX seam): + assert.equal(m.getState().status, "submitted"); + assert.match((m.getState().nativeMessages.at(-1) as { id?: string }).id ?? "", /^c_/); + assert.equal(m.getState().messages.at(-1)!.parts[0]!.text, "my topic"); + await pending; + assert.deepEqual(JSON.parse(posts[0]!), { chatId: "chat_x", input: { type: "text", text: "my topic" } }); + assert.equal(m.getState().status, "ready"); + // The streamed assistant REPLACED through full partials (never deltas): + const view = m.getState().messages; + assert.equal(view.at(-1)!.role, "assistant"); + assert.equal(view.at(-1)!.parts.at(-1)!.text, "hello there"); + }); + + it("a 503 refusal surfaces the RAW body text (parseModelsRefusal's contract)", async () => { + globalThis.fetch = (async () => + new Response(JSON.stringify({ code: "models_not_configured", missing: ["strong"] }), { + status: 503, + })) as typeof globalThis.fetch; + const m = model(); + await m.send({ type: "text", text: "q" }); + const state = m.getState(); + assert.equal(state.status, "error"); + assert.equal( + state.error!.message, + JSON.stringify({ code: "models_not_configured", missing: ["strong"] }), + "the banner carries the raw envelope — the refusal flow JSON-parses it", + ); + }); + + it("the optimistic answer result attaches to the pending quiz part at once", async () => { + const quizCall = { + id: "call-1", + role: "assistant", + content: [ + { type: "text", text: "Q?" }, + { type: "toolCall", id: "call-1", name: "quiz", arguments: { question: "Q?", choices: ["a", "b"] } }, + ], + stopReason: "toolUse", + usage: { input: 0, output: 0, total: 0, cacheReads: 0, cacheWrites: 0 }, + provider: "openrouter", + model: "test", + timestamp: 0, + } as unknown as LhMessage; + let postBody = ""; + globalThis.fetch = (async (input: RequestInfo | URL, init?: RequestInit) => { + if (String(input) === "/api/chat") { + postBody = init?.body as string; + return sseResponse("[DONE]"); + } + throw new Error(`unexpected fetch: ${String(input)}`); + }) as typeof globalThis.fetch; + const m = model({ initialMessages: [newUserMessage("hi"), quizCall] }); + const pending = m.submitAnswer("quiz", "call-1", { correct: true } as never); + // The dock's AnsweredCard state — before any server round-trip: + const state = m.getState(); + const quizPart = state.messages + .flatMap((v) => v.parts) + .find((p) => p.type === "tool-quiz")!; + assert.equal(quizPart.state, "output-available"); + assert.deepEqual(quizPart.output, { correct: true }); + await pending; + assert.deepEqual(JSON.parse(postBody), { + chatId: "chat_x", + input: { type: "answer", toolCallId: "call-1", answer: { correct: true } }, + }); + }); + + it("ignores the question placeholder's clean execution end (fork 2) and keeps errors", async () => { + // The dock's mount gate keys on the part STATE the live execution + // layer drives MID-STREAM: a clean question execution stays + // input-available (the in-memory "Question posed" placeholder + // never shows); an INVALID call is output-error and NEVER mounts + // (the modal-realism crash pin). + const enc = new TextEncoder(); + let push: (e: unknown) => void = () => {}; + const body = new ReadableStream({ + start(controller) { + push = (e) => controller.enqueue(enc.encode(e === "[DONE]" ? "data: [DONE]\n\n" : `data: ${JSON.stringify(e)}\n\n`)); + }, + }); + globalThis.fetch = (async () => new Response(body, { status: 200 })) as typeof globalThis.fetch; + const m = model(); + const done = m.send({ type: "text", text: "q" }); + const settle = () => new Promise((r) => setTimeout(r, 0)); + push({ + type: "message_start", + message: { + role: "assistant", + id: "a1", + content: [ + { type: "toolCall", id: "q1", name: "quiz", arguments: {} }, + { type: "toolCall", id: "bad1", name: "quiz", arguments: {} }, + ], + }, + }); + await settle(); + push({ type: "tool_execution_end", toolCallId: "q1", toolName: "quiz", isError: false, result: { content: [{ type: "text", text: "Question posed" }], details: { placeholder: true } } }); + push({ type: "tool_execution_end", toolCallId: "bad1", toolName: "quiz", isError: true, result: { content: [{ type: "text", text: "correctAnswer does not match any option value" }], details: undefined } }); + await settle(); + const parts = m.getState().messages.flatMap((v) => v.parts).filter((p) => p.type === "tool-quiz"); + assert.equal(parts.length, 2); + assert.equal(parts[0]!.state, "input-available", "the clean placeholder is IGNORED — the dock mounts on it"); + assert.equal(parts[1]!.state, "output-error"); + assert.equal(parts[1]!.errorText, "correctAnswer does not match any option value"); + push({ type: "agent_end", messages: [] }); + push("[DONE]"); + await done; + }); + + it("a provider error rides VERBATIM (owner ruling) — no masking anywhere", async () => { + globalThis.fetch = (async () => + sseResponse( + partial("a1", [{ type: "text", text: "" }]), + { type: "message_end", message: { role: "assistant", id: "a1", content: [{ type: "text", text: "" }], stopReason: "error", errorMessage: "OpenRouter 429: rate limited" } }, + { type: "agent_end", messages: [] }, + "[DONE]", + )) as typeof globalThis.fetch; + const m = model(); + await m.send({ type: "text", text: "q" }); + assert.equal(m.getState().status, "ready"); + assert.equal(m.getState().error!.message, "OpenRouter 429: rate limited"); + }); +}); + +// ─── reattach (resumeStream) ──────────────────────────────────────────────── + +describe("resumeStream (the reattach path)", () => { + it("refetches the record, then GETs the stream endpoint", async () => { + const urls: string[] = []; + globalThis.fetch = (async (input: RequestInfo | URL) => { + const url = String(input); + urls.push(url); + if (url === "/api/chats/chat_x") return Response.json({ messages: [] }); + if (url === "/api/chats/chat_x/stream") + return dripped(sse({ type: "message_start", message: { role: "assistant", id: "a1", content: [{ type: "text", text: "hi" }] } }, "[DONE]"), 8); + throw new Error(`unexpected fetch: ${url}`); + }) as typeof globalThis.fetch; + const m = model(); + await m.resumeStream(); + assert.deepEqual(urls, ["/api/chats/chat_x", "/api/chats/chat_x/stream"]); + assert.equal(m.getState().status, "ready"); + }); + + it("turn-settled fires the settle hook and ends cleanly", async () => { + let settled = 0; + globalThis.fetch = (async (input: RequestInfo | URL) => { + const url = String(input); + if (url === "/api/chats/chat_x") return Response.json({ messages: [] }); + if (url === "/api/chats/chat_x/stream") return dripped(sse({ type: "turn-settled" }, "[DONE]"), 8); + throw new Error(`unexpected fetch: ${url}`); + }) as typeof globalThis.fetch; + const m = model({ onReattachSettled: () => settled++ }); + await m.resumeStream(); + assert.equal(settled, 1); + assert.equal(m.getState().status, "ready"); + assert.equal(m.getState().error, null); + }); + + it("a replayed CONTINUATION mounts BESIDE the seeded message (the SDK replace-last seam is dead)", async () => { + // The seeded record ends with an assistant message; the replayed + // turn continues with its OWN assistant message — the view groups + // them as one document with a step separator (pixel-identical to a + // live continuation; the settle sync lands the canon). + globalThis.fetch = (async (input: RequestInfo | URL) => { + const url = String(input); + if (url === "/api/chats/chat_x") return Response.json({ messages: [assistantMsg("seed", "the question")] }); + if (url === "/api/chats/chat_x/stream") + return sseResponse( + { type: "message_start", message: { role: "assistant", id: "run1", content: [{ type: "text", text: "the reaction" }] } }, + { type: "message_end", message: { role: "assistant", id: "run1", content: [{ type: "text", text: "the reaction" }], stopReason: "stop" } }, + { type: "agent_end", messages: [] }, + "[DONE]", + ); + throw new Error(`unexpected fetch: ${url}`); + }) as typeof globalThis.fetch; + const m = model({ initialMessages: [assistantMsg("seed", "the question")] }); + await m.resumeStream(); + const view = m.getState().messages; + assert.equal(view.length, 1, "one continuous assistant document"); + assert.deepEqual( + view[0]!.parts.map((p) => p.type), + ["step-start", "text", "step-start", "text"], + "seeded message keeps rendering; the run mounts with a step separator", + ); + }); + + it("survives a body split byte-by-byte across reads (the framing pin)", async () => { + globalThis.fetch = (async (input: RequestInfo | URL) => { + const url = String(input); + if (url === "/api/chats/chat_x") return Response.json({ messages: [] }); + if (url === "/api/chats/chat_x/stream") + return dripped( + sse( + { type: "message_start", message: { role: "assistant", id: "a1", content: [] } }, + partial("a1", [{ type: "text", text: "hi" }]), + { type: "message_end", message: { role: "assistant", id: "a1", content: [{ type: "text", text: "hi" }], stopReason: "stop" } }, + ) + sse("[DONE]"), + 1, + ); + throw new Error(`unexpected fetch: ${url}`); + }) as typeof globalThis.fetch; + const m = model(); + await m.resumeStream(); + assert.equal(m.getState().messages.at(-1)!.parts.at(-1)!.text, "hi"); + }); + + it("an unhappy endpoint surfaces the error banner", async () => { + globalThis.fetch = (async (input: RequestInfo | URL) => { + const url = String(input); + if (url === "/api/chats/chat_x") return Response.json({ messages: [] }); + if (url === "/api/chats/chat_x/stream") return new Response("boom", { status: 500 }); + throw new Error(`unexpected fetch: ${url}`); + }) as typeof globalThis.fetch; + const m = model(); + await m.resumeStream(); + const state = m.getState(); + assert.equal(state.status, "error"); + assert.match(state.error!.message, /boom/); + }); +}); + +// ─── adoption (append-only ground rules, owner 2026-09-15) ────────────────── + +function userText(id: string, text: string): LhMessage { + return { ...newUserMessage(text), id }; +} + +describe("adoptRecord — append-only adoption", () => { + it("splices a server-added steer between the record and the live tail", () => { + const m = model({ initialMessages: [userText("u1", "topic")] }); + m.applyWireEvent({ type: "message_start", message: { role: "assistant", id: "a_live", content: [{ type: "text", text: "streaming" }] } }); + const steer = { ...newUserMessage("steered content"), id: "steer_1" } as LhMessage & { role: "user" }; + const record = [userText("u1", "topic"), steer]; + m.adoptRecord(record); + assert.deepEqual( + m.getState().nativeMessages.map((x) => x.id), + ["u1", "steer_1", "a_live"], + "the steer shows NOW, the live tail stays last", + ); + }); + + it("a missing base id (record rewrite) bails — the idle full sync is the net", () => { + const m = model({ initialMessages: [userText("u1", "a"), userText("u2", "b")] }); + m.adoptRecord([userText("u2", "b")]); // u1 vanished: a rewrite, not an append + assert.deepEqual( + m.getState().nativeMessages.map((x) => x.id), + ["u1", "u2"], + "the adoption is a no-op on a non-append", + ); + }); + + it("absorbs the optimistic entries the record has taken over (no doubled bubbles)", () => { + // A text send's optimistic bubble (local c_ id): the write-ahead's + // server copy absorbs it by CONTENT — the bubble never doubles. + const optimistic = { ...newUserMessage("my question"), id: "c_local_1" } as LhMessage; + const serverCopy = { ...newUserMessage("my question"), id: "srv_9" } as LhMessage; + assert.ok(absorbed(optimistic, [userText("u1", "topic"), serverCopy]), "content match absorbs the local copy"); + assert.ok(!absorbed(serverCopy, [optimistic]), "server stamps are never absorbed"); + assert.ok(!absorbed({ ...newUserMessage("my question") }, [serverCopy]), "an id-less message is never absorbed"); + // The answer result absorbs by toolCallId (ids differ, content may too — + // the toolCallId is the identity of an answer). + const answer = { ...newAnswerResult("call-1", "quiz", { correct: true }), id: "c_local_2" } as LhMessage; + assert.ok( + absorbed(answer, [{ ...newAnswerResult("call-1", "quiz", { correct: true }), id: "srv_10" } as LhMessage]), + "the answer result absorbs by toolCallId", + ); + // A verdict marker absorbs by accepted (the timestamps differ by design). + const verdict = { ...newVerdictMessage(true, "2026-09-26T00:00:00.000Z"), id: "c_local_3" } as LhMessage; + const serverVerdict = { ...newVerdictMessage(true, "2026-09-26T00:00:01.000Z"), id: "srv_11" } as LhMessage; + assert.ok(absorbed(verdict, [serverVerdict]), "the verdict absorbs by accepted"); + }); + + it("setMessages is the full snapshot swap (the idle settle sync)", () => { + const m = model({ initialMessages: [userText("u1", "old")] }); + m.setMessages([userText("u1", "old"), userText("u2", "new"), assistantMsg("a1", "reply")]); + assert.deepEqual( + m.getState().nativeMessages.map((x) => (x as { id?: string }).id), + ["u1", "u2", "a1"], + ); + }); +}); + +// ─── the continuation fold (mount/sync seam) ──────────────────────────────── + +function assistantWithCall(id: string, callId: string, name = "research"): LhMessage { + return { + id, + role: "assistant", + content: [{ type: "toolCall", id: callId, name, arguments: { topic: "x" } }], + stopReason: "toolUse", + usage: { input: 0, output: 0, total: 0, cacheReads: 0, cacheWrites: 0 }, + provider: "openrouter", + model: "test", + timestamp: 0, + } as unknown as LhMessage; +} + +function resultMsg(callId: string, name = "research"): LhMessage { + return { + ...newAnswerResult(callId, name, { ok: true }), + id: `r_${callId}`, + } as unknown as LhMessage; +} + +describe("needsContinuation (the native lastAssistantMessageIsCompleteWithToolCalls)", () => { + it("a record ending on a fully-answered tool call continues", () => { + const messages = [userText("u1", "topic"), assistantWithCall("a1", "call-r1"), resultMsg("call-r1")]; + assert.equal(needsContinuation(messages), true); + }); + + it("text-tailed records never continue", () => { + const messages = [userText("u1", "topic"), assistantMsg("a1", "done talking")]; + assert.equal(needsContinuation(messages), false); + }); + + it("a dangling question never fires a turn (the dock owns it)", () => { + const messages = [userText("u1", "topic"), assistantWithCall("a1", "call-q1", "quiz")]; + assert.equal(needsContinuation(messages), false); + }); + + it("parallel quizzes: only ALL-answered continues (the dock still owns the open one)", () => { + const messages = [ + userText("u1", "topic"), + { + ...assistantWithCall("a1", "call-a"), + content: [ + { type: "toolCall", id: "call-a", name: "quiz", arguments: {} }, + { type: "toolCall", id: "call-b", name: "quiz", arguments: {} }, + ], + } as unknown as LhMessage, + resultMsg("call-a", "quiz"), + ]; + assert.equal(needsContinuation(messages), false); + const both = [...messages, resultMsg("call-b", "quiz")]; + assert.equal(needsContinuation(both), true, "once the dock answers B, the agent owes the reaction"); + }); + + it("an orphan result (no asking assistant) never continues", () => { + assert.equal(needsContinuation([userText("u1", "topic"), resultMsg("call-x")]), false); + }); +}); + +// ─── the pending-call tool scan ───────────────────────────────────────────── + +describe("pendingCallTool", () => { + it("finds the dangling call's tool name (the optimistic answer's stamp)", () => { + const messages = [assistantWithCall("a1", "call-q1", "ask_user_question")]; + assert.equal(pendingCallTool(messages, "call-q1"), "ask_user_question"); + assert.equal(pendingCallTool(messages, "call-zz"), null); + }); +}); diff --git a/tests/stream-activity.test.ts b/tests/stream-activity.test.ts index 5938d47..2c2896a 100644 --- a/tests/stream-activity.test.ts +++ b/tests/stream-activity.test.ts @@ -7,11 +7,11 @@ */ import assert from "node:assert/strict"; import { describe, it } from "node:test"; -import type { UIMessage } from "ai"; +import type { LhViewMessage } from "../src/core/view.ts"; import { streamingToolActivity } from "../src/core/pending.ts"; -function msg(role: string, parts: unknown[]): UIMessage { - return { id: "m1", role, parts } as unknown as UIMessage; +function msg(role: string, parts: unknown[]): LhViewMessage { + return { id: "m1", role, parts } as unknown as LhViewMessage; } function toolPart(tool: string, state: string) {