teacher-prose-only extraction view: coverage, fingerprints, tokens, and batches derive from the teacher's text blocks alone (owner 2026-09-28)
This commit is contained in:
@@ -330,7 +330,7 @@ Sub-agents **verify, never one-shot**: the demo-builder headlessly executes its
|
|||||||
flowchart TB
|
flowchart TB
|
||||||
TRIG["trigger seam (turn-service): coverageOf(record, coverageLedger)<br/>— uncovered tokens ≥ graph.extractionChunkTokens (settings)?"] -->|"yes, single-flight per source"| RUN["extraction.runExtraction(chatId)"]
|
TRIG["trigger seam (turn-service): coverageOf(record, coverageLedger)<br/>— uncovered tokens ≥ graph.extractionChunkTokens (settings)?"] -->|"yes, single-flight per source"| RUN["extraction.runExtraction(chatId)"]
|
||||||
IMPORT["POST /api/graph/import (notes front door)<br/>deterministic src id = sha256(path) — idempotent re-imports"] -->|"importNotes: shared pipeline, second extractor prompt"| RUN
|
IMPORT["POST /api/graph/import (notes front door)<br/>deterministic src id = sha256(path) — idempotent re-imports"] -->|"importNotes: shared pipeline, second extractor prompt"| RUN
|
||||||
RUN --> BATCH["batchRuns — greedy token-budgeted batches of oldest uncovered messages"]
|
RUN --> BATCH["batchRuns — greedy token-budgeted batches of oldest uncovered TEACHER-PROSE messages"]
|
||||||
BATCH --> EXT["EXTRACTOR (LLM) — sees ONLY the session markdown view<br/>(tree-blind, graph-blind; shared engine: core/session-markdown.ts)"]
|
BATCH --> EXT["EXTRACTOR (LLM) — sees ONLY the session markdown view<br/>(tree-blind, graph-blind; shared engine: core/session-markdown.ts)"]
|
||||||
EXT --> AUD["AUDITOR (LLM, agentic loop) — sees findings ONLY, never the source;<br/>explores the graph via find / list_sessions / list_propositions tools,<br/>judges identity (link-or-create, equivalence bias) + may draw bridging edges"]
|
EXT --> AUD["AUDITOR (LLM, agentic loop) — sees findings ONLY, never the source;<br/>explores the graph via find / list_sessions / list_propositions tools,<br/>judges identity (link-or-create, equivalence bias) + may draw bridging edges"]
|
||||||
AUD --> MERGE["MERGER (deterministic, core/graph.ts) — the ONLY ledger writer:<br/>identity via validated linkTo · bridging-edge resolution · dedupe edges · coverage checkpoint"]
|
AUD --> MERGE["MERGER (deterministic, core/graph.ts) — the ONLY ledger writer:<br/>identity via validated linkTo · bridging-edge resolution · dedupe edges · coverage checkpoint"]
|
||||||
@@ -338,7 +338,7 @@ flowchart TB
|
|||||||
MERGE -->|"failure at ANY stage"| NOTHING["checkpoints nothing — the next trigger re-runs"]
|
MERGE -->|"failure at ANY stage"| NOTHING["checkpoints nothing — the next trigger re-runs"]
|
||||||
```
|
```
|
||||||
|
|
||||||
**Raw reasoning chains only** (2026-09-27 cleanup): the extractor reads ONLY the session — the whole thing, rendered as readable markdown by the shared engine (`core/session-markdown.ts`; the same renderer will serve future user-facing session export) — and proposes free-form propositions (true statements; a misconception is stated as its true correction) and `follows`/`supports` edges. The submit schemas are slim (owner, 2026-09-28): propositions carry id/name/statement/optional aliases and edges carry from/to/type — the `messages` and `evidence` fields are gone (provenance is chat-level; quote fields were never persisted and weak models burned retries on them), and a `prepareArguments` heal at the pre-validation seam rescues stringified-payload submits. Every node is an objective, world-facing fact. No quiz outcomes, no understanding assessments — the events ledger and `UnderstandingLevel` are deleted; the learner's relationship to a proposition becomes practice-module territory (assessment records, later).
|
**Raw reasoning chains only** (2026-09-27 cleanup; teacher-prose view 2026-09-28): the extractor reads ONLY the session — and since the 2026-09-28 ruling, ONLY the teacher agent's prose: plain paragraphs under the session title, no learner messages, no thinking, no tool calls or results, no quiz blocks. Coverage fingerprints, slot tokens, and batch budgets all derive from that same selection (`messageViewLines` in `core/coverage.ts`), so "covered" is exactly "what the extractor saw" — a quiz answer or learner edit can never open a run, and only a teacher-prose edit re-opens its message. The old call+result inseparability bond is deleted (nothing left to keep together); batching is pure greedy fill. The shared markdown engine (`core/session-markdown.ts`) keeps the full session format for the future user-facing export and is no longer used by extraction. The extractor proposes free-form propositions (true statements; a misconception is stated as its true correction) and `follows`/`supports` edges. The submit schemas are slim (owner, 2026-09-28): propositions carry id/name/statement/optional aliases and edges carry from/to/type — the `messages` and `evidence` fields are gone (provenance is chat-level; quote fields were never persisted and weak models burned retries on them), and a `prepareArguments` heal at the pre-validation seam rescues stringified-payload submits. Every node is an objective, world-facing fact. No quiz outcomes, no understanding assessments — the events ledger and `UnderstandingLevel` are deleted; the learner's relationship to a proposition becomes practice-module territory (assessment records, later).
|
||||||
|
|
||||||
**Serialized audit** (owner ruling 2026-09-28): extraction may run in parallel across sources, but the audit→merge section is a process-wide critical section — a FIFO mutex on `globalThis` (the same module-instance idiom as the run registries). Before any auditor LLM call or graph read, a run checks the gate and waits (`waiting` phase, visible in the status feed) until every earlier run's merge has landed, so each auditor explores the graph as it actually stands — two concurrent audits would each link into a graph the other is about to change, the structural mint of duplicate nodes. The lock is reacquired per batch (interleaved runs each see the latest graph instead of starving) and is abort-aware on both sides of the queue.
|
**Serialized audit** (owner ruling 2026-09-28): extraction may run in parallel across sources, but the audit→merge section is a process-wide critical section — a FIFO mutex on `globalThis` (the same module-instance idiom as the run registries). Before any auditor LLM call or graph read, a run checks the gate and waits (`waiting` phase, visible in the status feed) until every earlier run's merge has landed, so each auditor explores the graph as it actually stands — two concurrent audits would each link into a graph the other is about to change, the structural mint of duplicate nodes. The lock is reacquired per batch (interleaved runs each see the latest graph instead of starving) and is abort-aware on both sides of the queue.
|
||||||
|
|
||||||
|
|||||||
@@ -26,7 +26,8 @@ import { getGraph, mergeFindings, type EmissionStats, type GraphDeps } from "../
|
|||||||
import type { GraphStore } from "../../core/ports.ts";
|
import type { GraphStore } from "../../core/ports.ts";
|
||||||
import { settleGraphLayout } from "../../core/layout-settle.ts";
|
import { settleGraphLayout } from "../../core/layout-settle.ts";
|
||||||
import { batchRuns, coverageOf, messageViewLines } from "../../core/coverage.ts";
|
import { batchRuns, coverageOf, messageViewLines } from "../../core/coverage.ts";
|
||||||
import { sessionMarkdown } from "../../core/session-markdown.ts";
|
import { teacherProseBlocks } from "../../core/coverage.ts";
|
||||||
|
import type { LhMessage } from "../../core/native-messages.ts";
|
||||||
import { auditorGraphTools } from "../tools/graph.ts";
|
import { auditorGraphTools } from "../tools/graph.ts";
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -83,22 +84,29 @@ import { auditorGraphTools } from "../tools/graph.ts";
|
|||||||
// ─── the source view (what the extractor sees) ──────────────────────────────
|
// ─── the source view (what the extractor sees) ──────────────────────────────
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* A session rendered as readable markdown — the EXTRACTOR's view (owner
|
* The extraction view: the session as TEACHER PROSE ONLY (owner ruling,
|
||||||
* ruling 2026-09-27): the whole session goes in, formatted by the ONE
|
* 2026-09-28 — supersedes the 2026-09-27 whole-session markdown view):
|
||||||
* shared session-markdown engine (core/session-markdown.ts — the same
|
* plain prose paragraphs under the session title. The shared markdown
|
||||||
* engine the user-facing session export will use), with the message-id
|
* engine (core/session-markdown.ts) keeps the full session format for
|
||||||
* anchors preserved (the extractor reports them back per
|
* the future user-facing export; the extractor no longer uses it.
|
||||||
* proposition/edge — they become provenance). TREE-BLIND by design:
|
*/
|
||||||
* no plan tree, no mark_done.
|
|
||||||
*
|
/** The whole-record view (used by callers that render a full session;
|
||||||
* Presentation-only: coverage fingerprints hash core/coverage.ts's line
|
* the pipeline itself batches). TEACHER PROSE ONLY (owner, 2026-09-28).
|
||||||
* format, which is SEPARATE and unchanged — re-formatting the view never
|
|
||||||
* re-opens covered history (the deliberate 2026-09-27 split).
|
|
||||||
*/
|
*/
|
||||||
export function transcriptView(record: ChatRecord): string {
|
export function transcriptView(record: ChatRecord): string {
|
||||||
// anchors OFF (submit-schema slim, 2026-09-28): propositions no longer
|
return proseView(record.messages, record.title);
|
||||||
// carry message ids, so the [id] anchors are noise for the extractor.
|
}
|
||||||
return sessionMarkdown(record.messages, { title: record.title, anchors: false });
|
|
||||||
|
/** The extraction view under the teacher-prose ruling (owner, 2026-09-28):
|
||||||
|
* ONLY the teacher agent's text blocks — no learner messages, no
|
||||||
|
* thinking, no tool calls or results, no quiz blocks. Plain paragraphs
|
||||||
|
* under the session title; the shared markdown engine stays reserved for
|
||||||
|
* the future user-facing export. */
|
||||||
|
function proseView(messages: ReadonlyArray<LhMessage>, title?: string): string {
|
||||||
|
const prose = teacherProseBlocks(messages);
|
||||||
|
const head = title && title.length > 0 ? `# ${title}\n\n` : "";
|
||||||
|
return head + prose.join("\n\n");
|
||||||
}
|
}
|
||||||
|
|
||||||
// ─── the extractor ──────────────────────────────────────────────────────────
|
// ─── the extractor ──────────────────────────────────────────────────────────
|
||||||
@@ -866,15 +874,12 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
|
|||||||
`${covered.uncoveredCount} messages, ${batches.length} batch(es) of ≤${chunkTokens} tok`,
|
`${covered.uncoveredCount} messages, ${batches.length} batch(es) of ≤${chunkTokens} tok`,
|
||||||
);
|
);
|
||||||
for (const batch of batches) {
|
for (const batch of batches) {
|
||||||
// The batch's source view: the ONE shared markdown engine
|
// The batch's source view: TEACHER PROSE ONLY (owner,
|
||||||
// (owner 2026-09-27) over the batch's messages. Anchors OFF
|
// 2026-09-28) — plain paragraphs, same selection as the
|
||||||
// (2026-09-28): propositions report no message ids, so the
|
// coverage fingerprints (messageViewLines parity).
|
||||||
// [id] anchors are noise — same as transcriptView. The
|
const view = proseView(
|
||||||
// fingerprint's line format is separate and unchanged (no
|
|
||||||
// mass re-extraction).
|
|
||||||
const view = sessionMarkdown(
|
|
||||||
batch.map((slot) => slot.message),
|
batch.map((slot) => slot.message),
|
||||||
{ title: record.title, anchors: false },
|
record.title,
|
||||||
);
|
);
|
||||||
setPhase(chatId, "extracting", "chat");
|
setPhase(chatId, "extracting", "chat");
|
||||||
const findings = await extract(chatId, view);
|
const findings = await extract(chatId, view);
|
||||||
|
|||||||
+57
-149
@@ -1,45 +1,31 @@
|
|||||||
/**
|
/**
|
||||||
* Extraction coverage (vision-extraction-coverage.md, settled 2026-09-15) —
|
* Extraction coverage — the pure core. THE TEACHER'S PROSE IS THE UNIT
|
||||||
* the pure core. THE MESSAGE IS THE UNIT OF COVERAGE (Wave D, owner ruling
|
* (owner ruling, 2026-09-28): the extractor sees ONLY the teacher agent's
|
||||||
* 2026-09-26: the native message, not the old projected group): a per-chat
|
* text blocks — no learner messages, no thinking, no tool calls or
|
||||||
* append-only ledger records which messages have been extracted, by CONTENT
|
* results, no quiz blocks. Everything else is INVISIBLE to coverage: it
|
||||||
* FINGERPRINT over exactly what the extraction view sees; "uncovered" is
|
* neither opens nor joins a run, so answering a quiz or editing a learner
|
||||||
* DERIVED at read time (fingerprint the current transcript, diff against the
|
* message triggers nothing. "Covered" is by construction exactly "what
|
||||||
* ledger) — never dual-tracked. Runs are token-budgeted batches of the oldest
|
* the extractor saw": the fingerprint (sha256 of a message's view lines)
|
||||||
* uncovered messages, and THE ONE BATCH RULE (owner): a tool-call message and
|
* and the extractor's prompt share ONE selection — the teacher prose.
|
||||||
* its tool-result message(s) are INSEPARABLE — they always appear in the same
|
|
||||||
* chunk (a covered partner rides along as context; checkpoints are
|
|
||||||
* idempotent, so re-marking it is a no-op).
|
|
||||||
*
|
*
|
||||||
* The fingerprint and the LLM prompt share ONE line builder —
|
* A per-chat append-only ledger records which messages have been
|
||||||
* messageViewLines — so "covered" and "what the model saw" cannot drift
|
* extracted by CONTENT FINGERPRINT; "uncovered" is DERIVED at read time
|
||||||
* apart. The line logic is tree-blind, identical to the extractor's view:
|
* (fingerprint the current transcript, diff against the ledger) — never
|
||||||
* user/teacher text verbatim, quiz outcomes as one-line summaries, other
|
* dual-tracked. Runs are token-budgeted batches of the oldest uncovered
|
||||||
* clean tool results as one generic line each, markers invisible (a
|
* teacher-prose messages: greedy consecutive fill, a message is never
|
||||||
* plan-block marker does not re-open a covered message).
|
* split, a single over-budget message rides alone. (The old
|
||||||
|
* inseparability bond — call + results in one chunk — died with the
|
||||||
|
* ruling: tool results are invisible now, so nothing needs keeping
|
||||||
|
* together.)
|
||||||
*/
|
*/
|
||||||
|
|
||||||
import { createHash } from "node:crypto";
|
import { createHash } from "node:crypto";
|
||||||
|
|
||||||
import type { CoverageEntry } from "./contracts/graph.ts";
|
import type { CoverageEntry } from "./contracts/graph.ts";
|
||||||
import type { LhMessage } from "./native-messages.ts";
|
import type { LhMessage } from "./native-messages.ts";
|
||||||
import type { ToolResultMessage } from "@earendil-works/pi-ai";
|
|
||||||
|
|
||||||
export type { CoverageEntry };
|
export type { CoverageEntry };
|
||||||
|
|
||||||
/** The quiz outcome line — both the fingerprint and the LLM view see the
|
|
||||||
* quiz answer, so an answered quiz re-opens (the answer is its own native
|
|
||||||
* message now; its fingerprint is new the moment it lands). */
|
|
||||||
function quizLine(result: ToolResultMessage): string {
|
|
||||||
const out = result.details as
|
|
||||||
| Partial<{ questionText: string; correct: boolean; dontKnow: boolean; note: string }>
|
|
||||||
| undefined;
|
|
||||||
const question = typeof out?.questionText === "string" ? out.questionText : "(no question)";
|
|
||||||
const outcome = out?.dontKnow === true ? "did not know" : out?.correct === true ? "answered correctly" : "answered incorrectly";
|
|
||||||
const note = typeof out?.note === "string" && out.note ? ` (note: ${out.note})` : "";
|
|
||||||
return `QUIZ [${result.toolCallId}] ${question} — the learner ${outcome}${note}`;
|
|
||||||
}
|
|
||||||
|
|
||||||
/** The user-facing text of a native message (string or text blocks). */
|
/** The user-facing text of a native message (string or text blocks). */
|
||||||
function textOf(content: unknown): string[] {
|
function textOf(content: unknown): string[] {
|
||||||
if (typeof content === "string") return content.trim() ? [content] : [];
|
if (typeof content === "string") return content.trim() ? [content] : [];
|
||||||
@@ -52,47 +38,46 @@ function textOf(content: unknown): string[] {
|
|||||||
|
|
||||||
/**
|
/**
|
||||||
* The extraction-relevant lines of ONE native message — the shared seam
|
* The extraction-relevant lines of ONE native message — the shared seam
|
||||||
* between transcriptView (the LLM's prompt) and messageFingerprint
|
* between the extractor's prompt and messageFingerprint (coverage).
|
||||||
* (coverage). Same logic, same order, so a covered message is by
|
* TEACHER PROSE ONLY (owner, 2026-09-28): an assistant message yields one
|
||||||
* construction exactly what the extractor saw. `mid` is the provenance tag
|
* line per text block; everything else (user messages, tool results,
|
||||||
* (the message id; the transcript view passes an index fallback for legacy
|
* markers) yields NOTHING and is invisible to coverage. `mid` is the
|
||||||
* id-less messages, coverage passes the id — persisted messages always have
|
* message id — part of the hash input, so the same prose in two messages
|
||||||
* ids).
|
* is two slots (per-message coverage).
|
||||||
*/
|
*/
|
||||||
export function messageViewLines(message: LhMessage, mid: string): string[] {
|
export function messageViewLines(message: LhMessage, mid: string): string[] {
|
||||||
const lines: string[] = [];
|
const lines: string[] = [];
|
||||||
const role = (message as { role?: string }).role;
|
if ((message as { role?: string }).role !== "assistant") return lines;
|
||||||
if (role === "user") {
|
|
||||||
for (const text of textOf((message as { content: unknown }).content)) {
|
|
||||||
lines.push(`LEARNER [${mid}] ${text.trim()}`);
|
|
||||||
}
|
|
||||||
} else if (role === "assistant") {
|
|
||||||
for (const text of textOf((message as { content: unknown }).content)) {
|
for (const text of textOf((message as { content: unknown }).content)) {
|
||||||
lines.push(`TEACHER [${mid}] ${text.trim()}`);
|
lines.push(`TEACHER [${mid}] ${text.trim()}`);
|
||||||
}
|
}
|
||||||
} else if (role === "toolResult") {
|
|
||||||
const result = message as ToolResultMessage;
|
|
||||||
if (result.isError) return lines; // a failed call is retry context, not learning evidence
|
|
||||||
lines.push(
|
|
||||||
result.toolName === "quiz" ? quizLine(result) : `(${result.toolName} tool result delivered)`,
|
|
||||||
);
|
|
||||||
}
|
|
||||||
// custom markers are invisible to the extraction view.
|
|
||||||
return lines;
|
return lines;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The teacher's prose blocks of a transcript, in order — the extractor's
|
||||||
|
* prompt material. SAME selection as messageViewLines (the parity
|
||||||
|
* doctrine survives the ruling in this one function pair), rendered
|
||||||
|
* without the provenance prefix: plain paragraphs.
|
||||||
|
*/
|
||||||
|
export function teacherProseBlocks(messages: ReadonlyArray<LhMessage>): string[] {
|
||||||
|
const out: string[] = [];
|
||||||
|
for (const message of messages) {
|
||||||
|
if ((message as { role?: string }).role !== "assistant") continue;
|
||||||
|
for (const text of textOf((message as { content: unknown }).content)) out.push(text.trim());
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
/** Tokens of a message's view — chars/4, ceil, the app-wide estimate. */
|
/** Tokens of a message's view — chars/4, ceil, the app-wide estimate. */
|
||||||
export function viewTokens(lines: string[]): number {
|
export function viewTokens(lines: string[]): number {
|
||||||
return Math.ceil(lines.join("\n").length / 4);
|
return Math.ceil(lines.join("\n").length / 4);
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* A message's coverage fingerprint: sha256 of role + its view lines (with
|
* A message's coverage fingerprint: sha256 of role + its view lines (the
|
||||||
* the message's real id as the provenance tag). Changes exactly when the
|
* teacher prose, with the message id as the tag). Changes exactly when
|
||||||
* extraction view of the message changes. (Wave D: the unit is the native
|
* the teacher prose of the message changes — nothing else can re-open it.
|
||||||
* message — an answered quiz is the answer's OWN toolResult message, so
|
|
||||||
* answering opens exactly that new message; the asking message is pulled
|
|
||||||
* into the batch by the inseparability rule, not re-opened.)
|
|
||||||
*/
|
*/
|
||||||
export function messageFingerprint(message: LhMessage): string {
|
export function messageFingerprint(message: LhMessage): string {
|
||||||
const h = createHash("sha256");
|
const h = createHash("sha256");
|
||||||
@@ -122,11 +107,11 @@ export function coveredSet(entries: ReadonlyArray<CoverageEntry>, chatId: string
|
|||||||
|
|
||||||
/** The derived state: which slots are uncovered, plus the token totals.
|
/** The derived state: which slots are uncovered, plus the token totals.
|
||||||
* "Uncovered" needs no ledger of its own — this diff IS the coverage view.
|
* "Uncovered" needs no ledger of its own — this diff IS the coverage view.
|
||||||
* `uncovered` is INDEXES into `slots` (the batcher needs positions for the
|
* `uncovered` is INDEXES into `slots` (the batcher batches by position).
|
||||||
* inseparability rule). Messages with NO view lines (custom markers, an
|
* Messages with NO view lines — every learner message, every tool result,
|
||||||
* error result) are invisible to coverage entirely: the extractor can
|
* markers, prose-less assistant turns — are invisible to coverage
|
||||||
* never see them, so they are neither covered nor uncovered — a marker
|
* entirely: the extractor can never see them, so they are neither covered
|
||||||
* attach never opens (or joins) an extraction run. */
|
* nor uncovered. */
|
||||||
export function coverageOf(
|
export function coverageOf(
|
||||||
messages: ReadonlyArray<LhMessage>,
|
messages: ReadonlyArray<LhMessage>,
|
||||||
entries: ReadonlyArray<CoverageEntry>,
|
entries: ReadonlyArray<CoverageEntry>,
|
||||||
@@ -166,86 +151,15 @@ export function coverageOf(
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
// ─── the inseparability rule (owner, Wave D fork 3) ─────────────────────────
|
|
||||||
//
|
|
||||||
// A tool-call message and its tool-result message(s) always appear in the
|
|
||||||
// same chunk. Bonding is adjacency-honest: results follow their call's
|
|
||||||
// assistant message contiguously in the native record (the agent loop
|
|
||||||
// appends the results batch; the question gate guarantees nothing can land
|
|
||||||
// between a quiz call and its learner answer). A result whose call's
|
|
||||||
// assistant message has NO other bond partners still pulls that message in
|
|
||||||
// (the extractor needs the question with its answer).
|
|
||||||
|
|
||||||
/** The call ids an assistant message carries. */
|
|
||||||
function callIdsOf(message: LhMessage): Set<string> {
|
|
||||||
const out = new Set<string>();
|
|
||||||
const content = (message as { content?: unknown }).content;
|
|
||||||
if (!Array.isArray(content)) return out;
|
|
||||||
for (const block of content as Array<{ type?: string; id?: string }>) {
|
|
||||||
if (block.type === "toolCall" && typeof block.id === "string") out.add(block.id);
|
|
||||||
}
|
|
||||||
return out;
|
|
||||||
}
|
|
||||||
|
|
||||||
/** The contiguous bond-range containing slot i: an assistant-with-calls
|
|
||||||
* message plus its immediately-following results; a toolResult message
|
|
||||||
* plus the assistant message that made its call (and that call's sibling
|
|
||||||
* results). Anything else bonds to itself alone. */
|
|
||||||
function bondRange(messages: ReadonlyArray<LhMessage>, i: number): [number, number] {
|
|
||||||
const roleOf = (m: LhMessage | undefined) => (m ? (m as { role?: string }).role : undefined);
|
|
||||||
const m = messages[i];
|
|
||||||
if (!m) return [i, i];
|
|
||||||
if (roleOf(m) === "assistant") {
|
|
||||||
const calls = callIdsOf(m);
|
|
||||||
if (calls.size === 0) return [i, i];
|
|
||||||
let end = i;
|
|
||||||
while (
|
|
||||||
roleOf(messages[end + 1]) === "toolResult" &&
|
|
||||||
calls.has((messages[end + 1] as ToolResultMessage).toolCallId)
|
|
||||||
) {
|
|
||||||
end++;
|
|
||||||
}
|
|
||||||
return [i, end];
|
|
||||||
}
|
|
||||||
if (roleOf(m) === "toolResult") {
|
|
||||||
let start = i;
|
|
||||||
while (roleOf(messages[start - 1]) === "toolResult") start--;
|
|
||||||
const owner = messages[start - 1];
|
|
||||||
if (
|
|
||||||
owner &&
|
|
||||||
roleOf(owner) === "assistant" &&
|
|
||||||
callIdsOf(owner).has((m as ToolResultMessage).toolCallId)
|
|
||||||
) {
|
|
||||||
const [s, e] = bondRange(messages, start - 1);
|
|
||||||
return [s, Math.max(e, i)];
|
|
||||||
}
|
|
||||||
return [start, i]; // orphaned results (no asking message): keep the contiguous result run together
|
|
||||||
}
|
|
||||||
return [i, i];
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Batch the uncovered messages into token-budgeted RUNS: consecutive slots,
|
* Batch the uncovered teacher-prose messages into token-budgeted RUNS:
|
||||||
* oldest first, greedy fill — a message is never split, and a bond-range
|
* consecutive slots, oldest first, greedy fill — a message is never split.
|
||||||
* (call + results) never splits either (owner fork 3): a selected uncovered
|
* A single message larger than the budget becomes its own (over-budget)
|
||||||
* slot EXPANDS to its whole bond-range, covered partners riding as context
|
* run rather than being dropped. Runs are ephemeral — the durable record
|
||||||
* (checkpoint idempotency makes re-marking them a no-op). A single
|
* is per-message coverage, checkpointed after each run merges.
|
||||||
* bond-range larger than the budget becomes its own (over-budget) run
|
|
||||||
* rather than being dropped. Runs are ephemeral — the durable record is
|
|
||||||
* per-message coverage, checkpointed after each run merges.
|
|
||||||
*/
|
*/
|
||||||
export function batchRuns(slots: ReadonlyArray<CoverageSlot>, uncovered: ReadonlyArray<number>, budget: number): CoverageSlot[][] {
|
export function batchRuns(slots: ReadonlyArray<CoverageSlot>, uncovered: ReadonlyArray<number>, budget: number): CoverageSlot[][] {
|
||||||
if (budget <= 0 || uncovered.length === 0) return [];
|
if (budget <= 0 || uncovered.length === 0) return [];
|
||||||
const messages = slots.map((s) => s.message);
|
|
||||||
// Expand: every uncovered index marks its whole bond-range as needed.
|
|
||||||
const needed = new Set<number>();
|
|
||||||
for (const i of uncovered) {
|
|
||||||
const [start, end] = bondRange(messages, i);
|
|
||||||
for (let j = start; j <= end; j++) needed.add(j);
|
|
||||||
}
|
|
||||||
// Group the needed slots in order, greedy by budget, never splitting a
|
|
||||||
// bond-range: a range that would overflow starts a new run (and an
|
|
||||||
// over-budget range rides alone).
|
|
||||||
const runs: CoverageSlot[][] = [];
|
const runs: CoverageSlot[][] = [];
|
||||||
let current: CoverageSlot[] = [];
|
let current: CoverageSlot[] = [];
|
||||||
let currentTokens = 0;
|
let currentTokens = 0;
|
||||||
@@ -254,17 +168,11 @@ export function batchRuns(slots: ReadonlyArray<CoverageSlot>, uncovered: Readonl
|
|||||||
current = [];
|
current = [];
|
||||||
currentTokens = 0;
|
currentTokens = 0;
|
||||||
};
|
};
|
||||||
for (let i = 0; i < slots.length; i++) {
|
for (const i of uncovered) {
|
||||||
if (!needed.has(i)) continue;
|
const slot = slots[i]!;
|
||||||
const [start, end] = bondRange(messages, i);
|
if (current.length > 0 && currentTokens + slot.tokens > budget) flush();
|
||||||
const rangeTokens = slots.slice(start, end + 1).reduce((t, s) => t + s.tokens, 0);
|
current.push(slot);
|
||||||
if (current.length > 0 && currentTokens + rangeTokens > budget) flush();
|
currentTokens += slot.tokens;
|
||||||
// The whole range enters together — skip past its end in the outer loop.
|
|
||||||
for (let j = start; j <= end; j++) {
|
|
||||||
current.push(slots[j]!);
|
|
||||||
currentTokens += slots[j]!.tokens;
|
|
||||||
}
|
|
||||||
i = end;
|
|
||||||
}
|
}
|
||||||
flush();
|
flush();
|
||||||
return runs;
|
return runs;
|
||||||
|
|||||||
+60
-121
@@ -12,17 +12,16 @@ import {
|
|||||||
coverageSlots,
|
coverageSlots,
|
||||||
messageFingerprint,
|
messageFingerprint,
|
||||||
messageViewLines,
|
messageViewLines,
|
||||||
|
teacherProseBlocks,
|
||||||
} from "../src/core/coverage.ts";
|
} from "../src/core/coverage.ts";
|
||||||
import type { CoverageEntry } from "../src/core/contracts/graph.ts";
|
import type { CoverageEntry } from "../src/core/contracts/graph.ts";
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Extraction coverage — the pure core (vision-extraction-coverage.md,
|
* Extraction coverage — the pure core under the TEACHER-PROSE ruling
|
||||||
* settled 2026-09-15; Wave D native rebase 2026-09-26). THE NATIVE MESSAGE
|
* (owner, 2026-09-28): the extractor sees only the teacher's text
|
||||||
* IS THE UNIT OF COVERAGE (owner ruling); the fingerprint and the LLM prompt
|
* blocks; fingerprints, slots, tokens, and batching all derive from that
|
||||||
* share ONE line builder (messageViewLines), so "covered" is by construction
|
* one selection. Everything else is invisible — it neither opens nor
|
||||||
* exactly "what the extractor saw". Batches obey the ONE INSEPARABILITY RULE
|
* joins a run.
|
||||||
* (owner fork 3): a tool-call message and its tool-result messages always
|
|
||||||
* appear in the same chunk.
|
|
||||||
*/
|
*/
|
||||||
|
|
||||||
function user(id: string, text: string): LhMessage {
|
function user(id: string, text: string): LhMessage {
|
||||||
@@ -61,13 +60,14 @@ function slot(messages: ReadonlyArray<LhMessage>, id: string) {
|
|||||||
|
|
||||||
describe("messageFingerprint — what changes it, what never does", () => {
|
describe("messageFingerprint — what changes it, what never does", () => {
|
||||||
it("a text edit changes the fingerprint; role participates", () => {
|
it("a text edit changes the fingerprint; role participates", () => {
|
||||||
const a = messageFingerprint(user("m1", "hello world"));
|
const a = messageFingerprint(teacher("m1", "hello world"));
|
||||||
const b = messageFingerprint(user("m1", "hello worlds"));
|
const b = messageFingerprint(teacher("m1", "hello worlds"));
|
||||||
const same = messageFingerprint(user("m1", "hello world"));
|
const same = messageFingerprint(teacher("m1", "hello world"));
|
||||||
assert.notEqual(a, b);
|
assert.notEqual(a, b);
|
||||||
assert.equal(a, same);
|
assert.equal(a, same);
|
||||||
// Role is hashed too — a learner and a teacher line never collide.
|
// Role is hashed too — the same words from a LEARNER have no view
|
||||||
assert.notEqual(a, messageFingerprint(teacher("m1", "hello world")));
|
// lines at all (teacher prose only), so the fingerprints differ.
|
||||||
|
assert.notEqual(a, messageFingerprint(user("m1", "hello world")));
|
||||||
});
|
});
|
||||||
|
|
||||||
it("the message id is the provenance tag — part of exactly what the extractor saw", () => {
|
it("the message id is the provenance tag — part of exactly what the extractor saw", () => {
|
||||||
@@ -75,9 +75,9 @@ describe("messageFingerprint — what changes it, what never does", () => {
|
|||||||
// different id is a DIFFERENT fingerprint: coverage matches the
|
// different id is a DIFFERENT fingerprint: coverage matches the
|
||||||
// extraction view byte-for-byte. (Ids are stable in practice — the
|
// extraction view byte-for-byte. (Ids are stable in practice — the
|
||||||
// write-ahead persist owns them.)
|
// write-ahead persist owns them.)
|
||||||
assert.notEqual(messageFingerprint(user("m1", "same")), messageFingerprint(user("m2", "same")));
|
assert.notEqual(messageFingerprint(teacher("m1", "same")), messageFingerprint(teacher("m2", "same")));
|
||||||
// The same id + content re-fingerprints equal.
|
// The same id + content re-fingerprints equal.
|
||||||
assert.equal(messageFingerprint(user("m1", "same")), messageFingerprint(user("m1", "same")));
|
assert.equal(messageFingerprint(teacher("m1", "same")), messageFingerprint(teacher("m1", "same")));
|
||||||
});
|
});
|
||||||
|
|
||||||
it("marker messages are invisible — a plan-block attach never re-opens anything", () => {
|
it("marker messages are invisible — a plan-block attach never re-opens anything", () => {
|
||||||
@@ -102,41 +102,42 @@ describe("messageFingerprint — what changes it, what never does", () => {
|
|||||||
);
|
);
|
||||||
});
|
});
|
||||||
|
|
||||||
it("an answered quiz is its OWN message — the asking message never re-opens", () => {
|
it("quiz Q&A is INVISIBLE under the teacher-prose ruling (owner, 2026-09-28)", () => {
|
||||||
// Wave D: the answer arrives as the call's toolResult message; the
|
// Neither the asking message nor the graded answer produces view
|
||||||
// asking assistant message's fingerprint is STABLE (the old wire
|
// lines — a quiz exchange can never open or join an extraction run.
|
||||||
// merged the answer into the asking group and re-opened it).
|
|
||||||
const asking = assistantWithCalls("a1", [{ id: "q1", name: "quiz", args: { question: "Q?" } }]);
|
const asking = assistantWithCalls("a1", [{ id: "q1", name: "quiz", args: { question: "Q?" } }]);
|
||||||
const answer = result("r1", "q1", "quiz", { questionText: "Q?", correct: true });
|
const answer = result("r1", "q1", "quiz", { questionText: "Q?", correct: true });
|
||||||
const fpAsking = messageFingerprint(asking);
|
assert.deepEqual(messageViewLines(asking, "a1"), []);
|
||||||
assert.equal(fpAsking, messageFingerprint(assistantWithCalls("a1", [{ id: "q1", name: "quiz", args: { question: "Q?" } }])));
|
assert.deepEqual(messageViewLines(answer, "r1"), []);
|
||||||
// The answer's own line carries the graded outcome:
|
assert.equal(slot([asking, answer], "a1").tokens, 0);
|
||||||
const lines = messageViewLines(answer, "r1");
|
assert.equal(slot([asking, answer], "r1").tokens, 0);
|
||||||
assert.deepEqual(lines, ["QUIZ [q1] Q? — the learner answered correctly"]);
|
|
||||||
assert.ok(slot([asking, answer], "r1").tokens > 0, "the quiz outcome counts toward coverage tokens");
|
|
||||||
});
|
});
|
||||||
|
|
||||||
it("quiz outcome wording — dontKnow and incorrect each get their line", () => {
|
it("learner messages and ALL tool results are invisible — clean or failed", () => {
|
||||||
const dk = result("r1", "q1", "quiz", { questionText: "Q?", dontKnow: true });
|
assert.deepEqual(messageViewLines(user("u1", "teach me"), "u1"), []);
|
||||||
assert.deepEqual(messageViewLines(dk, "r1"), ["QUIZ [q1] Q? — the learner did not know"]);
|
|
||||||
const wrong = result("r2", "q2", "quiz", { questionText: "W?", correct: false, note: "mixed up signs" });
|
|
||||||
assert.deepEqual(messageViewLines(wrong, "r2"), ["QUIZ [q2] W? — the learner answered incorrectly (note: mixed up signs)"]);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("other clean tool results get one generic line; errors are invisible", () => {
|
|
||||||
const demo = result("r1", "j1", "demo", { title: "T" });
|
const demo = result("r1", "j1", "demo", { title: "T" });
|
||||||
assert.deepEqual(messageViewLines(demo, "r1"), ["(demo tool result delivered)"]);
|
assert.deepEqual(messageViewLines(demo, "r1"), []);
|
||||||
const failed = {
|
const failed = {
|
||||||
...newAnswerResult("j2", "visual", { oops: true }),
|
...newAnswerResult("j2", "visual", { oops: true }),
|
||||||
isError: true,
|
isError: true,
|
||||||
} as LhMessage;
|
} as LhMessage;
|
||||||
assert.deepEqual(messageViewLines(failed, "r9"), [], "a failed call is retry context, not learning evidence");
|
assert.deepEqual(messageViewLines(failed, "r9"), []);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("teacherProseBlocks — the prompt material: teacher text blocks only, plain, in order", () => {
|
||||||
|
const blocks = teacherProseBlocks([
|
||||||
|
user("u1", "learner asks"),
|
||||||
|
teacher("t1", " first insight "),
|
||||||
|
assistantWithCalls("a1", [{ id: "q1", name: "quiz" }]),
|
||||||
|
teacher("t2", "second insight"),
|
||||||
|
]);
|
||||||
|
assert.deepEqual(blocks, ["first insight", "second insight"]);
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
describe("coverageOf — uncovered is DERIVED, never dual-tracked", () => {
|
describe("coverageOf — uncovered is DERIVED, never dual-tracked", () => {
|
||||||
it("diffs fingerprints: covered stays, uncovered counts, tokens split", () => {
|
it("diffs fingerprints: covered stays, uncovered counts, tokens split", () => {
|
||||||
const messages = [user("m1", "aaaa"), user("m2", "bbbb"), user("m3", "cccc")];
|
const messages = [teacher("m1", "aaaa"), teacher("m2", "bbbb"), teacher("m3", "cccc")];
|
||||||
const fp1 = slot(messages, "m1").fp;
|
const fp1 = slot(messages, "m1").fp;
|
||||||
const fp2 = slot(messages, "m2").fp;
|
const fp2 = slot(messages, "m2").fp;
|
||||||
const entries: CoverageEntry[] = [
|
const entries: CoverageEntry[] = [
|
||||||
@@ -156,7 +157,7 @@ describe("coverageOf — uncovered is DERIVED, never dual-tracked", () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
it("a mutated covered message re-opens itself, and only itself", () => {
|
it("a mutated covered message re-opens itself, and only itself", () => {
|
||||||
const messages = [user("m1", "aaaa"), user("m2", "bbbb")];
|
const messages = [teacher("m1", "aaaa"), teacher("m2", "bbbb")];
|
||||||
const entries: CoverageEntry[] = coverageSlots(messages).map((s) => ({
|
const entries: CoverageEntry[] = coverageSlots(messages).map((s) => ({
|
||||||
chatId: "c1",
|
chatId: "c1",
|
||||||
fp: s.fp,
|
fp: s.fp,
|
||||||
@@ -164,7 +165,7 @@ describe("coverageOf — uncovered is DERIVED, never dual-tracked", () => {
|
|||||||
foldedAt: "2026-09-15T00:00:00.000Z",
|
foldedAt: "2026-09-15T00:00:00.000Z",
|
||||||
}));
|
}));
|
||||||
assert.equal(coverageOf(messages, entries, "c1").uncoveredCount, 0);
|
assert.equal(coverageOf(messages, entries, "c1").uncoveredCount, 0);
|
||||||
const mutated = [user("m1", "aaaa!"), messages[1]!];
|
const mutated = [teacher("m1", "aaaa!"), messages[1]!];
|
||||||
const cov = coverageOf(mutated, entries, "c1");
|
const cov = coverageOf(mutated, entries, "c1");
|
||||||
assert.deepEqual(cov.uncovered, [0]);
|
assert.deepEqual(cov.uncovered, [0]);
|
||||||
assert.equal(cov.slots[0]!.message.id, "m1");
|
assert.equal(cov.slots[0]!.message.id, "m1");
|
||||||
@@ -176,10 +177,10 @@ describe("batchRuns — token-budgeted, consecutive, never splits a message", ()
|
|||||||
// Each message's view line is ~14 tok (chars/4) → budget 45 fits
|
// Each message's view line is ~14 tok (chars/4) → budget 45 fits
|
||||||
// three, the fourth overflows: [m1,m2,m3] + [m4].
|
// three, the fourth overflows: [m1,m2,m3] + [m4].
|
||||||
const slots = coverageSlots([
|
const slots = coverageSlots([
|
||||||
user("m1", "a".repeat(40)),
|
teacher("m1", "a".repeat(40)),
|
||||||
user("m2", "b".repeat(40)),
|
teacher("m2", "b".repeat(40)),
|
||||||
user("m3", "c".repeat(40)),
|
teacher("m3", "c".repeat(40)),
|
||||||
user("m4", "d".repeat(40)),
|
teacher("m4", "d".repeat(40)),
|
||||||
]);
|
]);
|
||||||
const runs = batchRuns(slots, [0, 1, 2, 3], 45);
|
const runs = batchRuns(slots, [0, 1, 2, 3], 45);
|
||||||
assert.deepEqual(
|
assert.deepEqual(
|
||||||
@@ -189,7 +190,7 @@ describe("batchRuns — token-budgeted, consecutive, never splits a message", ()
|
|||||||
});
|
});
|
||||||
|
|
||||||
it("a single message larger than the budget becomes its own over-budget run", () => {
|
it("a single message larger than the budget becomes its own over-budget run", () => {
|
||||||
const slots = coverageSlots([user("m1", "x".repeat(400)), user("m2", "y".repeat(20))]);
|
const slots = coverageSlots([teacher("m1", "x".repeat(400)), teacher("m2", "y".repeat(20))]);
|
||||||
assert.deepEqual(
|
assert.deepEqual(
|
||||||
batchRuns(slots, [0, 1], 10).map((r) => r.map((s) => s.message.id)),
|
batchRuns(slots, [0, 1], 10).map((r) => r.map((s) => s.message.id)),
|
||||||
[["m1"], ["m2"]],
|
[["m1"], ["m2"]],
|
||||||
@@ -197,104 +198,42 @@ describe("batchRuns — token-budgeted, consecutive, never splits a message", ()
|
|||||||
});
|
});
|
||||||
|
|
||||||
it("an empty or non-positive budget never drops messages into a void", () => {
|
it("an empty or non-positive budget never drops messages into a void", () => {
|
||||||
assert.deepEqual(batchRuns(coverageSlots([user("m1", "hi")]), [0], 0), []);
|
assert.deepEqual(batchRuns(coverageSlots([teacher("m1", "hi")]), [0], 0), []);
|
||||||
assert.deepEqual(batchRuns([], [], 4096), []);
|
assert.deepEqual(batchRuns([], [], 4096), []);
|
||||||
});
|
});
|
||||||
|
|
||||||
it("THE BOND RULE (owner fork 3): a call and its results never split, even under a tiny budget", () => {
|
it("learner messages and tool results never join a batch — teacher prose only (owner, 2026-09-28)", () => {
|
||||||
// The call's assistant message is COVERED; only the (new) answer is
|
// The inseparability bond died with the ruling: tool results are
|
||||||
// uncovered — the asking message rides into the chunk anyway, and
|
// invisible, so nothing needs keeping together. Batches are pure
|
||||||
// the two NEVER split despite a budget of 1.
|
// greedy runs over the TEACHER slots alone.
|
||||||
const messages = [
|
const messages = [
|
||||||
user("u1", "topic"),
|
user("u1", "learner asks"),
|
||||||
assistantWithCalls("a1", [{ id: "q1", name: "quiz", args: { question: "Q?" } }]),
|
|
||||||
result("r1", "q1", "quiz", { questionText: "Q?", correct: true }),
|
|
||||||
];
|
|
||||||
const slots = coverageSlots(messages);
|
|
||||||
const cov = coverageOf(
|
|
||||||
messages,
|
|
||||||
coverageSlots([messages[0]!, messages[1]!]).map((s) => ({
|
|
||||||
chatId: "c1",
|
|
||||||
fp: s.fp,
|
|
||||||
tokens: s.tokens,
|
|
||||||
foldedAt: "2026-09-26T00:00:00.000Z",
|
|
||||||
})),
|
|
||||||
"c1",
|
|
||||||
);
|
|
||||||
assert.deepEqual(cov.uncovered, [2], "only the answer is uncovered");
|
|
||||||
const runs = batchRuns(slots, cov.uncovered, 1);
|
|
||||||
assert.deepEqual(
|
|
||||||
runs.map((r) => r.map((s) => s.message.id)),
|
|
||||||
[["a1", "r1"]],
|
|
||||||
"the covered asking message rides along as context — one chunk, over budget, never split",
|
|
||||||
);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("the reverse bond: an uncovered CALL pulls its covered results into the chunk", () => {
|
|
||||||
const messages = [
|
|
||||||
{
|
|
||||||
id: "a1",
|
|
||||||
role: "assistant",
|
|
||||||
content: [
|
|
||||||
{ type: "text", text: "Let me research that and build a demo." },
|
|
||||||
{ type: "toolCall", id: "j1", name: "research", arguments: { topic: "x" } },
|
|
||||||
{ type: "toolCall", id: "j2", name: "demo", arguments: { title: "T" } },
|
|
||||||
],
|
|
||||||
} as unknown as LhMessage,
|
|
||||||
result("r1", "j1", "research", { summary: "s" }),
|
|
||||||
result("r2", "j2", "demo", { done: true }),
|
|
||||||
];
|
|
||||||
const slots = coverageSlots(messages);
|
|
||||||
const cov = coverageOf(
|
|
||||||
messages,
|
|
||||||
coverageSlots([messages[1]!, messages[2]!]).map((s) => ({
|
|
||||||
chatId: "c1",
|
|
||||||
fp: s.fp,
|
|
||||||
tokens: s.tokens,
|
|
||||||
foldedAt: "2026-09-26T00:00:00.000Z",
|
|
||||||
})),
|
|
||||||
"c1",
|
|
||||||
);
|
|
||||||
assert.deepEqual(cov.uncovered, [0], "only the calling message is uncovered");
|
|
||||||
const runs = batchRuns(slots, cov.uncovered, 10_000);
|
|
||||||
assert.deepEqual(
|
|
||||||
runs.map((r) => r.map((s) => s.message.id)),
|
|
||||||
[["a1", "r1", "r2"]],
|
|
||||||
"all three travel together (checkpoint idempotency makes re-marking the results a no-op)",
|
|
||||||
);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("adjacent messages WITHOUT a bond still batch greedily (only calls bond)", () => {
|
|
||||||
// Two plain teacher messages after a bonded pair: the pair is one
|
|
||||||
// range, the plain messages group by budget alone.
|
|
||||||
const messages = [
|
|
||||||
assistantWithCalls("a1", [{ id: "q1", name: "quiz" }]),
|
|
||||||
result("r1", "q1", "quiz", { questionText: "Q?", correct: true }),
|
|
||||||
teacher("t1", "x".repeat(40)),
|
teacher("t1", "x".repeat(40)),
|
||||||
|
result("r1", "q1", "quiz", { questionText: "Q?", correct: true }),
|
||||||
teacher("t2", "y".repeat(40)),
|
teacher("t2", "y".repeat(40)),
|
||||||
];
|
];
|
||||||
const slots = coverageSlots(messages);
|
const cov = coverageOf(messages, [], "c1");
|
||||||
assert.equal(slots[0]!.tokens, 0, "a call-only assistant message has no view lines");
|
assert.deepEqual(cov.uncovered, [1, 3], "only the teacher-prose slots are uncovered");
|
||||||
// budget 30: the pair (~12) + t1 (~14) = 26 fits; t2 overflows.
|
const runs = batchRuns(cov.slots, cov.uncovered, 15);
|
||||||
const runs = batchRuns(slots, [0, 2, 3], 30);
|
|
||||||
assert.deepEqual(
|
assert.deepEqual(
|
||||||
runs.map((r) => r.map((s) => s.message.id)),
|
runs.map((r) => r.map((sl) => sl.message.id)),
|
||||||
[["a1", "r1", "t1"], ["t2"]],
|
[["t1"], ["t2"]],
|
||||||
|
"each run is teacher prose alone — the learner message and the quiz result ride nowhere",
|
||||||
);
|
);
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
describe("batchTag — deterministic, ref-stable, mutation-sensitive", () => {
|
describe("batchTag — deterministic, ref-stable, mutation-sensitive", () => {
|
||||||
it("the same members in the same order → the same 12-hex tag; a different order is a different batch", () => {
|
it("the same members in the same order → the same 12-hex tag; a different order is a different batch", () => {
|
||||||
const slots = coverageSlots([user("m1", "aaaa"), user("m2", "bbbb")]);
|
const slots = coverageSlots([teacher("m1", "aaaa"), teacher("m2", "bbbb")]);
|
||||||
assert.equal(batchTag(slots), batchTag(coverageSlots([user("m1", "aaaa"), user("m2", "bbbb")])));
|
assert.equal(batchTag(slots), batchTag(coverageSlots([teacher("m1", "aaaa"), teacher("m2", "bbbb")])));
|
||||||
assert.notEqual(batchTag(slots), batchTag([...slots].reverse()), "batches are ordered runs");
|
assert.notEqual(batchTag(slots), batchTag([...slots].reverse()), "batches are ordered runs");
|
||||||
assert.match(batchTag(slots), /^[0-9a-f]{12}$/);
|
assert.match(batchTag(slots), /^[0-9a-f]{12}$/);
|
||||||
});
|
});
|
||||||
|
|
||||||
it("a re-opened member changes the tag — its observations land as new events", () => {
|
it("a re-opened member changes the tag — its observations land as new events", () => {
|
||||||
const slots = coverageSlots([user("m1", "aaaa"), user("m2", "bbbb")]);
|
const slots = coverageSlots([teacher("m1", "aaaa"), teacher("m2", "bbbb")]);
|
||||||
const mutated = coverageSlots([user("m1", "aaaa!"), user("m2", "bbbb")]);
|
const mutated = coverageSlots([teacher("m1", "aaaa!"), teacher("m2", "bbbb")]);
|
||||||
assert.notEqual(batchTag(slots), batchTag(mutated));
|
assert.notEqual(batchTag(slots), batchTag(mutated));
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|||||||
+39
-38
@@ -186,6 +186,7 @@ function fixtureChat() {
|
|||||||
chatStore.saveMessages(chat.id, [
|
chatStore.saveMessages(chat.id, [
|
||||||
user("u1", "teach me the chain rule"),
|
user("u1", "teach me the chain rule"),
|
||||||
...assistant("a1", [
|
...assistant("a1", [
|
||||||
|
{ type: "text", text: "The chain rule composes rates of change along a path." },
|
||||||
quizPart("probe-1", { question: "What is a derivative, roughly?" }, {
|
quizPart("probe-1", { question: "What is a derivative, roughly?" }, {
|
||||||
questionId: "probe-1", kind: "quiz", questionText: "What is a derivative, roughly?",
|
questionId: "probe-1", kind: "quiz", questionText: "What is a derivative, roughly?",
|
||||||
correct: false, dontKnow: false, answeredAt: "2026-08-22T01:00:00.000Z",
|
correct: false, dontKnow: false, answeredAt: "2026-08-22T01:00:00.000Z",
|
||||||
@@ -315,35 +316,28 @@ function silentModel(): PiEngine {
|
|||||||
|
|
||||||
// ─── transcriptView (pure) ───────────────────────────────────────────────────
|
// ─── transcriptView (pure) ───────────────────────────────────────────────────
|
||||||
|
|
||||||
describe("transcriptView (pure — the shared markdown engine, owner 2026-09-27)", () => {
|
describe("transcriptView (pure — teacher prose only, owner 2026-09-28)", () => {
|
||||||
it("renders anchorless speaker blocks + check blocks — tree-blind, plan ops folded", () => {
|
it("renders ONLY the teacher's prose — no learner, no checks, no tool notes", () => {
|
||||||
const { record } = fixtureChat();
|
const { record } = fixtureChat();
|
||||||
const view = transcriptView(record);
|
const view = transcriptView(record);
|
||||||
|
|
||||||
// The document header is the session title.
|
// The document header is the session title; the body is the
|
||||||
|
// teacher's text blocks as plain paragraphs.
|
||||||
assert.match(view, /^# Extraction fixture$/m);
|
assert.match(view, /^# Extraction fixture$/m);
|
||||||
|
assert.match(view, /The chain rule composes rates of change along a path\./);
|
||||||
|
assert.match(view, /The derivative measures how fast a function changes\./);
|
||||||
|
|
||||||
// Conversation verbatim, each block headed by its speaker only —
|
// EVERYTHING else is gone: learner text, quiz blocks, tool notes,
|
||||||
// the [id] anchors are GONE (submit-schema slim, 2026-09-28:
|
// speaker headers — the ruling strips it all.
|
||||||
// propositions no longer report message ids).
|
assert.doesNotMatch(view, /teach me the chain rule/);
|
||||||
assert.match(view, /### Learner\n\nteach me the chain rule/);
|
assert.doesNotMatch(view, /go on/);
|
||||||
assert.match(view, /### Teacher\n\nThe derivative measures how fast a function changes\./);
|
assert.doesNotMatch(view, /check me/);
|
||||||
|
assert.doesNotMatch(view, /### /);
|
||||||
// Checks render as readable blocks (question + outcome).
|
assert.doesNotMatch(view, /Quiz/);
|
||||||
assert.match(view, /### Quiz\n/);
|
assert.doesNotMatch(view, /plan_init/);
|
||||||
assert.match(view, /\*\*Q:\*\* What is a derivative, roughly\?/);
|
|
||||||
assert.match(view, /The learner answered incorrectly\./);
|
|
||||||
assert.match(view, /The learner answered correctly\./);
|
|
||||||
|
|
||||||
// TREE-BLIND: no plan tree, no tree edges, no mark_done lines — plan
|
|
||||||
// ops fold to the generic one-liner like every other non-quiz tool.
|
|
||||||
assert.doesNotMatch(view, /FINAL PLAN TREE/);
|
|
||||||
assert.doesNotMatch(view, /TREE EDGES/);
|
|
||||||
assert.match(view, /\*plan_init result delivered\*/);
|
|
||||||
assert.doesNotMatch(view, /output-available/);
|
|
||||||
});
|
});
|
||||||
|
|
||||||
it("summarizes job tool results as one line and drops empty text", () => {
|
it("job tool results and empty text vanish — only prose paragraphs remain", () => {
|
||||||
const chat = chatStore.createChat("View noise fixture");
|
const chat = chatStore.createChat("View noise fixture");
|
||||||
createdChats.push(chat.id);
|
createdChats.push(chat.id);
|
||||||
chatStore.saveMessages(chat.id, [
|
chatStore.saveMessages(chat.id, [
|
||||||
@@ -356,12 +350,14 @@ describe("transcriptView (pure — the shared markdown engine, owner 2026-09-27)
|
|||||||
output: { brief: "long brief", sources: [] },
|
output: { brief: "long brief", sources: [] },
|
||||||
},
|
},
|
||||||
{ type: "text", text: " " },
|
{ type: "text", text: " " },
|
||||||
|
{ type: "text", text: "Prose survives." },
|
||||||
]),
|
]),
|
||||||
].flat());
|
].flat());
|
||||||
const view = transcriptView(chatStore.getChat(chat.id)!);
|
const view = transcriptView(chatStore.getChat(chat.id)!);
|
||||||
assert.match(view, /\*research result delivered\*/);
|
assert.match(view, /Prose survives\./);
|
||||||
|
assert.doesNotMatch(view, /research/);
|
||||||
assert.doesNotMatch(view, /long brief/);
|
assert.doesNotMatch(view, /long brief/);
|
||||||
assert.doesNotMatch(view, /### Teacher/);
|
assert.doesNotMatch(view, /### /);
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -1154,11 +1150,13 @@ function passThroughAuditStep(): FauxResponseStep {
|
|||||||
function chunkyChat() {
|
function chunkyChat() {
|
||||||
const chat = chatStore.createChat("Chunky coverage fixture");
|
const chat = chatStore.createChat("Chunky coverage fixture");
|
||||||
createdChats.push(chat.id);
|
createdChats.push(chat.id);
|
||||||
// ~1230 tok per message (chars/4): three fill the 4096 budget, the
|
// ~1230 tok per TEACHER message (chars/4): three fill the 4096 budget,
|
||||||
// fourth would overflow — deterministic batches [m1,m2,m3] + [m4,m5].
|
// the fourth would overflow — deterministic batches [m1,m2,m3] + [m4,m5].
|
||||||
|
// Teacher prose only (owner, 2026-09-28): learner messages are
|
||||||
|
// invisible to coverage, so the fixture is all-teacher.
|
||||||
chatStore.saveMessages(
|
chatStore.saveMessages(
|
||||||
chat.id,
|
chat.id,
|
||||||
[1, 2, 3, 4, 5].map((n) => user(`u${n}`, `topic ${n}: ${"detail ".repeat(700)}${n}`)),
|
[1, 2, 3, 4, 5].flatMap((n) => assistant(`a${n}`, [{ type: "text", text: `topic ${n}: ${"detail ".repeat(700)}${n}` }])),
|
||||||
);
|
);
|
||||||
return { chatId: chat.id, record: chatStore.getChat(chat.id)! };
|
return { chatId: chat.id, record: chatStore.getChat(chat.id)! };
|
||||||
}
|
}
|
||||||
@@ -1180,10 +1178,11 @@ describe("the chunked pipeline (extraction coverage)", () => {
|
|||||||
// Two batches: the first held messages 1-3, the second 4-5 — a
|
// Two batches: the first held messages 1-3, the second 4-5 — a
|
||||||
// message is never split, batches are consecutive oldest-first.
|
// message is never split, batches are consecutive oldest-first.
|
||||||
assert.equal(views.length, 2, "one extractor call per batch");
|
assert.equal(views.length, 2, "one extractor call per batch");
|
||||||
assert.ok(views[0]!.includes("### Learner") && views[0]!.includes("topic 1:"));
|
assert.ok(views[0]!.includes("topic 1:") && views[0]!.includes("topic 3:"), "teacher prose only, first three");
|
||||||
assert.ok(!views[0]!.includes("topic 4:"), "the batch stops at the token budget");
|
assert.ok(!views[0]!.includes("topic 4:"), "the batch stops at the token budget");
|
||||||
assert.ok(views[1]!.includes("### Learner") && views[1]!.includes("topic 4:"));
|
assert.ok(views[1]!.includes("topic 4:") && views[1]!.includes("topic 5:"));
|
||||||
assert.ok(!views[1]!.includes("topic 3:"));
|
assert.ok(!views[1]!.includes("topic 3:"));
|
||||||
|
assert.doesNotMatch(views[0]!, /### /, "plain paragraphs — no speaker headers");
|
||||||
// The document header is the session title.
|
// The document header is the session title.
|
||||||
assert.ok(views[0]!.startsWith("# Chunky coverage fixture"));
|
assert.ok(views[0]!.startsWith("# Chunky coverage fixture"));
|
||||||
|
|
||||||
@@ -1263,8 +1262,8 @@ describe("the chunked pipeline (extraction coverage)", () => {
|
|||||||
});
|
});
|
||||||
await finisher.runExtraction(chatId);
|
await finisher.runExtraction(chatId);
|
||||||
assert.equal(reRunViews.length, 1);
|
assert.equal(reRunViews.length, 1);
|
||||||
assert.ok(reRunViews[0]!.includes("### Learner") && reRunViews[0]!.includes("topic 4:"));
|
assert.ok(reRunViews[0]!.includes("topic 4:") && reRunViews[0]!.includes("topic 5:"));
|
||||||
assert.ok(!reRunViews[0]!.includes("### Learner · u1"), "the covered remainder never re-enters the prompt");
|
assert.ok(!reRunViews[0]!.includes("topic 1:"), "the covered remainder never re-enters the prompt");
|
||||||
assert.equal(ledgers.readCoverage().length, 5, "the re-run closed the gap");
|
assert.equal(ledgers.readCoverage().length, 5, "the re-run closed the gap");
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -1273,7 +1272,10 @@ describe("the chunked pipeline (extraction coverage)", () => {
|
|||||||
// the ONLY thing later runs can see.
|
// the ONLY thing later runs can see.
|
||||||
const chat = chatStore.createChat("Re-open fixture");
|
const chat = chatStore.createChat("Re-open fixture");
|
||||||
createdChats.push(chat.id);
|
createdChats.push(chat.id);
|
||||||
chatStore.saveMessages(chat.id, [user("s1", "short topic one"), user("s2", "short topic two")]);
|
chatStore.saveMessages(chat.id, [
|
||||||
|
...assistant("s1", [{ type: "text", text: "short topic one" }]),
|
||||||
|
...assistant("s2", [{ type: "text", text: "short topic two" }]),
|
||||||
|
].flat());
|
||||||
const chatId = chat.id;
|
const chatId = chat.id;
|
||||||
const ledgers = memoryLedgers();
|
const ledgers = memoryLedgers();
|
||||||
const pipeline = createExtractionPipeline({
|
const pipeline = createExtractionPipeline({
|
||||||
@@ -1298,10 +1300,9 @@ describe("the chunked pipeline (extraction coverage)", () => {
|
|||||||
const afterAttach = coverageOf(messages(), ledgers.readCoverage(), chatId);
|
const afterAttach = coverageOf(messages(), ledgers.readCoverage(), chatId);
|
||||||
assert.equal(afterAttach.uncoveredCount, 0, "data parts don't change the fingerprint");
|
assert.equal(afterAttach.uncoveredCount, 0, "data parts don't change the fingerprint");
|
||||||
|
|
||||||
// A quiz answer merging into an ASKING message changes its view →
|
// A teacher-prose edit changes the message's view → exactly that
|
||||||
// exactly that message re-opens, and the next run extracts it alone.
|
// message re-opens, and the next run extracts it alone. (Under the
|
||||||
// (Native: the answer arrives as the call's tool result — the
|
// 2026-09-28 ruling only TEACHER prose can re-open anything.)
|
||||||
// projected group gains the answer line.)
|
|
||||||
const mutated = record().map((m) =>
|
const mutated = record().map((m) =>
|
||||||
(m as { id?: string }).id === "s1"
|
(m as { id?: string }).id === "s1"
|
||||||
? ({ ...m, content: [...(m as { content: unknown[] }).content, { type: "text", text: "the answer merged in late" }] } as never)
|
? ({ ...m, content: [...(m as { content: unknown[] }).content, { type: "text", text: "the answer merged in late" }] } as never)
|
||||||
@@ -1320,8 +1321,8 @@ describe("the chunked pipeline (extraction coverage)", () => {
|
|||||||
});
|
});
|
||||||
await reOpener.runExtraction(chatId);
|
await reOpener.runExtraction(chatId);
|
||||||
assert.equal(reRunViews.length, 1, "one batch: the single re-opened message");
|
assert.equal(reRunViews.length, 1, "one batch: the single re-opened message");
|
||||||
assert.ok(reRunViews[0]!.includes("the answer merged in late") && !reRunViews[0]!.includes("### Learner · s2"));
|
assert.ok(reRunViews[0]!.includes("the answer merged in late") && !reRunViews[0]!.includes("short topic two"));
|
||||||
assert.ok(reRunViews[0]!.includes("### Learner"), "the re-opened message renders");
|
assert.ok(reRunViews[0]!.includes("short topic one"), "the re-opened message renders its full prose");
|
||||||
assert.equal(coverageOf(messages(), ledgers.readCoverage(), chatId).uncoveredCount, 0, "re-covered");
|
assert.equal(coverageOf(messages(), ledgers.readCoverage(), chatId).uncoveredCount, 0, "re-covered");
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|||||||
+10
-3
@@ -90,6 +90,11 @@ function user(id: string, text: string): LhMessage[] {
|
|||||||
return [{ id, role: "user", content: text, timestamp: Date.now() } as never];
|
return [{ id, role: "user", content: text, timestamp: Date.now() } as never];
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** A teacher-prose message (the coverage unit, owner 2026-09-28). */
|
||||||
|
function teacher(id: string, text: string): LhMessage[] {
|
||||||
|
return [{ id, role: "assistant", content: [{ type: "text", text }], timestamp: Date.now() } as never];
|
||||||
|
}
|
||||||
|
|
||||||
function assistant(id: string, calls: NativeCall[]): LhMessage[] {
|
function assistant(id: string, calls: NativeCall[]): LhMessage[] {
|
||||||
const content = calls.map((c) => ({ type: "toolCall", id: c.toolCallId, name: c.name, arguments: c.input as never }));
|
const content = calls.map((c) => ({ type: "toolCall", id: c.toolCallId, name: c.name, arguments: c.input as never }));
|
||||||
const out: LhMessage[] = [
|
const out: LhMessage[] = [
|
||||||
@@ -923,10 +928,12 @@ describe("single-node deletion (deleteNodeTraces + deleteGraphNode, node interlu
|
|||||||
describe("session coverage read (extraction-UX redesign, 2026-09-24)", () => {
|
describe("session coverage read (extraction-UX redesign, 2026-09-24)", () => {
|
||||||
it("one session's block: derived uncovered, the newest fold, honest nulls", () => {
|
it("one session's block: derived uncovered, the newest fold, honest nulls", () => {
|
||||||
// Empty coverage: everything uncovered, no fold yet, not running.
|
// Empty coverage: everything uncovered, no fold yet, not running.
|
||||||
|
// Teacher prose only (owner, 2026-09-28): coverage slots are the
|
||||||
|
// teacher's text blocks, so the fixture is all-teacher.
|
||||||
const chat = fixtureChat("coverage read", [
|
const chat = fixtureChat("coverage read", [
|
||||||
user("cov_u1", "tell me about coverage reads"),
|
teacher("cov_t1", "tell me about coverage reads"),
|
||||||
user("cov_u2", "and about per-session blocks"),
|
teacher("cov_t2", "and about per-session blocks"),
|
||||||
user("cov_u3", "a third distinct message"),
|
teacher("cov_t3", "a third distinct message"),
|
||||||
].flat());
|
].flat());
|
||||||
const empty = sessionCoverage(chat.id)!;
|
const empty = sessionCoverage(chat.id)!;
|
||||||
assert.ok(empty);
|
assert.ok(empty);
|
||||||
|
|||||||
@@ -116,12 +116,12 @@ describe("sessionMarkdown", () => {
|
|||||||
assert.equal(md, "### Learner · u1\n\ngo");
|
assert.equal(md, "### Learner · u1\n\ngo");
|
||||||
});
|
});
|
||||||
|
|
||||||
it("content parity with the coverage line builder (2026-09-27 split)", () => {
|
it("the seam split (owner, 2026-09-28): teacher prose renders in BOTH; the rest renders ONLY here", () => {
|
||||||
// Every message that produces view lines produces a markdown block,
|
// The extraction view became TEACHER-PROSE-ONLY: its line builder
|
||||||
// and every message that renders nothing renders nothing in either —
|
// yields lines for teacher text alone. This engine keeps the FULL
|
||||||
// same messages, two formats, one seam decision. The FINGERPRINT
|
// session format for the future user-facing export — so parity now
|
||||||
// stays on the line format (stable — no mass re-extraction when
|
// holds exactly for teacher prose, and everything else (learner,
|
||||||
// presentation changes); markdown is presentation only.
|
// quiz, tool results) is markdown-only.
|
||||||
const messages: LhMessage[] = [
|
const messages: LhMessage[] = [
|
||||||
user("u1", "teach me"),
|
user("u1", "teach me"),
|
||||||
teacher("a1", "The chain rule multiplies local rates."),
|
teacher("a1", "The chain rule multiplies local rates."),
|
||||||
@@ -130,9 +130,10 @@ describe("sessionMarkdown", () => {
|
|||||||
result("r3", "j2", "research", {}, true),
|
result("r3", "j2", "research", {}, true),
|
||||||
];
|
];
|
||||||
for (const message of messages) {
|
for (const message of messages) {
|
||||||
|
const isTeacher = (message as { role?: string }).role === "assistant";
|
||||||
const hasLines = messageViewLines(message, message.id).length > 0;
|
const hasLines = messageViewLines(message, message.id).length > 0;
|
||||||
const hasMarkdown = messageMarkdown(message, message.id) !== null;
|
const hasMarkdown = messageMarkdown(message, message.id) !== null;
|
||||||
assert.equal(hasMarkdown, hasLines, `parity for ${message.id}`);
|
assert.equal(hasLines, isTeacher && hasMarkdown, `view lines follow teacher prose for ${message.id}`);
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|||||||
Reference in New Issue
Block a user