sorokin.oleg.edu/learnharness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

feat: probe/teach agent split — explicit phase markers, position-only plan map

Browse Source 
Diagnosis (2026-09-11 user testing, minimal-effort GLM): sessions stuck on
'plan' forever. Two causes: (1) the three-phase view read the stored tree's
currentNodeId, which nothing ever writes post author-once (2026-09-03) — a
structural dead seam; (2) the teacher stochastically left quiz.nodeId
untagged, and done-by-quiz-result semantics would have falsely completed
nodes anyway.

The redesign (docs/probe-teach-split.md) abolishes derived occupancy in
favor of explicit events:

- Two agents over one transcript: createProberAgent (quiz without a
  nodeId field, ask, probe_done) and createTeacherAgent (full toolset,
  plan_init onward). The chat-handler construction branch keys on a
  probe_done part in the transcript. MODELS.prober added.
- probe_done: server-side seam tool — carries the learner assessment
  (strands + free-form edge read), refuses while a question is open or
  the probe is already concluded, and queues a handoff steer through the
  generalized wake pipeline (steers now dedupe by id, not jobId; job
  finalization only for job steers). The teacher auto-wakes and authors
  the plan; the learner sees the assessment via the probe-done card.
- phases: probe ends at a concluded probe_done, plan ends at a
  successful plan_init — strictly sequential, no skipping, no legacy
  compatibility (data wiped at the pivot per doctrine).
- Plan map = position, not completion: derivePlanStatuses reads only
  tagged quiz CALLS (state/outcome feed nothing); current = latest
  tagged node with consecutive duplicates collapsed, visited = earlier
  distinct stays (a revisit stays in both — current wins at render).
  PlanTree is structure-only; currentNodeId/doneNodeIds and the dead
  mutation helpers are gone. No fallback current.
- Prompts split per agent (prompts/prober/, prompts/teacher/ over the
  trimmed shared base); graph-context injection removed entirely
  (shelved, docs/probe-teach-split.md §5); knowledge-graph context no
  longer reaches either agent.
- Plan blocks are full-only snapshots at plan_init (neighborhood
  machinery retired; the persistent DAG window is the future UI wave);
  usage capture unchanged.

Gates: tsc clean, 294/294 tests (phases/plan-tree/plan-blocks/question-
tools rewritten to the new semantics; probe-done + handoff-wake tests
added), lint, build. Live GLM smoke PASS: prober concluded with an honest
assessment, teacher auto-woke, authored a 10-node plan with the block
attached, then tagged checks on slope-instinct → secant derived current/
visited exactly as designed; grading honest both ways (driver off-by-one
proved a wrong answer grades false).
This commit is contained in:
amosblomqvist
2026-09-11 06:44:20 +03:00
parent 741ff416d6
commit 5653c77a29
49 changed files with 1538 additions and 2038 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/core/phases.ts
+49 -117
View File
@@ -1,128 +1,69 @@
/**
* Probe/teach phase derivation (vision §2 "Two phases, clearly separated").
* Phase derivation (docs/probe-teach-split.md §4) — EXPLICIT EVENTS, never
* occupancy. The probe→teach agent split (2026-09-11) gave every phase
* boundary a transcript marker:
*
* The session's phase is DERIVED, never stored: `plan_init` IS the transition
* marker — the call already fires exactly once at the seam (verified live in
* Gate 1), so no new agent tool exists. Phase = "probe" until the first
* message containing a `plan_init` tool part; from that message onward,
* phase = "teach". Everything after it (checks, follow-up asks) is teaching.
* probe — until the first message containing a `tool-probe_done` part
* (the prober concludes; the assessment rides the tool part)
* plan — probe concluded, until the first successful `plan_init`
* (the teacher presents the map; interim rule until the
* accept-plan UI lands — then acceptance becomes the seam)
* teach — a plan exists
*
* Pure: no imports, no runtime deps — safe on server, client, and in tests.
* The former derived-occupancy seam (phase-of-tree via `currentNodeId`)
* is GONE: nothing ever writes currentNodeId (the stuck-on-plan bug — a
* session could never leave "plan"), and the stored tree no longer carries
* progress fields at all. No backwards compatibility: legacy sessions
* without probe_done simply read as probe (they are wiped data).
*
* Pure: no imports, no runtime deps — safe on server, client, and tests.
*/
/** The two phases of a session. */
export type Phase = "probe" | "teach";
/** Minimal structural slice of a UI message — anything with typed parts. */
export interface PhaseMessage {
parts: ReadonlyArray<{ type: string }>;
}
const PLAN_INIT_PART = "tool-plan_init";
/**
* The phase at `upToIndex` (inclusive; default: the last message).
*
* Scans messages[0..upToIndex] for the FIRST message containing a
* `tool-plan_init` part — any part state counts (the call itself is the
* transition; teaching has begun even while the call's input streams). The
* message that contains plan_init is already teach: the divider renders
* exactly there, so probe content sits above it, teaching below.
*
* Boundary semantics:
* - no plan_init anywhere → "probe" at every index (pre-plan sessions keep
* the old auto-open behavior for every question — backward compatible).
* - first plan_init in message i → phaseAt(m, i-1) === "probe",
* phaseAt(m, i) === "teach", and every index after.
* - multiple plan_init parts → the FIRST wins (a re-init attempt later in
* the transcript never moves the seam; the tool refuses re-init anyway).
*/
export function phaseAt(messages: readonly PhaseMessage[], upToIndex?: number): Phase {
const last = Math.min(upToIndex ?? messages.length - 1, messages.length - 1);
for (let i = 0; i <= last; i++) {
const parts = messages[i]?.parts;
if (parts?.some((p) => p.type === PLAN_INIT_PART)) return "teach";
}
return "probe";
}
// ─── The three-phase view (probe / plan / teach) ──────────────────────────────
//
// The session picker's color strips, the header chip, and the transcript
// dividers all speak THREE phases, derived data-only from the plan tree:
//
// probe — no plan output yet (the tree is empty or absent);
// plan — a tree exists but no node is occupied (plan_set_current not fired);
// teach — a node is occupied.
//
// The teacher's invisible PRE-plan planning (reasoning before plan_init
// fires) reads as probe until the plan lands — an accepted limitation: no
// observable signal marks "I'm now planning" (a teacher-side marker would be
// an agent change; see phases.md discussion in the transcript divider work).
/** The three phases of a session, as the picker / header / dividers see it. */
/** The three phases of a session, as the picker / header / dividers see them. */
export type Phase3 = "probe" | "plan" | "teach";
/** The advance-only order of phases: a session never visibly regresses. */
const PHASE3_RANK: Record<Phase3, number> = { probe: 0, plan: 1, teach: 2 };
/** Structural slice of a plan tree — anything with nodes + a current node. */
export interface PhaseTree {
nodes: ReadonlyArray<unknown>;
currentNodeId: string | null;
}
/**
* The phase of a tree, point-in-time: no nodes → probe, no current node →
* plan, an occupied node → teach. Used for the chats summary (the stored
* tree IS the session state there). Empty trees (emptyTree(), a failed
* plan_init) read as probe — a seam only counts once a plan exists.
* Minimal structural slice of a UI message for the phase scan: parts with
* a state and (for concluded calls) an output.
*/
export function phaseOfTree(tree: PhaseTree | null | undefined): Phase3 {
if (!tree || tree.nodes.length === 0) return "probe";
return tree.currentNodeId === null ? "plan" : "teach";
}
/**
* Minimal structural slice of a UI message for the three-phase scan: parts
* with a state and (when a plan op completed) its output.
*/
export interface Phase3Message {
export interface PhaseMessage {
parts: ReadonlyArray<{ type: string; state?: string; output?: unknown }>;
}
/** A successful plan-op output as it appears on the part (`PlanToolResult`). */
interface PlanOutput {
tree?: { nodes?: unknown };
}
/**
* The three-phase boundary semantics over the TRANSCRIPT (the same data the
* tree panel reads): scanning forward from 0 to `upToIndex` (default: the
* last message), the phase advances probe → plan at the first plan part
* with a NON-EMPTY tree available (a failed plan_init carries an empty tree
* and never marks the seam), and plan → teach at the first tree output with
* an occupied node. MONOTONIC by design: the phase only ever advances — a
* node removal mid-teaching (currentNodeId → null) is an edit, not a phase
* regression, so no "Plan" divider ever appears below teaching content.
*
* `phaseAt` (two-phase) stays as-is: the plan_init call is its seam and the
* question dock's reveal logic leans on it. The two views differ only in
* the window between plan_init and the first plan_set_current, where
* three-phase reads "plan".
*/
export function phaseAt3(messages: readonly Phase3Message[], upToIndex?: number): Phase3 {
/** The phase at `upToIndex` (inclusive; default: the last message). */
export function phaseAt3(messages: readonly PhaseMessage[], upToIndex?: number): Phase3 {
let phase: Phase3 = "probe";
const last = Math.min(upToIndex ?? messages.length - 1, messages.length - 1);
for (let i = 0; i <= last; i++) {
const parts = messages[i]?.parts;
if (!parts) continue;
for (const part of parts) {
if (!part.type.startsWith("tool-plan_") || part.state !== "output-available") continue;
const tree = (part.output as PlanOutput | undefined)?.tree as PhaseTree | undefined;
if (!tree || !Array.isArray(tree.nodes)) continue;
const candidate = phaseOfTree(tree);
if (PHASE3_RANK[candidate] > PHASE3_RANK[phase]) phase = candidate;
phase = advance(phase, messages[i]?.parts);
}
return phase;
}
/** One message's advance of the phase (pure fold step). */
function advance(
phase: Phase3,
parts: ReadonlyArray<{ type: string; state?: string; output?: unknown }> | undefined,
): Phase3 {
if (!parts) return phase;
for (const part of parts) {
// STRICTLY SEQUENTIAL: probe → plan only via a concluded probe_done,
// plan → teach only via a successful plan_init. A marker out of order
// (e.g. a legacy plan_init with no probe) advances nothing — the chain
// never skips, and the rank table below only guards regressions.
if (phase === "probe" && part.type === "tool-probe_done" && part.state === "output-available") {
phase = "plan";
}
if (phase === "plan" && part.type === "tool-plan_init" && part.state === "output-available") {
const tree = (part.output as PlanOutput | undefined)?.tree;
if (tree && Array.isArray(tree.nodes) && tree.nodes.length > 0) phase = "teach";
}
}
return phase;
@@ -130,24 +71,15 @@ export function phaseAt3(messages: readonly Phase3Message[], upToIndex?: number)
/**
* The phase at EVERY message index, in one forward fold — the transcript
* dividers render one per boundary, so phaseAt3(m, i) per index would
* rescan 0..i each time (quadratic); this is the same derivation, linear.
* out[i] === phaseAt3(messages, i) for every index.
* dividers render one per boundary, so phaseAt3(m, i) per index would rescan
* 0..i each time (quadratic); this is the same derivation, linear.
* out[i] === phaseAt3(messages, i) for every i.
*/
export function phaseTimeline(messages: readonly Phase3Message[]): Phase3[] {
export function phaseTimeline(messages: readonly PhaseMessage[]): Phase3[] {
const out: Phase3[] = [];
let phase: Phase3 = "probe";
for (let i = 0; i < messages.length; i++) {
const parts = messages[i]?.parts;
if (parts) {
for (const part of parts) {
if (!part.type.startsWith("tool-plan_") || part.state !== "output-available") continue;
const tree = (part.output as PlanOutput | undefined)?.tree as PhaseTree | undefined;
if (!tree || !Array.isArray(tree.nodes)) continue;
const candidate = phaseOfTree(tree);
if (PHASE3_RANK[candidate] > PHASE3_RANK[phase]) phase = candidate;
}
}
phase = advance(phase, messages[i]?.parts);
out.push(phase);
}
return out;