question tools: phase-visible names, hardened teach schemas, context-slim seams
The tools_fails_2 diagnosis (glm-5.3 dropping trailing required fields):
- probe-phase quiz/explain renamed to probe/recall — a shared name carried
the probe call shape into teaching (the first teach quiz dropped nodeId);
display seams accept the old names forever
- teach quiz+explain schemas lead with the required pair (type, nodeId);
explain gains the predict/check type and joins the reveal law (checks
collapse, predictions arrive open; no-type records render check-style)
- plan_init/probe_done results go context-slim (owner, context diet):
content = validated nodeIds + binding-id reminder / fixed ack; details
keeps the tree for the popup and {ok:true} — serializer-verified that
details never reaches the model; failure echoes dropped
- the cards carry the tool's own name (Probe/Recall) and the quiz type
reads on the label (Prediction/Check) in composer and transcript
This commit is contained in:
@@ -30,7 +30,7 @@ The map is a hypothesis about him, and the probe is only its first sketch — th
|
||||
|
||||
And when the ground shifts mid-lesson — a missing foundation, a confusion deeper than the map assumed — never rush the repair: no wall of text that "covers" the missing ground in one turn to get back on plan. The plan serves his understanding, never the reverse. Update the map if the structure changed (sometimes nothing changed and no rewrite is needed), then build the missing ground the only way anything is built here — one reasoning step at a time, each step landing and verified, at the session's one and only pace. Detours are not interruptions to get through; they are the teaching. A map that can't change is a wrong map.
|
||||
|
||||
**Every quizzing call carries `nodeId`** (required — the quizzing tool is one of `quiz` or `explain` depending on the mechanic the learner chose; a call without it is rejected): that's what moves the map. Its live position is simply the latest tagged node, so asking about the next node IS how the lesson gets there. A tag is a position, not a verdict — nodes are never marked finished. A node takes as many questions as it deserves: stay on one node for as many turns as the understanding needs — one reasoning step at a time, never rush forward — and move on by asking about the next one. Off-map checks (no plan node fits) use ask_user_question.
|
||||
**Every quizzing call carries `nodeId` AND `type`** (both required — the quizzing tool is one of `quiz` or `explain` depending on the mechanic the learner chose; a call missing either is rejected): that's what moves the map. `type` is the question's role in the teaching cycle — `predict` before you teach the step, `check` after. Its live position is simply the latest tagged node, so asking about the next node IS how the lesson gets there. A tag is a position, not a verdict — nodes are never marked finished. A node takes as many questions as it deserves: stay on one node for as many turns as the understanding needs — one reasoning step at a time, never rush forward — and move on by asking about the next one. Off-map checks (no plan node fits) use ask_user_question.
|
||||
|
||||
## Presenting the plan — always, before any teaching
|
||||
|
||||
|
||||
+45
-20
@@ -108,11 +108,14 @@ export const askUserQuestionTool: AgentTool<typeof askInputSchema> = {
|
||||
// mechanical delta.
|
||||
|
||||
/** The quiz mechanics every variant shares; `delta` appends the variant's
|
||||
* own fields. All mechanics, zero doctrine. */
|
||||
function quizDescription(delta: { nodeId: boolean; types: boolean }): string {
|
||||
* own fields (its NAME among them — the probe-phase tools self-reference
|
||||
* as probe/recall, owner 2026-10-09). All mechanics, zero doctrine. */
|
||||
function quizDescription(delta: { name: "quiz" | "probe"; nodeId: boolean; types: boolean }): string {
|
||||
return (
|
||||
"Ask the learner a GRADED question with a known correct answer, then instantly grade and give feedback. " +
|
||||
"Unlike ask_user_question (which collects preferences/decisions with no right answer), quiz always has a " +
|
||||
"Unlike ask_user_question (which collects preferences/decisions with no right answer), " +
|
||||
delta.name +
|
||||
" always has a " +
|
||||
"correct answer supplied by you, marks the learner's selection right/wrong, reveals the correct answer, and " +
|
||||
"can show an explanation. Options-only: single-select or multi-select, plus an automatic 'I don't know' " +
|
||||
"choice so the learner can signal a genuine gap instead of guessing. An always-present optional thinking " +
|
||||
@@ -124,37 +127,51 @@ function quizDescription(delta: { nodeId: boolean; types: boolean }): string {
|
||||
(delta.nodeId
|
||||
? "Set `nodeId` (required) to the plan node this question is about — validated at call time against the session's plan map. For off-map checks use ask_user_question. "
|
||||
: "") +
|
||||
"ONE question per turn: call a single quiz, then end your turn — his answer determines the next."
|
||||
"ONE question per turn: call a single " +
|
||||
delta.name +
|
||||
", then end your turn — his answer determines the next."
|
||||
);
|
||||
}
|
||||
|
||||
/** The explain mechanics (free-recall quizzing): the commitment rule is
|
||||
* the GRADING CONTRACT — mechanics, not doctrine. The softer doctrine
|
||||
* (near-miss reading, genuine gaps) lives in the shared prompt chunk. */
|
||||
function explainDescription(nodeId: boolean): string {
|
||||
* (near-miss reading, genuine gaps) lives in the shared prompt chunk.
|
||||
* `name`: the teach tool self-references as explain, the probe variant
|
||||
* as recall (owner 2026-10-09). */
|
||||
function explainDescription(name: "explain" | "recall", nodeId: boolean): string {
|
||||
return (
|
||||
"Ask the learner to EXPLAIN something in his own words — a graded check with NO options: he types a free-text " +
|
||||
"answer. Unlike ask_user_question (no right answer), explain always has one. THE COMMITMENT RULE: " +
|
||||
"answer. Unlike ask_user_question (no right answer), " +
|
||||
name +
|
||||
" always has one. THE COMMITMENT RULE: " +
|
||||
"correctAnswer is the FULL correct answer in your own words, reasoning included, COMMITTED before he ever " +
|
||||
"sees the question. When his answer returns, grade it strictly against the commitment — never revise it " +
|
||||
"after seeing his words. He sees the committed answer on submit. " +
|
||||
(nodeId
|
||||
? "Set `nodeId` (required) to the plan node this question is about — validated at call time against the session's plan map. "
|
||||
: "") +
|
||||
"ONE question per turn: call a single explain, then end your turn — his answer determines the next."
|
||||
"ONE question per turn: call a single " + name + ", then end your turn — his answer determines the next."
|
||||
);
|
||||
}
|
||||
|
||||
/** The prober's multiple-choice quizzing tool — named `probe`, not
|
||||
* `quiz` (owner, 2026-10-09): the probe and teach schemas differ
|
||||
* (neither `type` nor `nodeId` exists while probing), and a shared name
|
||||
* invited the model to carry its probe-phase call shape into teaching
|
||||
* — the first teach quiz after plan_init dropped `nodeId` on exactly
|
||||
* that habit (the tools_fails_2 incident). Distinct names make the
|
||||
* phase boundary visible in the transcript itself: history written by
|
||||
* `probe` never lies about what `quiz` requires. */
|
||||
export const probeQuizTool: AgentTool<typeof probeQuizInputSchema> = {
|
||||
name: "quiz",
|
||||
label: "quiz",
|
||||
description: quizDescription({ nodeId: false, types: false }),
|
||||
name: "probe",
|
||||
label: "probe",
|
||||
description: quizDescription({ name: "probe", nodeId: false, types: false }),
|
||||
parameters: probeQuizInputSchema,
|
||||
prepareArguments: prepareQuestionArguments,
|
||||
executionMode: "sequential",
|
||||
execute: async (_toolCallId, input) => {
|
||||
const problems = quizInputProblems(input);
|
||||
if (problems.length > 0) return invalidInputResult("quiz", problems);
|
||||
if (problems.length > 0) return invalidInputResult("probe", problems);
|
||||
return questionResult();
|
||||
},
|
||||
};
|
||||
@@ -167,7 +184,7 @@ export function teachExplainToolFor(
|
||||
return {
|
||||
name: "explain",
|
||||
label: "explain",
|
||||
description: explainDescription(true),
|
||||
description: explainDescription("explain", true),
|
||||
parameters: explainInputObject,
|
||||
prepareArguments: prepareQuestionArguments,
|
||||
executionMode: "sequential",
|
||||
@@ -183,16 +200,22 @@ export function teachExplainToolFor(
|
||||
|
||||
/** The prober's free-recall variant (no nodeId — same pattern as the
|
||||
* probe quiz). */
|
||||
/** The prober's free-recall variant — named `recall`, not `explain`
|
||||
* (owner, 2026-10-09), for the same phase-boundary reason as `probe`:
|
||||
* the teach explain adds `type` and `nodeId`; a shared name carried the
|
||||
* probe call shape into teaching. Old probe-phase records keep the
|
||||
* `explain` name and parse identically (the display seam accepts
|
||||
* both). */
|
||||
export const probeExplainTool: AgentTool<typeof explainInputBase> = {
|
||||
name: "explain",
|
||||
label: "explain",
|
||||
description: explainDescription(false),
|
||||
name: "recall",
|
||||
label: "recall",
|
||||
description: explainDescription("recall", false),
|
||||
parameters: explainInputBase,
|
||||
prepareArguments: prepareQuestionArguments,
|
||||
executionMode: "sequential",
|
||||
execute: async (_toolCallId, input) => {
|
||||
const problems = explainInputProblems(input);
|
||||
if (problems.length > 0) return invalidInputResult("explain", problems);
|
||||
if (problems.length > 0) return invalidInputResult("recall", problems);
|
||||
return questionResult();
|
||||
},
|
||||
};
|
||||
@@ -239,14 +262,16 @@ export function createProberTools(
|
||||
});
|
||||
// Quizzing mechanics (owner 2026-09-27): free-recall swaps the probe
|
||||
// quiz for the probe explain — one quizzing tool at a time, whichever
|
||||
// the settings arm. ask stays always-on.
|
||||
// the settings arm. ask stays always-on. The registry keys are the
|
||||
// PHASE names (probe/recall, owner 2026-10-09) — distinct from the
|
||||
// teach tools so the schema boundary is visible to the model.
|
||||
const mechanics = quizMechanics(enabled);
|
||||
// The armed quizzing tool, widened to the registry's heterogeneous
|
||||
// AgentTool record BEFORE the filter (a conditional spread of two
|
||||
// differently-typed literals makes the whole object's type a union —
|
||||
// unusable as Record<string, AgentTool<any>> downstream).
|
||||
const quizzing: Record<string, AgentTool<any>> =
|
||||
mechanics === "free" ? { explain: probeExplainTool } : { quiz: probeQuizTool };
|
||||
mechanics === "free" ? { recall: probeExplainTool } : { probe: probeQuizTool };
|
||||
// healToolRecord (owner, 2026-10-09 — the whitehead generalization):
|
||||
// the standard argument heal over the WHOLE registry — every tool,
|
||||
// not just the one the latest wobble incident landed on.
|
||||
@@ -313,7 +338,7 @@ export function teachQuizToolFor(ctx: Pick<PlanToolContext, "chatId" | "chatStor
|
||||
return {
|
||||
name: "quiz",
|
||||
label: "quiz",
|
||||
description: quizDescription({ nodeId: true, types: true }),
|
||||
description: quizDescription({ name: "quiz", nodeId: true, types: true }),
|
||||
parameters: quizInputObject,
|
||||
prepareArguments: prepareQuestionArguments,
|
||||
executionMode: "sequential",
|
||||
|
||||
+34
-11
@@ -3,9 +3,17 @@ import type { AgentTool } from "@earendil-works/pi-agent-core";
|
||||
import { serverExecute } from "../tool-execute.ts";
|
||||
import type { PlanToolResult, PlanTree } from "../../core/contracts.ts";
|
||||
import type { ChatStore } from "../../core/ports.ts";
|
||||
import { createTree, emptyTree, PlanTreeError } from "../../core/plan-tree.ts";
|
||||
import { createTree, PlanTreeError } from "../../core/plan-tree.ts";
|
||||
import { parseMermaidPlan } from "./mermaid-plan.ts";
|
||||
|
||||
/** The binding-id reminder riding plan_init's slim content (owner,
|
||||
* 2026-10-09): plan_init is the exact moment the node ids become
|
||||
* load-bearing — every later quiz/explain call must reference one. The
|
||||
* field-drop incidents (tools_fails_2) showed the turn's tail is where
|
||||
* discipline slips; the reminder lands right there. */
|
||||
const PLAN_NODE_IDS_REMINDER =
|
||||
"These are the plan's node ids — the only valid nodeId values for every quiz and explain call from here on. Reference them exactly.";
|
||||
|
||||
/** What the plan tools close over (the plan-tree half of the tool context). */
|
||||
export interface PlanToolContext {
|
||||
chatId: string;
|
||||
@@ -33,10 +41,13 @@ export interface PlanToolContext {
|
||||
*
|
||||
* The former mutation tools (add/update/remove node/edge, set_current,
|
||||
* mark_done) are gone by design: restructuring is a rewrite, tracking is
|
||||
* derived. SERVER-SIDE with execute: parse → validate → persist → return
|
||||
* the FULL canonicalized tree (PlanToolResult), so the tree panel renders the
|
||||
* latest state straight from the transcript's tool outputs (the plan
|
||||
* window and the plan blocks).
|
||||
* derived. SERVER-SIDE with execute: parse → validate → persist. The
|
||||
* result is CONTEXT-SLIM (owner, context diet 2026-10-09): the model
|
||||
* WROTE the mermaid one message ago, so content never echoes the tree —
|
||||
* it carries the validated nodeIds + the binding-nodeId reminder (the
|
||||
* exact moment the ids become load-bearing); `details` keeps the full
|
||||
* tree for the record and the plan popup (latestTree reads it — the
|
||||
* agent-invisible channel, serializer-verified).
|
||||
*/
|
||||
const planInitInput = Type.Object({
|
||||
mermaid: Type.String({
|
||||
@@ -60,25 +71,37 @@ export function planTools(ctx: PlanToolContext): Record<string, AgentTool<any>>
|
||||
"`-->` edges meaning 'from must be understood before to' (chains fine). 5–15 nodes: a map, not a syllabus — one reasoning step per node, compress. " +
|
||||
"Every check you fire names the node it assesses via `nodeId`; where the lesson stands derives from those tags.",
|
||||
parameters: planInitInput,
|
||||
execute: serverExecute("plan_init", planInitInput, async ({ mermaid }): Promise<PlanToolResult> => {
|
||||
execute: serverExecute("plan_init", planInitInput, async ({ mermaid }) => {
|
||||
const { chatId, chatStore } = ctx;
|
||||
const record = chatStore.getChat(chatId);
|
||||
if (!record) return { ok: false, error: `Chat not found: ${chatId}`, tree: emptyTree() };
|
||||
const current = record.tree ?? emptyTree();
|
||||
if (!record) return { ok: false, error: `Chat not found: ${chatId}` };
|
||||
|
||||
const parsed = parseMermaidPlan(mermaid);
|
||||
if (!parsed.ok || !parsed.plan) {
|
||||
// The parser collects every problem; the teacher fixes and retries.
|
||||
return { ok: false, error: parsed.problems.join("\n"), tree: current };
|
||||
// No tree echo — the model needs the problems, not a restatement of
|
||||
// the map it is rewriting (owner, context diet 2026-10-09).
|
||||
return { ok: false, error: parsed.problems.join("\n") };
|
||||
}
|
||||
try {
|
||||
// createTree validates the structure (duplicate ids, missing
|
||||
// edge endpoints, self-loops, cycles → PlanTreeError).
|
||||
const next = createTree(parsed.plan.nodes, parsed.plan.edges);
|
||||
chatStore.saveTree(chatId, next);
|
||||
return { ok: true, tree: next };
|
||||
// Context diet: content = the validated nodeIds + the reminder (the
|
||||
// ids just became binding for every quiz/explain call); details =
|
||||
// the full tree for the record/UI — the plan popup's latestTree
|
||||
// seam, agent-invisible (the serializer sends content only).
|
||||
return {
|
||||
text: JSON.stringify({
|
||||
ok: true,
|
||||
nodeIds: next.nodes.map((n) => n.id),
|
||||
reminder: PLAN_NODE_IDS_REMINDER,
|
||||
}),
|
||||
details: { ok: true, tree: next },
|
||||
};
|
||||
} catch (err) {
|
||||
if (err instanceof PlanTreeError) return { ok: false, error: err.message, tree: current };
|
||||
if (err instanceof PlanTreeError) return { ok: false, error: err.message };
|
||||
throw err;
|
||||
}
|
||||
}),
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import type { AgentTool } from "@earendil-works/pi-agent-core";
|
||||
import { serverExecute } from "../tool-execute.ts";
|
||||
import { probeDoneInputSchema, type ProbeDoneResult } from "../../core/contracts.ts";
|
||||
import { probeDoneInputSchema, PROBE_DONE_ACK_TEXT } from "../../core/contracts.ts";
|
||||
import { danglingQuestionOf, hasProbeDone, type LhMessage } from "../../core/native-messages.ts";
|
||||
import type { ChatStore } from "../../core/ports.ts";
|
||||
import { buildProbeHandoffSteer, type SteerMessage } from "../../core/contracts.ts";
|
||||
@@ -37,7 +37,10 @@ function probeConcluded(messages: readonly LhMessage[]): boolean {
|
||||
* steer through the wake pipeline that fires the teacher's first turn
|
||||
* (plan authoring) the moment the prober's turn ends. Auto-wake is the
|
||||
* user-decided trigger — the learner experiences assessment → plan as
|
||||
* one continuous flow.
|
||||
* one continuous flow. CONTEXT-SLIM (owner, context diet 2026-10-09):
|
||||
* the call's arguments already carry the assessment (durable, streamed
|
||||
* live), so the agent sees the fixed PROBE_DONE_ACK_TEXT — never an
|
||||
* echo. details = {ok:true} only.
|
||||
*
|
||||
* The first probe_done in the transcript is also the construction seam:
|
||||
* from that message on, every request builds the TEACHER agent.
|
||||
@@ -58,7 +61,7 @@ export function probeDoneTool(ctx: ProbeDoneContext): Record<string, AgentTool<a
|
||||
// The core-owned schema (contracts/probe.ts) — the SAME schema the UI
|
||||
// parse seam (tool-parts.ts) runs, so boundary-valid ⇔ consumer-parseable.
|
||||
parameters: probeDoneInputSchema,
|
||||
execute: serverExecute("probe_done", probeDoneInputSchema, async (input): Promise<ProbeDoneResult> => {
|
||||
execute: serverExecute("probe_done", probeDoneInputSchema, async (input) => {
|
||||
const record = ctx.chatStore.getChat(ctx.chatId);
|
||||
if (!record) return { ok: false, error: `Chat not found: ${ctx.chatId}` };
|
||||
|
||||
@@ -81,7 +84,14 @@ export function probeDoneTool(ctx: ProbeDoneContext): Record<string, AgentTool<a
|
||||
// constructs the teacher off the transcript seam) — record the
|
||||
// assessment anyway; the wake is an optimization, not a gate.
|
||||
}
|
||||
return { ok: true, assessment: input.assessment };
|
||||
// Context diet (owner, 2026-10-09): the assessment lives ONCE — as
|
||||
// this call's arguments. content is the fixed ack (the agent never
|
||||
// sees an echo); details = {ok:true} for the record/UI — the
|
||||
// probe-done card reads the call's input, which it always preferred.
|
||||
return {
|
||||
text: PROBE_DONE_ACK_TEXT,
|
||||
details: { ok: true },
|
||||
};
|
||||
}),
|
||||
},
|
||||
};
|
||||
|
||||
Reference in New Issue
Block a user