tools: the standard argument heal — every registry, both seams

The whitehead incident (2026-10-09): glm-5.3 emitted quiz type with
quotes baked in ('"predict"') — rejected ten times, byte-identical
retries, the lesson never started — and correctAnswer with the JSON
key-value colon glued on ([':parmenides']). Third GLM wobble incident,
and each prior heal was wired only where its incident landed; this one
landed precisely where the practice heal never reached.

Layer 1 — healEncodedArguments moves to core/argument-heal (the display
seam needs it too — the tool-parts law) with withArgumentHeal +
healToolRecord (agent/tools/heal) wired at EVERY registry: prober,
teacher, practice, bareQuestionTools (it had no prepareArguments at
all), demo-builder, and submit-agent (composing after
healStringifiedArrays; the auditor's extraTools wrapped).

Layer 2 — stripModelValueDebris (core/value-debris): leading/trailing
colons, quotes, backticks, whitespace strip at the MEMBERSHIP seams —
coerceCorrectAnswer (boundary check + display + grading in one),
normalizeQuizType (persisted wobbled records keep their role),
resolveTargetId (copy-syntax on targets).

Layer 3 — schema description hygiene: the quiz type and practice
mode/outcome/rung/missClass values now read as bare words — the quoted
JSON-key style invited quote-copying.

Display mirror — parseQuizInput heals the raw record on read, so a
boundary-healed quiz whose record keeps raw args renders instead of
becoming an invisible pending quiz (learner lockout).

Pins: tests/tool-heal.test.ts (the incident shapes end-to-end through
pi-ai validation, the wiring, the strips, the mirror). The stringy-
options pins flip to boundary-parity per tool-parts' own law; the
chat-flow garbage pin keeps its honest-reject intent with an unhealable
shape. 1471 tests, typecheck, lint green.
This commit is contained in:
amosblomqvist
2026-10-09 04:54:34 +03:00
parent e4207e869f
commit c23eb1f15b
15 changed files with 446 additions and 136 deletions
+96
View File
@@ -0,0 +1,96 @@
/**
* THE WEAK-MODEL ARGUMENT HEAL — the pure core (owner, 2026-10-09 —
* generalize the 2026-10-04 practice heal; see agent/tools/heal.ts for
* the registry-wide wiring story). Lives in core because BOTH seams need
* it (the tool-parts law: display parses mirror boundary coercion or a
* healed-at-boundary call whose record keeps the RAW arguments renders
* as an invisible pending quiz):
*
* - the agent boundary: pi's prepareArguments seam, BEFORE TypeBox
* validation (withArgumentHeal / healToolRecord);
* - the display parse: tool-parts' parseQuizInput, before
* boundaryParse — the persisted record keeps raw arguments, so the
* same wobble heals on read.
*
* Born of the GLM incident (owner, 2026-10-04): 19 consecutive rejected
* `problem` calls, every one carrying `"mode": "\"recall\""` — a
* JSON-string-encoded enum — while TypeBox's error text `must be equal
* to constant` never named the expected values, so the model re-emitted
* the identical payload each time; the 2026-10-09 whitehead session ran
* the same loop on quiz `type` for ten turns. A STRING value where the
* schema expects a literal union / boolean / null / number / array is
* JSON.parse-d and kept when the parsed value satisfies the expectation.
* Well-formed args pass through UNTOUCHED (same reference); unhealable
* values stay for validation to reject honestly.
*/
/** One property schema of a TypeBox object (the shapes this fork emits:
* literal unions carry anyOf[{const}], scalars carry type). */
type PropSchema = {
type?: string;
const?: unknown;
anyOf?: PropSchema[];
};
/** The literal values a property accepts ([] when it is not a literal
* union / bare literal). */
function literalsOf(prop: PropSchema): unknown[] {
if (prop.anyOf) return prop.anyOf.flatMap((m) => literalsOf(m));
return "const" in prop ? [prop.const] : [];
}
/** Does the property accept null (Type.Null standalone or as a union
* member)? */
function acceptsNull(prop: PropSchema): boolean {
if (prop.type === "null") return true;
return (prop.anyOf ?? []).some((m) => m.type === "null");
}
/** JSON.parse that never throws (undefined for malformed input). */
function tryParse(raw: string): unknown {
try {
return JSON.parse(raw) as unknown;
} catch {
return undefined;
}
}
export function healEncodedArguments(
args: unknown,
schema: { properties?: Record<string, PropSchema> },
): unknown {
if (typeof args !== "object" || args === null || Array.isArray(args)) return args;
const properties = schema.properties;
if (!properties) return args;
let healed: Record<string, unknown> | null = null;
for (const [key, prop] of Object.entries(properties)) {
const raw = (args as Record<string, unknown>)[key];
if (typeof raw !== "string") continue;
const parsed = tryParse(raw);
let fixed: unknown;
const literals = literalsOf(prop);
if (literals.length > 0) {
if (literals.includes(raw)) continue; // already a valid member
if (literals.includes(parsed)) fixed = parsed;
}
// The literal branch does not cover null (Type.Null carries no
// const) — a union of literals ∪ null must also accept "null".
if (fixed === undefined && acceptsNull(prop) && parsed === null) fixed = null;
if (fixed === undefined && prop.type === "boolean" && typeof parsed === "boolean") {
fixed = parsed;
}
if (
fixed === undefined &&
(prop.type === "number" || prop.type === "integer") &&
typeof parsed === "number"
) {
fixed = parsed;
}
if (fixed === undefined && prop.type === "array" && Array.isArray(parsed)) fixed = parsed;
if (fixed !== undefined) {
healed ??= { ...(args as Record<string, unknown>) };
healed[key] = fixed;
}
}
return healed ?? args;
}
+6 -3
View File
@@ -1,4 +1,5 @@
import type { GraphNode } from "./contracts/graph.ts";
import { stripModelValueDebris } from "./value-debris.ts";
/**
* Selection-facing chain views — pure + browser-safe (the shell's chain
@@ -63,17 +64,19 @@ export function chainsForSelection(
* not unique; an ambiguous name resolves to the FIRST node in store
* order). Returns null when nothing matches — the caller rejects.
* Validation set is the WHOLE graph, not the session's picks: pickless
* sessions and future graph-tools poses stay valid. */
* sessions and future graph-tools poses stay valid. The debris strip
* (whitehead, 2026-10-09) heals copy-syntax (`:node-id`, quoted ids)
* before matching — targets are membership values like correctAnswer. */
export function resolveTargetId(
target: string,
nodes: ReadonlyArray<Pick<GraphNode, "id" | "name">>,
): string | null {
const t = target.trim();
const t = stripModelValueDebris(target);
if (t.length === 0) return null;
if (nodes.some((n) => n.id === t)) return t;
const lower = t.toLowerCase();
for (const n of nodes) {
if (n.name.trim().toLowerCase() === lower) return n.id;
if (stripModelValueDebris(n.name).toLowerCase() === lower) return n.id;
}
return null;
}
+24 -12
View File
@@ -18,6 +18,7 @@
*/
import { Type, type Static } from "typebox";
import { stripModelValueDebris } from "./value-debris.ts";
// ─── shared types ───────────────────────────────────────────────────────────
@@ -115,7 +116,9 @@ export function quizInputProblems(input: {
}): string[] {
const { problems, keys } = optionProblems(input.options);
const entries = coerceCorrectAnswer(input.correctAnswer).map((v) => v.trim());
// Debris (colons/quotes/whitespace) strips inside coerceCorrectAnswer —
// the shared membership seam (boundary, display, grading).
const entries = coerceCorrectAnswer(input.correctAnswer);
const unique = [...new Set(entries)];
if (unique.length === 0 || unique.some((v) => v.length === 0)) {
problems.push("correctAnswer is required — provide the value (or label) of the correct option(s).");
@@ -207,10 +210,14 @@ export const QUIZ_QUESTION_TYPES = ["predict", "check"] as const;
export type QuizQuestionType = (typeof QUIZ_QUESTION_TYPES)[number];
/** Normalize a persisted quiz `type` (display seam): old records carry
* none, and model wobble degrades to undefined — callers render "Quiz"
* and the check-style card. Only exact literals count. */
* none, and unhealable wobble degrades to undefined — callers render
* "Quiz" and the check-style card. The debris strip (whitehead,
* 2026-10-09) heals the recorded `"\"predict\""` wobble — a raw
* persisted record displays its intended role. */
export function normalizeQuizType(raw: unknown): QuizQuestionType | undefined {
return raw === "predict" ? "predict" : raw === "check" ? "check" : undefined;
if (typeof raw !== "string") return undefined;
const v = stripModelValueDebris(raw);
return v === "predict" ? "predict" : v === "check" ? "check" : undefined;
}
/** The reveal law as a pure predicate (owner 2026-10-03): a teach-phase
@@ -228,10 +235,10 @@ export const quizInputObject = Type.Object({
...quizInputBase.properties,
type: Type.Union([Type.Literal("predict"), Type.Literal("check")], {
description:
"REQUIRED — the question's role in the teaching cycle. \"predict\": posed BEFORE you teach the step " +
"(its own turn; the learner answers, then your teaching resolves it — arrives open, never collapsed). " +
"\"check\": posed AFTER the step's teaching is complete (arrives collapsed behind the \"Check your " +
"understanding\" reveal). ONE question per turn either way.",
"REQUIRED — the question's role in the teaching cycle, as a bare word (no quotes): predict = posed BEFORE " +
"you teach the step (its own turn; the learner answers, then your teaching resolves it — arrives open, never " +
"collapsed); check = posed AFTER the step's teaching is complete (arrives collapsed behind the Check your " +
"understanding reveal). ONE question per turn either way.",
}),
nodeId: Type.String({
description:
@@ -563,19 +570,24 @@ export function shuffleOptions(options: QuizOption[], rand: () => number = Math.
// ─── quiz grading ───────────────────────────────────────────────────────────
/** The harness sometimes delivers a multi-select correctAnswer array as a
* JSON-stringified string because the schema union lists String first. */
* JSON-stringified string because the schema union lists String first.
* Every entry carries the MODEL-VALUE DEBRIS STRIP (whitehead,
* 2026-10-09): glm-5.3 copied option values with the JSON key-value
* colon glued on (`[":parmenides"]`); membership comparisons on ALL
* three seams (boundary check, display parse, grading) route through
* here, so the strip heals new calls AND already-poisoned records. */
export function coerceCorrectAnswer(correctAnswer: string | string[]): string[] {
if (Array.isArray(correctAnswer)) return correctAnswer;
if (Array.isArray(correctAnswer)) return correctAnswer.map(stripModelValueDebris);
const trimmed = correctAnswer.trim();
if (trimmed.startsWith("[") && trimmed.endsWith("]")) {
try {
const parsed = JSON.parse(trimmed);
if (Array.isArray(parsed)) return parsed.map((v) => String(v));
if (Array.isArray(parsed)) return parsed.map((v) => stripModelValueDebris(String(v)));
} catch {
// not valid JSON — treat as a single literal value
}
}
return [correctAnswer];
return [stripModelValueDebris(correctAnswer)];
}
/** Resolve author-supplied value(s) to 1-based display indices. Keying by
+8 -1
View File
@@ -31,6 +31,7 @@ import { Value } from "typebox/value";
import type { TSchema } from "typebox";
import { quizInputBase, quizDisplayInputSchema, askInputSchema, askOptionProblems, quizInputProblems, explainInputBase, explainInputProblems, type AskToolInput, type QuizToolInput, type ExplainToolInput } from "./question-core.ts";
import { restoreMathEscapes } from "./math-delims.ts";
import { healEncodedArguments } from "./argument-heal.ts";
import { probeDoneInputSchema, type ProbeDoneInput } from "./contracts/probe.ts";
/** The pi-boundary parse step, mirrored here (live-smoke fix, 2026-09-26):
@@ -89,7 +90,13 @@ export function parseQuizInput(input: unknown): QuizToolInput | undefined {
// dock renders/gates by it (normalizeQuizType at the read seam); old and
// probe records carry none. nodeId stays an unknown key Value.Clean
// strips, exactly as before.
const cleaned = boundaryParse<QuizToolInput>(quizDisplayInputSchema, restoreMathEscapesDeep(input));
// The display mirror of the boundary heal (whitehead, 2026-10-09): the
// record keeps RAW arguments, so the recorded `type: "\"predict\""`
// wobble must heal HERE too or a posed quiz renders invisible (the
// tool-parts law). healEncodedArguments is the same pure pass the
// boundary's withArgumentHeal runs.
const healedInput = healEncodedArguments(restoreMathEscapesDeep(input), quizDisplayInputSchema);
const cleaned = boundaryParse<QuizToolInput>(quizDisplayInputSchema, healedInput);
if (!cleaned) return undefined;
if (quizInputProblems(cleaned).length > 0) return undefined;
return cleaned;
+26
View File
@@ -0,0 +1,26 @@
/**
* MODEL-VALUE DEBRIS STRIP (owner, 2026-10-09 — the whitehead incident):
* glm-5.3 emitted `correctAnswer: [":parmenides"]` — the option value with
* the JSON key-value COLON glued on — across four rejected quiz calls
* (and `type: "\"predict\""` — the literal with QUOTES baked in — across a
* ten-retry death loop; see tools/heal.ts for that half). Membership
* fields (correctAnswer against option values, quiz type against the
* literal pair, practice targets against proposition ids) are copied by
* the model from its own context, and the copy picks up surrounding
* syntax: colons, quotes, backticks, whitespace.
*
* THE SEAM: strip at every MEMBERSHIP comparison — never on free text
* (statements, questions, explanations may legitimately carry any of
* these characters). Pure + browser-safe: shared by the call boundary
* (quizInputProblems), the display parse (tool-parts), and grading
* (question-dock) — the three consumers the tool-parts law keeps in
* lockstep. Idempotent by construction.
*/
/** Strip the copy-syntax debris a model glues onto a membership value:
* any run of surrounding whitespace, colons, double/single quotes, and
* backticks, in any interleaving (`":value"`, `: value`, `` `value` ``).
* The interior is untouched. */
export function stripModelValueDebris(value: string): string {
return value.replace(/^[\s:."'`]+/, "").replace(/[\s:."'`]+$/, "");
}