sorokin.oleg.edu/learnharness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 7 Wiki Activity

free-recall explain tool: the quizzing-mechanic switch (owner 2026-09-27)

Browse Source 
Sycophancy mitigation as a structural commitment: the model writes its
full correctAnswer (plus explanation, in that order) BEFORE the learner
sees the question; the commitment rides the persisted call args, is
revealed on submit, and grades his free text in the model's next reply —
the client never grades free recall.

- tools.explain setting (segmented 'Quizzing — Multiple choice | Free
  recall' in Agent settings' Tools group): free arms explain and drops
  quiz for BOTH agents (prober variant carries no nodeId, same pattern
  as the probe quiz); mechanics keys never pass filterEnabledTools —
  quiz stays unwritable (the 2026-09-19 wedge ruling stands)
- doctrine lives pi-natively in the tool descriptions (owner ruling —
  no conditional system-prompt chunks): explain carries the commitment
  rule; the MC option-building craft moved out of the prompt into the
  quiz description; shared/20-question-tools.md slims to the
  quiz-agnostic core
- display seams: parseExplainInput (LaTeX-debris restoration included),
  dock text-mode with an 'I don't know' row (kbd row 0), AnsweredCard
  reveal (The answer / Why, no verdict chips), [Explain answer] format
- phases/plan seams recognize both quizzing tools' nodeId tags; explain
  answers deliberately feed no ledger evidence (no client grading to
  key on — a future ruling can parse the model's grade)

1049/1049 tests, typecheck/lint/build green.
This commit is contained in:
amosblomqvist
2026-09-27 08:55:13 +03:00
parent 07d2c84461
commit 2f39766528
25 changed files with 784 additions and 68 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/tool-toggles-bridge.test.ts
+7 -1
View File
@@ -81,10 +81,16 @@ describe("parseAgentSettingsBody: tool keys validate at write time", () => {
assert.deepEqual(r.ok && r.settings.tools, { research: false, pdf: false });
});
it("the quizzing-mechanics key (explain) is writable (owner 2026-09-27)", () => {
const r = parse({ explain: true });
assert.equal(r.ok, true);
assert.deepEqual(r.ok && r.settings.tools, { explain: true });
});
it("unknown keys are rejected with the supported list (the quiz wedge never persists)", () => {
const r = parse({ quiz: false });
assert.equal(r.ok, false);
assert.match(!r.ok && r.error, /unknown tool "quiz" \(supported: research, demo, visual, pdf\)/);
assert.match(!r.ok && r.error, /unknown tool "quiz" \(supported: research, demo, visual, pdf, explain\)/);
const typo = parse({ reasearch: false });
assert.equal(typo.ok, false);
assert.match(!typo.ok && typo.error, /unknown tool "reasearch"/);