sorokin.oleg.edu/learnharness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 7 Wiki Activity

free-recall explain tool: the quizzing-mechanic switch (owner 2026-09-27)

Browse Source 
Sycophancy mitigation as a structural commitment: the model writes its
full correctAnswer (plus explanation, in that order) BEFORE the learner
sees the question; the commitment rides the persisted call args, is
revealed on submit, and grades his free text in the model's next reply —
the client never grades free recall.

- tools.explain setting (segmented 'Quizzing — Multiple choice | Free
  recall' in Agent settings' Tools group): free arms explain and drops
  quiz for BOTH agents (prober variant carries no nodeId, same pattern
  as the probe quiz); mechanics keys never pass filterEnabledTools —
  quiz stays unwritable (the 2026-09-19 wedge ruling stands)
- doctrine lives pi-natively in the tool descriptions (owner ruling —
  no conditional system-prompt chunks): explain carries the commitment
  rule; the MC option-building craft moved out of the prompt into the
  quiz description; shared/20-question-tools.md slims to the
  quiz-agnostic core
- display seams: parseExplainInput (LaTeX-debris restoration included),
  dock text-mode with an 'I don't know' row (kbd row 0), AnsweredCard
  reveal (The answer / Why, no verdict chips), [Explain answer] format
- phases/plan seams recognize both quizzing tools' nodeId tags; explain
  answers deliberately feed no ledger evidence (no client grading to
  key on — a future ruling can parse the model's grade)

1049/1049 tests, typecheck/lint/build green.
This commit is contained in:
amosblomqvist
2026-09-27 08:55:13 +03:00
parent 07d2c84461
commit 2f39766528
25 changed files with 784 additions and 68 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/agent/prompts/prober/10-probe-doctrine.md
+2 -2
View File
@@ -1,6 +1,6 @@
# The probe
Your whole job is two measurements: **what he needs to learn** (his goal — `ask_user_question`) and **what he already knows** (a detailed map of his exact understanding — `quiz`). The whole lesson will be planned from your map — it must be precise.
Your whole job is two measurements: **what he needs to learn** (his goal — `ask_user_question`) and **what he already knows** (a detailed map of his exact understanding — the quizzing tool: `quiz` or `explain`, whichever the learner’s mechanic armed). The whole lesson will be planned from your map — it must be precise.
Work strictly one question at a time: ask, wait for the answer, then choose the next question from what it told you.
@@ -16,7 +16,7 @@ Work strictly one question at a time: ask, wait for the answer, then choose the
From the goal, what will the lesson depend on? To teach him this, where would the teaching start, and what must it build on? Each dependency is a strand — bound by relevance, nothing more.
## 4. Binary-search every strand — `quiz`
## 4. Binary-search every strand — the quizzing tool
The edge of a strand is found by bracketing: something he reliably gets **right** (a floor) and something he gets **wrong** or doesn't know (a ceiling). Narrow the bracket one question at a time: