Commit Graph
4 Commits
Author SHA1 Message Date
amosblomqvist 84f93999c5 fix: normalize model-visible images with Pi processing 2026-10-28 18:16:04 +02:00
amosblomqvist b87a6925af Wave A: pi-native tool layer — delete LhTool/toPiTool, TypeBox everywhere, terminate on questions
- src/agent/lh-tool.ts deleted: the zod-wrapped LhTool + toPiTool bridge
  (z.toJSONSchema per call, clientQuestion flag, hand-rolled result mapping)
  is gone. Tools are pi-agent-core native AgentTools now.
- New src/agent/tool-execute.ts owns what the wrapper added: the abort guard
  (runWithAbortGuard: run signal + 30s per-call timeout + 250ms grace,
  'Operation aborted' / timeout wording preserved), ToolInputError (the
  retryable '- problem at path' format), result mapping (string verbatim /
  object JSON / image parts, details rides the record's tool part output),
  and questionResult() — placeholder + terminate:true (owner Option A:
  belt of terminate + slim makeQuestionStopper with the clean-question
  rule; maxTurns deleted from the teaching turn entirely per owner).
- All 11 tool files + tools.ts registries converted to TypeBox parameters
  (typebox@1.3.7, pinned to pi-agent-core's copy). zod refinements
  (quizInputProblems, askOptionProblems, live-nodeId) moved from schemas
  into execute — the model sees the same retryable error shape.
- question-core + contracts/probe TypeBox-authored (single source shared
  with the UI); tool-parts parse seam runs the boundary's full chain
  (Convert → Check → Clean → refinement), fixing the live-smoke lockout:
  pi's boundary Value.Converts correctAnswer: 17 → '17' but the record
  keeps raw args, so a parse without Convert rejected a posed quiz
  (invisible pending question, turn ended). Pinned with the real smoke
  emission shape.
- Subagents converted: demo-builder/researcher AgentTool-native,
  visual-maker/pdf-reader declaration-only pi-ai Tools with local execute
  fns, extraction submit schemas re-authored in TypeBox (owner option 2:
  zod defaults → Type.Optional + fill at the submit seam).
- Accepted delta, flagged: TypeBox-native schemas skip pi's JSON-Schema
  primitive coercion fast path... (boundary still Value.Converts), and
  prober-quiz unknown keys ride through (Value.Clean strips at display).
- Tests: 988 passing (was 967 at the recon baseline); tool-abort and
  read-tool rewritten against the native seams; question-core/question-
  tools/tool-parts/demos/web-search/graph-tools/plan-tree/probe-done/
  tool-toggles converted to Value.Check/execute pins.

Live smoke (PORT=3004, real model): native ls turn streams end-to-end; a
real quiz pose (first attempt missing correctAnswer → tool-input-error →
model retry → posed) terminates the turn cleanly and persists pending.
2026-09-25 23:32:13 +03:00
amosblomqvist 7a3e19e95f feat(sources v1): the agent locates files itself — ls/find/grep + tool-call cards
User pivot (2026-09-19): no attachments UI at all. Pasting paths is native
for technical users and alien for everyone else - so the agent gets the
tools to find what the learner merely NAMES. The learner communicates in
messages; the agent locates and reads. Zero cross-stream seams: paths
already live in tool parts, so the sources "list" is the transcript.

- ls/find/grep: pi-verbatim schemas, descriptions, and output formats
  (fd's glob semantics, rg's path:line: match format, pi's truncation
  notices), ported to pure Node - no binaries, no shell-out, read-only by
  construction. Deltas from pi: home-anchored resolution (the harness has
  no cwd), .gitignore replaced by hard noise exclusions
  (node_modules/.git/Library/.Trash), 10s walk timeout with partial results.
- read: description swapped to pi-verbatim (doctrine moved to the prompt);
  relative paths now resolve against the learner's home like the rest.
- prompts/shared/05-learner-files.md: one minimal doctrine fragment
  composed into BOTH agents (composeInstructions grew a shared/ dir,
  before the agent's own fragments). teacher/30-jobs.md keeps only the
  pdf-dispatch addendum.
- ToolCallCard: the generic quiet tool-call card the user specced -
  collapsed one-liner (tool name + headline argument), expandable to full
  input/output, errors red, image payloads never render. Registered for
  read/ls/find/grep.
- ui-message.ts: TeacherUITools entries for the four tools (mechanical).
- tests: fs-tools suite (glob semantics, pi formats, exclusions, limits),
  registry/card renders, shared-fragment composition pins, tool-registry
  key updates, read re-anchor.
2026-09-19 16:50:20 +03:00
amosblomqvist 9a90dd51d9 feat(T2b): sources v0 — read tool + pdf-reader sub-agent
Learner source files, on the teacher's own terms (nothing auto-injects):

- read tool (src/agent/tools/read.ts): pi-read port — path/offset/limit,
  2000-line/50KB truncation with use-offset continuation notices, image
  files (jpg/png/gif/webp/bmp) returned as native image content parts
  (new LhToolImageResult escape hatch in lh-tool.ts). Always-on in both
  the prober and teacher registries (user-decided 2026-09).
- pdf-reader sub-agent (JobKind 'pdf', toggleable, ack-and-steer): triage
  via pdf_info (pdfjs text layer: per-page text/image/math-density),
  targeted pdf_extract, regex pdf_search (hits count as consultation),
  and pdf_render — pages painted by pdfjs onto a canvas shim in headless
  Chrome (the pdfium viewer doesn't paint headless), screenshot fed back
  image-in-the-loop (visual-maker idiom, providerSafePng applied). The
  unread-finish gate bounces a zero-consultation brief once. Teacher gets
  a text brief with page citations only; renders never leave the loop.
- MODELS.pdfReader = qwen/qwen3.8-flash (user override 2026-09-19: the
  look-at-renders workload wants the vision-proven model; catalog listed
  glm-5.3-flash as image-capable but qwen is the known-good daily driver).
- UI: PdfCompleteBadge + PdfBriefCard (steer-card, job-card), dispatch
  card + activity-dock kind entries (type-forced, flagged for UI review).
- Prompts: 'Learner source files' section in teacher/30-jobs.md; read
  line in prober/10-probe-doctrine.md. Doctrine rides tool descriptions
  per the self-contained-tools principle.
2026-09-19 15:24:26 +03:00