bab0d3f94d6a4e3900bdd8d91a3d5902250250d9
69
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 amosblomqvist
|
0b2e4e03fb |
sessions: export as markdown — the sidebar right-click, the route, the practice join (release polish 3)
Owner round (2026-10-06): markdown export of sessions, actioned from
the sidebar context menu — "Export as Markdown…" sits with the
management actions, above Delete (the destructive action stays last).
- GET /api/chats/:id/markdown (404 unknown): the whole session as one
attachment — text/markdown; charset=utf-8, filename from the slugified
title ("relational-databases-joins-normal-forms.md", non-ascii titles
fall back to "session.md"). Both session kinds export.
- The client is a same-origin anchor download; the route's
Content-Disposition names the file.
- sessionMarkdown's user-facing debut — minted ANCHORLESS (message-id
anchors are extraction provenance, not learner-facing). The engine
joins the core barrel (pure, browser-safe) for its first consumer.
The practice join (owner-approved mid-round): practice tool results are
bare acks — statement, outcome, verdict, and the model solution all
live in the assistant's call arguments. The engine now folds them the
same way the view seam folds cards: Problem blocks render the pose's
statement (solution never leaks at pose time); Concluded blocks render
the outcome as prose ("Solved with hints.") + verdict, then the
pose-committed model solution as a blockquoted reveal via
problemInputForConclude (clean-poses-only, its refusal law). A
dismissed problem concludes as "The learner skipped this problem." —
no reveal. Errored poses render nothing (the failed-tool law).
1427 tests (route pins: headers, slug, anchorless body, 404; engine
pins: pose-time statement, the reveal, the dismissal refusal, the
errored-pose silence); tsc, lint, build green.
|
||
|
amosblomqvist
|
a2db80b72d |
compaction: the usage-anchored trigger + visible session state (round 15)
The relational-databases incident, diagnosed and fixed: TRIGGER — pi's estimator law, adopted exactly: anchor on the last assistant message's provider-reported usage (totalTokens || the field sum) and chars/4-estimate ONLY what was appended after it. The anchor scan skips aborted/zero-usage assistants and STOPS at the newest compaction marker (pre-cut usage described context the next request will not carry — pi's distrust rule). The old estimator (JSON.stringify of every message /4) tripled the true context (measured: 84,412 'tokens' estimated vs 38,799 provider-reported on the incident record) — scaffolding, escaped LaTeX, thinking blocks and app-side details all counted. estimateMessageTokens now counts wire text only (text + thinking + toolCall name/args; content text for user/toolResult/custom). The unanchored fallback runs the plain heuristic over the CUT-HONORED projection plus the additive prompt term; the anchored path adds nothing (usage already billed prompt and tools). OBSERVABILITY — compaction is a visible session-wide state (owner spec): the transcript-bottom placeholder register gains COMPACTING CONTEXT… (BrailleSpinner + Shimmer, the THINKING placeholder's twin, mutually exclusive by construction — the gate precedes beginTurn), and the sidebar row shows 'Compacting context · <ago>' with the spinner (sessionMeta outranks busy; compacting-changed joins turn-started/turn-settled as refresh triggers). Liveness rides the API like turn-busy: ChatListItem.compacting + compactionInFlight on both GET /api/chats and GET /api/chats/[id] (initiallyCompacting covers the navigate-back-mid-compaction mount — the event already fired), exported through the server barrel. Also: the relational-databases record rolled back to pre-compaction (marker, post-marker turns, tree reset; backup in /tmp) for a clean retest. The summarizer-usage 'gap' was a misread — steps[] was populated all along. |
||
|
amosblomqvist
|
ffc4df1938 |
compaction: the surface — settings row, two ruled dividers, summary dialog, send gate (round 14 slice 3)
- settings: compaction.thresholdTokens row on the Agent tab (owner pick)
in a Context group — CommitInput mirroring the graph budget row, hint
resolved through the SAME core resolver the trigger reads; PUT rejects
below the 32k floor with the anti-churn rationale; own section
(compaction) in SETTINGS_SECTIONS + the settings-signal union
- view fold: one marker → two dividers — the minimal MOMENT divider at
the marker (retrying-message register, no rule lines) and the longer
CUT divider emitted immediately before the first kept message
(phase-divider geometry, chip carries '· Nk tok'); both carry the full
summary payload; full history stays visible to the user
- components/compaction-divider.tsx: the dividers + summary dialog
(pure exported body for SSR pins, the delete-dialog pattern; Esc +
Close dismiss, backdrop does not)
- chat.tsx: compaction-changed SSE flips the running state — composer
canSend, onSend/doSend/doSteerOrSend, and both docks' submit + hint
('compacting session…') gate on it (the server's registry mutex is the
structural gate; this is the affordance); dialog mounts beside the
abort dialog
- pins: settings-body (4), view fold (2), divider SSR (4)
|
||
|
amosblomqvist
|
85022ae143 |
practice hardening: weak-model argument heal + parked dismissals mid-turn
The GLM incident (2026-10-04): glm-5.3 sent mode as a JSON-string-encoded
enum ('"recall"') on 19 consecutive problem calls; TypeBox's error never
named the expected values, so the model re-emitted the identical payload
until the dev server was killed. And mid-turn the learner was trapped:
Esc's question/problem rungs only dismissed — the dismiss route 409'd
against the turn-owned record and nothing could stop the running turn.
- healEncodedArguments (practice.ts, the prepareArguments seam): a STRING
where the schema expects a literal union / boolean / null / number /
array is JSON.parse-d and kept when it satisfies the schema. Valid args
pass through untouched (same reference); unhealable values stay for
validation to reject honestly. Wired on BOTH problem and conclude.
- dismissal-park.ts: the dismiss route parks a mid-turn dismissal (202);
the turn's settle path (onFinish, after persistTurn, before endTurn —
the same ordering law as the record write itself) applies it against
the persisted record, only if the dismissed thing really is pending.
- confirmAbort: question/problem discard while busy now ALSO aborts the
in-flight turn; the dialog line says so (busy prop).
No error-result cap (owner ruling). 16 new pins (heal incl. pi-ai
validateToolArguments end-to-end; park registry, apply paths, the 202
route contract).
|
||
|
amosblomqvist
|
90893ae236 |
graph context: a separate marker message before the first user turn
The map picks no longer glue into the opening prompt as seed text
(owner, 2026-10-04 late). POST /api/chats accepts graphContext
{propositions}; the route resolves the picks against the live graph,
derives the reasoning chains among them (chainsForSelection →
formatChainLine), and writes a graph-context custom marker as
message[0] — before the learner's first user message, in either mode.
- Core: newGraphContextMessage (model-facing content = the
contextualization framing + proposition lines + chain lines;
details = the structured pick); toLlmMessages flattens it to a user
message like every marker, so the model reads it for free.
- View: projects to a user-side data-graph-context part.
- UI: GraphContextCard — minimal, muted, dashed, collapsed by default
(counts only); expand reveals name — statement rows and the chain
lines. Same compact-expandable pattern as the practice cards.
- Home: sends graphContext with the creation whenever picks exist;
the prompt param carries the learner's own words only.
|
||
|
amosblomqvist
|
7dcb006cf3 |
session kind: ChatRecord.kind teach|practice (absent = teach)
Practice sessions (owner, 2026-10-04): the marker rides the record and
the normalized summary (absent field reads as teach — pre-practice
records never migrate); POST /api/chats accepts {kind: 'practice'}.
The agent branch, tools, and UI land in the following slices.
|
||
|
amosblomqvist
|
814c2ad412 |
delete the batch practice-problems module (superseded by practice sessions)
Owner ruling 2026-10-04: the module is a sunk cost, retrievable from history if ever revisited. Removed: core practice contracts + pipeline helpers, the PracticeStore port + adapter + data dir path, server service + API routes (sets, clusters picker, practice aborts), the practice pages + dialog/exam/list UI, chain-table's Generate handoff, sidebar nav, activity-dock practice rows, and the partitioner/ problemGenerator/grader model roles (BACKGROUND_MODEL_ROLES 8 → 5). Survivors, moved to core/chains.ts (pure + browser-safe): ReasoningChain, chainsForSelection, formatChainLine — the chain table and graph emission still consume them. |
||
|
amosblomqvist
|
5271ca0e4a |
composer steering: the general steering machinery (owner arc 2026-10-03)
STEERING, NOT FOLLOW-UPS — pi-agent-core's two rails, used deliberately:
agent.steer() injects at the next response boundary INSIDE the live run
(one logical turn, the model changes course); followUp() would only extend
an ended run — not built. The same rail job markers already ride
(steeringMode 'all', the 2026-09-26 doctrine).
- POST /api/chats/[id]/steer: 404/400 honest guards; no live turn → 409
and the client silently falls back to a plain text send (the settle
race); live → steerLiveChat hands the message to the live agent's
steering queue AND parks it in the userSteers registry — the stranding
safety net. A steer stranded behind a question termination or abort
stays parked and rides the chat's NEXT run (handOffEligible), the same
deliver-or-requeue lifecycle as job markers. Reconciliation at settle:
delivered ids drop (reconcileUserSteers).
- Client: steer(text) queues + POSTs without flipping any turn state;
the drain's verbatim message events upsert the real user card by the
client-minted id (steer_<localid>) and the indicator fold drops the
line. canSend no longer gates on turnInFlight — Enter mid-turn steers;
Enter with an empty composer still stops; the square button stays Stop.
- The queued-steer indicator (owner spec): quiet one-liners above the
composer shell — right-aligned to where its corner rounding starts,
truncated with an ellipsis, fixed max width, ~half opacity, no
background, no container. Tunable later.
- Verified from pi-agent-core source: steering drains per completed turn
inside the loop; injected messages emit message_start/end verbatim;
finishTurn {action:'end'} (our question stopper) exits BEFORE the
steering poll — hence the registry.
|
||
|
amosblomqvist
|
673a430d6e |
question dismissal, A2: the silent write + the dismissal pause (owner design 2026-10-03)
- {type:"dismiss"} input event: parse + pure appends with the answer-event
orphan discipline (dangling-call 400, kind match, answer must be dismissed)
- POST /api/chats/[id]/dismiss: the no-turn composition — write-ahead append,
409 while a turn runs, emits record-changed (new bus word; idle views sync)
- needsContinuation: an all-dismissed trailing batch demands nothing;
endsInDismissal = the derived pause predicate (tail-based, no latch)
- wake pipeline: while the record ends in a dismissal the pump holds every
delivery in the steer queue — the user's next send is the wake (until the
text-turn steer claim lands, settlement re-pump delivers right after it)
- session-core.dismissQuestion: optimistic skipped card + silent POST;
chat.tsx confirmAbort question-mode swaps submitAnswer for it; ESC-abort
popup everywhere (AbortTurnDialog) with popup-Esc claimed via state gate
- parallel question calls dismiss call-by-call (results bind by id)
|
||
|
amosblomqvist
|
26131b3599 | push architecture slice 7: graph-status + practice surfaces ride the events channel; coverage polls accepted-by-design; stale poll comments + architecture.md observability section rewritten to the push model | ||
|
amosblomqvist
|
78c7393909 | push architecture slice 1: core event bus + /api/events SSE channel (notify->fetch infra, no consumers yet) | ||
|
amosblomqvist
|
e7f4923159 |
deletion screen minimal + 'node' as THE UI word (owner ruling)
Owner 2026-09-30: the confirm dialog's pre-flight impact read had to go — what full death takes is obvious, the per-open fetch flickered, and richer facts would only confuse as deletion semantics evolve (rewiring is on the table). The screen is now: Delete node? / Delete N nodes?, the names in the mono block, one permanence line, Cancel / Delete. No loading state, no gone state (a 404 mid-run lands in the honest failure line), no sessions join — the joins prop died too. The dead path is deleted end to end: GET /api/graph/nodes/[id], graphNodeImpact (server), nodeDeletionImpact (core) and their tests. WORDING: node/node(s) is THE UI word for graph objects, standardized across every surface's copy (owner, same ruling): the dialog titles, the chain table's counts/search/summary/delete aria-labels, the practice dialog's cluster counts, selection line, and empty-cluster hint. Internal identifiers (selection.propositions) unchanged. Rewritten dialog pins (tests/delete-node-dialog.test.tsx): minimal contract — no facts/loading/fetch copy, singular/plural titles, the cap, 409 dock phrasing, busy states, no 'proposition' left in the dialog. |
||
|
amosblomqvist
|
06032769d2 |
practice: the learner's creation note — optional free-form input carried to partitioner + generators (owner, 2026-09-29)
- dialog: optional 'Anything else?' textarea (typing-only, 2000-char mirror) - API: body gains note? — trimmed, whitespace-only omitted, hard 400 over 2000 - pipeline: note persisted on the set and threaded to both agent stages - agents: THE LEARNER'S NOTE prompt block (present only when set) + the weighed-never-owed stance pinned in both doctrines - fix en route: doubled THE PARAMETERS heading in both prompts since slice 3 (renderParams owns the heading; the literal blocks removed) |
||
|
amosblomqvist
|
cf0ec4a2f0 | practice UI round — top bars + sidebar reopen, one-attempt per-problem submit with instant reveal, score-only grading (owner 2026-09-29) | ||
|
amosblomqvist
|
88eb4b2ec0 |
practice restructure — per-problem jobs + ONE shared background-LLM pool (owner ruling 2026-09-29)
Rulings: each problem generation is its own visible dock job (the extraction per-chunk pattern), and practice shares extraction's semaphore — one process-wide 3-permit pool for ALL background LLM jobs; total background LLM concurrency bounded at 3. A. SHARED POOL: new src/agent/subagents/bg-llm-semaphore.ts — globalThis __lhLlmSlots, PERMIT_COUNT 3 (the ruling), the fresh-take/handed-off-permit deadlock discipline preserved for BOTH callers. extraction.ts's withExtractSlot is now a thin wrapper (job-row signal → withBgLlmSlot); __lhExtractSlots and __lhPracticeSlots both die. The partitioner now acquires a permit too (it's a background LLM job — it didn't before). B. REGISTRY ROWS: the aggregate setId:generation / setId:grading rows are replaced by setId:partition (phase partitioning, label 'Partitioning — title'), setId:gen:problemId (queued→generating, 'Problem i/N — title'), setId:grade:problemId (queued→grading, 'Grading i/N — title'). Each row carries its OWN abort controller and its OWN usage (one PracticeStepLog per row — the aggregate partitioner/generator/grader folding dies). Row key = feed id = abort key (the dock dup-key law). C. ABORT SEMANTICS: abortPracticeRow(key) routes by shape — gen row: THAT problem only (error 'Aborted', siblings continue, set settles ready with one errored problem); partition row: the whole run; grade row: that grade drops (answer kept, re-submittable). abortPractice(setId) composes from row aborts + the synchronous persisted marks (a set-abort's verdict stands before the settles re-read it — the settle step keeps an existing error status). /api/activity/abort kind practice now takes the ROW key (charset admits ':'). D. FEED + DOCK: ActivityPractice/FeedPractice carry per-row identity (id = row key, problemIndex/problemTotal); dock blobs render per row, labels carry the i/N; the aggregate row is gone. E. Boot sweep unchanged at set grain; the live-row check is now a key-prefix scan (any live row of the set protects it). The updatedAt heartbeat covers the new row scheme as-is. Tests (20 in the pipeline suite): per-problem row shape/uniqueness/ i-N labels (partition row frozen behind a gated partitioner; per-row usage ISOLATION pin — b's log never shows on c's row); abort-one-gen (sibling completes, set ready with one errored problem); partition- row abort; queued grade-row abort (dead-ticket path: answer kept, re-submittable, sibling lands); shared-pool identity + contention (practice queues behind extraction-style holders); registry/dock feed mapping. Test isolation hardened: unique per-store set ids, afterEach registry/lock/pool reset (a timed-out test can no longer poison the rest). architecture.md: jobs-pattern paragraph rewritten (per-problem rows, shared pool, abort routing). 1149/1149, tsc 0, lint 0, build 0. |
||
|
amosblomqvist
|
f39e1b4294 |
practice fix — creation is graph-domain: cluster picker, nodeIds-only API (owner ruling 2026-09-29)
Owner ruling: 'problem sets always take in nodes — graph objects' —
practice is graph-domain, not chat-domain. No session-created sets.
API: POST /api/practice/sets takes {nodeIds, params, title?} ONLY —
the chatId branch, UnknownChatError, and the any-provenance filter are
deleted; the selector is nodeIds (readNodes filtered to the selection,
unknown ids drop, empty live intersection → 400 empty_selection);
title optional from the UI (the picker's cluster label), dated default
stays. chatStore leaves the pipeline deps entirely (labels resolve
store-side in the clusters service).
GET /api/graph/clusters (new): the birth-session groups the graph view
renders — graphSnapshot's clusters verbatim (same ordering, same
MIN_CLUSTER_SIZE presentation filter), mapped to {key, kind
chat|notes, label (null when the naming record is gone — honest,
never guessed), nodeIds, nodeCount}; corrupt/empty graph → empty
list. GraphClusterGroup contract in core/contracts/graph.ts (client-
safe — the dialog imports it).
practice-dialog: session picker → cluster picker (rows show label +
proposition count, notes clusters marked); Create posts the cluster's
nodeIds + label as title + params. 400 empty-selection copy reworded
for the vanished-nodes edge ('Nothing practicable is left in that
cluster — the graph changed.'); 503 deep link unchanged. Preselected
mode gone — the dialog is the only creation entry for now (the graph-
view cluster entry comes later with free-form selection).
Sidebar: session right-click 'Practice this session's propositions'
removed (item + state + dialog wiring); the /practice nav row stays.
Tests: pipeline — chatId/any-provenance tests die, nodeIds selection
pins added (unknown-id filtering, dated-default title, title hint
rides, empty/vanished → EmptySelectionError); render — cluster picker
pins (label+count rows, notes kind, unlabeled cluster muted, empty
graph line), preselected pins removed; 16 render + 13 pipeline.
Docs: architecture.md §7 API+UI paragraph rewritten (graph-domain
ruling, /api/graph/clusters, cluster picker as the only entry, no
404-unknown-chat); README creation-flow line updated.
1130/1130, tsc 0, lint 0, build 0.
|
||
|
amosblomqvist
|
27c5d9df19 |
practice slice 4 — pipeline, registry, API, feed
src/agent/subagents/practice-pipeline.ts (sibling of practice.ts —
orchestration vs calls): createPracticePipeline(graphStore+practiceStore
+chatStore+PracticeModels). Fire-and-forget front doors returning
{set, done}: createProblemSet (refuse → resolve selection (chatId =
ANY-provenance membership per brief; nodeIds raw) → chains → snapshot →
createSet partitioning → registry row → runPartitioner → stubs land
(generating) → ONE generator per problem behind the process-wide
3-permit abort-aware semaphore (__lhPracticeSlots, extraction idiom) →
per-problem failure isolation → ready (errored problems included);
abortPractice (idempotent: controllers + 'Aborted' marks; dropped
grades keep answers, re-submittable); submitAnswers (ready-only,
already-graded rejects, one grading job per answer under the same
semaphore); deleteProblemSet (abort-first). Per-set write mutex on
globalThis (__lhPracticeLocks, audit-lock idiom); registry
__lhPracticeRuns one row per set per kind; boot sweep marks orphaned
partitioning/generating sets 'Interrupted (server restarted)'. Typed
errors: EmptySelection/UnknownChat/SetNotFound/SetNotReady/BadAnswers.
Usage folds stage logs into the registry row (v1: completion-folded).
src/server/practice.ts: thin service — strict params parsing, typed
error → response mapping (503 envelope family, 400/404/409), list/get.
Routes: POST+GET /api/practice/sets (202 {id}), GET+DELETE
/api/practice/sets/[id], POST /api/practice/sets/[id]/answers (202).
Composition root: chat-handler wires createPracticePipeline (engineFor
fresh + providerKeyMissing gate) exported as ; barrel re-
exports. Feed: observability activityFeed() += practice rows
(ActivityPractice); dock blobs (process 'practice', GraduationCap
icon, set-keyed abort); /api/activity/abort kind 'practice'.
tests/practice-pipeline.test.ts — 13 tests: end-to-end (stubs mid-
flight via gated generators, coverage complete), failure isolation,
3-permit semaphore pin, abort mid-generation, registry shape pins,
refusal-before-create, chatId any-provenance selection, typed errors,
answers→grades land + graded rejects, boot sweep (orphan + live-row),
delete. 1114/1114, tsc 0, lint 0, build 0.
|
||
|
amosblomqvist
|
4001c3465b |
Graph view: unique pill keys for chunk jobs (second duplicate-key consumer)
The graph page's pipeline pills keyed rows by run.chatId — the source id, shared by every chunk job of a fan-out (the dock got the same fix in 8f66f47; this consumer was missed). /api/graph/status rows now carry an id (the registry job key: source id, or 'src#i'), and the pills key by it. chatId stays the SOURCE id — the reveal refetch and per-run cost refresh track chatId SETS, which dedupe naturally and fire when the source's LAST row leaves the feed. |
||
|
amosblomqvist
|
d047e7b2dc |
Per-chunk jobs: parallel extracts under a 3-slot semaphore, serialized audits (owner 2026-09-28)
Every notes chunk and chat coverage batch is its own job — registry row,
usage record, failure domain. Extracts fan out process-wide behind a
3-permit extract semaphore on globalThis (new 'queued' phase); audits and
merges stay serialized behind the existing FIFO gate ('waiting'), so each
auditor explores the graph after every earlier job's merge.
- ExtractionPhase gains 'queued'; RunningExtraction gains part {i,n};
the feed shows one row per job with a '· i/N' suffix (status route)
- abortExtraction is source-wide: every job of a fan-out dies together;
merged jobs keep their coverage checkpoints
- failure isolation at the fan-out (Promise.allSettled + aggregation):
a failed chunk no longer kills its siblings — the run returns what
actually merged; job-level aborts log, pre-fan-out errors still throw
- usage ledgers: one record per job (source-keyed), failed jobs bill
their partial spend
- runJob/aggregateJobs helpers; setPhase/recordStep/extract/audit/
notesExtract key the registry by jobKey (src#i) — notesExtract's
signal lookup was the one real bug the new tests caught (source-id
lookup meant fan-out aborts never reached the model call)
- semaphore is handoff-correct (fresh takes increment; handed-off
permits don't double-count — an inflated counter deadlocks at 3)
- tests: isolation pins rewritten (stage failure now resolves zeros),
single-flight/registry tails updated, per-job usage counts, and a new
describe pinning the fan-out: 3 extracting + 5 queued, parts 1..8,
source-wide abort, per-job usage on failure, extract peak == 3,
audit peak == 1
- docs/architecture.md: serialized-audit paragraph rewritten for
per-chunk jobs; two-front-doors wording aligned
|
||
|
amosblomqvist
|
3e9a85d60f |
per-session models: every session owns its model + thinking level from birth
The reported bug (2026-09-28): every chat ran glm-5.3-flash regardless of any pick, while composer and settings UIs agreed with each other and lied about the run. Root cause: a fossil default-settings record (settings.json chat section, frozen pre-slots-wave) was stamped verbatim into every new chat at birth by the new-session flow, and resolveConversationSpec prefers the per-chat record over the strong slot forever. Owner redesign — models are per session, always; the strong slot is the new-session default: - POST /api/chats stamps the newborn verbatim from the strong slot (model + thinking level); no client settings PUT on create anymore - the composer control is dual-bound: in-session it reads/writes the session's own record (GET /api/chats/:id, PUT .../settings); on the draft surfaces it binds the strong slot (picking there = changing the default) - thinking levels follow the identical mechanics (confirmed by owner) - the zombie default-settings system dies whole: useDefaultSettings, /api/settings/defaults, src/store/default-settings.ts, the chat section of UserSettings (sanitizer strips fossil copies), the legacy default-settings.json fold, and DEFAULT_SETTINGS_FILE - catalogDefaultsFor collapses to the strong slot alone - thinking display for a legacy unstamped session resolves exactly like the engine (session ?? strong ?? model ladder) Tests: new chat-birth-stamp suite pins the stamp, the unconfigured boot, the unset level, and the copy-not-live-view regression; defaults pins rewritten strong-only; legacy-migration suite deleted. |
||
|
amosblomqvist
|
f3003d357a |
Wave D: native AgentEvent wire + client session rewrite (SDK purged)
Server: ui-stream.ts is verbatim pi AgentEvent passthrough (SSE data lines + [DONE]); only app-level events are the loop-level error and turn-settled terminal; onError carries raw provider text (owner ruling). Client: useChat + the AI-SDK transport replaced by components/session-core.ts (SessionModel: 409 identity-retry, optimistic c_ appends, absorbed() adoption, needsContinuation, shared consume for POST + reattach) and the session.ts hook; the golden-master grouping canon moved client-side into core/view.ts (projectView); scanners read the native record (phaseAt3Lh, danglingQuestionOf, graph toolParts, plan-blocks, runner, extraction). Coverage: the native message is the unit (fork 3) with the call+result inseparability bond rule; view-line-less messages (markers, error results) are invisible to coverage and never checkpointed. Deletes: chat-transport, projection, ui-message, their tests (ported to tests/session-core.test.ts), and the ai + @ai-sdk/react dependencies — the Vercel SDK is fully out of the tree. Wire pins in chat-flow rewritten onto the AgentEvent dialect (eventsOf parser). Gates: tsc clean, 994/994 tests, eslint clean, build green; live smoke on PORT=3004 (verbatim wire → native record). |
||
|
amosblomqvist
|
7ab47d4fcb |
Wave B: the native message record — input events, projection, adapter death
The chat record is now pi-native agent history (LhMessage[]), the AI SDK's
UIMessage transcript contract is gone from the server, and every client turn
arrives as ONE explicit input event.
Core (new):
- src/core/native-messages.ts — the LhMessage taxonomy (user/assistant/
toolResult + lhMarker customs: verdict, probe-handoff, subagent-steer,
subagent-abort, plan-block), constructors, and toLlmMessages — the single
model-facing seam (convertToLlm): strips ids, flattens markers to user
text, drops UI-only plan-blocks, synthesizes INTERRUPTED results for
dangling aborted calls, never throws.
- src/core/projection.ts — projectNative: the one-way native → UIMessage
projection (consecutive assistant+toolResult runs group into one
assistant view message; toolResults attach backward by toolCallId;
markers project to data parts; plan-blocks splice after the last
successful plan_init). Every existing consumer reads through it.
- src/core/input-events.ts — the ChatInput union (text/answer/verdict/
continue), parseChatInput validation, and inputAppends (answer targets
the dangling question call or 400s; verdict = optional note user
message + native verdict marker).
Server:
- chat-handler.ts — the turn root takes {chatId, input}; write-ahead
persists input appends + claimed steers before the turn; claimSteers
coalesces the whole queue with the handler's steerDelivered filter as
the dedupe seam; teacher-vs-prober branch on hasProbeDone.
- turn-service.ts — persistTurn stamps ids, drops clean question
placeholder toolResults (the record ends at the unanswered call),
emits plan-block markers idempotently (plan-blocks.ts rewrite), and
persists abort partials.
- ui-stream.ts — PiTurnSpec takes the native base; the wire dialect is
unchanged. jobs.ts steers are native markers (steer_<jobId> dedup ids);
waitForIdle reads danglingQuestionOf. teacher-agent.ts wires
convertToLlm: toLlmMessages. record-adapter.ts DELETED wholesale.
- GET /api/chats/[id] serves messages: projectNative(record.messages).
Client:
- chat-transport.ts — the 409 retry is IDENTITY (re-POST the same event
body after the turn settles — the event was never persisted
pre-acceptance; the merge logic is gone). The transport consumes the
stashed input event via prepareSendMessagesRequest and throws loudly on
a missing stash.
- chat.tsx — every send call site stashes exactly one ChatInput event
(text, answer, verdict, continue) before invoking the SDK send.
Tests: full native migration (chat-flow rewritten to input events and
native record pins; jobs/extraction/demos/graph/usage/plan-blocks/
probe-done/reattach-transport converted; record-adapter.test.ts deleted;
chat-transport tests pin the identity retry). 982/982, typecheck, lint,
build all green.
|
||
|
amosblomqvist
|
e79f85ebe4 |
Prompt-override orphan trap: delete clears the switch, writes heal orphans
Contract-fix interlude (2026-09-26), two bugs from UI's prompts-overhaul (6edc220): (a) ORPHAN-400 TRAP: an ON switch whose override file dies orphaned the record; every agent-section write then 400'd (ON-must-have-file) while the disabled toggle couldn't reach the OFF escape hatch — a user stuck unable to write settings. Verified live on 3004 before the fix. Two-layer fix: - DELETE /api/settings/prompts/[...segment] now clears the record's switch in the same operation (switch-first order: a crash between the two leaves the valid off+file state, never an orphan). Response gains switchCleared (additive). Overrules the 2026-09-19 switch-survives- delete note — the delete route and the record agree by construction. - PUT /api/settings/[section] agent writes HEAL an orphaned ON to OFF (key dropped) inside the write — never a 400 (defense in depth: disk wipes outside the app can still orphan). A valid ON with its file stays valid; the invariant: a user can never get stuck unable to write settings, and the record never contradicts file reality for long. Heal logs at info. (b) TEST WIPES REAL DATA: tests/prompt-overrides.test.ts wiped data/prompts/ (REAL user override files) in after() and its empty-list case — every npm test killed actual overrides and orphaned their ON switches (feeding trap a). Isolated via the established LH_DATA_DIR seam (mkdtemp + env set before the first config.ts-loading import; all app-module imports dynamic). after() now snapshots the REAL repo data/prompts before/after and fails the suite if it changed; the seam is pinned (writes land under the temp dir). Pins: delete-clears-switch (incl. idempotent no-op), the exact repro (disk-level wipe → mirror-the-orphan + tool toggle → 200, healed key, rest of the write landed), vanished-fragment ON heals, valid ON stays. Live: the 400-forever write now lands 200, heal info-logged, record clean. Gates: 967/967 (+5), typecheck, lint, build green. |
||
|
amosblomqvist
|
7cfd8b5bc7 |
Background turn runner: every turn is a server-side process, clients are views
Owner ruling 2026-09-25. runPiTurn executes a turn decoupled from any HTTP
response: an unbounded memory-only event log (finite by construction; the
30-min liveness TTL bounds the pathological case) plus a subscriber set.
The POST /api/chat stream and the new GET /api/chats/[id]/stream are both
subscribers — identical SSE dialect; late subscribers get FULL replay
(starting at the start chunk) then live until finish + [DONE]. A view
cancelling (reload, navigate) now detaches ONLY — never aborts the turn.
The remote kill is the abort channel: POST /api/activity/abort gains
kind: "turn" (id = chatId; not_running 409 like jobs); handle.abort sets
sawAbort + agent.abort(), every subscriber closes cleanly, the partial
persists through the usual settle (onFinish runs once, then endTurn,
then unregisterTurn LAST so a view that finds nothing finds the record
current). No running turn on reattach: the cheap terminal — one
{"type":"turn-settled"} event + [DONE] (settledTurnResponse).
The live smoke caught what the unit suite had masked: the agent_end
close() had been left behind in the restructure — ReferenceError every
turn, steered into the error path; fixed and pinned (clean turns must
never emit an error chunk; finishReason stop present).
|
||
|
amosblomqvist
|
e4049e4bf8 |
Key-presence refusal at every gate: a pinned model is not enough
Owner-diagnosed: the turn gate checked only model coordinates, so a pinned slot whose provider key was deleted passed the gate and died mid-stream masked to 'An error occurred.' The key is now gated where the model is — by what would RUN (env-first keyValue truth, so an env-shadowed key passes). - core/settings.ts: the provider_key_missing envelope family (code, provider, slots, human line naming the provider from AUTH_PROVIDER_KEYS labels) + ProviderKeyMissingError + isProviderKeyRefusal matcher. - chat-handler turn gate: refuses 503 before beginTurn when the resolved conversation's provider key is gone (slots: [strong]). - Job runner: same check via an injected dep (agent layer cannot read the key store) — a keyless engine refuses the spawn through the existing failure-shape convention, no phantom run. - Extraction pipeline: refuseIfStagesUnready refuses BEFORE any batch (first offender: extractor before auditor); the trigger skips info- level, /api/graph/import surfaces the 503 envelope per file. - Tests: 8 new pins (turn gate both ways, runner refusal, pipeline first-offender + pass-through, envelope factory, matcher); the three turn-driving suites seed OPENROUTER_API_KEY like any real setup. - Doc fix: stale EXTRACTION.chunkTokens comment in the manual-extract route (left over from eed4b4e). Known residual (pre-existing, flagged for master): the manual extract route logs BOTH config refusals as console.error (fire-and-forget path); the coverage ledger keeps everything uncovered either way. |
||
|
amosblomqvist
|
3327309e8b |
The all-sessions coverage GET dies: the per-session block is the coverage surface
Extraction-UX close-out interlude (master dispatch 2026-09-24, off UI's ff1ec7a confirmation): nothing client-side consumes GET /api/graph/coverage — the sidebar block and the chat-bar ring both read GET /api/chats/[id]'s coverage block. The dead route (app/api/graph/coverage/route.ts) is deleted; the manual-extract front door (POST /api/graph/coverage/extract) stays by design. No server helper was orphaned — the route was composition-only over shared seams (chatStore, extractionStatuses, stores, coverageOf). The sessionCoverage doc + the chats/[id] route comment now point at the per-session block as THE coverage surface. Pin (tests/coverage.test.ts): the route file's absence (prompt-overrides' structural-pin precedent) + the Extract POST's survival — a merge cannot resurrect the dead surface silently. |
||
|
amosblomqvist
|
2e9fd01c0c |
Extraction-UX data layer: the chunk budget becomes a graph setting; the chat view gets a per-session coverage read
Owner-locked extraction-UX redesign (dispatch 2026-09-24), part 1 (data
layer; the UI chunk follows).
THRESHOLD -> SETTING
- config.ts's EXTRACTION.chunkTokens dies; the default moves into
FACTORY_SETTINGS.graph.extractionChunkTokens (4096, the old constant's
value). GraphSettings gains the optional field, Required at resolution
via the standard ?? FACTORY pattern — pre-redesign records resolve the
default cleanly.
- FLOOR 500 (the implementer's call, documented on the field): every run
pays the graph-carrying prompt overhead out of the write channel's
money — a smaller budget fires penny-batches that cost more per
extracted token than the content is worth. The sanitizer CLAMPS up
(bd68486 clamp-never-reject discipline: fractions floor, junk drops);
the settings PUT REJECTS below-floor with the rationale in the message.
No ceiling: a large budget just delays extraction — the user's stated
cost/thoroughness tradeoff, not a misconfiguration to police.
- The trigger (turn-service.triggerExtraction) takes the budget as a
parameter — chat-handler passes effective.graph.extractionChunkTokens,
the fresh per-turn read, same as every other gate.
- The pipeline's BATCH side threads through the same number:
ExtractionPipelineDeps gains an OPTIONAL getChunkTokens getter with the
factory default (Teacher's 24 test fixtures stay untouched; the two
call-site lines inside extraction.ts are the only cross-band edits,
flagged). chat-handler + backfill/import-notes wire the live
effectiveSettings — trigger and batches share one budget, always.
PER-SESSION COVERAGE READ
- sessionCoverage(chatId) (server/chats.ts): the chat view's ring-meter
block — totalMessages/coveredCount/uncoveredCount/coveredTokens/
uncoveredTokens (the same coverageOf derivation /api/graph/coverage
serves, scoped to one chat), running (chat-kind extraction registered
for this id), and lastFoldedAt (the chat's newest coverage row's fold
— a cheap single pass; null before any fold). GET /api/chats/[id]
carries it as coverage: {...}; unknown chat -> null (the route 404s
first).
- /api/graph/coverage (the all-sessions list) compiles untouched — the
per-session read is a scoping of it, not a supersession; the UI's
sidebar/ring chunk decides which surface it consumes.
Tests: factory/sanitizer pins (clamp-up, fraction-floor, junk-drop),
resolution pins (absent -> 4096, stored 2000 wins), settings-body pins
(valid/absent/fraction/below-floor 400/junk 400), sessionCoverage pins
(derived split, newest-fold scoping, cross-session isolation, running via
the registry, null on unknown chat). 893/893.
|
||
|
amosblomqvist
|
2d133f1133 |
Single-node deletion: full death by node id (node interlude, owner-locked)
The remedy for undeletable nodes (preserve-graph orphans from session deletion; notes-import nodes with no chat behind them): DELETE /api/graph/nodes/[id], full-death semantics — the user pointing at the node IS the intent; references from other sessions do NOT preserve it. - core/graph-deletion.ts deleteNodeTraces: the node's every line, every edge touching it, every quiz/observation event naming it, its mapping lines, and its resting position — one synchronous atomic rewriteLedgers call; only the ledgers that changed rewrite (mtimes stay honest). Blast radius is exactly the rows naming the node id; notes source records stay (deleting a proposition is not deleting the file). Derived data self-heals on read: the birth cluster shrinks below MIN_CLUSTER_SIZE and stops rendering. - nodeDeletionImpact (GET /api/graph/nodes/[id]): the confirm dialog's pre-flight facts — edge count, quiz/observation history, referencing sessions. The snapshot's edges + nodeSessions cover part of this, but the raw event history was nowhere on the client — hence the read. - Guards (server/graph.ts deleteGraphNode), decided + documented: NO turn/jobs gates (they never write graph ledgers; the orchestration is synchronous end to end, nothing interleaves). THE EXTRACTION GATE: 409 while ANY pipeline run is registered — a mid-flight run holds pre-deletion findings and could re-reference the dying node. The run is never aborted (its spend belongs to another session — outside this deletion's blast radius; the UI retries once the dock drains. A run started after the deletion is legitimate re-learning: fresh ledgers, fresh node id, and mergeFindings' existingIds guard ignores linkTo ids that no longer exist — no dangling edges, no resurrection. Live-verified on dev data: impact read, full-death stats, honest 404 on repeat, and the file cluster fell 3→2 and unrendered itself. 859/859 tests, typecheck/lint/build green. EOF ) |
||
|
amosblomqvist
|
c86430e8bf |
Wave 7 chunk B: remove the embeddings machinery — the ladder collapses
The owner's wave-7 ruling: embeddings die entirely (git history is the
stash); clustering is birth-session derived (chunk A, 950ddfd), labels
are session names, and the prober's graph tools replace injection.
- settings: GraphSettings collapses to { enabled } — embedding,
embeddingModel, clustering, labeling all die. Migration story: stale
record keys are UNREAD and dropped by the sanitizer's reconstruction;
the next graph-section PUT (replace-whole) persists the collapsed
shape. The settings-panel GraphSection renders one master row.
- deleted: src/server/embeddings.ts (adapters, resolveEmbeddingPortFor,
embedNewNodes, embeddingPort, embeddingModelCatalog), the
GET /api/models/embeddings route, scripts/embed-graph.ts, tests for
all three, and the /api/models/embeddings picker surfaces in the UI.
- prices: EMBEDDING_PRICES overlay + callCost fallback die. cost-summary:
the embedding stage fold dies — back to three stages (extraction/
audit/naming) per the owner ruling; historical stages.embedding spend
in old run records is simply not displayed.
- ports: EmbeddingPort/EmbeddingResult/EmbeddingTask die. The embeddings
LEDGER surface (readEmbeddings/writeEmbeddings + EmbeddingsLedger +
EMBEDDINGS_FILE + graph-deletion's vector pruning) STAYS — inert data,
kept only so graph-context.ts compiles until Teacher's pass removes
it; the seam dies in the follow-up interlude.
- chat-handler (approved cross-lane unwiring, wave-6 precedent): the
extraction pipeline's embedder becomes a documented no-op (nothing
embedded, nothing spent, ledger untouched) until Teacher's pass
removes the phase; the graph-context injection dispatch gate, the
graphContext deps wiring, the setContextRespawn seam, and
turn-service's triggerGraphContext are gone — injection is dead
code pending Teacher's file deletions.
- scripts: backfill-graph/import-notes pass the same no-op embedder;
replay-layout derives clusters from birthSessionPartition (the same
partition the real settle uses) instead of the embeddings ledger.
- settings-body: parseGraphSettingsBody reads the master switch only —
old-shape keys in a body are ignored, not rejected; bad shapes 400.
- next.config: the @huggingface tracing-exclusion guards die with the
last reference.
Cross-lane flags: extraction.ts/graph-context.ts keep stale embed
comments + the embed phase until Teacher's full pass (their parallel
interlude owns those files — zero overlap kept by design);
semantic-clusters.ts + its test stay one more round (extraction.test.ts,
Teacher's file, still imports it — they delete it with the namer cases);
ChatSettings.graphContextTokens + the graph-context-control UI stay
until Teacher+UI.
Gates: tsc clean, eslint clean, 885/885 tests, build compiles.
Live-verified on the port-3004 dev tree: PUT /api/settings/graph with
dead keys collapses on write, bad shapes 400, GET reads {enabled: true},
/api/models/embeddings 404s.
|
||
|
amosblomqvist
|
0a4a9a11de |
BYOK provider embeddings (wave 6 chunk B): kill local transformers
Replace the @huggingface/transformers local embedder (~316MB ONNX cache) with an EmbeddingPort seam: graph.embeddingModel (nullable "provider:id" slug) selects an OpenRouter/OpenAI/Gemini adapter, resolved fresh per call from settings + the auth store so key rotations land immediately. - src/server/embeddings.ts rewritten: slug grammar, injected-fetch adapters (OpenRouter input_type, Gemini taskType batching, OpenAI index-ordered rows), honest resolution (null = skip-with-info, set model with missing key = honest error), embedNewNodes with model-change re-embed wipe + ghost pruning + usage steps, federated catalog (OpenRouter /api/v1/embeddings/models — VERIFIED live, the inverse path 404s; OpenAI text-embedding- filter; Gemini embedContent filter). - GET /api/models/embeddings: federated catalog, empty when no keys. - Extraction tail records stages.embedding usage into the run's money channel; EMBEDDING_PRICES overlay in core/prices.ts (outside the sync-generated block), callCost falls back to it; cost-summary folds the fourth stage; graph-context injection resolves its embedder per run, fails honestly when unconfigured, records query tokens. - Settings: GraphSettings.embeddingModel (null factory), section parser validates slug shape (id may contain colons — OpenRouter :free suffixes, live catalog fact). - Removal: @huggingface/transformers uninstalled, PATHS.modelCache and onnx tracing entries deleted, @huggingface tracing-excludes guard added, stale sharp/libvips comments updated to historical. Live-verified on port 3004: catalog federates 36 rows (33 OpenRouter + 3 OpenAI), embedNewNodes embedded 10 nodes via the free liquid/lfm-2.5 model (dim 1024, 435 prompt tokens, 1 call, $0), settings PUT rejects bad slugs with 400. Gates: typecheck, lint, 895/895 tests, build. |
||
|
amosblomqvist
|
a6338511d3 |
feat(graph): two-mode session deletion — the first destructive ledger operation
DELETE /api/chats/[id]?mode=preserve-graph|delete-nodes (mode required): - preserve-graph: only extraction bookkeeping dies (coverage rows, legacy checkpoints) — graph ledgers byte-untouched - delete-nodes: the chat's ledger traces die. Node survival = surviving provenance point / mapping line (notes imports carry empty provenance) / surviving event. The creator's aliases + provenance are stripped from a surviving node's create line (owner ruling 2026-09-23: name/statement/createdAt are the shared identity); orphan nodes, their edges, events, mappings, positions, embedding vectors and cluster members die; cluster labels stay (immutable) Gates: 404 unknown, 409 turn_in_flight / jobs_running (with jobIds), 503 extraction_winding_down after a 10s abort-and-poll; the purge is one sync block (ledgers → jobs → chat record last). rewriteLedgers is atomic (tmp + rename). graphRevision (max mtime over graph data files) rides the activity feed + the response as the change stamp. Anti-resurrection: the chat pipeline re-checks chat liveness at each merge — a run that slips past the wind-down aborts with nothing merged or checkpointed; its partial spend dies with the chat. |
||
|
amosblomqvist
|
9387418864 |
feat(models): clamp-on-write for pinned thinking levels + DEFAULT_THINKING_ORDER in the core barrel
Fix 1 — clamp, never reject (owner ruling, interlude 2026-09-22): a model swap with a pinned thinking level must JUST WORK. The owner hit the write-side 400 live (Claude Fable 5.1/xhigh → GLM 5.3 flash: "z-ai/glm-5.3-flash does not support thinking level xhigh"). The supported-level verdicts in validateChatSettings and validateModelsSettings are DEAD — the WRITE clamps the pin to the new model's nearest supported level (pi's clampThinkingLevel, the same translation piEngineForSpec applies at runtime) and stores the clamped result; the server is the truth — the record shows the clamped level after write. New pure helpers in model-catalog.ts, applied by ALL THREE write routes (section, defaults, per-chat — one verdict seam, so every surface that used the old 400 gets the clamp uniformly): - clampModelsLevels: every slot's level clamps against the SLOT'S OWN model (the parser already requires a model on each slot write). - clampChatLevel: the body's own model, else the record's resolved strong slot (the chain's fallback); no model anywhere → rides unchecked (runtime clamps at engine build; the turn gate refuses the model-less chat anyway). Fix 2 — DEFAULT_THINKING_ORDER moves from config.ts to core/settings (core barrel, AUTH_KEYS precedent — one source of truth, no drift). The UI imports it to auto-populate a thinking level on FRESH model picks; the ordering has no other client role. Legacy unset records keep the runtime defaultThinkingLevel fallback at engine build untouched. Import sites updated: agent/models.ts, tests. Verified live on 3004: the owner's exact case stores max (not 400); a bare chat-level write clamps to low against the strong slot's glm; unknown provider/model still 400s with the provider list. Tests: the level-verdict suites became clamp suites (the owner's case pinned verbatim: xhigh→max, minimal→low, off→low, supported pass-through, level-less slots ride); the machine-aware expectBareLevelVerdict chain is retired — clampChatLevel takes the strong slot as an explicit param, so the tests are pure again. Gates: typecheck, lint, 839/839, build. |
||
|
amosblomqvist
|
bfa906e2f8 |
feat(models): BYOK all the way down — factory pins deleted, per-need refusal gates
Owner decision 2026-09-22: 'it doesn't make sense to have any default
model configured anywhere; the application is bring your own keys.'
config.ts: MODELS deleted; ModelRole becomes an explicit 9-role union;
DEFAULT_THINKING_ORDER (medium, low, high, minimal, xhigh, max) bakes
the unset-level default — first entry the model supports; explicitly
set levels keep pi's native nearest-clamp.
core/settings-resolution: the factory layer dissolves — the chain is
role override → slot → NOTHING. resolveSettings(record) takes no
factory. Empty chains resolve to null/undefined; a thinkingLevel-only
slot is inert (stored, never run). resolveConversationSpec: chat.model
override over the strong slot, null when both empty. resolveRoleSpec
spans all nine roles for the job/extraction engines.
Refusal envelope (core/settings.ts): modelsNotConfigured(missing) —
{error, code: 'models_not_configured', missing: [slot tiers]} +
ModelsNotConfiguredError with .envelope. Per-need gates, each honest:
- chat-handler: universal turn gate (new + continuing turns) returns
503 before beginTurn — no liveness, no write-ahead persist, no graph
trigger before the gate fires.
- runner.runJob: unresolvable engine lands the job as error with the
envelope wording; no phantom loop, no result steer.
- extraction: refuseIfStagesUnconfigured throws before any stage
(turn-service info-skips so coverage retries next turn; the import
route surfaces per-file code+missing); labeler-less runs skip
cluster naming with a console line and still merge.
Write side: slot entries must carry a model — level-only slot writes
are 400s (absent slot key = clear). piEngineForSpec resolves unset
thinking levels via DEFAULT_THINKING_ORDER; piEngineFor now throws
honestly; thinkingLevelFor/modelIdFor deleted; teacher-agent, pi-run,
visual-maker, pdf-reader all read engine.level.
model-catalog: defaults/slots/rolePins null when the chain is empty
(contracts nullable) — no phantom ids in payloads. Stores drop
MODEL_PINS entirely.
Tests: pin-leaners converted to explicit injection (per-chat model
pins on scripted chats — exactly what the setup flow writes; faux
engines for agent-construction tests). New coverage: the 503 gate
envelope, the runner refusal, the extraction refusal + labeler skip.
Gates: typecheck, lint, 815/815, build.
|
||
|
amosblomqvist
|
f9e3a4c43d |
feat(models): strong/cheap/vision model slots + multi-provider engine federation
Model slots wave (2026-09-20), chunk B:
- settings: new `models` section {strong, cheap, vision} slots (each
field independent, same shape as a role override); ROLE_SLOT maps all
nine roles to tiers (strong=prober+teacher, cheap=researcher/
demoBuilder/extractor/auditor/labeler, vision=visualMaker+pdfReader);
resolution chain per role: override → slot → factory pin.
- config: ModelSpec gains provider (all pins openrouter, the BYOK
default); MODEL_PINS in stores.ts now spans all nine roles (the strong
slot's factory rep IS the prober pin).
- engine: piModels() registers EVERY builtin provider catalog;
piEngineForSpec(resolved spec) is the new engine factory (pi-ai
resolves per-provider keys from env itself); piEngineFor stays as the
config-pin fallback. chat-handler resolves the conversation chain
(chat override → strong slot → pin) and injects engineFor into the job
runner, closing the old gap where per-role overrides never reached
job engines.
- /api/models: models federate ONLY keyed providers (providers surface
with keyPresent, env-wins, fresh per call); slots (resolved tier pins)
join rolePins; catalogDefaultsFor inherits the strong slot; the models
settings section gets the same catalog gate as chat.
- pricing: PRICES/CONTEXT_LIMITS uniformly keyed provider:id
(regenerated from ALL pi-ai catalogs, 1354 keys); usage stamps
(TurnUsage, JobUsage, extraction stages) switch to provider:id —
owner decision: no legacy compat, old stamps render unpriced.
|
||
|
amosblomqvist
|
f1373d5de5 |
feat(search): Tavily + Brave behind a search-provider port; Google CSE retires
Search port + model slots wave, chunk A (owner design, 2026-09-20):
- core/search.ts: SearchHit + SearchProvider port + SearchStatus —
pure, browser-safe; adapters arrive as injected dependencies.
- server/search.ts: Tavily + Brave adapters (wire shapes VERIFIED
against docs.tavily.com's OpenAPI + Brave's dashboard docs — Tavily
auth is the Bearer header, not the legacy body api_key; Brave is
X-Subscription-Token) + engine resolution per owner decisions: a key
ALONE activates its engine (no engine choice needed), explicit
agent.search.engine pins the preference (both keys: Tavily first),
a pinned engine with a missing key is honestly unavailable — no
silent fallback, no default engine.
- Curated keys catalog: GOOGLE_SEARCH_API_KEY/GOOGLE_CSE_ID retire
(CSE closed to new customers, dies Jan 2027 — stored values become
preserved unknown vars); TAVILY_API_KEY (tvly- shape) + BRAVE_API_KEY
(length guard) join. UI keys-section TOOL_IDS now derives from the
catalog — cannot drift.
- GET /api/search: {available, engine, reason} — the honest status for
the settings gray-out; researcher-disable semantics ride the same
signal (Teacher stream rewires the tool onto the port).
- agent.search settings field (schema + sanitizer + strict parser);
web.ts's WebSearchResult is now the core SearchHit (re-exported).
|
||
|
amosblomqvist
|
ec38f91a2b |
feat(keys): free-form model-provider keys — pi-ai provider table on /api/keys
Provider-keys interlude (owner spec, 2026-09-19): any pi-compatible
provider's key is manageable, not just the curated set.
GET /api/keys now also returns providers: {id, envVar, label}[] — every
pi-ai provider that takes a plain api-key env var (33 today), derived
from pi-ai's own provider→env-var table. No-drift mechanism: the table
is generated from pi-ai (builtinProviders + findEnvKeys over a
truthy-for-every-var proxy env) and pinned by
tests/auth-provider-keys.test.ts, which re-derives the truth at test
time and asserts EXACT equality — a pi-ai upgrade fails the gate loudly.
Owner-spec exclusions: github-copilot (OAuth), amazon-bedrock (AWS),
google-vertex (ADC), azure-openai-responses (base URL), the cloudflare
pair; anthropic keeps ANTHROPIC_API_KEY only. OPENROUTER_API_KEY
graduates from the curated catalog to the provider table (dedupe — the
UI pins it always-visible); curated stays Deepgram + the Google search
pair. Statuses: curated + present vars + unknown file vars — absent
provider keys never spam rows; the dropdown is discovery.
Write side: PUT/DELETE accept provider env vars (generic truncation
guard — trim, no whitespace, ≥12; family checks stay on the curated
three). Present-in-env provider vars surface as status rows so
env-wins/shadow semantics stay honest.
|
||
|
amosblomqvist
|
a3b6a495d3 | Merge branch 'feat/edge-glyph' | ||
|
amosblomqvist
|
3e263a18cc |
Graph pipeline cost metrics: capture naming spend, fold all channels, cost pill + panel
- Cluster namer bills the money channel now: per-step usage + model captured via onStep, persisted as NamingRunUsage records to data/graph/naming-runs.jsonl (idempotent per startedAt; failed/aborted runs persist too — money spent either way). callUsage promoted to extraction.ts module scope for reuse. - core/cost-summary.ts: the pure fold behind the pill — chat runs, notes-source runs, and naming runs into stage totals (extraction / audit / naming) + newest-first recent runs. Dollars INFERRED read-time via prices.ts, never stored; unpriced models excluded-but-counted (render '—', never a silent zero); tokens always sum. - GET /api/graph/costs (src/server graphCosts composition, thin route). - Graph view: bottom-left cost pill (cumulative dollars headline) + expandable breakdown panel (stage rows with tokens secondary, recent runs with time/title/merged/cost); fetched on mount, refetched when a pipeline run finishes; merged into one bottom-left stack with the running-pipeline pills. - Tests: pure-fold suite, naming capture wiring (full importNotes -> naming path), store idempotency + strict reads. |
||
|
amosblomqvist
|
cbaf6df454 |
feat(keys): first-class BYOK key management — store, catalog, /api/keys
The auth/keys wave (2026-09-19): the server now owns the flat auth.json map directly (LH_AUTH_FILE from main.cjs; repo-root auth.json in dev, gitignored). The store reflects itself into the child's env at boot with fill-gap precedence — real env (incl. .env.local in dev) always wins — so every consumer keeps its plain process.env reads and saved keys take effect immediately, no restart; main.cjs's spawn-time injection retired (an env copy would shadow every settings-page change). Curated 4-key catalog (OpenRouter, Deepgram, Google search pair) with shape-only validation (no live pings — owner decision), 400s on bad pastes, last-4+length masking (plaintext never leaves the store), unknown file vars preserved + surfaced. Routes: GET /api/keys, PUT/DELETE /api/keys/:id. Tests: catalog/store/composition, with the env-sanitizing prologue (a dev shell may export real keys; tests use throwaway values only). |
||
|
amosblomqvist
|
cb4fe25c89 |
wave 2 (platform stream): settings storage & plumbing foundation (P2)
- src/core/settings.ts: versioned DELTA record schema + sanitizer (unknown keys preserved)
— general/chat/agent{models,tools,prompts}/appearance/graph sections
- src/core/settings-resolution.ts: pure resolveSettings (factory defaults → user delta),
extraction/graph-context channel predicates
- src/store/settings.ts + prompt-overrides.ts: SettingsStore/PromptOverrideStore adapters
wired through stores.ts (effectiveSettings() fresh per call); lazy migration of
default-settings.json into record.chat
- app/api/settings/** routes (record, section bodies, prompt overrides per segment)
- chat-handler: graph gates at trigger call-sites + toolToggles injection seam
- config.ts: SETTINGS_FILE + PROMPTS_OVERRIDE_DIR paths
587/587 tests; typecheck/lint/build clean
|
||
|
amosblomqvist
|
487f5d4cea |
wave 1 (platform stream): background-observability accuracy + owner-driven follow-ups
- observability: liveness-gated stale-job recovery, extraction run registry on globalThis with explicit 'embedding' phase, /api/jobs force-dynamic, client refresh nudges at every seam - headless: providerSafePng canonicalizer (multi-IDAT Chrome PNGs → single-IDAT) fixing qwen 400s - composer: Enter routes through pure enterAction — accepts a pending plan verdict (owner decision) - markdown chrome: session-scoped mermaid overlay + md-blocks component set + pan-zoom math, corner-copy (clipboard grant in electron main), math/table/code block fixes, heading weight 900 543/543 tests; typecheck/lint/build clean |
||
|
amosblomqvist
|
bd3086194f |
feat: no baked-in default model — the new-session controls ARE the default settings page
Owner reframe (2026-09-18): the model + thinking + context controls on
the home graph view and /chat/new are not a draft overlay on a baked-in
default — they ARE the settings page for the default model, and the
user's last choice IS the default. BYOK first boot has no keys and
possibly no OpenRouter, so there is NO default to show: the controls
render 'no model' ('—') until the first pick.
The flash was the deeper symptom: the surfaces rendered from client
state that started empty and fetched truth afterward, so every visit
flashed the config pin (GLM/low — a dev artifact) for a few frames.
Fix at the root: both pages are now force-dynamic server components
that read the record (readDefaultSettings → initialSettings) and hand
it to the client — the first frame shows truth. Dynamic routes aren't
cached client-side (staleTimes.dynamic = 0), so every navigation
re-reads the record; the useDefaultSettings hook keeps only save-side
state (optimistic set + PUT). The GET-on-mount and its touch-guard are
gone.
ModelControl gains noDefaultModel (the default surfaces only): unset
settings render no model/level, never the catalog pin; the thinking
menu no-ops until a model exists. Chat surfaces keep the catalog
fallback — the engine runs the config pin when a session record has no
override, so the control keeps showing what will actually run.
Renamed draft→default throughout: data/default-settings.json,
GET/PUT /api/settings/defaults, useDefaultSettings. New sessions
still inherit the default on creation exactly as before.
Live-verified: first-boot 'no model' with zero flash, picker +
Shift+Tab cycle, values held across session→/chat/new→home roundtrips
and a reload (10×100ms samples, never the old default), zero console
errors.
|
||
|
amosblomqvist
|
a94a108e74 |
feat: draft settings persistence — shared model/thinking/context record for the new-session surfaces
Both new-session pages (the home graph view's topic dock and /chat/new)
held their model + thinking + knowledge-context choices in plain
useState(undefined) — every navigation reset them to config defaults.
Now both surfaces read/write ONE server-side record
(data/draft-settings.json via GET/PUT /api/settings/draft, the
render-settings pattern): whatever the learner last left the controls on
is what every later visit shows. useDraftSettings loads on mount with a
touch-guard (a slow GET never clobbers a fast choice), saves
optimistically. Session creation still carries the choice into the new
chat exactly as before; per-session settings are untouched.
The PUT validation is the exact per-chat contract — parseSettingsBody
extracted verbatim from app/api/chats/[id]/settings and shared by both
routes (unknown model/level → 400 with the supported list, never a
silent clamp). 400s on garbage, {} = the clear-overrides write.
Live-verified: Cmd+K+Enter model pick, Shift+Tab thinking cycle, context
slider — all held across home → session → home → /chat/new, and carried
into a created session's record.
|
||
|
amosblomqvist
|
573e369db5 |
fix: dictation cloud path — bearer subprotocol, access_token grant field
Two live-verified Deepgram facts from the first real mic-test round:
- /v1/auth/grant returns { access_token }, not { token } — the route now
parses the right field.
- Minted JWTs authenticate over the WebSocket via the "bearer"
subprotocol; ["token", jwt] is rejected at the handshake (token is
for raw API keys only, as in the pi dictate extension).
Also documents the env-precedence gotcha in docs/research/stt.md: a
DEEPGRAM_API_KEY exported in the shell (e.g. .zshrc) silently beats
.env.local in next dev and the desktop authEnv() seam.
|
||
|
amosblomqvist
|
e3bdbc2f2c |
feat: voice dictation (STT wave 1 — Deepgram cloud)
Two-band architecture from docs/research/stt.md: renderer captures mic audio in an AudioWorklet (linear16 16kHz mono, ~128ms frames, per-quantum RMS for the meter) and streams it directly to Deepgram over WebSocket — audio never loops through our server. The real key stays server-side; /api/stt/token mints a short-lived grant token (TTL 300s) per run. Patterns ported from the owner's pi dictate extension (tuned in real use): generation counter guarding every async handler, the latest-RMS-never-resets meter (-50..-10dB window, 60ms tick), mic-first stop + CloseStream + 3s safety net, endpointing=300 with the utterance_end_ms=1000 backstop. UI (owner placement): mic button in the stats bar's middle slot on all three surfaces (home graph-view, chat, draft); while recording the meter unfurls to fill the row's free width with the interim ghost at its flank. Esc ends dictation from the composer. Token mint comes before getUserMedia so a keyless app fails fast without a TCC prompt; one automatic reconnect on abnormal socket close, errors surface inline. Live-smoked the full error path in the dev shell (bar renders on all three pages, 403 grant failure shows inline, mic untouched). The full audio path needs a Deepgram key with Member permission to mint tokens. |
||
|
amosblomqvist
|
769a3da186 |
feat: session rename + graph chrome polish (popovers dismiss, no-shift no-match, coverage reseat)
Rename (owner ask): clicking the header title opens edit mode with the whole title selected — typing replaces, arrows move inside. Enter or a click away commits (PUT /api/chats/:id); Escape and an empty commit discard. renameChat joins the ChatStore port; like saveSettings it is NOT conversation activity — updatedAt stays put so sidebar order never churns on a rename. The sidebar patch is instant: the header dispatches lh:chat-renamed and SessionsSidebar folds it into its snapshot (the 2.5s poll is only a backstop). Sidebar right-click rename is DELIBERATELY deferred: it needs a real context-menu system, which gets its own pass later (owner decision, 2026-09-17). Graph chrome: - The extraction-coverage button moves between search and import; the gear keeps the far-right corner. Coverage still renders on an EMPTY graph (that is exactly when it matters). - The no-match search note no longer shifts its siblings: the chrome row is items-start (every button is 44px tall, so top alignment pins the row while the search container grows downward). - Popovers dismiss properly: pointerdown anywhere outside the control's root (button + panel) closes it, and so does Escape. Two panels can never be open at once — the sibling chrome buttons are outside the open root, so pointerdown closes the first before the click opens the second. New shared hook components/popover-dismiss.ts, wired into both the settings gear and the coverage control. Gates: typecheck, eslint, 498/498 (+2 rename store tests), build. Live-verified: select-all/focus/Enter/Escape/empty/blur round-trip, updatedAt untouched, sidebar instant patch; chrome order + frozen positions under a no-match query; popover close via canvas, search input, Esc, mutual exclusion, toggle intact. |
||
|
amosblomqvist
|
5c380de669 |
Merge lh-ui refinement wave (feat/ui-polish @ 8163437) into main
11 commits on fbeca36: minimal-color rounds 1-2 (phase color retired, the braille spinner is the one working indicator; the chip is label-only), typography rounds 1-5 (three-font rule enforced — CMU titles, Source Sans session titles, JetBrains mono; snapped type ladder in globals.css with the xxs rung merged into xs; lh-input caret at the font-metric floor; session titles at prose size, message counts and active-row weight dropped, ISO dates), header rounds 6/6b/6c/6d (60px top bar aligned to the sidebar LH band; the open-sidebar toggle matches the close glyph at size-4 both ways; SidebarToggle always-mounted with a lockstep animated slot — no more one-frame title jump; title at text-3xl). Semantic change folded in: `busy` in the chats list now means exactly "a teach turn is in flight" — background jobs no longer spin the session row (the sidebar's Background agents section is their channel). Merge fallout fixed in the merge: components/graph-coverage.tsx (main's coverage panel) held the last three text-xxs uses; with the rung deleted they silently lost their size — mechanically swapped to text-xs, matching lh-ui's own swap in graph-context-marker.tsx. No file overlap with main's waves since fbeca36 (extraction coverage, lh-graph merge, world-scale bake) — clean auto-merge. Gates: typecheck, eslint, 496/496 tests, build. Live-verified on the dev server: 60px header with 32px CMU title, label-only chip, snapped ladder active (--lh-prose-size 16px, no --text-xxs rule anywhere), sidebar rows with ISO dates, animated sidebar toggle with no header jump. |
||
|
amosblomqvist
|
2678f04e7a |
feat: extraction coverage — message-granular ledger + token-batched runs (d132373 vision)
THE MESSAGE IS THE UNIT OF COVERAGE (vision-extraction-coverage.md): a per-chat append-only ledger (coverage.jsonl) records which messages the pipeline has extracted, by content fingerprint over exactly what the extraction view sees (one shared line builder — core/coverage.ts); the uncovered remainder is DERIVED at read time, never dual-tracked. A quiz answer merging into the asking message re-opens only that message; plan-block data-part attach never re-opens anything. - The trigger (turn-service) replaces every-3-quiz-answers with uncovered-tokens >= EXTRACTION.chunkTokens (4096) — quiz-less sessions extract too, content size is the honest signal. - runExtraction is now chunked: token-budgeted batches of the oldest uncovered consecutive messages (a tool call + result never split), each merged batch checkpointing its messages' coverage atomically — an abort or error loses at most the in-flight batch; merged batches stay covered. - Observation refs are batch-tagged (obs:<batchTag>:<sourceId>): re-runs of the same batch dedupe; a re-opened message's fresh understanding lands as a new event. Fixes the old obs:p1 position-collision. - New API: GET /api/graph/coverage (exact per-session derived coverage + running flags, most-uncovered first) and POST /api/graph/coverage/extract (manual run for one session, 409 while in flight). - Graph view chrome: the coverage button opens a panel with every session's covered/uncovered tokens and a per-session Extract button — the manual answer to aborted runs, errors, and dormant tails. Renders even when the graph is empty: an empty graph with uncovered sessions is exactly when it matters. Polls only while open. - Notes imports keep the whole-file checkpoint path (coverage undefined). - Tests: pure-core coverage suite (11), chunked-pipeline suite (3: multi-batch + no-op, abort-keeps-earlier-batches + remainder re-run, re-open alone + data-part invisibility), panel render suite (5); pendingQuizAnswers and its tests deleted as dead code. 486/486. Live-smoked on real GLM: manual extract of "teach me modal realism" ran one batch, wrote 4 coverage entries, panel flipped to "1.3k covered · complete" with the button disabled, graph refetched (nodes 11), usage persisted with real stats. |
||
|
amosblomqvist
|
326222f53c |
Minimal-color round 1 (owner): phase color retired — the braille spinner is the one working indicator
Nothing exists purely for decoration: phase color-coding is retired app-wide (phases only advance — every session ends Teaching — so phase-as-signal was decoration with a one-time function), and what state genuinely deserved — 'a turn is in flight' — is now carried by MOTION alone, in the terminal's own glyph language. - BrailleSpinner (components/braille-spinner.tsx): the canonical cli-spinner sequence (⠋⠙⠹⠸⠼⠴⠦⠧⠇⠏) as exact dot bitmaps in a 2×4 braille cell, 80ms/frame (800ms cycle). Pure CSS: one shared keyframes, frames staggered via animation-delay; no JS timer, no re-render per frame — cheaper than the old breathing dot. Static contexts (SSR, prefers-reduced-motion) show frame 0 — a legible glyph, not a blank. - Sidebar rows: spinner ONLY while a teach turn is in flight; idle and question-pending render NOTHING (a resting state has no scan value — the meta line's 'Waiting for your answer' text carries it). The three-state phase dot (pulse/solid/hollow) and PHASE_FILL/RING maps are gone. - Top bar PhaseChip: label-only pill; the old dot slot becomes the spinner's home while a turn is in flight (busy prop wired from ChatView's busy = localStream || turnInFlight). Transcript PhaseDividers drop phase colors for monochrome rules + labels. - API narrowing (app/api/chats/route.ts + ChatListItem contract): busy = isTurnRunning ONLY — background jobs no longer count (the sidebar's Background agents section is their channel; a per-row spinner would double-signal them). - globals.css: --phase-* tokens + lh-dot-breathe removed (dead after the sweep — no remaining users). Live-verified: idle/pending rows render no indicator, /chat/new chip is label-only, 10 CSSAnimations at 800ms with correct 80ms stagger, static frame 0 under reduced motion. Gates: typecheck, lint, 470/470, build. Lane crossings to flag at merge: app/api/chats/route.ts + the ChatListItem contract comment (src/core) — the busy-semantics narrowing; components/chat.tsx one-line busy prop. |
||
|
amosblomqvist
|
1d45d77303 |
feat: graph context injection — the session's prior knowledge, steered in at session start (vision-context-injection.md)
One shot per chat, fired at the first turn's onFinish: the session's topic (its only typed message) is embedded as an EmbeddingGemma retrieval query; the semantically closest graph propositions are selected under the session's token budget (pure budget, no similarity threshold — the decided doctrine) and delivered as a marker-only steer through the wake pipeline, queuing behind pending questions like every sub-agent result. - contracts: ChatSettings.graphContextTokens (default 1024, max 4096, step 128, 0 = off); GraphContextData + graphContextText; the graph-context JobKind + buildGraphContextSteer (id steer_ctx_<chatId>) - core/graph-context.ts: brute-force cosine top-K, greedy chars/4 fill under budget, most-recent UnderstandingLevel fold, count cap 64, parse-don't-cast at the marker seam - server: embedQuery (role-asymmetric query format, same warm singleton — vector-space parity by construction); the runner (startGraphContext- Injection one-shot gates; runGraphContextJob validates ledger parity); wake branches (chatIdOf/jobIdOf, finalizeJob done, recoverStaleJobs re-deliver or contextRespawn); abortJob kills context jobs SILENTLY (nothing was promised in the lesson); turn-service trigger seam - adapter: data-graph-context flattens to the canonical block (malformed parts have no model face) - UI: the budget slider (home dock + /chat/new draft, the session-data strip); the quiet SystemLine "~537 · 10 propositions"; dock kind "Knowledge context"; model-picker PUT now spreads settings (a model pick can no longer clear the sibling override) Live-smoked end to end on the real embedder + real GLM: seeded ledger (10 propositions), session on "modal logic: possible worlds and the necessity box" — job in the dock, queued behind the prober's question, delivered on the answer, SystemLine in the transcript, wake turn reads the block and continues the probe. Gates: tsc, lint, 459/459, build. |