2cba283009aa0f04b4c080818f18c198655b6080
7
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 amosblomqvist
|
2d133f1133 |
Single-node deletion: full death by node id (node interlude, owner-locked)
The remedy for undeletable nodes (preserve-graph orphans from session deletion; notes-import nodes with no chat behind them): DELETE /api/graph/nodes/[id], full-death semantics — the user pointing at the node IS the intent; references from other sessions do NOT preserve it. - core/graph-deletion.ts deleteNodeTraces: the node's every line, every edge touching it, every quiz/observation event naming it, its mapping lines, and its resting position — one synchronous atomic rewriteLedgers call; only the ledgers that changed rewrite (mtimes stay honest). Blast radius is exactly the rows naming the node id; notes source records stay (deleting a proposition is not deleting the file). Derived data self-heals on read: the birth cluster shrinks below MIN_CLUSTER_SIZE and stops rendering. - nodeDeletionImpact (GET /api/graph/nodes/[id]): the confirm dialog's pre-flight facts — edge count, quiz/observation history, referencing sessions. The snapshot's edges + nodeSessions cover part of this, but the raw event history was nowhere on the client — hence the read. - Guards (server/graph.ts deleteGraphNode), decided + documented: NO turn/jobs gates (they never write graph ledgers; the orchestration is synchronous end to end, nothing interleaves). THE EXTRACTION GATE: 409 while ANY pipeline run is registered — a mid-flight run holds pre-deletion findings and could re-reference the dying node. The run is never aborted (its spend belongs to another session — outside this deletion's blast radius; the UI retries once the dock drains. A run started after the deletion is legitimate re-learning: fresh ledgers, fresh node id, and mergeFindings' existingIds guard ignores linkTo ids that no longer exist — no dangling edges, no resurrection. Live-verified on dev data: impact read, full-death stats, honest 404 on repeat, and the file cluster fell 3→2 and unrendered itself. 859/859 tests, typecheck/lint/build green. EOF ) |
||
|
amosblomqvist
|
d1b46132b9 |
Wave 7 close-out: the embedding ledger seam dies; notes birth via mapping lines
Part 1 — the ledger + remnants interlude (master-dispatched, owner ruling 2026-09-23: nothing configures the graph beyond chat models): - The embedding ledger surface is gone: readEmbeddings/writeEmbeddings ports, the EmbeddingsLedger contract, EMBEDDINGS_FILE (config, store impls, graphRevision source list), and graph-deletion's vector-pruning step. On-disk data files stay untouched (inert history). - The extraction embedder seam dies wholesale: ExtractionPipelineDeps drops the embedder dep (chat-handler + backfill/import-notes stubs go with it), the 'embedding' phase leaves ExtractionPhase, the usage capture/persist helpers die, and stages.embedding leaves ExtractionRunUsage (historical records keep their shape — reads are lenient). Extraction is two stages again, like the local-embedder era. - Consumer-less remnants: latestQuizOutcomes + QUIZ_EVENT_KINDS, clusterContentId, matchClusterLabelings + Community + ClusterLabeling, the readClusterLabels/appendClusterLabel port seams + store impls + LEDGER_FILES.clusterLabels. NamingRunUsage/readNamingRuns STAY (standing ruling — the cost pill renders historical naming spend); clusterHue + MIN_CLUSTER_SIZE stay (live). - The 'labeler' role dies: BACKGROUND_MODEL_ROLES, ROLE_SLOT, ModelRole, the catalog pin projections, and the UI roster strings. RolePins keys shrink with the union. The notes birth fallback (owner-approved 2026-09-23, surfaced by the interlude's live smoke: production notes imports create provenance-EMPTY nodes (importNotes passes messages: null — the file's anchor is the mapping line), so the provenance-only birth rule left every imported file unclustered and invisible to the prober's session list, against the locked file-birth ruling. A provenance-empty node now births under its FIRST mapping line's chatId (ledger order); only notes-born nodes ever hit the fallback (chat nodes always carry provenance; a re-encountered node's link-append re-opens the provenance rule). Deletion-safe: mapping lines of deleted chats are stripped alongside their provenance. Live-verified on port 3004: a fresh notes import clusters under its src_ id with the file path as the label, end to end with no embedder. 845/845 tests, typecheck/lint/build green. EOF ) |
||
|
amosblomqvist
|
a6338511d3 |
feat(graph): two-mode session deletion — the first destructive ledger operation
DELETE /api/chats/[id]?mode=preserve-graph|delete-nodes (mode required): - preserve-graph: only extraction bookkeeping dies (coverage rows, legacy checkpoints) — graph ledgers byte-untouched - delete-nodes: the chat's ledger traces die. Node survival = surviving provenance point / mapping line (notes imports carry empty provenance) / surviving event. The creator's aliases + provenance are stripped from a surviving node's create line (owner ruling 2026-09-23: name/statement/createdAt are the shared identity); orphan nodes, their edges, events, mappings, positions, embedding vectors and cluster members die; cluster labels stay (immutable) Gates: 404 unknown, 409 turn_in_flight / jobs_running (with jobIds), 503 extraction_winding_down after a 10s abort-and-poll; the purge is one sync block (ledgers → jobs → chat record last). rewriteLedgers is atomic (tmp + rename). graphRevision (max mtime over graph data files) rides the activity feed + the response as the change stamp. Anti-resurrection: the chat pipeline re-checks chat liveness at each merge — a run that slips past the wind-down aborts with nothing merged or checkpointed; its partial spend dies with the chat. |
||
|
amosblomqvist
|
3e263a18cc |
Graph pipeline cost metrics: capture naming spend, fold all channels, cost pill + panel
- Cluster namer bills the money channel now: per-step usage + model captured via onStep, persisted as NamingRunUsage records to data/graph/naming-runs.jsonl (idempotent per startedAt; failed/aborted runs persist too — money spent either way). callUsage promoted to extraction.ts module scope for reuse. - core/cost-summary.ts: the pure fold behind the pill — chat runs, notes-source runs, and naming runs into stage totals (extraction / audit / naming) + newest-first recent runs. Dollars INFERRED read-time via prices.ts, never stored; unpriced models excluded-but-counted (render '—', never a silent zero); tokens always sum. - GET /api/graph/costs (src/server graphCosts composition, thin route). - Graph view: bottom-left cost pill (cumulative dollars headline) + expandable breakdown panel (stage rows with tokens secondary, recent runs with time/title/merged/cost); fetched on mount, refetched when a pipeline run finishes; merged into one bottom-left stack with the running-pipeline pills. - Tests: pure-fold suite, naming capture wiring (full importNotes -> naming path), store idempotency + strict reads. |
||
|
Amos
|
af90fadcda |
Extraction pipeline: extractor → auditor → merger — the sole graph writer (stage 5)
The knowledge graph's write channel is now the pivot's target
architecture (vision §1 construction):
- src/agent/subagents/extraction.ts — the pipeline factory
(createExtractionPipeline({ chatStore, graphStore, extractor?,
auditor? }); models env LH_EXTRACT_MODEL ?? LH_MODEL):
* transcriptView(): the compact view both LLM stages see — final plan
tree with node ids, conversation verbatim, QUIZ lines with outcomes,
mark_done lines; job tool parts are one-liners, plan ops fold into
the tree line.
* Extractor: transcript → SessionFindings (propositions incl.
misconceptions and prose-only supports edges, attribution for
UNTAGGED probes, all with verbatim evidence) via one zod-validated
submit_findings call.
* Auditor: transcript + findings → the verified subset + drops (the
faithfulness net) via submit_review.
- src/core/graph.ts — emitChat/foldStaleChats/replayFinalTree retire;
mergeFindings() replaces them as the pipeline's deterministic apply
stage: exact/alias identity resolution (one mapping line per (chat,
graph node)), edge dedupe, and event rebuild FROM THE TRANSCRIPT's tool
parts — kinds/timestamps/refs/notes deterministic, the LLM's only event
say being WHICH proposition an untagged probe tested (retroactive —
the vision's open question #3, resolved structurally: tagged events
resolve deterministically; attribution rescues untagged and
tagged-but-unmapped ones; unattributable land null + question text).
pendingQuizAnswers() is the trigger helper.
- Trigger (turn-service.ts triggerExtraction, replacing the old
foldTurnIntoGraph body): once a chat accumulates
EXTRACTION.everyNQuizAnswers (LH_EXTRACT_N, default 3) new answered
quizzes, fire the pipeline async — single-flight per chat, failure
checkpoints nothing and re-runs on the next trigger. Deliberately a v1
placeholder isolated in ONE seam, trigger-agnostic pipeline.
- GET /api/graph is now a pure read (foldStaleChats gone).
- NodeMapping.treeNodeId → sourceId (the mapping key is now the writer's
local proposition id — tree node when tree-derived); store strict-read
schema updated. Local ledger data regenerates through the pipeline.
- Tests: graph.test.ts rewritten around mergeFindings (identity,
attribution paths incl. ghost-rescue, idempotency, cross-session
provenance, trigger counting), graph-context.test.ts merge section
converted to findings-driven, NEW extraction.test.ts — transcriptView
units + the full pipeline with mock models (audit drops, failure
checkpoints nothing, single-flight). 195/195, tsc/eslint/build clean.
Probe attribution (vision open question): probes attach retroactively by
the extractor against the whole transcript; a pre-tree probe maps to a
proposition the tree only articulates later. Fuzzy/embedding merge
adjudication stays deferred (conservative exact/alias only).
|
||
|
Amos
|
1167157c23 |
Storage ports: ChatStore/GraphStore/JobStore + adapters + core inversion (stage 3)
Per-entity semantic ports (src/core/ports.ts), one adapter family
(src/store/*, fs JSON/JSONL), exactly one composition point
(src/server/stores.ts). Dependency direction is now machine-checked:
adapters implement core interfaces; core never imports an adapter.
The inversion:
- core/graph.ts (emitChat/getGraph/graphSnapshot/foldStaleChats) is now
pure — it takes the storage ports as explicit parameters (GraphDeps).
No store imports; newGraphNodeId is local; node id format unchanged.
- core/graph-context.ts buildGraphContext() no longer defaults to the
live ledgers: the chat-handler hands in getGraph(graphStore) per turn.
- store adapters export exactly one object each implementing their port
(chatStore/graphStore/jobStore); bodies otherwise verbatim.
NodeMapping + SessionCheckpoint moved store/graph -> core/contracts
(they ARE contract shapes); CreateJobInit/JobPatch live in ports.
- titleFromPrompt moved store/chats -> core/contracts (pure helper,
planNodeStatus precedent). Store modules now contain persistence only.
- server/stores.ts is the composition root; server/{chats,graph,jobs}.ts
and chat-handler wire from it; agent (plan tools, sub-agent runner)
call the port objects. The shell surface (@/src/server exports) is
unchanged.
Lint law extended: the whole of src/core is now a purity zone (was a
file glob with a graph/graph-context exemption). Verified it fires on
core->store imports; tsc clean, 190/190 tests, eslint clean, prod build
clean (prerender 6/6). Tests pass deps explicitly — graph.test uses the
same fs-adapter pair stores.ts wires in production; graph-context.test
keeps its in-memory ledgers as a GraphStore implementation.
|
||
|
Amos
|
e92140f993 |
Shell/core import law: public API barrels + ESLint enforcement (stage 2)
The shell now imports through exactly two barrels, enforced by
import/no-restricted-paths (npm run lint):
components/** -> "@/src/core" (pure core API, browser-safe)
app/** -> "@/src/core" + "@/src/server" (server entries)
Zone violations are lint errors with architecture-pointer messages; the
pure-core modules (contracts, question-core, plan-tree, phases, pending,
math-delims, ui-message) may never import agent/server/store.
Supporting moves (direction: IO layers -> core):
- core/index.ts: the core public API barrel
- core/ui-message.ts: moved from src/agent (it was shell-facing all along)
- quiz/ask input zod schemas moved agent/tools -> core/question-core
(the question contract, alongside the grading logic)
- ChatRecord/ChatSummary moved store/chats -> core/contracts (single schema)
- GraphSnapshot/NodeStateClass moved core/graph -> core/contracts (the
/api/graph payload IS a contract; the pure renderer needs only these)
- src/server/{chats,graph,jobs}.ts + index.ts: thin server entries for the
transport layer; GET /api/graph's fold side-effect now lives behind
graphView() instead of inline in the route
Gates: tsc clean, 190/190 tests, eslint clean (law verified to fire on
store/agent/subpath violations), production build clean.
(Repo repair en route: a git object (app/chat/[id] tree) was missing from
the object store — reconstructed byte-exactly via mktree from its blob;
the broken node_modules that stalled the build was cleanly reinstalled.)
|