delete the mappings ledger — per-job in-memory resolution + notes fingerprint guard
The persisted (chatId, sourceId)→node table was a shared namespace across independent chunk/batch jobs: each chunk's extractor mints local ids fresh (p1..pN), so later chunks collided with earlier ones — propositions silently skipped as 'already resolved' (coverage checkpointed anyway: permanent silent loss) and edges resolved onto the wrong nodes (the 'everything links to everything' mush). Owner ruling 2026-09-29: mergeFindings resolves THIS job's ids in memory, nothing persists between jobs; cross-job identity is the auditor's linkTo verdicts alone. - contracts: NodeMapping dies; NotesSource.contentFp — sha256 of the text at the last run, the re-import guard (unchanged re-import = TRUE no-op, no model calls, no usage; changed text re-runs; legacy unstamped sources re-run once) - merger: job-local resolution map, in-submit duplicate ids skip once, edges resolve extractor-local ids only - ports/store: appendMapping/readMappings, LEDGER_FILES.mappings, rewriteLedgers.mappings gone (stale data/graph/mappings.jsonl just stops being read) - session-clusters: birthSessionOf is provenance-only — the legacy mapping-line fallback for provenance-empty notes nodes dies - deletion: no mapping rows to strip; stats lose mappingsRemoved - replay-layout batches nodes by first provenance point - graph.test pins the new independence semantics (re-merge re-creates; duplicate ids within one submit resolve once); extraction tests pin per-chunk merges + the fingerprint guard
This commit is contained in:
@@ -58,7 +58,6 @@ export type ModelRole =
|
||||
export const LEDGER_FILES = {
|
||||
nodes: path.join(PATHS.graph, "nodes.jsonl"),
|
||||
edges: path.join(PATHS.graph, "edges.jsonl"),
|
||||
mappings: path.join(PATHS.graph, "mappings.jsonl"),
|
||||
sessions: path.join(PATHS.graph, "sessions.jsonl"),
|
||||
/** Per-message extraction coverage (contracts/graph.ts CoverageEntry;
|
||||
* the checkpoint's replacement for chat extractions — notes keep
|
||||
|
||||
@@ -329,7 +329,7 @@ Sub-agents **verify, never one-shot**: the demo-builder headlessly executes its
|
||||
```mermaid
|
||||
flowchart TB
|
||||
TRIG["trigger seam (turn-service): coverageOf(record, coverageLedger)<br/>— uncovered tokens ≥ graph.extractionChunkTokens (settings)?"] -->|"yes, single-flight per source"| RUN["extraction.runExtraction(chatId)"]
|
||||
IMPORT["POST /api/graph/import (notes front door)<br/>deterministic src id = sha256(path) — idempotent re-imports"] -->|"importNotes: shared pipeline, second extractor prompt"| RUN
|
||||
IMPORT["POST /api/graph/import (notes front door)<br/>deterministic src id = sha256(path); content-fp guard skips unchanged re-imports"] -->|"importNotes: shared pipeline, second extractor prompt"| RUN
|
||||
RUN --> BATCH["batchRuns — greedy token-budgeted batches of oldest uncovered TEACHER-PROSE messages"]
|
||||
BATCH --> EXT["EXTRACTOR (LLM) — sees ONLY the session markdown view<br/>(tree-blind, graph-blind; shared engine: core/session-markdown.ts)"]
|
||||
EXT --> AUD["AUDITOR (LLM, agentic loop) — sees findings ONLY, never the source;<br/>explores the graph via find / list_sessions / list_propositions tools,<br/>judges identity ONLY (link-or-create, equivalence bias) — never mints edges"]
|
||||
@@ -342,11 +342,11 @@ flowchart TB
|
||||
|
||||
**Serialized audit, parallel extracts — per-chunk jobs** (owner ruling 2026-09-28): every chunk (notes file) and every batch (chat coverage) is its own JOB — registry row, usage record, failure domain. Extracts fan out in parallel across all jobs and sources, throttled by a process-wide 3-permit extract semaphore on `globalThis` (the provider-facing rate stance; a job waiting for a permit shows `queued`). The audit→merge section remains a process-wide FIFO critical section — a mutex on `globalThis` (the same module-instance idiom as the run registries): before any auditor LLM call or graph read, a job waits (`waiting` phase) until every earlier job's merge has landed, so each auditor explores the graph as it actually stands — two concurrent audits would each link into a graph the other is about to change, the structural mint of duplicate nodes. Failure isolation: one job failing (bad chunk, degraded model output) is absorbed at the fan-out — its messages stay uncovered and honestly re-runnable while its siblings still merge; the run returns what actually merged. Abort is source-wide: every job of a source's fan-out dies together, and merged jobs keep their checkpoints. The status feed shows one row per job (`file.md · 2/5` suffixes for chunked sources), and each job persists its own usage record (failed jobs bill their partial spend too).
|
||||
|
||||
**Division of trust** (the pipeline's core rule): the extractor is fully trusted and is the only stage that ever sees the source. The auditor never sees the transcript — it receives the findings JSON alone and explores the existing graph through read-only tools (`find`, `list_sessions`, `list_propositions` — statement text included, no timestamps) to reach its verdicts. VERDICTS-ONLY (owner ruling 2026-09-28): the auditor's submit is `{ links }` — identity links only (which findings propositions ARE existing nodes). The edges argument is GONE (owner, same evening): the auditor never mints edges — cross-source connection happens solely through identified equivalences, and relations between propositions come from the extractor alone. It never restates the findings; the merger joins findings + verdicts, so an order of magnitude less is emitted and there is no structural way for the auditor to alter or drop extractor output. Equivalence bias (owner ruling 2026-09-27): paraphrases, rewordings, and compressions of the same claim MERGE; only genuinely distinct propositions stay apart. The deterministic merger validates and appends: link resolution, dedupe by `(from, to, type)`, coverage checkpoints. There is no drop channel — nothing rejects the extractor's output.
|
||||
**Division of trust** (the pipeline's core rule): the extractor is fully trusted and is the only stage that ever sees the source. The auditor never sees the transcript — it receives the findings JSON alone and explores the existing graph through read-only tools (`find`, `list_sessions`, `list_propositions` — statement text included, no timestamps) to reach its verdicts. VERDICTS-ONLY (owner ruling 2026-09-28): the auditor's submit is `{ links }` — identity links only (which findings propositions ARE existing nodes). The edges argument is GONE (owner, same evening): the auditor never mints edges — cross-source connection happens solely through identified equivalences, and relations between propositions come from the extractor alone. It never restates the findings; the merger joins findings + verdicts, so an order of magnitude less is emitted and there is no structural way for the auditor to alter or drop extractor output. Equivalence bias (owner ruling 2026-09-27): paraphrases, rewordings, and compressions of the same claim MERGE; only genuinely distinct propositions stay apart. The deterministic merger validates and appends: link resolution, edge dedupe by `(from, to, type)`, coverage checkpoints. There is no drop channel — nothing rejects the extractor's output. **Per-job in-memory resolution** (owner, 2026-09-29): the mappings ledger is deleted — each job resolves its own proposition ids against the graph in memory and merges independently, so no job can see or collide with another's ids (the persisted table once silently dropped same-id propositions across chunks and welded wrong edges). Cross-job and cross-run identity is the auditor's links alone, plus the coverage ledger (nothing re-extracts) and the notes content-fingerprint guard.
|
||||
|
||||
**Coverage, not quiz-counting** (supersedes the v1 every-N-quiz-answers trigger): each message's *view* (the compact line format `core/coverage.ts` has always hashed) is hashed into a per-chat fingerprint ledger (`coverage.jsonl`); "covered" means exactly "a run committed on this content." The prompt view is the richer markdown document — a deliberate 2026-09-27 split: re-formatting presentation never re-opens covered history. Uncovered = fingerprint diff, never dual-tracked; runs are greedy token-budgeted batches (`graph.extractionChunkTokens` — a settings knob, default 4096, floor 500, read fresh each turn) of the oldest uncovered messages — quiz-less sessions extract too. A coarse `sessions.jsonl` checkpoint still marks each merged run. The trigger seam (`turn-service.ts`) is trigger-agnostic from the pipeline's side: when a better mechanism lands (session end, idle, explicit flush), only the seam changes. Single-flight per source; the run registry powers the coverage UI and true abort.
|
||||
|
||||
**Two front doors, one pipeline**: chat transcripts (quiz-triggered) and raw personal notes (`POST /api/graph/import` — each file becomes a `NotesSource` playing the `chatId` role, deterministic id from the path hash). Notes chunk like chats (owner, 2026-09-28): large files split at line boundaries under the same token budget, and each chunk/batch fans out as its own job exactly like a chat run — parallel extracts under the semaphore, serialized audits under the gate. No single model call carries a whole file. Both submit tools require their top-level fields (`edges`/`links` present even when empty): a payload missing them fails validation rather than silently merging structure-less nodes.
|
||||
**Two front doors, one pipeline**: chat transcripts (quiz-triggered) and raw personal notes (`POST /api/graph/import` — each file becomes a `NotesSource` playing the `chatId` role, deterministic id from the path hash; a content-fingerprint stamp makes an unchanged re-import a TRUE no-op — no model calls, no usage — while changed text re-runs the whole file, owner 2026-09-29). Notes chunk like chats (owner, 2026-09-28): large files split at line boundaries under the same token budget, and each chunk/batch fans out as its own job exactly like a chat run — parallel extracts under the semaphore, serialized audits under the gate. No single model call carries a whole file. Both submit tools require their top-level fields (`edges`/`links` present even when empty): a payload missing them fails validation rather than silently merging structure-less nodes.
|
||||
|
||||
**The read channel is pure derivation.** `GET /api/graph` reads the ledgers (`graphSnapshot`: merged nodes/edges, birth-session clusters, positions) — the home view never triggers a write. Node detail cards show a node's edges — outgoing under "What follows from this" / "What this supports", incoming under "Follows from" / "Supported by" (an edge A→B means A implies or supports B, owner semantics 2026-09-28) — and its birth sessions.
|
||||
|
||||
@@ -409,7 +409,7 @@ The Electron main is a launcher + security guard, never a host for app logic: si
|
||||
| **One writer per ledger** — the merger is the sole graph writer; chat records written only by the turn pipeline; job records only by the runner; positions PUT is the only sanctioned graph-adjacent exception |
|
||||
| **The store never decides** — adapters persist and shape-guard; merge/dedup/derivation semantics are reader-side, in core |
|
||||
| **Derived, never dual-tracked** — append-only evidence; every state recomputed at read (snapshot, plan statuses, phases, costs, coverage) |
|
||||
| **Idempotency via deterministic keys** — steer ids, ledger refs, coverage fingerprints, mapping `(chatId, sourceId)`, cluster content hashes; re-running anything converges |
|
||||
| **Idempotency via deterministic keys** — steer ids, ledger refs, coverage fingerprints, source content fingerprints, cluster content hashes; re-running anything converges |
|
||||
| **LLMs judge; code mechanizes** — grading, merging, fingerprints, layout are deterministic; a failed LLM stage checkpoints nothing |
|
||||
| **The persisted record is the only source of truth** — agents are stateless per-turn rebuilds; no unanswered tool call reaches a provider |
|
||||
| **Failure containment** — side channels (context, extraction, usage, naming) are fire-and-forget behind try/catch seams; a side-channel failure never touches the response or the persist; the turn registry clears on every path |
|
||||
|
||||
@@ -19,7 +19,7 @@ const depthMode: "off" | "vertical" | "radial" =
|
||||
depthArg === "--depth=vertical" ? "vertical" : depthArg === "--depth=radial" ? "radial" : "off";
|
||||
import { incrementalLayout } from "../src/core/layout.ts";
|
||||
import { birthSessionPartition } from "../src/core/session-clusters.ts";
|
||||
import type { GraphNode, NodeMapping, Position } from "../src/core/contracts.ts";
|
||||
import type { GraphNode, Position } from "../src/core/contracts.ts";
|
||||
|
||||
function main() {
|
||||
const t0 = Date.now();
|
||||
@@ -35,17 +35,17 @@ function main() {
|
||||
const edges = graphStore.readEdges();
|
||||
// The cluster source (wave 7): the birth-session partition, derived on
|
||||
// read — same as the real settle (core/layout-settle.ts).
|
||||
const birthClusters = birthSessionPartition(nodes, graphStore.readMappings());
|
||||
const birthClusters = birthSessionPartition(nodes);
|
||||
|
||||
// Batches: each imported file, in real import order (sources.json
|
||||
// importedAt). A node belongs to the file whose mapping first claims it
|
||||
// (mapping line order = arrival); nodes no mapping covers (legacy) ride
|
||||
// importedAt). A node belongs to the session of its FIRST provenance
|
||||
// point (line order = arrival); nodes with no provenance (legacy) ride
|
||||
// the final batch.
|
||||
const sources = [...graphStore.readSources()].sort((a, b) => a.importedAt.localeCompare(b.importedAt));
|
||||
const mappings: NodeMapping[] = graphStore.readMappings();
|
||||
const batchOf = new Map<string, string>(); // nodeId → srcId
|
||||
for (const m of mappings) {
|
||||
if (!batchOf.has(m.graphNodeId)) batchOf.set(m.graphNodeId, m.chatId);
|
||||
for (const node of nodes) {
|
||||
const birth = node.provenance[0]?.chatId;
|
||||
if (birth !== undefined && !batchOf.has(node.id)) batchOf.set(node.id, birth);
|
||||
}
|
||||
|
||||
const delivered = new Set<string>();
|
||||
|
||||
@@ -1221,11 +1221,19 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
|
||||
|
||||
/** Import ONE raw notes file: register the source (deterministic id per
|
||||
* file path — re-import keeps the same identity and its usage history)
|
||||
* and run the pipeline over it. */
|
||||
* and run the pipeline over it. Re-import guard (owner, 2026-09-29):
|
||||
* the file's content fingerprint decides — unchanged text is a TRUE
|
||||
* no-op (the mappings ledger used to dedupe this at merge time by
|
||||
* accident); changed text re-runs the whole file. */
|
||||
async function importNotes(input: NotesImportInput): Promise<EmissionStats> {
|
||||
const id = `src_${crypto.createHash("sha256").update(input.filePath).digest("hex").slice(0, 12)}`;
|
||||
const contentFp = crypto.createHash("sha256").update(input.text).digest("hex").slice(0, 16);
|
||||
if (graphStore.readSources().find((s) => s.id === id)?.contentFp === contentFp) {
|
||||
console.info(`[extraction ${id}] re-import with unchanged content (fp match) — skipping`);
|
||||
return { nodes: 0, edges: 0 };
|
||||
}
|
||||
const importedAt = new Date().toISOString();
|
||||
graphStore.upsertSource({ id, filePath: input.filePath, importedAt });
|
||||
graphStore.upsertSource({ id, filePath: input.filePath, importedAt, contentFp });
|
||||
return runNotesExtraction({ id, ...input, importedAt });
|
||||
}
|
||||
|
||||
|
||||
@@ -58,6 +58,12 @@ export interface NotesSource {
|
||||
filePath: string;
|
||||
/** ISO — when this file was imported (the checkpoint's foldedAt). */
|
||||
importedAt: string;
|
||||
/** sha256 of the imported text at the last EXTRACTION run — the
|
||||
* same-path re-import guard (owner, 2026-09-29): a matching
|
||||
* fingerprint is a TRUE no-op (no model calls, no usage record);
|
||||
* a mismatch or a legacy missing stamp re-runs the whole file.
|
||||
* The mappings ledger used to give this idempotency by accident. */
|
||||
contentFp?: string;
|
||||
/** The run's usage log — same append-only discipline as
|
||||
* ChatRecord.extractions, idempotent per startedAt. */
|
||||
extractions?: ExtractionRunUsage[];
|
||||
@@ -109,18 +115,6 @@ export interface GraphEdge {
|
||||
provenance: ProvenancePoint[];
|
||||
}
|
||||
|
||||
/** mappings.jsonl line — the session→graph identity map: the extraction
|
||||
* pipeline's proposition (identified by its run-local source id) → the
|
||||
* graph node it resolved to (created or linked). The audit trail of the
|
||||
* auditor's identity verdicts; also the chat-scoped resolution table the
|
||||
* merger uses to point events and edges at node ids. */
|
||||
export interface NodeMapping {
|
||||
chatId: string;
|
||||
/** The extractor's run-local proposition id ("p1", …). */
|
||||
sourceId: string;
|
||||
graphNodeId: string;
|
||||
}
|
||||
|
||||
/** sessions.jsonl line — the emission checkpoint for one chat (which chats
|
||||
* have been folded, and through which updatedAt — never fold twice). */
|
||||
export interface SessionCheckpoint {
|
||||
|
||||
@@ -13,7 +13,7 @@
|
||||
*
|
||||
* preserve-graph the chat's extraction bookkeeping dies — its coverage
|
||||
* rows and its legacy checkpoint lines — and NOTHING else:
|
||||
* nodes, edges, mappings stay byte-identical.
|
||||
* nodes and edges stay byte-identical.
|
||||
* The graph keeps the session's contributions; the
|
||||
* snapshot simply no longer resolves the dead chatId to
|
||||
* a title (nodeSessions filters unknown ids at read
|
||||
@@ -22,8 +22,8 @@
|
||||
* delete-nodes "this session never happened": every trace of the
|
||||
* chat leaves the ledgers — its provenance points on
|
||||
* every node and edge line, its link lines (their
|
||||
* phrasings die with them), its mapping lines (the audit trail), its
|
||||
* coverage/checkpoint rows — and orphans die: a node
|
||||
* phrasings die with them), its coverage/checkpoint
|
||||
* rows — and orphans die: a node
|
||||
* whose references all came from this chat disappears
|
||||
* entirely, pruned from the positions and embedding
|
||||
* ledgers too (dead members would break cluster hulls).
|
||||
@@ -57,7 +57,6 @@ import type {
|
||||
CoverageEntry,
|
||||
GraphEdge,
|
||||
GraphNode,
|
||||
NodeMapping,
|
||||
SessionCheckpoint,
|
||||
} from "./contracts.ts";
|
||||
import type { GraphStore } from "./ports.ts";
|
||||
@@ -74,8 +73,6 @@ export interface ChatDeletionStats {
|
||||
nodesRemoved: number;
|
||||
/** Edge assertion LINES removed (empty-provenance or dead endpoint). */
|
||||
edgesRemoved: number;
|
||||
/** Mapping lines removed (the chat's audit trail). */
|
||||
mappingsRemoved: number;
|
||||
/** Coverage rows removed. */
|
||||
coverageRemoved: number;
|
||||
/** Checkpoint lines removed. */
|
||||
@@ -85,7 +82,6 @@ export interface ChatDeletionStats {
|
||||
const EMPTY_STATS: ChatDeletionStats = {
|
||||
nodesRemoved: 0,
|
||||
edgesRemoved: 0,
|
||||
mappingsRemoved: 0,
|
||||
coverageRemoved: 0,
|
||||
checkpointsRemoved: 0,
|
||||
};
|
||||
@@ -121,7 +117,6 @@ export function deleteChatTraces(graphStore: GraphStore, chatId: string, mode: C
|
||||
// ── delete-nodes: the full purge ──────────────────────────────────────
|
||||
const nodeLines = graphStore.readNodes();
|
||||
const edgeLines = graphStore.readEdges();
|
||||
const mappings = graphStore.readMappings();
|
||||
|
||||
// Does the chat touch the graph at all? (A never-extracted chat still
|
||||
// needs its coverage rows dropped above, but no ledger rewrite — keeps
|
||||
@@ -129,8 +124,7 @@ export function deleteChatTraces(graphStore: GraphStore, chatId: string, mode: C
|
||||
// changed.)
|
||||
const touched =
|
||||
nodeLines.some((line) => line.provenance.some((pt) => pt.chatId === chatId)) ||
|
||||
edgeLines.some((line) => line.provenance.some((pt) => pt.chatId === chatId)) ||
|
||||
mappings.some((m) => m.chatId === chatId);
|
||||
edgeLines.some((line) => line.provenance.some((pt) => pt.chatId === chatId));
|
||||
if (!touched) {
|
||||
if (stats.coverageRemoved > 0 || stats.checkpointsRemoved > 0) {
|
||||
graphStore.rewriteLedgers({ coverage, checkpoints });
|
||||
@@ -153,9 +147,6 @@ export function deleteChatTraces(graphStore: GraphStore, chatId: string, mode: C
|
||||
for (const line of nodeLines) {
|
||||
if (survives(line.provenance)) live.add(line.id);
|
||||
}
|
||||
for (const mapping of mappings) {
|
||||
if (mapping.chatId !== chatId) live.add(mapping.graphNodeId);
|
||||
}
|
||||
|
||||
// 2. Node lines. Per-line (each line is ONE extraction run's append, so
|
||||
// per-session alias provenance exists at line granularity):
|
||||
@@ -209,17 +200,12 @@ export function deleteChatTraces(graphStore: GraphStore, chatId: string, mode: C
|
||||
outEdges.push(points.length === line.provenance.length ? line : { ...line, provenance: points });
|
||||
}
|
||||
|
||||
// 4. Mappings: the chat's rows die outright.
|
||||
const outMappings = mappings.filter((m) => m.chatId !== chatId);
|
||||
|
||||
stats.nodesRemoved = [...new Set(nodeLines.map((line) => line.id))].filter((id) => !live.has(id)).length;
|
||||
stats.edgesRemoved = edgeLines.length - outEdges.length;
|
||||
stats.mappingsRemoved = mappings.length - outMappings.length;
|
||||
|
||||
graphStore.rewriteLedgers({
|
||||
nodes: outNodes,
|
||||
edges: outEdges,
|
||||
mappings: outMappings,
|
||||
coverage,
|
||||
checkpoints,
|
||||
});
|
||||
@@ -272,14 +258,11 @@ export function deleteChatTraces(graphStore: GraphStore, chatId: string, mode: C
|
||||
export interface NodeDeletionStats {
|
||||
/** Edge assertion LINES removed (every edge touching the node). */
|
||||
edgesRemoved: number;
|
||||
/** Mapping lines removed (the node's audit trail). */
|
||||
mappingsRemoved: number;
|
||||
}
|
||||
|
||||
/** The confirm dialog's pre-flight facts (the GET on the same node id):
|
||||
* what full death would take with it. sessions is the deduped set of
|
||||
* chats/notes sources that reference the node — provenance points and
|
||||
* mapping lines count. */
|
||||
* chats/notes sources that reference the node — provenance points count. */
|
||||
export interface NodeDeletionImpact {
|
||||
edges: number;
|
||||
sessions: string[];
|
||||
@@ -298,9 +281,6 @@ export function nodeDeletionImpact(graphStore: GraphStore, nodeId: string): Node
|
||||
for (const line of nodeLinesOf(graphStore, nodeId)) {
|
||||
for (const pt of line.provenance) sessions.add(pt.chatId);
|
||||
}
|
||||
for (const mapping of graphStore.readMappings()) {
|
||||
if (mapping.graphNodeId === nodeId) sessions.add(mapping.chatId);
|
||||
}
|
||||
return {
|
||||
edges: edges.length,
|
||||
sessions: [...sessions].sort(),
|
||||
@@ -318,10 +298,7 @@ export function deleteNodeTraces(graphStore: GraphStore, nodeId: string): NodeDe
|
||||
if (nodeLines.length === 0) return null;
|
||||
|
||||
const edgeLines = graphStore.readEdges();
|
||||
const mappings = graphStore.readMappings();
|
||||
|
||||
const outEdges = edgeLines.filter((line) => line.from !== nodeId && line.to !== nodeId);
|
||||
const outMappings = mappings.filter((m) => m.graphNodeId !== nodeId);
|
||||
|
||||
// Only the touched ledgers: the node's own lines always die (its file
|
||||
// must move), everything else rewrites only when rows actually drop.
|
||||
@@ -329,7 +306,6 @@ export function deleteNodeTraces(graphStore: GraphStore, nodeId: string): NodeDe
|
||||
graphStore.rewriteLedgers({
|
||||
nodes: outNodes,
|
||||
...(outEdges.length !== edgeLines.length ? { edges: outEdges } : {}),
|
||||
...(outMappings.length !== mappings.length ? { mappings: outMappings } : {}),
|
||||
});
|
||||
|
||||
// The resting position dies with the node.
|
||||
@@ -341,6 +317,5 @@ export function deleteNodeTraces(graphStore: GraphStore, nodeId: string): NodeDe
|
||||
|
||||
return {
|
||||
edgesRemoved: edgeLines.length - outEdges.length,
|
||||
mappingsRemoved: mappings.length - outMappings.length,
|
||||
};
|
||||
}
|
||||
|
||||
+33
-44
@@ -55,7 +55,7 @@ import type {
|
||||
GraphSnapshot,
|
||||
} from "./contracts.ts";
|
||||
import { MIN_CLUSTER_SIZE } from "./clusters.ts";
|
||||
import { birthMappingIndex, birthSessionOf, birthSessionPartition } from "./session-clusters.ts";
|
||||
import { birthSessionOf, birthSessionPartition } from "./session-clusters.ts";
|
||||
import type { ChatStore, GraphStore } from "./ports.ts";
|
||||
|
||||
/** The storage ports every graph operation needs, passed explicitly. */
|
||||
@@ -101,21 +101,23 @@ const samePoint = (a: ProvenancePoint, b: ProvenancePoint): boolean =>
|
||||
// ─── the merger (extraction pipeline stage 3 — deterministic) ────────────────
|
||||
|
||||
/**
|
||||
* Merge one chat's audited findings into the graph ledgers — the pipeline's
|
||||
* apply stage. Deterministic + idempotent: re-merging the same verdicts (or
|
||||
* a later re-extraction of the same chat) appends nothing but the new.
|
||||
* Merge one job's audited findings into the graph ledgers — the pipeline's
|
||||
* apply stage. Deterministic. Each chunk/batch job merges INDEPENDENTLY
|
||||
* (owner, 2026-09-29): resolution is a job-local in-memory map from this
|
||||
* submit's proposition ids to graph node ids — nothing persists between
|
||||
* jobs, so no job can see (or collide with) another's ids. Cross-job
|
||||
* identity is the auditor's alone: it links this job's propositions to
|
||||
* existing nodes before this function is ever called.
|
||||
*
|
||||
* Identity is the auditor's link-or-create verdict: `linkTo` an existing
|
||||
* node (validated against the ledger — an unknown id falls back to create)
|
||||
* appends a LINK line (the session's phrasings join the node's aliases and
|
||||
* the provenance unions; getGraph merges lines by id); absent → a new node.
|
||||
* Mapping lines record each verdict: one per (chat, sourceId). Idempotent:
|
||||
* a proposition id already mapped for this chat reuses its resolution.
|
||||
* Identity inside the job is the auditor's link-or-create verdict: `linkTo`
|
||||
* an existing node (validated against the ledger — an unknown id falls
|
||||
* back to create) appends a LINK line (the job's phrasings join the node's
|
||||
* aliases and the provenance unions; getGraph merges lines by id);
|
||||
* absent → a new node.
|
||||
*
|
||||
* `chatId` is the run's source key: a real chat, or a notes source id
|
||||
* ("src_…") for the format-agnostic import path — same ledgers, same
|
||||
* idempotency, one graph. The `source` parameter carries the checkpoint
|
||||
* stamp; the merger itself is chat-store-free.
|
||||
* `chatId` is the job's source key: a real chat, or a notes source id
|
||||
* ("src_…") for the format-agnostic import path. The `source` parameter
|
||||
* carries the checkpoint stamp; the merger itself is chat-store-free.
|
||||
*/
|
||||
export function mergeFindings(
|
||||
chatId: string,
|
||||
@@ -127,26 +129,22 @@ export function mergeFindings(
|
||||
const { graphStore: ledgers } = deps;
|
||||
const stats: EmissionStats = { nodes: 0, edges: 0 };
|
||||
|
||||
// 1. Resolution: this run's propositions → graph node ids. Existing
|
||||
// mapping lines for THIS chat are loaded first — the audit trail doubles
|
||||
// as the resolution table, which makes re-merging the same chat
|
||||
// idempotent (a re-extraction re-merges through the auditor's linkTo
|
||||
// verdicts instead). Fresh sourceIds resolve as the verdicts say.
|
||||
// 1. Resolution: THIS job's propositions → graph node ids, built
|
||||
// in-memory as the propositions resolve (owner, 2026-09-29: the
|
||||
// mappings ledger is gone — a persisted table was a shared namespace
|
||||
// across independent jobs, and cross-job id collisions silently
|
||||
// dropped propositions and welded wrong edges). Dedupe across runs
|
||||
// is the coverage ledger (nothing re-extracts) and the auditor's
|
||||
// equivalence links (re-encountered knowledge merges in).
|
||||
const existingIds = new Set(ledgers.readNodes().map((n) => n.id));
|
||||
const mapping = new Map<string, string>();
|
||||
const seenMappingKeys = new Set<string>();
|
||||
for (const m of ledgers.readMappings()) {
|
||||
if (m.chatId !== chatId) continue;
|
||||
mapping.set(m.sourceId, m.graphNodeId);
|
||||
seenMappingKeys.add(m.sourceId);
|
||||
}
|
||||
const now = new Date().toISOString();
|
||||
const linkOf = new Map(verdicts.links.map((l) => [l.id, l.linkTo]));
|
||||
|
||||
// VERDICTS-ONLY (owner, 2026-09-28): the propositions come from the
|
||||
// extractor's findings verbatim; the auditor's links join here by id.
|
||||
for (const p of findings.propositions) {
|
||||
if (mapping.has(p.id)) continue; // already resolved for this chat — idempotent
|
||||
if (mapping.has(p.id)) continue; // duplicate id WITHIN this submit — first resolution stands
|
||||
const points = pointOf(chatId);
|
||||
const linkTo = linkOf.get(p.id);
|
||||
let graphNodeId: string;
|
||||
@@ -180,20 +178,14 @@ export function mergeFindings(
|
||||
existingIds.add(graphNodeId);
|
||||
}
|
||||
mapping.set(p.id, graphNodeId);
|
||||
if (!seenMappingKeys.has(p.id)) {
|
||||
ledgers.appendMapping({ chatId, sourceId: p.id, graphNodeId });
|
||||
seenMappingKeys.add(p.id);
|
||||
}
|
||||
}
|
||||
|
||||
// 2. Edges — the extractor's own first, then the auditor's BRIDGING
|
||||
// edges (endpoints may be local ids and/or EXISTING graph node ids —
|
||||
// the auditor welding the batch into the graph it explored). Both
|
||||
// resolve through the mapping, both carry the run's chat-level
|
||||
// provenance point, and both dedupe by (from,to,type) already
|
||||
// asserted by THIS chat: an echoed duplicate (a weak auditor
|
||||
// restating an extractor edge) merges to one append; an assertion
|
||||
// by another chat stays untouched (provenance unions at read time).
|
||||
// 2. Edges — the extractor's own, the ONLY edge source (the auditor's
|
||||
// edges argument is gone, owner 2026-09-28 evening). Endpoints are
|
||||
// this job's local proposition ids and resolve through the job-local
|
||||
// map; edges carry the run's chat-level provenance point and dedupe
|
||||
// by (from,to,type) already asserted by THIS chat — an assertion by
|
||||
// another chat stays untouched (provenance unions at read time).
|
||||
const assertedByChat = new Set<string>();
|
||||
for (const line of ledgers.readEdges()) {
|
||||
const key = `${line.from}\u0000${line.to}\u0000${line.type}`;
|
||||
@@ -295,7 +287,6 @@ export function graphSnapshot(deps: GraphDeps): GraphSnapshot {
|
||||
}
|
||||
chats.add(chatId);
|
||||
};
|
||||
for (const m of deps.graphStore.readMappings()) touch(m.graphNodeId, m.chatId);
|
||||
for (const node of nodes) {
|
||||
for (const pt of node.provenance) touch(node.id, pt.chatId);
|
||||
}
|
||||
@@ -339,7 +330,7 @@ export function graphSnapshot(deps: GraphDeps): GraphSnapshot {
|
||||
// presentation rule (a region needs >= 3 propositions for a hull). A
|
||||
// birth session whose record is gone (deleted chat) keeps its members
|
||||
// but renders unlabeled — honest, never guessed.
|
||||
const communities = birthSessionPartition(nodes, deps.graphStore.readMappings());
|
||||
const communities = birthSessionPartition(nodes);
|
||||
const clusters: Cluster[] = communities
|
||||
.filter((community) => community.members.length >= MIN_CLUSTER_SIZE)
|
||||
.map((community) => ({
|
||||
@@ -396,8 +387,7 @@ export interface GraphSessionEntry {
|
||||
export function graphSessionIndex(deps: GraphDeps): GraphSessionEntry[] {
|
||||
const counts = new Map<string, number>();
|
||||
const born = getGraph(deps.graphStore);
|
||||
const mappingFallback = birthMappingIndex(deps.graphStore.readMappings());
|
||||
for (const cluster of birthSessionPartition(born.nodes, deps.graphStore.readMappings())) {
|
||||
for (const cluster of birthSessionPartition(born.nodes)) {
|
||||
counts.set(cluster.id, cluster.members.length);
|
||||
}
|
||||
const chats = new Map(deps.chatStore.listChats().map((chat) => [chat.id, chat]));
|
||||
@@ -445,8 +435,7 @@ export interface GraphSessionProposition {
|
||||
/** The propositions BORN in one session — what that session first
|
||||
* articulated. Members sorted by node id — deterministic. */
|
||||
export function graphSessionPropositions(deps: GraphDeps, sessionId: string): GraphSessionProposition[] {
|
||||
const mappingFallback = birthMappingIndex(deps.graphStore.readMappings());
|
||||
const born = getGraph(deps.graphStore).nodes.filter((node) => birthSessionOf(node, mappingFallback) === sessionId);
|
||||
const born = getGraph(deps.graphStore).nodes.filter((node) => birthSessionOf(node) === sessionId);
|
||||
return born
|
||||
.map((node) => ({ nodeId: node.id, name: node.name }))
|
||||
.sort((a, b) => (a.nodeId < b.nodeId ? -1 : 1));
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import type { GraphEdge, GraphNode, NodeMapping, Position } from "./contracts.ts";
|
||||
import type { GraphEdge, GraphNode, Position } from "./contracts.ts";
|
||||
import { incrementalLayout } from "./layout.ts";
|
||||
import { birthSessionPartition } from "./session-clusters.ts";
|
||||
|
||||
@@ -19,7 +19,6 @@ import { birthSessionPartition } from "./session-clusters.ts";
|
||||
export function settleGraphLayout(graphStore: {
|
||||
readNodes(): GraphNode[];
|
||||
readEdges(): GraphEdge[];
|
||||
readMappings(): NodeMapping[];
|
||||
readPositions(): Record<string, Position>;
|
||||
writePositions(positions: Record<string, Position>): void;
|
||||
}): void {
|
||||
@@ -39,7 +38,7 @@ export function settleGraphLayout(graphStore: {
|
||||
// core/session-clusters.ts, wave 7) — one cluster concept everywhere.
|
||||
// The unique list is the deduped set the partition needs (lines may
|
||||
// repeat an id — first defines).
|
||||
const communities = birthSessionPartition(unique, graphStore.readMappings());
|
||||
const communities = birthSessionPartition(unique);
|
||||
const next = incrementalLayout(unique, edges, previous, { clusters: communities });
|
||||
graphStore.writePositions(Object.fromEntries(next));
|
||||
}
|
||||
@@ -19,7 +19,6 @@ import type {
|
||||
JobKind,
|
||||
JobRecord,
|
||||
ExtractionRunUsage,
|
||||
NodeMapping,
|
||||
PlanTree,
|
||||
Position,
|
||||
CoverageEntry,
|
||||
@@ -84,10 +83,6 @@ export interface GraphStore {
|
||||
|
||||
/** Append one quiz outcome / understanding observation. */
|
||||
|
||||
/** Append one identity-map line (chat's tree node → graph node). */
|
||||
appendMapping(mapping: NodeMapping): void;
|
||||
readMappings(): NodeMapping[];
|
||||
|
||||
/** Append one emission checkpoint. */
|
||||
appendCheckpoint(checkpoint: SessionCheckpoint): void;
|
||||
readCheckpoints(): SessionCheckpoint[];
|
||||
@@ -127,7 +122,6 @@ export interface GraphStore {
|
||||
rewriteLedgers(next: {
|
||||
nodes?: GraphNode[];
|
||||
edges?: GraphEdge[];
|
||||
mappings?: NodeMapping[];
|
||||
coverage?: CoverageEntry[];
|
||||
checkpoints?: SessionCheckpoint[];
|
||||
}): void;
|
||||
|
||||
@@ -21,24 +21,18 @@
|
||||
*
|
||||
* Notes imports are birth-sessions too (owner ruling 2026-09-23): their
|
||||
* nodes carry a "src_…" source id instead of a chat id; the caller labels
|
||||
* them with the file's path (kind: file in the session views). The
|
||||
* mapping-line fallback (owner-approved 2026-09-23, surfaced by the
|
||||
* interlude's live smoke): production notes imports create provenance-
|
||||
* EMPTY nodes (the notes front door passes messages: null — the file's
|
||||
* anchor is the mapping line), so a provenance-empty node's birth session
|
||||
* is its FIRST mapping line's chatId in ledger order. Only notes-born
|
||||
* nodes ever hit the fallback — chat extraction always writes provenance
|
||||
* points, and a re-encountered node's link-append adds one, which
|
||||
* re-opens the provenance rule above. Deletion-safe: a deleted chat's
|
||||
* mapping lines are stripped alongside its provenance; notes mappings
|
||||
* survive because files are never deleted.
|
||||
* them with the file's path (kind: file in the session views). The old
|
||||
* mapping-line fallback for provenance-empty notes nodes is GONE (owner,
|
||||
* 2026-09-29): every merge writes provenance points, and the mappings
|
||||
* ledger itself died with per-job in-memory resolution — provenance is
|
||||
* the one birth record.
|
||||
*
|
||||
* Browser-safe pure logic (no node builtins, no fs): same contract style
|
||||
* as the rest of the core graph derivation. Not re-exported through the
|
||||
* core barrel — the shell reads clusters from the snapshot; only the
|
||||
* server-side derivations (snapshot, settle) and tests import this.
|
||||
*/
|
||||
import type { GraphNode, NodeMapping } from "./contracts.ts";
|
||||
import type { GraphNode } from "./contracts.ts";
|
||||
|
||||
/** One birth-session cluster: the session id + the SORTED member node
|
||||
* ids born in it. The shape is deliberately the { members } contract
|
||||
@@ -51,44 +45,26 @@ export interface BirthSessionCluster {
|
||||
members: string[];
|
||||
}
|
||||
|
||||
/** A node's birth session: its first surviving provenance point's chatId;
|
||||
* for a provenance-empty node, its first mapping line's chatId (the notes
|
||||
* fallback — see the module doc). null when neither exists. */
|
||||
export function birthSessionOf(node: GraphNode, mappingFallback?: ReadonlyMap<string, string>): string | null {
|
||||
const first = node.provenance[0];
|
||||
if (first !== undefined) return first.chatId;
|
||||
return mappingFallback?.get(node.id) ?? null;
|
||||
}
|
||||
|
||||
/** The notes fallback index: nodeId → the chatId of the node's FIRST
|
||||
* mapping line in ledger order. */
|
||||
export function birthMappingIndex(mappings: readonly NodeMapping[]): Map<string, string> {
|
||||
const index = new Map<string, string>();
|
||||
for (const mapping of mappings) {
|
||||
if (!index.has(mapping.graphNodeId)) index.set(mapping.graphNodeId, mapping.chatId);
|
||||
}
|
||||
return index;
|
||||
/** A node's birth session: its first surviving provenance point's chatId.
|
||||
* null when the node carries no provenance at all (legacy data). */
|
||||
export function birthSessionOf(node: GraphNode): string | null {
|
||||
return node.provenance[0]?.chatId ?? null;
|
||||
}
|
||||
|
||||
/**
|
||||
* The birth-session partition over the given node LINES: dedupes by id
|
||||
* (first line defines — the getGraph rule), groups by birth session, and
|
||||
* returns the groups size-desc with members sorted (deterministic output:
|
||||
* ties break by session id ascending). Nodes with no surviving provenance
|
||||
* fall back to their first mapping line (notes-born); a node with neither
|
||||
* is left unclustered — it belongs to no session the graph can name.
|
||||
* ties break by session id ascending). A node with no provenance is left
|
||||
* unclustered — it belongs to no session the graph can name.
|
||||
*/
|
||||
export function birthSessionPartition(
|
||||
nodes: readonly GraphNode[],
|
||||
mappings: readonly NodeMapping[] = [],
|
||||
): BirthSessionCluster[] {
|
||||
const mappingFallback = birthMappingIndex(mappings);
|
||||
export function birthSessionPartition(nodes: readonly GraphNode[]): BirthSessionCluster[] {
|
||||
const members = new Map<string, string[]>();
|
||||
const seen = new Set<string>();
|
||||
for (const node of nodes) {
|
||||
if (seen.has(node.id)) continue;
|
||||
seen.add(node.id);
|
||||
const birth = birthSessionOf(node, mappingFallback);
|
||||
const birth = birthSessionOf(node);
|
||||
if (birth === null) continue;
|
||||
const bucket = members.get(birth);
|
||||
if (bucket) bucket.push(node.id);
|
||||
|
||||
@@ -57,7 +57,6 @@ export function graphRevision(): number {
|
||||
for (const file of [
|
||||
LEDGER_FILES.nodes,
|
||||
LEDGER_FILES.edges,
|
||||
LEDGER_FILES.mappings,
|
||||
LEDGER_FILES.sessions,
|
||||
LEDGER_FILES.coverage,
|
||||
POSITIONS_FILE,
|
||||
|
||||
@@ -10,8 +10,6 @@
|
||||
* edges.jsonl GraphEdge — one line per edge ASSERTION (a second
|
||||
* session asserting the same (from,to,type)
|
||||
* appends another line; reads merge)
|
||||
* mappings.jsonl NodeMapping — the session→graph identity map:
|
||||
* (chatId, sourceId) → graphNodeId
|
||||
* sessions.jsonl SessionCheckpoint— emission checkpoints (foldedAt)
|
||||
*
|
||||
* Plus config.SOURCES_FILE (sources.json): the imported-notes bookkeeping
|
||||
@@ -34,7 +32,6 @@ import type {
|
||||
GraphNode,
|
||||
ExtractionRunUsage,
|
||||
NotesSource,
|
||||
NodeMapping,
|
||||
Position,
|
||||
ProvenancePoint,
|
||||
CoverageEntry,
|
||||
@@ -107,9 +104,6 @@ const isGraphEdge = (v: unknown): v is GraphEdge =>
|
||||
Array.isArray(v.provenance) &&
|
||||
v.provenance.every(isProvenancePoint);
|
||||
|
||||
const isNodeMapping = (v: unknown): v is NodeMapping =>
|
||||
isRecord(v) && isString(v.chatId) && isString(v.sourceId) && isString(v.graphNodeId);
|
||||
|
||||
const isCheckpoint = (v: unknown): v is SessionCheckpoint =>
|
||||
isRecord(v) && isString(v.chatId) && isString(v.foldedAt);
|
||||
|
||||
@@ -137,13 +131,6 @@ export const graphStore: GraphStore = {
|
||||
return readValid(LEDGER_FILES.edges, isGraphEdge);
|
||||
},
|
||||
|
||||
appendMapping(mapping: NodeMapping): void {
|
||||
appendLine(LEDGER_FILES.mappings, mapping);
|
||||
},
|
||||
readMappings(): NodeMapping[] {
|
||||
return readValid(LEDGER_FILES.mappings, isNodeMapping);
|
||||
},
|
||||
|
||||
appendCheckpoint(checkpoint: SessionCheckpoint): void {
|
||||
appendLine(LEDGER_FILES.sessions, checkpoint);
|
||||
},
|
||||
@@ -198,14 +185,12 @@ export const graphStore: GraphStore = {
|
||||
rewriteLedgers(next: {
|
||||
nodes?: GraphNode[];
|
||||
edges?: GraphEdge[];
|
||||
mappings?: NodeMapping[];
|
||||
coverage?: CoverageEntry[];
|
||||
checkpoints?: SessionCheckpoint[];
|
||||
}): void {
|
||||
const files = {
|
||||
nodes: LEDGER_FILES.nodes,
|
||||
edges: LEDGER_FILES.edges,
|
||||
mappings: LEDGER_FILES.mappings,
|
||||
coverage: LEDGER_FILES.coverage,
|
||||
checkpoints: LEDGER_FILES.sessions,
|
||||
} as const;
|
||||
|
||||
+36
-26
@@ -35,7 +35,7 @@ import type {
|
||||
ExtractionRunUsage,
|
||||
GraphEdge,
|
||||
GraphNode,
|
||||
NodeMapping,
|
||||
|
||||
NotesSource,
|
||||
SessionCheckpoint,
|
||||
SessionFindings,
|
||||
@@ -65,7 +65,6 @@ function fauxEngine(steps: FauxResponseStep[]): PiEngine {
|
||||
function memoryLedgers() {
|
||||
const nodes: GraphNode[] = [];
|
||||
const edges: GraphEdge[] = [];
|
||||
const mappings: NodeMapping[] = [];
|
||||
const checkpoints: SessionCheckpoint[] = [];
|
||||
const coverage: CoverageEntry[] = [];
|
||||
const sources = new Map<string, NotesSource>();
|
||||
@@ -75,8 +74,6 @@ function memoryLedgers() {
|
||||
readNodes: () => nodes.map((n) => ({ ...n })),
|
||||
appendEdge: (e: GraphEdge) => void edges.push(e),
|
||||
readEdges: () => edges.map((e) => ({ ...e })),
|
||||
appendMapping: (m: NodeMapping) => void mappings.push(m),
|
||||
readMappings: () => mappings.map((m) => ({ ...m })),
|
||||
appendCheckpoint: (c: SessionCheckpoint) => void checkpoints.push(c),
|
||||
readCheckpoints: () => checkpoints.map((c) => ({ ...c })),
|
||||
// Extraction coverage (vision-extraction-coverage.md) — append-only;
|
||||
@@ -108,7 +105,6 @@ function memoryLedgers() {
|
||||
rewriteLedgers: (next) => {
|
||||
if (next.nodes) nodes.splice(0, nodes.length, ...next.nodes.map((n) => ({ ...n, provenance: [...n.provenance], aliases: [...n.aliases] })));
|
||||
if (next.edges) edges.splice(0, edges.length, ...next.edges.map((e) => ({ ...e, provenance: [...e.provenance] })));
|
||||
if (next.mappings) mappings.splice(0, mappings.length, ...next.mappings.map((m) => ({ ...m })));
|
||||
if (next.coverage) coverage.splice(0, coverage.length, ...next.coverage.map((c) => ({ ...c })));
|
||||
if (next.checkpoints) checkpoints.splice(0, checkpoints.length, ...next.checkpoints.map((c) => ({ ...c })));
|
||||
},
|
||||
@@ -419,11 +415,11 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
|
||||
// Link-append: the phrasing joined the existing node's aliases and
|
||||
// the provenance unioned; p2's create line is its own node.
|
||||
assert.equal(ledgers.readNodes().length, 3); // pre-seeded + link line + p2's create
|
||||
const mappings = ledgers.readMappings();
|
||||
assert.equal(mappings.length, 2);
|
||||
assert.deepEqual(mappings[0], { chatId, sourceId: "p1", graphNodeId: existingId });
|
||||
assert.equal(mappings[1]!.sourceId, "p2");
|
||||
assert.match(mappings[1]!.graphNodeId, /^n_/);
|
||||
const linkedLines = ledgers.readNodes().filter((n) => n.id === existingId);
|
||||
assert.ok(
|
||||
linkedLines.some((line) => line.provenance.some((pt) => pt.chatId === chatId)),
|
||||
"the link-append unions provenance",
|
||||
);
|
||||
|
||||
// COVERAGE checkpointed per message (not a whole-chat checkpoint):
|
||||
// one entry per transcript message, fingerprinted content each —
|
||||
@@ -1021,7 +1017,6 @@ describe("the notes front door (importNotes — format-agnostic)", () => {
|
||||
const node = ledgers.readNodes()[0]!;
|
||||
assert.equal(node.statement, "Vim starts in Normal mode, where keys are commands.");
|
||||
assert.deepEqual(node.provenance, [{ chatId: source.id, messageId: "" }]);
|
||||
assert.deepEqual(ledgers.readMappings(), [{ chatId: source.id, sourceId: "p1", graphNodeId: node.id }]);
|
||||
// The checkpoint stamps the import.
|
||||
assert.equal(ledgers.readCheckpoints()[0]!.chatId, source.id);
|
||||
});
|
||||
@@ -1062,10 +1057,12 @@ describe("the notes front door (importNotes — format-agnostic)", () => {
|
||||
assert.ok(calls >= 2, `expected ≥2 extractor calls, got ${calls}`);
|
||||
assert.match(views[0]!, new RegExp(`part 1/${calls}`));
|
||||
assert.match(views.at(-1)!, new RegExp(`part ${calls}/${calls}`));
|
||||
// Same findings in every chunk → ONE node total (mapping dedupe
|
||||
// across chunks), stats never double-count.
|
||||
assert.deepEqual(stats, { nodes: 1, edges: 0 });
|
||||
assert.equal(ledgers.readNodes().length, 1);
|
||||
// Same findings in every chunk → one node PER CHUNK: jobs merge
|
||||
// independently (owner, 2026-09-29 — the mappings ledger is gone);
|
||||
// cross-chunk identity is the auditor's linkTo verdict, and these
|
||||
// fixtures submit none.
|
||||
assert.deepEqual(stats, { nodes: calls, edges: 0 });
|
||||
assert.equal(ledgers.readNodes().length, calls);
|
||||
});
|
||||
|
||||
it("required top-level fields (owner, 2026-09-28): a findings submit MISSING edges fails validation — no silent edge-less merge", async () => {
|
||||
@@ -1098,7 +1095,7 @@ describe("the notes front door (importNotes — format-agnostic)", () => {
|
||||
assert.equal(ledgers.readCoverage().length, 0, "nothing checkpointed — the messages stay re-runnable");
|
||||
});
|
||||
|
||||
it("re-importing the same file is idempotent: same source identity, refs dedupe, usage history grows", async () => {
|
||||
it("re-importing is fingerprint-guarded: unchanged text is a TRUE no-op; changed text re-runs (owner, 2026-09-29)", async () => {
|
||||
const ledgers = memoryLedgers();
|
||||
const mk = () =>
|
||||
createExtractionPipeline({
|
||||
@@ -1110,12 +1107,23 @@ describe("the notes front door (importNotes — format-agnostic)", () => {
|
||||
const first = await mk().importNotes({ filePath: "/vault/learn vim.md", text: NOTES_TEXT });
|
||||
assert.deepEqual(first, { nodes: 1, edges: 0 });
|
||||
|
||||
// Unchanged text: the content fingerprint matches — no extraction
|
||||
// at all (no model calls, no usage record; the mappings ledger
|
||||
// used to give this no-op at merge time by accident).
|
||||
const second = await mk().importNotes({ filePath: "/vault/learn vim.md", text: NOTES_TEXT });
|
||||
assert.deepEqual(second, { nodes: 0, edges: 0 }, "the same source id dedupes everything");
|
||||
assert.deepEqual(second, { nodes: 0, edges: 0 }, "unchanged content is a true no-op");
|
||||
|
||||
assert.equal(ledgers.readSources().length, 1, "one source identity — the path hashes to the same id");
|
||||
assert.equal(ledgers.readSources()[0]!.extractions?.length, 2, "both runs billed their usage");
|
||||
assert.equal(ledgers.readSources()[0]!.extractions?.length, 1, "only the real run billed its usage");
|
||||
assert.equal(ledgers.readNodes().length, 1, "no duplicate node");
|
||||
|
||||
// Changed text under the same path: the fingerprint misses — the
|
||||
// whole file re-runs (fresh merge; the auditor's links would dedupe
|
||||
// identity on a real model).
|
||||
const third = await mk().importNotes({ filePath: "/vault/learn vim.md", text: `${NOTES_TEXT}\nA new closing line.` });
|
||||
assert.deepEqual(third, { nodes: 1, edges: 0 }, "changed content re-extracts");
|
||||
assert.equal(ledgers.readSources()[0]!.extractions?.length, 2, "the re-run bills its usage");
|
||||
assert.equal(ledgers.readNodes().length, 2);
|
||||
});
|
||||
|
||||
it("aborts like a chat run: true cancellation, nothing merged, partial usage in the source record", async () => {
|
||||
@@ -1221,9 +1229,11 @@ describe("the chunked pipeline (extraction coverage)", () => {
|
||||
assert.ok(views[0]!.startsWith("# Chunky coverage fixture"));
|
||||
|
||||
// Batch 1 merged the full findings (verdicts-only: the extractor's
|
||||
// output always fully merges); batch 2's identical ids mapped to
|
||||
// the same nodes (chat-scoped mapping) — nothing new.
|
||||
assert.deepEqual(stats, { nodes: 2, edges: 1 });
|
||||
// output always fully merges); batch 2's identical ids are a
|
||||
// SEPARATE job's namespace (owner, 2026-09-29 — no mapping table)
|
||||
// — its propositions merge again as fresh nodes. Cross-batch
|
||||
// identity is the auditor's linkTo verdict, never the merger's.
|
||||
assert.deepEqual(stats, { nodes: 4, edges: 2 });
|
||||
|
||||
// Per-batch coverage: every message checkpointed, nothing legacy.
|
||||
const coverage = ledgers.readCoverage();
|
||||
@@ -1464,10 +1474,11 @@ describe("per-chunk parallel jobs (owner, 2026-09-28)", () => {
|
||||
|
||||
assert.equal(extractPeak, 3, "8 chunks, 3 permits — the peak pins both the fan-out and the cap");
|
||||
assert.equal(auditPeak, 1, "audits serialize behind the gate — never two at once");
|
||||
// Identical findings in every chunk: one merge, then dedupe — the
|
||||
// aggregate counts real emissions, never double.
|
||||
assert.deepEqual(stats, { nodes: 2, edges: 1 });
|
||||
assert.equal(ledgers.readNodes().length, 2);
|
||||
// Identical findings in every chunk, one merge PER chunk (jobs are
|
||||
// independent — owner, 2026-09-29): 8 chunks × 2 propositions,
|
||||
// 8 × 1 edge. The auditor's links, not the merger, dedupe identity.
|
||||
assert.deepEqual(stats, { nodes: 16, edges: 8 });
|
||||
assert.equal(ledgers.readNodes().length, 16);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1496,7 +1507,6 @@ describe("the session-deletion guard (chat deleted mid-run)", () => {
|
||||
// trace for a later snapshot to resurrect.
|
||||
assert.equal(ledgers.readNodes().length, 0);
|
||||
assert.equal(ledgers.readEdges().length, 0);
|
||||
assert.equal(ledgers.readMappings().length, 0);
|
||||
assert.equal(ledgers.readCoverage().length, 0);
|
||||
assert.equal(ledgers.readCheckpoints().length, 0);
|
||||
|
||||
|
||||
@@ -37,14 +37,12 @@ function fakeStores(
|
||||
nodes: GraphNode[],
|
||||
over: {
|
||||
chats?: Array<Pick<ChatSummary, "id" | "title" | "createdAt" | "updatedAt">>;
|
||||
mappings?: Array<{ chatId: string; sourceId: string; graphNodeId: string }>;
|
||||
} = {},
|
||||
): { chatStore: ChatStore; graphStore: GraphStore } {
|
||||
const edges: GraphEdge[] = [];
|
||||
const graphStore: GraphStore = {
|
||||
readNodes: () => nodes,
|
||||
readEdges: () => edges,
|
||||
readMappings: () => over.mappings ?? [],
|
||||
readCheckpoints: () => [],
|
||||
readCoverage: () => [],
|
||||
readSources: () => [],
|
||||
@@ -217,12 +215,7 @@ describe("auditor graph tools", () => {
|
||||
});
|
||||
|
||||
it("list_propositions: FULL statement text (identity work), no dates", async () => {
|
||||
const { chatStore: cs, graphStore: gs } = fakeStores(nodes, {
|
||||
mappings: [
|
||||
{ chatId: "c1", sourceId: "p1", graphNodeId: "n1" },
|
||||
{ chatId: "c1", sourceId: "p2", graphNodeId: "n2" },
|
||||
],
|
||||
});
|
||||
const { chatStore: cs, graphStore: gs } = fakeStores(nodes);
|
||||
const tools = auditorGraphTools({ chatStore: cs, graphStore: gs });
|
||||
const out = (await tools.list_propositions!.execute!("tc", { session: "c1" })).details as string;
|
||||
assert.match(out, /2 proposition\(s\) born in this session:/);
|
||||
|
||||
+57
-61
@@ -272,14 +272,8 @@ let chat3Id = "";
|
||||
* emissions run concurrently against the same files — scope, never assume). */
|
||||
function scopedGraph(chatIds: string[]) {
|
||||
const graph = getGraph(graphStore);
|
||||
const nodeIds = new Set(
|
||||
graphStore
|
||||
.readMappings()
|
||||
.filter((m) => chatIds.includes(m.chatId))
|
||||
.map((m) => m.graphNodeId),
|
||||
);
|
||||
return {
|
||||
nodes: graph.nodes.filter((n) => nodeIds.has(n.id)),
|
||||
nodes: graph.nodes.filter((n) => n.provenance.some((pt) => chatIds.includes(pt.chatId))),
|
||||
edges: graph.edges.filter((e) => e.provenance.some((pt) => chatIds.includes(pt.chatId))),
|
||||
};
|
||||
}
|
||||
@@ -312,21 +306,45 @@ describe("the merger (mergeFindings)", () => {
|
||||
assert.equal(edge.type, "follows");
|
||||
assert.deepEqual(edge.provenance, [{ chatId: chat1.id, messageId: "" }]);
|
||||
|
||||
// Mapping lines: one per (chat, sourceId).
|
||||
assert.deepEqual(
|
||||
graphStore.readMappings().map((m) => m.sourceId),
|
||||
["p1", "p2", "p3"],
|
||||
);
|
||||
});
|
||||
|
||||
it("is idempotent — re-merging the same verdicts appends nothing", () => {
|
||||
const stats = mergeFindings(chat1.id, fixtureFindings(), fixtureVerdicts(), graphDeps, sessionSource(chat1.id));
|
||||
assert.deepEqual(stats, { nodes: 0, edges: 0 });
|
||||
const graph = scopedGraph([chat1.id]);
|
||||
assert.equal(graph.nodes.length, 3);
|
||||
assert.equal(graph.edges.length, 1);
|
||||
it("merges INDEPENDENTLY per call — no cross-run identity (owner, 2026-09-29)", () => {
|
||||
// The mappings ledger is gone: resolution is a job-local in-memory
|
||||
// map, so a second merge of the same verdicts knows nothing of the
|
||||
// first. Cross-run identity is the AUDITOR's job (linkTo) plus the
|
||||
// coverage ledger (nothing re-extracts) — never the merger's.
|
||||
// (Own chat + unique statements — this suite stacks on shared
|
||||
// files, so nothing here may touch the other tests' counts.)
|
||||
const rep = fixtureChat("Independence fixture", [user("u1", "rep")].flat());
|
||||
const findings: SessionFindings = {
|
||||
propositions: [
|
||||
{ id: "p1", name: "Rep one", statement: "Independence one statement", aliases: [] },
|
||||
{ id: "p2", name: "Rep two", statement: "Independence two statement", aliases: [] },
|
||||
],
|
||||
edges: [{ from: "p1", to: "p2", type: "follows" }],
|
||||
};
|
||||
const first = mergeFindings(rep.id, findings, { links: [] }, graphDeps, sessionSource(rep.id));
|
||||
assert.deepEqual(first, { nodes: 2, edges: 1 });
|
||||
const second = mergeFindings(rep.id, findings, { links: [] }, graphDeps, sessionSource(rep.id));
|
||||
assert.deepEqual(second, { nodes: 2, edges: 1 }, "nothing remembered — everything re-creates");
|
||||
const graph = scopedGraph([rep.id]);
|
||||
assert.equal(graph.nodes.length, 4, "the propositions merge again as fresh nodes");
|
||||
assert.equal(graph.edges.length, 2, "fresh endpoints — fresh edge assertions");
|
||||
// exactly one checkpoint line per merge, though — checkpoints are a log
|
||||
assert.equal(graphStore.readCheckpoints().filter((c) => c.chatId === chat1.id).length, 2);
|
||||
assert.equal(graphStore.readCheckpoints().filter((c) => c.chatId === rep.id).length, 2);
|
||||
});
|
||||
|
||||
it("is idempotent WITHIN one submit — a repeated proposition id resolves once", () => {
|
||||
const dup = fixtureChat("Duplicate-id fixture", [user("u1", "dup")].flat());
|
||||
const findings: SessionFindings = {
|
||||
propositions: [
|
||||
{ id: "p1", name: "Dup", statement: "Dup statement", aliases: [] },
|
||||
{ id: "p1", name: "Dup", statement: "Dup statement", aliases: [] },
|
||||
],
|
||||
edges: [],
|
||||
};
|
||||
const stats = mergeFindings(dup.id, findings, { links: [] }, graphDeps, sessionSource(dup.id));
|
||||
assert.deepEqual(stats, { nodes: 1, edges: 0 }, "the duplicate id is skipped — first resolution stands");
|
||||
});
|
||||
|
||||
it("applies a linkTo verdict: phrasings join aliases, provenance unions, no duplicate node", () => {
|
||||
@@ -374,9 +392,6 @@ describe("the merger (mergeFindings)", () => {
|
||||
alpha.provenance.some((pt) => pt.chatId === chat2.id),
|
||||
"provenance unions",
|
||||
);
|
||||
// The mapping records the verdict: (chat2, q1) → the linked node.
|
||||
const m = graphStore.readMappings().find((x) => x.chatId === chat2.id)!;
|
||||
assert.deepEqual(m, { chatId: chat2.id, sourceId: "q1", graphNodeId: alphaId });
|
||||
});
|
||||
|
||||
it("resolves imported-notes ids to their file paths in the snapshot (static provenance)", () => {
|
||||
@@ -514,11 +529,6 @@ describe("the merger (mergeFindings)", () => {
|
||||
|
||||
const node = getGraph(graphStore).nodes.find((n) => n.statement.startsWith("Vim starts"))!;
|
||||
assert.deepEqual(node.provenance, [{ chatId: "src_notesfixture", messageId: "" }], "one chat-level point — the source key, no message anchors");
|
||||
assert.deepEqual(graphStore.readMappings().find((m) => m.chatId === "src_notesfixture"), {
|
||||
chatId: "src_notesfixture",
|
||||
sourceId: "p1",
|
||||
graphNodeId: node.id,
|
||||
});
|
||||
// The checkpoint stamps the import time.
|
||||
assert.equal(graphStore.readCheckpoints().find((c) => c.chatId === "src_notesfixture")?.foldedAt, "2026-09-13T00:00:00.000Z");
|
||||
});
|
||||
@@ -591,7 +601,7 @@ describe("session deletion (deleteChatTraces + deleteChatSession, wave 6)", () =
|
||||
* asserts X→Y; dB links X (its own phrasings), creates Z, asserts X→Z.
|
||||
* Deleting dA must: keep X (dB's link line survives, dA's create-line
|
||||
* aliases stripped — the identity stays), kill Y (orphan), keep Z and
|
||||
* dB's edge, kill dA's events/mappings/coverage/checkpoints. */
|
||||
* dB's edge, kill dA's coverage/checkpoints. */
|
||||
function buildTwoSessionScenario() {
|
||||
const dA = fixtureChat("Deletion session A", [
|
||||
user("du1", "teach me the deletion topic"),
|
||||
@@ -611,13 +621,12 @@ describe("session deletion (deleteChatTraces + deleteChatSession, wave 6)", () =
|
||||
edges: [{ from: "p1", to: "p2", type: "follows", }],
|
||||
};
|
||||
mergeFindings(dA.id, findingsA, { links: [] }, graphDeps, sessionSource(dA.id));
|
||||
// Resolve through THIS session's mappings — the shared ledger holds
|
||||
// lookalike statements from earlier tests, so statement-lookup would
|
||||
// link dB to the wrong node.
|
||||
const mappedId = (sourceId: string): string =>
|
||||
graphStore.readMappings().find((m) => m.chatId === dA.id && m.sourceId === sourceId)!.graphNodeId;
|
||||
const xId = mappedId("p1");
|
||||
const yId = mappedId("p2");
|
||||
// Resolve through THIS session's provenance — the shared ledger
|
||||
// holds lookalike statements from earlier tests, so scope by chat.
|
||||
const bornOf = (chatId: string, statement: string): string =>
|
||||
getGraph(graphStore).nodes.find((n) => n.statement === statement && n.provenance.some((pt) => pt.chatId === chatId))!.id;
|
||||
const xId = bornOf(dA.id, "Shared core statement");
|
||||
const yId = bornOf(dA.id, "Solo fact statement");
|
||||
|
||||
const dB = fixtureChat("Deletion session B", [
|
||||
user("eu1", "more on the shared core"),
|
||||
@@ -637,7 +646,7 @@ describe("session deletion (deleteChatTraces + deleteChatSession, wave 6)", () =
|
||||
edges: [{ from: "q1", to: "q2", type: "follows", }],
|
||||
};
|
||||
mergeFindings(dB.id, findingsB, { links: [{ id: "q1", linkTo: xId }] }, graphDeps, sessionSource(dB.id));
|
||||
const zId = graphStore.readMappings().find((m) => m.chatId === dB.id && m.sourceId === "q2")!.graphNodeId;
|
||||
const zId = bornOf(dB.id, "B extra statement");
|
||||
|
||||
// Coverage rows (both chats) + positions garnish — the deletion
|
||||
// prunes both ledgers.
|
||||
@@ -658,7 +667,6 @@ describe("session deletion (deleteChatTraces + deleteChatSession, wave 6)", () =
|
||||
assert.deepEqual(stats, {
|
||||
nodesRemoved: 1,
|
||||
edgesRemoved: 1,
|
||||
mappingsRemoved: 2,
|
||||
coverageRemoved: 2,
|
||||
checkpointsRemoved: 1,
|
||||
} satisfies ChatDeletionStats);
|
||||
@@ -680,9 +688,6 @@ describe("session deletion (deleteChatTraces + deleteChatSession, wave 6)", () =
|
||||
// Edges: dA's assertion X→Y died; dB's X→Z survived.
|
||||
const scoped = scopedGraph([dA.id, dB.id]);
|
||||
assert.deepEqual(scoped.edges, [{ from: xId, to: zId, type: "follows", provenance: [{ chatId: dB.id, messageId: "" }] }]);
|
||||
// Mappings: dA's audit trail died; dB's stays.
|
||||
const mappings = graphStore.readMappings().filter((m) => [dA.id, dB.id].includes(m.chatId));
|
||||
assert.deepEqual(mappings, [{ chatId: dB.id, sourceId: "q1", graphNodeId: xId }, { chatId: dB.id, sourceId: "q2", graphNodeId: zId }]);
|
||||
// Positions: the orphan pruned.
|
||||
const positions = graphStore.readPositions();
|
||||
assert.equal(positions[yId], undefined);
|
||||
@@ -699,7 +704,7 @@ describe("session deletion (deleteChatTraces + deleteChatSession, wave 6)", () =
|
||||
graphStore.appendNode({ id: "n_del_append_probe", name: "Probe", statement: "Probe statement", aliases: [], createdAt: "2026-09-23T00:04:00.000Z", provenance: [{ chatId: "chat_foreign_probe", messageId: "" }] });
|
||||
assert.equal(graphStore.readNodes().length, nodesBefore + 1);
|
||||
// Idempotent: a second run finds nothing.
|
||||
assert.deepEqual(deleteChatTraces(graphStore, dA.id, "delete-nodes"), { ...stats, nodesRemoved: 0, edgesRemoved: 0, mappingsRemoved: 0, coverageRemoved: 0, checkpointsRemoved: 0 } satisfies ChatDeletionStats);
|
||||
assert.deepEqual(deleteChatTraces(graphStore, dA.id, "delete-nodes"), { ...stats, nodesRemoved: 0, edgesRemoved: 0, coverageRemoved: 0, checkpointsRemoved: 0 } satisfies ChatDeletionStats);
|
||||
});
|
||||
|
||||
it("preserve-graph: only the extraction bookkeeping dies — the graph ledgers are byte-identical", () => {
|
||||
@@ -707,20 +712,17 @@ describe("session deletion (deleteChatTraces + deleteChatSession, wave 6)", () =
|
||||
const before = {
|
||||
nodes: graphStore.readNodes(),
|
||||
edges: graphStore.readEdges(),
|
||||
mappings: graphStore.readMappings(),
|
||||
positions: graphStore.readPositions(),
|
||||
};
|
||||
const stats = deleteChatTraces(graphStore, dA.id, "preserve-graph");
|
||||
assert.deepEqual(stats, {
|
||||
nodesRemoved: 0,
|
||||
edgesRemoved: 0,
|
||||
mappingsRemoved: 0,
|
||||
coverageRemoved: 2,
|
||||
checkpointsRemoved: 1,
|
||||
} satisfies ChatDeletionStats);
|
||||
assert.deepEqual(graphStore.readNodes(), before.nodes);
|
||||
assert.deepEqual(graphStore.readEdges(), before.edges);
|
||||
assert.deepEqual(graphStore.readMappings(), before.mappings);
|
||||
assert.deepEqual(graphStore.readPositions(), before.positions);
|
||||
assert.equal(graphStore.readCoverage().some((row) => row.chatId === dA.id), false);
|
||||
assert.equal(graphStore.readCoverage().some((row) => row.chatId === dB.id), true);
|
||||
@@ -765,15 +767,15 @@ describe("session deletion (deleteChatTraces + deleteChatSession, wave 6)", () =
|
||||
// ─── single-node deletion (node interlude, full death) ───────────────────────
|
||||
|
||||
describe("single-node deletion (deleteNodeTraces + deleteGraphNode, node interlude)", () => {
|
||||
/** The full-death scenario: node X is born in a notes file (the
|
||||
* production orphan shape — EMPTY provenance, the mapping line is the
|
||||
* anchor), re-encountered by a chat (link line: alias + provenance),
|
||||
* asserted in an edge, and evaluated by two sessions. */
|
||||
/** The full-death scenario: node X is born in a notes file (EMPTY
|
||||
* provenance — a legacy line; the mappings anchor is gone, owner
|
||||
* 2026-09-29), re-encountered by a chat (link line: alias +
|
||||
* provenance), and asserted in edges by two sessions. */
|
||||
function buildNodeScenario() {
|
||||
// Clean slate per test: each test in this describe builds its own
|
||||
// scenario from empty ledgers (the suite's isolation covers the
|
||||
// whole file; this covers describe-internal stacking).
|
||||
graphStore.rewriteLedgers({ nodes: [], edges: [], mappings: [] });
|
||||
graphStore.rewriteLedgers({ nodes: [], edges: [] });
|
||||
graphStore.writePositions({});
|
||||
graphStore.appendNode({
|
||||
id: "n_dead",
|
||||
@@ -804,10 +806,6 @@ describe("single-node deletion (deleteNodeTraces + deleteGraphNode, node interlu
|
||||
graphStore.appendEdge({ from: "n_dead", to: "n_keep", type: "follows", provenance: [{ chatId: "chat_node_a", messageId: "m1" }] });
|
||||
graphStore.appendEdge({ from: "n_keep", to: "n_dead", type: "supports", provenance: [{ chatId: "chat_node_b", messageId: "m3" }] });
|
||||
graphStore.appendEdge({ from: "n_keep", to: "n_other", type: "follows", provenance: [{ chatId: "chat_node_b", messageId: "m3" }] });
|
||||
// Mappings: the notes anchor + the chat's audit trail.
|
||||
graphStore.appendMapping({ chatId: "src_nodefile", sourceId: "p1", graphNodeId: "n_dead" });
|
||||
graphStore.appendMapping({ chatId: "chat_node_a", sourceId: "p1", graphNodeId: "n_dead" });
|
||||
graphStore.appendMapping({ chatId: "chat_node_a", sourceId: "p2", graphNodeId: "n_keep" });
|
||||
// Resting positions for both.
|
||||
graphStore.writePositions({ n_dead: { x: 1, y: 1 }, n_keep: { x: 2, y: 2 } });
|
||||
}
|
||||
@@ -816,9 +814,9 @@ describe("single-node deletion (deleteNodeTraces + deleteGraphNode, node interlu
|
||||
buildNodeScenario();
|
||||
assert.deepEqual(nodeDeletionImpact(graphStore, "n_dead"), {
|
||||
edges: 2,
|
||||
// The file anchor + the re-encounter — every referencing
|
||||
// session, deduped and sorted.
|
||||
sessions: ["chat_node_a", "src_nodefile"],
|
||||
// The re-encounter — the only session provenance can name (the
|
||||
// legacy empty-provenance line names none).
|
||||
sessions: ["chat_node_a"],
|
||||
});
|
||||
// Unknown node: null (the route's 404).
|
||||
assert.equal(nodeDeletionImpact(graphStore, "n_missing"), null);
|
||||
@@ -828,7 +826,7 @@ describe("single-node deletion (deleteNodeTraces + deleteGraphNode, node interlu
|
||||
buildNodeScenario();
|
||||
const keepLineBefore = JSON.stringify(graphStore.readNodes().find((n) => n.id === "n_keep"));
|
||||
const stats = deleteNodeTraces(graphStore, "n_dead");
|
||||
assert.deepEqual(stats, { edgesRemoved: 2, mappingsRemoved: 2 } satisfies NodeDeletionStats);
|
||||
assert.deepEqual(stats, { edgesRemoved: 2 } satisfies NodeDeletionStats);
|
||||
|
||||
// The node's EVERY line is gone; the neighbor is byte-identical
|
||||
// (full death is node-scoped — other nodes' sessions keep their
|
||||
@@ -844,8 +842,6 @@ describe("single-node deletion (deleteNodeTraces + deleteGraphNode, node interlu
|
||||
graphStore.readEdges().map((e) => [e.from, e.to]),
|
||||
[["n_keep", "n_other"]],
|
||||
);
|
||||
// The audit trail: X's mappings gone, the neighbor's stay.
|
||||
assert.deepEqual(graphStore.readMappings().map((m) => m.graphNodeId), ["n_keep"]);
|
||||
// The resting position died with the node.
|
||||
const positions = graphStore.readPositions();
|
||||
assert.equal(positions["n_dead"], undefined);
|
||||
@@ -866,7 +862,7 @@ describe("single-node deletion (deleteNodeTraces + deleteGraphNode, node interlu
|
||||
provenance: [{ chatId: "chat_node_solo", messageId: "m1" }],
|
||||
});
|
||||
const stats = deleteNodeTraces(graphStore, "n_solo");
|
||||
assert.deepEqual(stats, { edgesRemoved: 0, mappingsRemoved: 0 } satisfies NodeDeletionStats);
|
||||
assert.deepEqual(stats, { edgesRemoved: 0 } satisfies NodeDeletionStats);
|
||||
assert.equal(graphStore.readNodes().some((n) => n.id === "n_solo"), false);
|
||||
});
|
||||
|
||||
@@ -916,7 +912,7 @@ describe("single-node deletion (deleteNodeTraces + deleteGraphNode, node interlu
|
||||
const revisionBefore = graphRevision();
|
||||
const outcome = deleteGraphNode("n_orch");
|
||||
assert.ok(outcome.ok);
|
||||
assert.deepEqual(outcome.stats, { edgesRemoved: 0, mappingsRemoved: 0 });
|
||||
assert.deepEqual(outcome.stats, { edgesRemoved: 0 });
|
||||
assert.ok(outcome.graphRevision > 0);
|
||||
assert.ok(outcome.graphRevision >= revisionBefore, "the version stamp moved");
|
||||
assert.equal(graphStore.readNodes().some((n) => n.id === "n_orch"), false);
|
||||
|
||||
@@ -108,7 +108,6 @@ describe("settleGraphLayout", () => {
|
||||
const store = {
|
||||
readNodes: () => lines.map((l) => ({ ...l })),
|
||||
readEdges: () => [edge("a", "b")],
|
||||
readMappings: () => [],
|
||||
readPositions: () => ({} as Record<string, Position>),
|
||||
writePositions: (p: Record<string, Position>) => void written.push(p),
|
||||
};
|
||||
@@ -122,7 +121,6 @@ describe("settleGraphLayout", () => {
|
||||
settleGraphLayout({
|
||||
readNodes: () => [],
|
||||
readEdges: () => [],
|
||||
readMappings: () => [],
|
||||
readPositions: () => ({} as Record<string, Position>),
|
||||
writePositions: (p) => void written.push(p),
|
||||
});
|
||||
|
||||
@@ -38,14 +38,12 @@ function fakeStores(
|
||||
over: {
|
||||
chats?: Array<Pick<ChatSummary, "id" | "title" | "createdAt" | "updatedAt">>;
|
||||
sources?: Array<Pick<NotesSource, "id" | "filePath" | "importedAt">>;
|
||||
mappings?: Array<{ chatId: string; sourceId: string; graphNodeId: string }>;
|
||||
} = {},
|
||||
): GraphDeps {
|
||||
const edges: GraphEdge[] = [];
|
||||
const graphStore: GraphStore = {
|
||||
readNodes: () => nodes,
|
||||
readEdges: () => edges,
|
||||
readMappings: () => over.mappings ?? [],
|
||||
readCheckpoints: () => [],
|
||||
readCoverage: () => [],
|
||||
readSources: () =>
|
||||
@@ -103,46 +101,15 @@ describe("birthSessionPartition", () => {
|
||||
});
|
||||
});
|
||||
|
||||
// ─── the notes mapping fallback (owner-approved 2026-09-23) ─────────────────
|
||||
|
||||
describe("the notes mapping-line fallback", () => {
|
||||
// The PRODUCTION notes shape: importNotes passes messages: null, so
|
||||
// nodes are born with EMPTY provenance — the file's anchor is the
|
||||
// mapping line (interlude live-smoke finding: the provenance-only rule
|
||||
// left every imported file unclustered and invisible to the prober).
|
||||
it("empty-provenance nodes birth under their first mapping line's session", () => {
|
||||
const nodes = [
|
||||
node("a", "Alpha", []),
|
||||
node("b", "Beta", []),
|
||||
node("c", "Gamma", []),
|
||||
node("d", "Delta", [pt("chat1")]), // provenance wins — mapping ignored
|
||||
];
|
||||
const mappings = [
|
||||
{ chatId: "src_notes", sourceId: "p1", graphNodeId: "a" },
|
||||
{ chatId: "src_notes", sourceId: "p2", graphNodeId: "b" },
|
||||
{ chatId: "src_notes", sourceId: "p3", graphNodeId: "c" },
|
||||
// A chat mapping for a node that ALSO has provenance: never consulted.
|
||||
{ chatId: "chat2", sourceId: "p4", graphNodeId: "d" },
|
||||
// A mapping for a node that does not exist: ignored.
|
||||
{ chatId: "src_notes", sourceId: "p5", graphNodeId: "ghost" },
|
||||
];
|
||||
const clusters = birthSessionPartition(nodes, mappings);
|
||||
assert.deepEqual(clusters, [
|
||||
{ id: "src_notes", members: ["a", "b", "c"] },
|
||||
{ id: "chat1", members: ["d"] },
|
||||
], "size-desc, then id asc — src_notes first");
|
||||
// No mapping and no provenance: unclustered.
|
||||
assert.deepEqual(birthSessionPartition([node("x", "X", [])], mappings), []);
|
||||
});
|
||||
// ─── notes files as birth sessions (production shape) ────────────────────────
|
||||
|
||||
describe("notes files as birth sessions", () => {
|
||||
// The mapping-line fallback is GONE (owner, 2026-09-29): the mappings
|
||||
// ledger itself died with per-job in-memory resolution, and every merge
|
||||
// writes provenance points — the src id IS the birth session.
|
||||
it("graphSessionIndex lists imported files as kind:file with birth counts (production shape)", () => {
|
||||
const nodes = [node("a", "Alpha", []), node("b", "Beta", []), node("c", "Gamma", [])];
|
||||
const nodes = [node("a", "Alpha", [pt("src_abc")]), node("b", "Beta", [pt("src_abc")]), node("c", "Gamma", [pt("src_abc")])];
|
||||
const deps = fakeStores(nodes, {
|
||||
mappings: [
|
||||
{ chatId: "src_abc", sourceId: "p1", graphNodeId: "a" },
|
||||
{ chatId: "src_abc", sourceId: "p2", graphNodeId: "b" },
|
||||
{ chatId: "src_abc", sourceId: "p3", graphNodeId: "c" },
|
||||
],
|
||||
sources: [{ id: "src_abc", filePath: "/vault/learn vim.md", importedAt: "2026-09-22T00:00:00.000Z" }],
|
||||
});
|
||||
const index = graphSessionIndex(deps);
|
||||
@@ -159,11 +126,7 @@ describe("the notes mapping-line fallback", () => {
|
||||
});
|
||||
|
||||
it("graphSessionPropositions lists the file's births under the src id (a pure birth list, 2026-09-27)", () => {
|
||||
const deps = fakeStores([node("a", "Alpha", []), node("b", "Beta", [])], {
|
||||
mappings: [
|
||||
{ chatId: "src_abc", sourceId: "p1", graphNodeId: "a" },
|
||||
{ chatId: "src_abc", sourceId: "p2", graphNodeId: "b" },
|
||||
],
|
||||
const deps = fakeStores([node("a", "Alpha", [pt("src_abc")]), node("b", "Beta", [pt("src_abc")])], {
|
||||
sources: [{ id: "src_abc", filePath: "/vault/learn vim.md", importedAt: "2026-09-22T00:00:00.000Z" }],
|
||||
});
|
||||
assert.deepEqual(graphSessionPropositions(deps, "src_abc"), [
|
||||
|
||||
Reference in New Issue
Block a user