amosblomqvist
1b50013551
The graph becomes raw reasoning chains only. Quiz outcomes and understanding observations are no longer formalized; assessment moves to the future practice module. Events ledger removed end-to-end: LedgerEvent/LedgerEventKind/ UnderstandingLevel/ExtractionAttribution/AuditResult.dropped and snapshot nodeObservations deleted from contracts; quizEvent builder and the events step of mergeFindings gone; appendEvent/readEvents dropped from the port and store; LEDGER_FILES loses events.jsonl with a one-time purge of a stale file next to the nodes ledger; graph-deletion and the delete dialog lose event purge + quiz-answer impact counts; GraphSessionProposition becomes a pure birth list. Whole-session markdown source view: new core/session-markdown.ts is the shared formatting engine (id-anchored speaker blocks, quiz/ recall/question blocks, tool one-liners) — the extractor's prompt view and the future user-facing export render from one engine. Coverage fingerprints deliberately stay on the old line format (content-hash semantics; re-formatting never re-opens covered history). The whole session goes to the extractor — no phase cut. Auditor refocused as the identity linker: never sees the source; explores the graph via new auditor-specific tools in agent/tools/graph.ts (find — glob over name/statement/aliases; list_sessions; list_propositions with full statement text; no timestamps) instead of an injected full-graph dump; equivalence bias — paraphrases merge, only genuinely distinct propositions stay apart; no drop channel, the extractor is fully trusted; may draw bridging edges (typed follows/supports) between existing nodes and the batch, resolved by the merger with chat-scoped message-less provenance. Agentic loop, maxTurns 16. UI: the node detail card gains an edges section above the sessions list (follows-from / supported-by / what-follows / what-it-supports, rendered only when present, click-to-select neighbors); the understanding-note chips are gone. Prober list_propositions reworded to what a session covered — names only, no levels. docs/architecture.md + README updated to the as-built pipeline. Gates: typecheck, 1059/1059, lint, build.
43 lines
1.5 KiB
TypeScript
43 lines
1.5 KiB
TypeScript
/**
|
|
* One-time backfill: wipe the legacy ledgers and re-extract every chat with
|
|
* the redesigned pipeline (vision §1, commit b07c907). Run from the master
|
|
* tree only — data never crosses trees:
|
|
*
|
|
* npx tsx --env-file=.env.local scripts/backfill-graph.ts
|
|
*/
|
|
import { chatStore } from "../src/store/chats.ts";
|
|
import { graphStore } from "../src/store/graph.ts";
|
|
import { createExtractionPipeline } from "../src/agent/subagents/extraction.ts";
|
|
import { getGraph } from "../src/core/graph.ts";
|
|
import { effectiveSettings } from "../src/server/stores.ts";
|
|
|
|
async function main() {
|
|
const only = process.argv[2]; // optional: extract just one chat id
|
|
const chats = only ? chatStore.listChats().filter((c) => c.id === only) : chatStore.listChats();
|
|
console.log(`[backfill] ${chats.length} chats to extract`);
|
|
const pipeline = createExtractionPipeline({
|
|
chatStore,
|
|
graphStore,
|
|
// Live budget: the graph section's setting, same as the app's trigger
|
|
getChunkTokens: () => effectiveSettings().graph.extractionChunkTokens,
|
|
});
|
|
for (const chat of chats) {
|
|
const t0 = Date.now();
|
|
try {
|
|
const stats = await pipeline.runExtraction(chat.id);
|
|
console.log(
|
|
`[backfill] ${chat.id} (${chat.title.slice(0, 40)}): ` +
|
|
`${stats.nodes} nodes, ${stats.edges} edges in ${((Date.now() - t0) / 1000).toFixed(1)}s`,
|
|
);
|
|
} catch (err) {
|
|
console.error(`[backfill] ${chat.id} FAILED:`, err);
|
|
}
|
|
}
|
|
const graph = getGraph(graphStore);
|
|
console.log(
|
|
`[backfill] done: ${graph.nodes.length} nodes, ${graph.edges.length} edges`,
|
|
);
|
|
}
|
|
|
|
main();
|