digest-aware serialized extraction + chain simplification replace the spine stage

Owner live-test verdict on the spine (2026-09-30): no perceptible gain —
the real problems (restatement nodes, shallow re-derivations, same
conclusion from many angles) are mint-time issues a post-hoc chain
minter cannot fix. This arc moves assembly INTO the extractor and adds
the deterministic dedupe the owner specified:

- Digest-aware extraction: each stretch's extractor prompt carries THE
  GRAPH SO FAR (its own source's merged propositions + chains, ids
  verbatim); build on existing ids instead of restating, lean on
  established conclusions as premises. Mixed-namespace chain endpoints
  (fresh p-ids OR existing n_-ids) validated at the submit seam.
- Serialized per source: a source's batches/chunks run strictly in
  order (each extract reads every earlier merge); cross-source runs
  stay parallel; source-wide abort stops the loop. The spine stage,
  role, phase, usage stage, cost row, and setup roster are gone.
- Core-ideas prompt: goal reframed as uncovering core ideas and showing
  how everything derives from the foundations; direction upward, prefer
  lean derivations, never re-derive what the graph already holds.
- Chain simplification (merger maintenance): simplifyChains drops
  provably redundant chains at every merge — prefer-fewest-premises +
  ancestry-cover ([A,B]⟹X with [X,C]⟹Y covers [A,B,C]⟹Y, transitive);
  same-key provenance lines drop together; rewriteLedgers under the
  audit lock; droppedChains counted separately in stats.

Chunk budget stays 8192 (owner: keep for now, maybe lower later).
This commit is contained in:
amosblomqvist
2026-09-30 17:44:37 +03:00
parent 52c02533d1
commit 91341a93da
18 changed files with 734 additions and 879 deletions
+4 -8
View File
@@ -309,14 +309,11 @@ export function displayCostTotals(costs: Pick<GraphCostSummary, "stages">): {
totalDollars: number;
unpricedRuns: number;
} {
// The three live stages (spine joined 2026-09-30); empty spines
// contribute zero — the fold leaves them at 0 when a run never
// called the spine (first-stretch skip / pre-spine records).
// The two live stages (the spine came and went on one day,
// 2026-09-30 — digest-aware extraction replaced it).
return {
totalDollars:
costs.stages.extraction.dollars + costs.stages.spine.dollars + costs.stages.audit.dollars,
unpricedRuns:
costs.stages.extraction.unpricedRuns + costs.stages.spine.unpricedRuns + costs.stages.audit.unpricedRuns,
totalDollars: costs.stages.extraction.dollars + costs.stages.audit.dollars,
unpricedRuns: costs.stages.extraction.unpricedRuns + costs.stages.audit.unpricedRuns,
};
}
@@ -373,7 +370,6 @@ function GraphCostControl({ costs }: { costs: GraphCostSummary }) {
function GraphCostPanel({ costs }: { costs: GraphCostSummary }) {
const stageRows: [string, GraphCostSummary["stages"]["extraction"]][] = [
["Extraction", costs.stages.extraction],
["Assembly", costs.stages.spine],
["Audit", costs.stages.audit],
];
// The displayed figures come from displayCostTotals (naming excluded —