Extraction observability: usage capture, live status payload, true abort (Wave 4)

Extraction is real spend and background work; the dock needs to see
and stop it. User decisions: persist all cost data (chat record),
per-stage usage granularity, and TRUE cancellation for abort.

USAGE CAPTURE (live + persisted):
- extraction.ts: per-call onStepFinish captures each billed step's
  usage (SDK LanguageModelUsage -> CallUsage: input/output/cache
  read+write/reasoning) into the registry's live per-stage usage
  (extractor/auditor; the merger is deterministic — no model call).
- /api/graph/status runs now carry usage (model + steps per stage);
  cost/context folds are the consumer's (core/prices.ts — nothing
  stores dollars).
- ChatRecord.extractions: append-only per-run usage log (mirrors the
  turns log): startedAt key, at, durationMs, merge stats when the
  merger ran, stages. Persisted on success AND failure — a failed run
  still spent money. Idempotent per startedAt; the list payload
  (ChatSummary) never leaks it.
- EmissionStats moved to contracts/graph.ts (its contract home);
  core/graph.ts re-exports for existing importers.
- Verified end-to-end against the real pipeline (backfill one chat):
  per-step token counts, reasoning tokens, and the auditor's growing
  prompt-cache reads captured and persisted.

TRUE ABORT (the dock's abort button, user decision):
- RunningExtraction owns one AbortController per run, shared across
  phases; abortExtraction(chatId) fires it and returns whether a run
  was aborted (idempotent no-op otherwise). The pending model call
  receives the signal — the provider cancels the request, spend
  stops. throwIfAborted() at phase boundaries ends runs aborted in
  sync gaps.
- Abort is a deliberate stop, not a failure: same ledger discipline
  (nothing merged, nothing checkpointed, partial usage persisted) but
  the run reports zero emissions instead of throwing. Exported via
  src/server/observability.ts for master's /api/activity/abort route.
- Tests: mid-run abort (hanging mock that rejects on the signal),
  live-accumulation + persisted-record assertions, store-seam
  idempotency, and the summary-payload leak guard.
This commit is contained in:
amosblomqvist
2026-09-11 03:25:25 +03:00
parent 83a00800e9
commit 9174cde55e
10 changed files with 330 additions and 31 deletions
+4 -9
View File
@@ -42,6 +42,7 @@ import { randomBytes } from "node:crypto";
import { isToolUIPart, type UIMessage } from "ai";
import type {
AuditResult,
EmissionStats,
GraphEdge,
GraphNode,
ProvenancePoint,
@@ -108,15 +109,6 @@ const samePoint = (a: ProvenancePoint, b: ProvenancePoint): boolean =>
// ─── the merger (extraction pipeline stage 3 — deterministic) ────────────────
export interface EmissionStats {
/** New graph nodes appended. */
nodes: number;
/** New edge assertions appended. */
edges: number;
/** New ledger events appended. */
events: number;
}
/**
* Merge one chat's audited findings into the graph ledgers — the pipeline's
* apply stage. Deterministic + idempotent: re-merging the same verdicts (or
@@ -470,3 +462,6 @@ export function graphSnapshot(deps: GraphDeps): GraphSnapshot {
return { nodes, edges, sessions, nodeSessions, nodeObservations };
}
// Re-export: the contract owns the shape; legacy importers still reach it.
export type { EmissionStats };