BYOK provider embeddings (wave 6 chunk B): kill local transformers

Replace the @huggingface/transformers local embedder (~316MB ONNX cache)
with an EmbeddingPort seam: graph.embeddingModel (nullable "provider:id"
slug) selects an OpenRouter/OpenAI/Gemini adapter, resolved fresh per
call from settings + the auth store so key rotations land immediately.

- src/server/embeddings.ts rewritten: slug grammar, injected-fetch
  adapters (OpenRouter input_type, Gemini taskType batching, OpenAI
  index-ordered rows), honest resolution (null = skip-with-info, set
  model with missing key = honest error), embedNewNodes with model-change
  re-embed wipe + ghost pruning + usage steps, federated catalog
  (OpenRouter /api/v1/embeddings/models — VERIFIED live, the inverse
  path 404s; OpenAI text-embedding- filter; Gemini embedContent filter).
- GET /api/models/embeddings: federated catalog, empty when no keys.
- Extraction tail records stages.embedding usage into the run's money
  channel; EMBEDDING_PRICES overlay in core/prices.ts (outside the
  sync-generated block), callCost falls back to it; cost-summary folds
  the fourth stage; graph-context injection resolves its embedder per
  run, fails honestly when unconfigured, records query tokens.
- Settings: GraphSettings.embeddingModel (null factory), section parser
  validates slug shape (id may contain colons — OpenRouter :free
  suffixes, live catalog fact).
- Removal: @huggingface/transformers uninstalled, PATHS.modelCache and
  onnx tracing entries deleted, @huggingface tracing-excludes guard
  added, stale sharp/libvips comments updated to historical.

Live-verified on port 3004: catalog federates 36 rows (33 OpenRouter +
3 OpenAI), embedNewNodes embedded 10 nodes via the free liquid/lfm-2.5
model (dim 1024, 435 prompt tokens, 1 call, $0), settings PUT rejects
bad slugs with 400.

Gates: typecheck, lint, 895/895 tests, build.
This commit is contained in:
amosblomqvist
2026-09-23 03:53:57 +03:00
parent a6338511d3
commit 0a4a9a11de
29 changed files with 1233 additions and 841 deletions
+24 -9
View File
@@ -1,20 +1,35 @@
/**
* Embed the whole graph (backfill / cold start): one-time pass that
* fills the embedding ledger for every node — the same embedNewNodes the
* post-merge hook runs, just with an empty (or stale-model) ledger so
* every node is "new". First run downloads the model weights (~300MB)
* to the local HF cache; subsequent runs embed only what's missing.
* Embed the whole graph (backfill / cold start / model change): one-time
* pass that fills the embedding ledger for every node — the same
* embedNewNodes the post-merge hook runs, just with an empty (or
* stale-model) ledger so every node is "new".
*
* BYOK (wave 6, 2026-09-23): vectors come from the configured provider
* (graph.embeddingModel, keys from the auth store) — this script resolves
* the live state up front so an unconfigured machine says so before
* anything bills. A model CHANGE re-embeds everything (ledger staleness).
*
* npx tsx scripts/embed-graph.ts
*/
import { embedNewNodes, embeddingPort } from "../src/server/embeddings.ts";
import { graphStore } from "../src/store/graph.ts";
import { embedNewNodes, EMBEDDING_MODEL, EMBEDDING_DIM } from "../src/server/embeddings.ts";
async function main() {
const resolved = embeddingPort();
if (!resolved.ok) {
console.error(`[embed] cannot run — ${resolved.reason}`);
process.exit(1);
}
const t0 = Date.now();
console.log(`[embed] model: ${EMBEDDING_MODEL} (dim ${EMBEDDING_DIM}, q8)`);
const { embedded, total } = await embedNewNodes(graphStore);
console.log(`[embed] embedded ${embedded} nodes → ledger holds ${total} vectors (${((Date.now() - t0) / 1000).toFixed(1)}s)`);
console.log(`[embed] model: ${resolved.port.model} (provider embeddings, batch 32)`);
const { embedded, total, usage } = await embedNewNodes(graphStore);
console.log(
`[embed] embedded ${embedded} nodes → ledger holds ${total} vectors (${((Date.now() - t0) / 1000).toFixed(1)}s)`,
);
if (usage) {
const tokens = usage.steps.reduce((sum, s) => sum + (s.inputTokens ?? 0), 0);
console.log(`[embed] embedding spend: ${tokens} prompt tokens over ${usage.steps.length} call(s)`);
}
}
main().catch((err) => {