BYOK provider embeddings (wave 6 chunk B): kill local transformers

Replace the @huggingface/transformers local embedder (~316MB ONNX cache)
with an EmbeddingPort seam: graph.embeddingModel (nullable "provider:id"
slug) selects an OpenRouter/OpenAI/Gemini adapter, resolved fresh per
call from settings + the auth store so key rotations land immediately.

- src/server/embeddings.ts rewritten: slug grammar, injected-fetch
  adapters (OpenRouter input_type, Gemini taskType batching, OpenAI
  index-ordered rows), honest resolution (null = skip-with-info, set
  model with missing key = honest error), embedNewNodes with model-change
  re-embed wipe + ghost pruning + usage steps, federated catalog
  (OpenRouter /api/v1/embeddings/models — VERIFIED live, the inverse
  path 404s; OpenAI text-embedding- filter; Gemini embedContent filter).
- GET /api/models/embeddings: federated catalog, empty when no keys.
- Extraction tail records stages.embedding usage into the run's money
  channel; EMBEDDING_PRICES overlay in core/prices.ts (outside the
  sync-generated block), callCost falls back to it; cost-summary folds
  the fourth stage; graph-context injection resolves its embedder per
  run, fails honestly when unconfigured, records query tokens.
- Settings: GraphSettings.embeddingModel (null factory), section parser
  validates slug shape (id may contain colons — OpenRouter :free
  suffixes, live catalog fact).
- Removal: @huggingface/transformers uninstalled, PATHS.modelCache and
  onnx tracing entries deleted, @huggingface tracing-excludes guard
  added, stale sharp/libvips comments updated to historical.

Live-verified on port 3004: catalog federates 36 rows (33 OpenRouter +
3 OpenAI), embedNewNodes embedded 10 nodes via the free liquid/lfm-2.5
model (dim 1024, 435 prompt tokens, 1 call, $0), settings PUT rejects
bad slugs with 400.

Gates: typecheck, lint, 895/895 tests, build.
This commit is contained in:
amosblomqvist
2026-09-23 03:53:57 +03:00
parent a6338511d3
commit 0a4a9a11de
29 changed files with 1233 additions and 841 deletions
+24 -8
View File
@@ -5,7 +5,7 @@ import { piEngineForSpec, type PiEngine } from "../agent/models.ts";
import { createJobRunner } from "../agent/subagents/runner.ts";
import { createExtractionPipeline } from "../agent/subagents/extraction.ts";
import { runGraphContextJob, type GraphContextDeps } from "../agent/subagents/graph-context.ts";
import { EMBEDDING_DIM, EMBEDDING_MODEL, embedNewNodes, embedQuery } from "./embeddings.ts";
import { embedNewNodes, embeddingPort } from "./embeddings.ts";
import type { ModelRole } from "../../config.ts";
import { claimSteers, deliverJobResult, setContextRespawn, setJobRespawn, setTurnRunner } from "./jobs.ts";
import { recordToPiMessages } from "./pi/record-adapter.ts";
@@ -63,8 +63,8 @@ const jobRunner = createJobRunner({
* (/api/graph/import) hit the same single-flight registry. */
export const extraction = createExtractionPipeline({
...stores,
// The REAL embedder (local transformer model): post-merge embedding
// batch — semantic clusters are computed and persisted here.
// The provider-embedder batch (BYOK wave 6): post-merge embedding —
// resolves the model fresh per call; null model = info-skip inside.
embedder: embedNewNodes,
// BYOK (owner, 2026-09-22): the extraction stages resolve the chain —
// role override → slot → NOTHING — read FRESH per stage call; an empty
@@ -75,14 +75,27 @@ export const extraction = createExtractionPipeline({
});
/** Graph context injection (vision-context-injection.md): the runner's
* deps — same storage ports, the REAL embedder as the query seam (same
* warm singleton as extraction, so vector-space parity holds by
* construction), and the wake pipeline as the delivery seam. */
* deps — same storage ports, the provider-embedder RESOLVER as the query
* seam (BYOK wave 6: fresh setting + key reads per run, so vector-space
* parity with the ledger is checked against the CURRENT model; a key or
* settings PUT between dispatch and spawn lands), and the wake pipeline
* as the delivery seam. */
export const graphContext: GraphContextDeps = {
chatStore: stores.chatStore,
jobStore: stores.jobStore,
graphStore: stores.graphStore,
embedder: { model: EMBEDDING_MODEL, dim: EMBEDDING_DIM, embedQuery },
embedder: () => {
const resolved = embeddingPort();
if (!resolved.ok) return null;
const port = resolved.port;
return {
model: port.model,
embedQuery: async (text: string) => {
const result = await port.embed([text], "query");
return { vector: result.vectors[0]!, promptTokens: result.promptTokens };
},
};
},
deliver: deliverJobResult,
};
@@ -221,7 +234,10 @@ export async function handleChatRequest(
// GRAPH GATES (settings-storage wave, 2026-09-19): the read channel
// needs BOTH the master switch and the embedding rung (injection is a
// vector query); with graph off there is no context job at all.
if (graphContextEnabled(effective.graph)) {
// BYOK (wave 6): an unconfigured embedding model means no vectors to
// query — the dispatch skips honestly (the runner's own resolution is
// the belt-and-suspenders for respawn races).
if (graphContextEnabled(effective.graph) && effective.graph.embeddingModel !== null) {
triggerGraphContext(graphContext, chatId);
}