Key-presence refusal at every gate: a pinned model is not enough

Owner-diagnosed: the turn gate checked only model coordinates, so a
pinned slot whose provider key was deleted passed the gate and died
mid-stream masked to 'An error occurred.' The key is now gated where
the model is — by what would RUN (env-first keyValue truth, so an
env-shadowed key passes).

- core/settings.ts: the provider_key_missing envelope family (code,
  provider, slots, human line naming the provider from AUTH_PROVIDER_KEYS
  labels) + ProviderKeyMissingError + isProviderKeyRefusal matcher.
- chat-handler turn gate: refuses 503 before beginTurn when the resolved
  conversation's provider key is gone (slots: [strong]).
- Job runner: same check via an injected dep (agent layer cannot read
  the key store) — a keyless engine refuses the spawn through the
  existing failure-shape convention, no phantom run.
- Extraction pipeline: refuseIfStagesUnready refuses BEFORE any batch
  (first offender: extractor before auditor); the trigger skips info-
  level, /api/graph/import surfaces the 503 envelope per file.
- Tests: 8 new pins (turn gate both ways, runner refusal, pipeline
  first-offender + pass-through, envelope factory, matcher); the three
  turn-driving suites seed OPENROUTER_API_KEY like any real setup.
- Doc fix: stale EXTRACTION.chunkTokens comment in the manual-extract
  route (left over from eed4b4e).

Known residual (pre-existing, flagged for master): the manual extract
route logs BOTH config refusals as console.error (fire-and-forget path);
the coverage ledger keeps everything uncovered either way.
This commit is contained in:
amosblomqvist
2026-09-24 22:51:12 +03:00
parent 085fdc1a98
commit e4049e4bf8
13 changed files with 411 additions and 36 deletions
+59
View File
@@ -58,6 +58,7 @@ import { persistTurn, withMessageIds } from "../src/server/turn-service.ts";
import { chatStore } from "../src/store/chats.ts";
import { jobStore } from "../src/store/jobs.ts";
import { createJobRunner, pdfModelOverride, researchModelOverride } from "../src/agent/subagents/runner.ts";
import { providerKeyMissing, type ModelSlotName } from "../src/core/settings.ts";
import type { PiEngine } from "../src/agent/models.ts";
import type { SearchProvider } from "../src/core/search.ts";
import { PATHS } from "../config.ts";
@@ -65,6 +66,11 @@ import { PATHS } from "../config.ts";
const JOBS_DIR = path.join(PATHS.data, "jobs");
const createdChats: string[] = [];
// The key-presence turn gate (provider-key interlude, 2026-09-25): scripted
// turns pin openrouter per chat — the gate reads env-first truth, so the
// key must be present in the test process like any real setup.
process.env.OPENROUTER_API_KEY ??= "test-key-presence-gate";
const createdJobs: string[] = [];
function trackChat(id: string) {
@@ -1391,3 +1397,56 @@ describe("BYOK job refusal (no factory floor)", () => {
assert.equal(result.sources.length, 0, "the failure result matches the research convention");
});
});
// ─── provider-key: the key-presence job refusal (owner, 2026-09-25) ─────────
describe("provider-key job refusal (key-presence interlude)", () => {
it("an engine whose provider key is gone refuses the spawn — envelope wording, no phantom run", async () => {
const chat = chatStore.createChat("Test: keyless spawn");
trackChat(chat.id);
chatStore.saveMessages(chat.id, [
{ id: "u1", role: "user", parts: [{ type: "text", text: "hi" }] },
{
id: "a1",
role: "assistant",
parts: [{ type: "tool-research", toolCallId: "call-km1", state: "output-available", input: RESEARCH_INPUT, output: "Dispatched." }],
},
] as never);
const job = jobStore.createJob({ kind: "research", chatId: chat.id, toolCallId: "call-km1" });
trackJob(job.id);
const delivery = fakeDeliver();
// The chain RESOLVES (an engine exists) — the KEY is what's missing.
// The injected check is the server seam's stand-in: it sees the
// engine's provider + the kind's slot tier, exactly as wired.
const engine = fauxEngine([]);
const seen: Array<[string, ModelSlotName]> = [];
const runner = createJobRunner({
chatStore,
jobStore,
deliver: delivery.deliver,
engineFor: () => engine,
providerKeyMissing: (provider, tier) => {
seen.push([provider, tier]);
return providerKeyMissing(provider, [tier]);
},
});
await runner.runJob(job.id);
// The refusal lands through the SAME convention as every failed job:
// honest record, kind failure shape steered, terminal flip after.
const done = jobStore.getJob(job.id)!;
assert.equal(done.status, "error", "the job lands error, never a phantom run");
assert.equal(done.error, providerKeyMissing(engine.model.provider, ["cheap"]).error, "the envelope's error line verbatim");
assert.deepEqual(seen, [[engine.model.provider, "cheap"]], "the check saw the engine's provider + the researcher's tier");
assert.equal(delivery.steers.length, 1, "the refusal STEERED — the teacher is not stranded");
const text = delivery.steers[0]!.parts[0];
assert.ok(
text.type === "text" && text.text.includes(`Research failed: No ${engine.model.provider} key is set`),
"the steer carries the kind's failure shape",
);
const result = done.result as ResearchResult;
assert.equal(result.sources.length, 0, "the failure result matches the research convention");
});
});