learnharness/docs/custom-providers.md

Custom and local models

LearnHarness supports one custom OpenAI-compatible endpoint with multiple models. The setup and Settings UI are based on PR #62 by Daniel Buckner and Abhishek Thakur. Pi owns inference, message conversion, tools, streaming, and reasoning-effort handling.

Configure an endpoint

1. Open Settings → Providers → + Add a provider → Custom endpoint. The same editor is available from the first-run provider picker.
2. Enter the server's complete API base URL, for example http://localhost:11434/v1. LearnHarness does not append /v1.
3. Choose Keyless or API key:
   • Keyless never uses a saved or environment custom-provider key for authentication. Pi's OpenAI client receives an internal dummy credential.
   • API key uses the effective CUSTOM_OPENAI_API_KEY. A real environment value wins over a file-saved value; the editor identifies environment ownership. Any nonempty trimmed custom key is accepted—there is no arbitrary length floor.
   • Changing the URL requires a new effective key or explicit consent to reuse the existing effective key. A key is never silently forwarded to a new URL. Clearing authentication selects Keyless.
4. Enter optional model IDs, separated by commas or whitespace. This supports servers whose chat endpoint works but whose /models endpoint is absent.
5. Click Connect, then choose models for Strong, Cheap, and Vision.

HTTP and HTTPS, including localhost/private-network servers, are supported. URLs with embedded credentials, query strings, or fragments are rejected. Put a key in the authentication field, not the URL. Discovery redirects are rejected.

Connect requests GET <base>/models. Discovery metadata wins over duplicate manually entered IDs. A failed probe can fall back to entered IDs; a failed probe for the same URL preserves its previous model snapshot and pins. A different URL starts with its own models. 401/403 always fails, even with entered IDs. Response-size, model-count, and redirect violations also fail explicitly rather than silently truncating or bypassing a discovery limit.

Configuration is not a continuous health check. An endpoint may be configured while offline, and model capability or output quality is not proven by listing its ID. No backend/model installation, downloads, or OAuth flows are performed.

Model budgets and reasoning support

Settings exposes offline-editable pins per model:

• Context budget: user pin → reported context → 8192 fallback.
• Maximum output: user pin → reported output limit → 4096 fallback.
• Reasoning support: user declaration → reported support.

Every positive integer token pin is accepted, including a 512-token output limit. Blank fields remove that pin. The editor identifies reported, fallback, and user-pin values. These are request/compaction budgets; changing them does not resize the backend's actual context. Configure the server's active context separately and enter budgets that match it. Reconnecting prunes pins only to the resulting model IDs; pin saves do not contact the server.

Reasoning support is a capability declaration, not an execution toggle. The existing Pi-backed effort picker remains available. In particular, Off may not disable a custom backend's default thinking: an absent effort parameter is not a universal disable command. There is no compatibility/thinking-map editor, Ollama preset, or model-name inference. Backends requiring additional nonstandard headers or compatibility configuration are outside this release's custom-endpoint UI.

Images and Vision

All custom models can be selected for Vision; models that report image support appear first. The endpoint owner is responsible for choosing an appropriate model. Built-in model filtering is unchanged.

Missing discovery metadata does not silently remove images: every custom-engine image-bearing request attempts delivery, including teaching read results, rendered PDF pages, and visual-maker screenshots. A request-local model clone permits Pi's native image encoding; registered/catalog metadata is not promoted to claim that every model understands images. A text-only backend may reject the request; that error is surfaced, with no silent text-only fallback. This does not establish real-model image-understanding quality.

Credentials, reloads, and concurrent editing

Nonsecret configuration is stored in the existing settings.json.customProvider; file-saved credentials use the existing auth.json key seam. A reserved internal string entry, __learnharness_custom_provider_binding, binds authorization to the endpoint, mode, and a random nonsecret credentialBindingId in custom settings. The file key and its binding are published in one existing atomic auth-map write. The internal entry is not an environment variable or an editable/displayed key. Neither keys nor raw upstream errors are returned by the custom-provider API.

Every startup and upstream operation requires matching binding IDs, URL, and mode before using credentials. A crash between auth/settings writes, or a failed settings write whose auth rollback also fails, leaves the custom provider disabled rather than sending a new key to an old endpoint. Reload and explicitly reconnect to repair authorization. A shadowed file key retains its own authorized URL: if an environment key is reused for another URL and later disappears, the old file key is not silently forwarded there. Keyless also requires its nonsecret binding, but never resolves a stored/environment key. Disconnect leaves an internal disabled marker so a latent environment key cannot revive an interrupted disconnect.

Use the custom endpoint editor to change custom credentials; generic /api/keys PUT/DELETE cannot bypass binding. No credential-derived fingerprint is stored in settings or a public revision. These guards cover process crashes and restart using the existing atomic rename; they are not a power-loss/fsync transaction system. Missing auth storage can initialize normally; unreadable or malformed private storage fails closed without overwriting its contents.

Each upstream model operation resolves the current custom model, endpoint, effective credential, limits, and original saved effort preference against Pi's current model metadata. Already-started requests keep their setup. A removed model or disconnected provider fails rather than falling back; saved model selections and history are not rewritten.

Edits use an opaque nonsecret ETag/If-Match revision. A stale edit returns 409 with reload/retry guidance. A connect also checks configuration and effective credentials after discovery, and merges only the custom section into the latest settings record, preserving intervening unrelated edits. Failed mutations never claim success.

Desktop requests use the existing desktop capability. Browser mutations must be same-origin JSON requests. Limits are 15 seconds for discovery, 8 MiB for its response, 10,000 listed models, and 64 KiB for configuration-write bodies. These are protections for the new endpoint, not a multiuser authentication system.

Pricing and previous development configuration

Custom model pricing is unknown/unpriced, not $0—even for an endpoint that may be locally hosted. Pi-required internal zero-rate placeholders are not presented as free inference. There is no rate editor in this release.

The earlier, unshipped app-data models.json feature is retired and is not read. No user file is deleted or renamed, and there is no importer, coexistence mode, profile reset, history rewrite, or silent model remapping. Existing development profiles need manual endpoint configuration and model reselection. Unbound early GUI development configurations also require explicit reconnect/new-key entry or the existing reuse-consent checkbox, even at the same URL; they are never silently trusted or migrated. This release supports one OpenAI-compatible custom endpoint, not multiple arbitrary protocols.