feat: finish donor-led custom endpoint integration
This commit is contained in:
amosblomqvist
+113
-139
@@ -1,159 +1,133 @@
|
||||
# Custom model providers
|
||||
# Custom and local models
|
||||
|
||||
LearnHarness can register several independently named custom providers using its
|
||||
existing Pi AI runtime. This is configuration wiring, not another agent runtime:
|
||||
Pi owns the model protocols, request conversion, streaming, tools, thinking and
|
||||
usage accounting. There is no additional dependency on Pi coding-agent.
|
||||
LearnHarness supports **one custom OpenAI-compatible endpoint with multiple models**.
|
||||
The setup and Settings UI are based on PR #62 by Daniel Buckner and Abhishek Thakur.
|
||||
Pi owns inference, message conversion, tools, streaming, and reasoning-effort handling.
|
||||
|
||||
## Configuration location
|
||||
## Configure an endpoint
|
||||
|
||||
Create `models.json` in the LearnHarness app-data directory (the directory
|
||||
containing `settings.json`; `LH_DATA_DIR` relocates it). LearnHarness does not read
|
||||
`~/.pi/agent/models.json`, install a model server, or edit this file for you.
|
||||
Custom-provider setup UI is not included yet. Once configured, models appear in
|
||||
the existing model selectors when their credential/header references resolve.
|
||||
1. Open **Settings → Providers → + Add a provider → Custom endpoint**. The same
|
||||
editor is available from the first-run provider picker.
|
||||
2. Enter the server's complete API base URL, for example
|
||||
`http://localhost:11434/v1`. LearnHarness does **not** append `/v1`.
|
||||
3. Choose **Keyless** or **API key**:
|
||||
- **Keyless** never uses a saved or environment custom-provider key for
|
||||
authentication. Pi's OpenAI client receives an internal dummy credential.
|
||||
- **API key** uses the effective `CUSTOM_OPENAI_API_KEY`. A real environment
|
||||
value wins over a file-saved value; the editor identifies environment ownership.
|
||||
Any nonempty trimmed custom key is accepted—there is no arbitrary length floor.
|
||||
- Changing the URL requires a new effective key or explicit consent to reuse
|
||||
the existing effective key. A key is never silently forwarded to a new URL.
|
||||
Clearing authentication selects Keyless.
|
||||
4. Enter optional **model IDs**, separated by commas or whitespace. This supports
|
||||
servers whose chat endpoint works but whose `/models` endpoint is absent.
|
||||
5. Click **Connect**, then choose models for Strong, Cheap, and Vision.
|
||||
|
||||
The file uses Pi's `providers` / `models` structure. For example:
|
||||
HTTP and HTTPS, including localhost/private-network servers, are supported. URLs
|
||||
with embedded credentials, query strings, or fragments are rejected. Put a key in
|
||||
the authentication field, not the URL. Discovery redirects are rejected.
|
||||
|
||||
```json
|
||||
{
|
||||
"providers": {
|
||||
"my-local-server": {
|
||||
"baseUrl": "http://127.0.0.1:11434/v1",
|
||||
"api": "openai-completions",
|
||||
"apiKey": "unused-local-key",
|
||||
"compat": {
|
||||
"supportsDeveloperRole": false,
|
||||
"supportsReasoningEffort": false,
|
||||
"maxTokensField": "max_tokens"
|
||||
},
|
||||
"models": [
|
||||
{
|
||||
"id": "your-installed-model",
|
||||
"name": "My local model",
|
||||
"contextWindow": 8192,
|
||||
"maxTokens": 1024,
|
||||
"cost": {
|
||||
"input": 0,
|
||||
"output": 0,
|
||||
"cacheRead": 0,
|
||||
"cacheWrite": 0
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
"my-gateway": {
|
||||
"baseUrl": "https://your-gateway.example/v1",
|
||||
"api": "openai-completions",
|
||||
"apiKey": "$MY_GATEWAY_API_KEY",
|
||||
"headers": { "X-Tenant": "${MY_GATEWAY_TENANT}" },
|
||||
"models": [
|
||||
{
|
||||
"id": "your-gateway-model-id",
|
||||
"contextWindow": 32768,
|
||||
"maxTokens": 4096,
|
||||
"cost": {
|
||||
"input": 1,
|
||||
"output": 3,
|
||||
"cacheRead": 0,
|
||||
"cacheWrite": 0
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
Connect requests `GET <base>/models`. Discovery metadata wins over duplicate manually
|
||||
entered IDs. A failed probe can fall back to entered IDs; a failed probe for the
|
||||
same URL preserves its previous model snapshot and pins. A different URL starts
|
||||
with its own models. **401/403 always fails**, even with entered IDs. Response-size,
|
||||
model-count, and redirect violations also fail explicitly rather than silently
|
||||
truncating or bypassing a discovery limit.
|
||||
|
||||
**Replace all example IDs, limits, prices and compatibility flags with the actual
|
||||
server/model configuration.** These numbers are examples, not recommendations or
|
||||
automatically detected capabilities. Rates are USD per million tokens. Explicit
|
||||
zero rates are allowed; LearnHarness does not assume that custom gateways are
|
||||
free. Verify the endpoint's billing independently.
|
||||
Configuration is not a continuous health check. An endpoint may be configured while
|
||||
offline, and model capability or output quality is not proven by listing its ID.
|
||||
No backend/model installation, downloads, or OAuth flows are performed.
|
||||
|
||||
Use the exact API base URL your server documents. LearnHarness does not append
|
||||
`/v1`, discover models, infer capabilities from names, or download models. Ollama,
|
||||
vLLM, LM Studio, 9router and other gateways work only to the extent that their
|
||||
configured endpoint implements the selected Pi protocol and compatibility flags.
|
||||
For a keyless OpenAI-compatible local server, supply an explicit nonempty dummy
|
||||
`apiKey` as above: the native Pi OpenAI client requires a key. Omitting `apiKey`
|
||||
leaves a provider unavailable rather than implying unauthenticated access.
|
||||
## Model budgets and reasoning support
|
||||
|
||||
## Supported fields
|
||||
Settings exposes offline-editable pins per model:
|
||||
|
||||
Provider fields: `name`, `baseUrl`, `api`, `apiKey`, `headers`, `authHeader`,
|
||||
`compat`, and the explicit `models` array. Each model requires `id`,
|
||||
`contextWindow`, `maxTokens`, and `cost` with all four base rates. Unlike Pi
|
||||
coding-agent's defaults, LearnHarness never invents limits or prices.
|
||||
- **Context budget**: user pin → reported context → **8192 fallback**.
|
||||
- **Maximum output**: user pin → reported output limit → **4096 fallback**.
|
||||
- **Reasoning support**: user declaration → reported support.
|
||||
|
||||
Optional Pi model fields are `name`, `api`, `baseUrl`, `reasoning`, `input`,
|
||||
`thinkingLevelMap`, `inputLimits`, `promptCache`, `samplingParams`,
|
||||
`samplingParamsByThinkingLevel`, `headers`, and `compat`. Cost tiers and Pi's
|
||||
exported compatibility schemas are supported. A model's `api` and `baseUrl`
|
||||
override its provider's values; model headers override provider headers. Names
|
||||
default to model IDs, `reasoning` defaults to false, and `input` defaults to
|
||||
`["text"]`. Declare image input only when the server/model supports it. Existing
|
||||
onboarding and vision-slot requirements are unchanged.
|
||||
Every positive integer token pin is accepted, including a 512-token output limit.
|
||||
Blank fields remove that pin. The editor identifies reported, fallback, and user-pin
|
||||
values. **These are request/compaction budgets; changing them does not resize the
|
||||
backend's actual context.** Configure the server's active context separately and
|
||||
enter budgets that match it. Reconnecting prunes pins only to the resulting model
|
||||
IDs; pin saves do not contact the server.
|
||||
|
||||
The native chat transports available in the pinned Pi AI 1.4.0 dependency are:
|
||||
`openai-completions`, `openai-responses`, `anthropic-messages`,
|
||||
`google-generative-ai`, `azure-openai-responses`, `google-vertex`,
|
||||
`bedrock-converse-stream`, `mistral-conversations`, `openai-codex-responses`, and
|
||||
`pi-messages`. Protocol availability does **not** establish account access,
|
||||
OAuth support, cloud credentials, subscription permission, or vendor approval.
|
||||
This feature adds no login flows. In particular, an Anthropic Messages endpoint
|
||||
is not a Claude Agent SDK or Claude Code runtime.
|
||||
Reasoning support is a **capability declaration, not an execution toggle**. The
|
||||
existing Pi-backed effort picker remains available. In particular, **Off may not
|
||||
disable a custom backend's default thinking**: an absent effort parameter is not a
|
||||
universal disable command. There is no compatibility/thinking-map editor, Ollama
|
||||
preset, or model-name inference. Backends requiring additional nonstandard headers
|
||||
or compatibility configuration are outside this release's custom-endpoint UI.
|
||||
|
||||
Use new provider IDs; built-in and subscription-route IDs cannot be replaced.
|
||||
Provider IDs must be nonempty and cannot contain `:` (the provider/model
|
||||
coordinate separator). Model IDs must be unique within a provider. Built-in
|
||||
provider overrides, `modelOverrides`, OAuth definitions, executable plugins, and
|
||||
automatic discovery are outside this phase.
|
||||
## Images and Vision
|
||||
|
||||
## Credentials and trust
|
||||
All custom models can be selected for Vision; models that report image support
|
||||
appear first. The endpoint owner is responsible for choosing an appropriate model.
|
||||
Built-in model filtering is unchanged.
|
||||
|
||||
API keys and header values accept literals, `$NAME`, and `${NAME}`. References
|
||||
use LearnHarness's existing key lookup: environment first, then the existing
|
||||
flat `auth.json` file, respecting the fresh-user environment-ignore setting.
|
||||
There is no auth migration or new secret file format. Prefer references over
|
||||
putting secrets into `models.json`; protect any file containing literal secrets.
|
||||
Unknown variable references keep the provider unavailable. All required header
|
||||
references, including model-specific ones, must resolve before a provider becomes
|
||||
available. An empty key does not configure a provider.
|
||||
Missing discovery metadata does not silently remove images: **every custom-engine
|
||||
image-bearing request attempts delivery**, including teaching `read` results,
|
||||
rendered PDF pages, and visual-maker screenshots. A request-local model clone permits
|
||||
Pi's native image encoding; registered/catalog metadata is not promoted to claim
|
||||
that every model understands images. A text-only backend may reject the request;
|
||||
that error is surfaced, with no silent text-only fallback. This does not establish
|
||||
real-model image-understanding quality.
|
||||
|
||||
Pi's `$$` and `$!` literal escapes are supported. `!command` credential/header
|
||||
values are rejected, never executed; resolved environment/file values are data,
|
||||
not commands. Optional `authHeader: true` adds `Authorization: Bearer <key>` in
|
||||
addition to the selected native transport's normal authentication behavior.
|
||||
## Credentials, reloads, and concurrent editing
|
||||
|
||||
Only trusted people should edit this file: its endpoints receive model prompts,
|
||||
attachments, tool results and configured credentials. Only HTTP(S) URLs are
|
||||
accepted, without embedded URL credentials or fragments. This is not an
|
||||
arbitrary-endpoint discovery or config-editing API and does not add a network
|
||||
sandbox, endpoint reachability checks, or vendor policy exemptions.
|
||||
Nonsecret configuration is stored in the existing `settings.json.customProvider`;
|
||||
file-saved credentials use the existing `auth.json` key seam. A reserved internal
|
||||
string entry, `__learnharness_custom_provider_binding`, binds authorization to the
|
||||
endpoint, mode, and a random nonsecret `credentialBindingId` in custom settings.
|
||||
The file key and its binding are published in one existing atomic auth-map write.
|
||||
The internal entry is not an environment variable or an editable/displayed key.
|
||||
Neither keys nor raw upstream errors are returned by the custom-provider API.
|
||||
|
||||
## Reloading and failures
|
||||
Every startup and upstream operation requires matching binding IDs, URL, and mode
|
||||
before using credentials. A crash between auth/settings writes, or a failed
|
||||
settings write whose auth rollback also fails, leaves the custom provider disabled
|
||||
rather than sending a new key to an old endpoint. Reload and explicitly reconnect
|
||||
to repair authorization. A shadowed file key retains its own authorized URL: if an
|
||||
environment key is reused for another URL and later disappears, the old file key
|
||||
is not silently forwarded there. Keyless also requires its nonsecret binding, but
|
||||
never resolves a stored/environment key. Disconnect leaves an internal disabled
|
||||
marker so a latent environment key cannot revive an interrupted disconnect.
|
||||
|
||||
Changes take effect on the next catalog, settings-validation, or inference
|
||||
operation without restarting. Credential/header reference changes are also
|
||||
picked up through the existing key lookup. Every upstream model operation
|
||||
revalidates the same saved provider/model ID against the current Pi catalog,
|
||||
including later requests inside a long-running agent/tool loop. It uses the
|
||||
current endpoint, model metadata and resolved credentials. A removed model,
|
||||
invalid configuration or missing credential blocks the next operation, without
|
||||
falling back. An HTTP operation already in flight retains its own native Pi
|
||||
setup and is not implicitly cancelled by a file edit.
|
||||
Use the custom endpoint editor to change custom credentials; generic
|
||||
`/api/keys` PUT/DELETE cannot bypass binding. No credential-derived fingerprint is
|
||||
stored in settings or a public revision. These guards cover process crashes and
|
||||
restart using the existing atomic rename; they are not a power-loss/fsync
|
||||
transaction system. Missing auth storage can initialize normally; unreadable or
|
||||
malformed private storage fails closed without overwriting its contents.
|
||||
|
||||
Deleting the file means no custom providers. Invalid JSON, invalid metadata,
|
||||
reserved IDs, unsupported configuration, or an unreadable file disables the
|
||||
custom routes, **not** built-in providers. The file accepts Pi's line comments
|
||||
and trailing commas. `GET /api/models` includes an optional sanitized
|
||||
`customProvidersError` diagnostic; rendering it in setup/settings UI is deferred.
|
||||
Correcting the file restores valid providers on the next operation.
|
||||
Each upstream model operation resolves the current custom model, endpoint, effective
|
||||
credential, limits, and original saved effort preference against Pi's current model
|
||||
metadata. Already-started requests keep their setup. A removed model or disconnected
|
||||
provider fails rather than falling back; saved model selections and history are not
|
||||
rewritten.
|
||||
|
||||
Saved selections and thinking preferences are not erased when a provider/model
|
||||
is removed or becomes unavailable. New work using an unavailable selection fails
|
||||
rather than falling back to another model. Select another configured model to
|
||||
repair the slot. Custom-provider removal does not rewrite historical records;
|
||||
like existing API-key models, historical cost estimates use the current catalog
|
||||
and become unknown when the relevant model definition is unavailable.
|
||||
Edits use an opaque nonsecret ETag/If-Match revision. A stale edit returns **409** with
|
||||
reload/retry guidance. A connect also checks configuration and effective credentials
|
||||
after discovery, and merges only the custom section into the latest settings record,
|
||||
preserving intervening unrelated edits. Failed mutations never claim success.
|
||||
|
||||
Desktop requests use the existing desktop capability. Browser mutations must be
|
||||
same-origin JSON requests. Limits are **15 seconds** for discovery, **8 MiB** for its
|
||||
response, **10,000** listed models, and **64 KiB** for configuration-write bodies.
|
||||
These are protections for the new endpoint, not a multiuser authentication system.
|
||||
|
||||
## Pricing and previous development configuration
|
||||
|
||||
Custom model pricing is **unknown/unpriced**, not $0—even for an endpoint that may
|
||||
be locally hosted. Pi-required internal zero-rate placeholders are not presented as
|
||||
free inference. There is no rate editor in this release.
|
||||
|
||||
The earlier, unshipped app-data `models.json` feature is retired and is **not read**.
|
||||
No user file is deleted or renamed, and there is no importer, coexistence mode,
|
||||
profile reset, history rewrite, or silent model remapping. Existing development
|
||||
profiles need manual endpoint configuration and model reselection. Unbound early
|
||||
GUI development configurations also require explicit reconnect/new-key entry or
|
||||
the existing reuse-consent checkbox, even at the same URL; they are never silently
|
||||
trusted or migrated. This release
|
||||
supports one OpenAI-compatible custom endpoint, not multiple arbitrary protocols.
|
||||
|
||||
Reference in New Issue
Block a user