The hand-packed icns was pixel-clean (verified by unpacking and scanning
every size), but macOS 26 composites legacy .icns icons into the Liquid
Glass system at render time; stacked on Icon Composer's baked PNG-export
lighting, that clips the rim highlights into the near-corner specks and
abrupt highlight ends (owner report 2026-10-13). The fix is the real
layered icon: mac.icon now points at the Icon Composer document
(build/icon.icon, copied from ~/Documents/Affinity/lh-logo) and
electron-builder compiles it via actool into Assets.car — macOS 26+
renders the true Liquid Glass design (dark/tinted variants included).
actool's backwards-compat icns caps at 256px, so scripts/after-pack.cjs
swaps in the existing full-res build/icon.icns before signing: pre-Tahoe
macOS keeps today's quality, Tahoe gets the layered icon.
Windows/Linux get the owner's flat design (rounded squircle with baked
transparency): scripts/make-icons.mjs renders build/icon-flat.svg via
resvg into build/icon.ico (16→256 PNG entries) + build/icon.png (512),
and electron-builder.yml pins win.icon/linux.icon.
desktop:dist injects DEVELOPER_DIR (actool needs Xcode; the machine
selects CommandLineTools by default, CI's setup-xcode takes precedence
via the :- default). The release workflow's mac job selects Xcode 26+
explicitly. Verified locally: mac --dir build (Assets.car + checksummed
full-res fallback + CFBundleIconName/IconFile) and the win-unpacked exe
carries our ico bytes; NSIS/AppImage assembly is CI's (cross-build
tooling can't run on arm64 macOS). Runbook updated. 1610/1610.
- desktop:dist cleans dist/ + .next/standalone before next build: the
trace sweep was shipping the PREVIOUS build's DMG inside every new one
(standalone/dist = 8.0GB inside the app; measured 7.4GB DMG → 144MB).
- the same sweep shipped the repo-root auth.json (dev BYOK keys,
byte-identical) in every DMG ever built — excluded in the trace and
REFUSED by a new build-server.mjs guard (.env* belt-and-braces).
- WARNING (next.config.ts): dir-style trace excludes (dist/** etc.,
"./"-anchored or not) strip server-external packages to bare
package.jsons — pi-ai/pi-agent-core lost their dist/, the packaged
server 500'd on ERR_MODULE_NOT_FOUND. Only the three verified-safe
entries stay.
- new scripts/smoke-server.mjs gates desktop:dist between payload
curation and packaging: boots the standalone exactly as the Electron
shell does (temp LH_DATA_DIR, empty auth) and requires 200s on / +
graph/models/settings. Caught both defects above on its first day.
- entitlements: allow-unsigned-executable-memory dropped (2019-era
relic per current electron-builder docs; weakens hardened runtime).
- pi-ai + pi-agent-core pinned exact 1.0.2 — persisted chat records
carry their wire shapes; post-release a bump is a migration event.
Gates: typecheck, lint, 1459/1459 tests, full desktop:dist + smoke PASS.
Owner ruling (2026-09-30, live-testing loop — supersedes 02a58db's
exemption): implication nodes ARE depth-anchored after all. Their anchor
is the average of the HIGHEST premise row and the conclusion's row —
depth(imp) = (max(premise depths) + depth(conclusion)) / 2 — so the
junction always sits strictly above every premise and strictly below
the conclusion (a conclusion is >= highest premise + 1 in the logical
graph). Uniform across modes: a half-row in vertical, a ring midway in
radial (owner confirmed both).
Implementation:
- layout.ts: new ImpLink type; SimOptions/incrementalLayout gain
impLinks (the derived chain linkage). The rank map is now computed
once alongside the orphan mask: proposition ranks via Tarjan over the
logical edges, then each supplied link adds its midpoint entry. An
imp node WITH an anchor is structured (leaves the orphan mask —
gravity projections treat it like any ranked node); WITHOUT supplied
links, arg_ ids keep the legacy 02a58db exemption (honest fallback
for callers that predate the field). The depth force reuses the
precomputed map — one pass, shared.
- deriveView (core/graph.ts) emits impLinks (one per multi-premise
live chain — single-premise chains mint no imp node); GraphSnapshot
carries them; graph-view maps them into CanvasData; settleGraphLayout
and replay-layout forward them (replay activates a junction's link
exactly when its chain is delivered).
Tests (+3, 1 rewritten): the 02a58db exemption pin became the MIDPOINT
pin (premises row 0, conclusion row -500, junction pinned to -250);
mixed-depth pin (premises at 0 and 1, conclusion 2, junction at the 1.5
half-row — highest premise, not average-of-premises); legacy fallback
pin (no impLinks = never depth-anchored); deriveView impLinks emission
pin (multi-premise only). The horizontal-pole and ORPHAN RULE pins from
5554c3a survive unchanged — proposition orphans are untouched.
Owner-approved brief (orchestrator, physics arc): two laws.
1. IMP-NODE CLUSTER ASSIGNMENT. Imp nodes were unclustered (the partition
runs over stored propositions only) — no centroid pull, outside the
painted hulls, bundles sitting on the periphery pointing outward.
Now: a derived imp node's cluster is a function of its derivation —
MAJORITY among its PREMISES' birth sessions (owner: premises, NOT the
conclusion), ties to the chain's OWN birth (its provenance's first
point — the asserting session; deterministic, coincides with the
unanimous common case). Imp nodes are always re-derived, never
birthed, so the assignment must be exactly this.
Seam chosen: core/session-clusters.ts exports impClusterKey + a
shared attachImpToCommunities that mutates a communities list's
members IN PLACE. graphSnapshot attaches AFTER the hull filter (a
2-proposition community + its bundle never resurrects a hull — the
filter counts propositions); settleGraphLayout attaches to the
unfiltered partition (settle pulls every community; hulls are a
canvas-only concern). Both cluster-membership consumers — the sim's
centroid pull and buildRegions' rasterization — read it through the
members lists that went through the one helper, so the resting
ledger and the live sim agree on where bundles belong.
2. DEPTHS OVER THE LOGICAL GRAPH. inferenceDepths ran Tarjan over the
DRAWN edges (premise→imp→conclusion), inflating every multi-premise
conclusion one hop (depth(imp)+1 instead of depth(premises)+1) and
ranking the geometry point itself. Now:
- deriveView additionally emits logicalEdges — every chain contracted
to direct premise→conclusion arrows over PROPOSITION ids only
(per-premise, deduped — a conclusion's rank is max over premises,
which a single arbitrary premise edge could not express).
- GraphSnapshot carries logicalEdges; createGraphSimulation /
incrementalLayout accept a logicalEdges option: the depth
machinery (orphan detection AND the rank pass) reads it, springs
keep the drawn edges. Absent = edges (legacy behavior).
- Imp nodes join the ORPHAN-EXEMPTION mask explicitly — never pinned
to a level, keep the full regular forces (gravity, cluster pull,
springs), settle naturally between premise and conclusion levels;
the rank map is computed over propositions only, so arg_ ids are
absent from it by construction.
Wiring: settle + canvas (CanvasData.logicalEdges → buildSim) +
replay-layout all pass the logical set.
EXTRA FIX (in-slice discovery, one line, graph-server domain):
server/graph.ts graphClusters — the practice picker's /api/graph/
clusters — partitioned snap.NODES (the canvas image, imp nodes riding
it since chains-primary), leaking arg_ ids into nodeIds: pickable ids
that silently vanish inside chainsForSelection while nodeCount lied.
Now partitions snap.propositions — same purity law as the chain
table's 767a559 fix.
Tests (+13): impClusterKey pins (unanimous; cross-session majority; tie
→ chain's own birth, not the conclusion's); attach pins (members gain
the imp id, sorted; single-premise attaches nothing — no node exists;
dangling mints nothing; home absent from the list stays unattached);
snapshot pins (imp id rides cluster members to both consumers; the hull
filter counts propositions); logical-depth pins (multi-premise
conclusion at premises'+1 NOT +2; imp absent from the rank map);
depth-force exemption pin (rows pin crisply, arg_x floats unpinned —
old behavior stamped it onto the conclusion's row); deriveView logical
arrows (per-premise, deduped, proposition ids only).
Owner rulings 2026-09-30: reasoning chains are the stored truth; the graph
canvas renders a lossy derived image. No migration, no backcompat — old
graph data was wiped.
- contracts: GraphChain (arg_-minted id, premises, conclusion) replaces
typed edges; GraphEdge survives only as the untyped derived arrow;
GraphSnapshot carries stored chains; supports edges retired entirely
- store: chains.jsonl ledger (edges.jsonl purged at load alongside legacy
events.jsonl); readChains/appendChain seam; merge dedupes by
(sorted premises, conclusion), re-assertions append under the same arg_ id
- core: deriveView mints implication nodes (name ⟹, statement via
formatChainLine) plus premise→imp→conclusion arrows; clusters stay
proposition-only (birthSessionPartition); chainsForSelection now groups
STORED chains (chain rides iff every member selected) — independent
arguments for one conclusion never union; chainsToGraph/chainsToEdges die
- extraction: extractor submits chains natively (expandFindings deleted);
auditor receives propositions only; supports fan-out gone
- deletion: full death of a proposition kills every referencing chain;
arg_-prefixed node deletes kill that chain; chainsRemoved stats
- UI: every arrow renders the ⟹ double shaft; supports toggle + thickness
slider and detail-card edge sections deleted (card keeps name +
proposition); chain table + practice pipeline read stored chains
- tests: 1188 green across the new seams
PRICES must stay a generated static table (core is client-safe; the
renderer folds costs and reading the catalog JSON needs fs), so the
forgetting-the-sync failure mode is guarded instead of eliminated.
- sync-pricing.ts: buildPrices() extracted (the pure catalog walk),
exported for the test; main() now guarded by an entry-point check so
importing the module runs nothing
- tests/prices-sync.test.ts: asserts PRICES equals EXACTLY what the
script would generate now — unpriced new models, drifted prices, and
keys gone from the catalog all fail with 'run npx tsx
scripts/sync-pricing.ts'. Verified it fires by deleting the
deepseek-v4.1-flash entry (1 unpriced, actionable message)
- sync script's embedded header template corrected to match prices.ts
(no bare-slug legacy claim) + drift-guard note, so the next
regeneration keeps the honest docs
The persisted (chatId, sourceId)→node table was a shared namespace across
independent chunk/batch jobs: each chunk's extractor mints local ids fresh
(p1..pN), so later chunks collided with earlier ones — propositions silently
skipped as 'already resolved' (coverage checkpointed anyway: permanent silent
loss) and edges resolved onto the wrong nodes (the 'everything links to
everything' mush). Owner ruling 2026-09-29: mergeFindings resolves THIS job's
ids in memory, nothing persists between jobs; cross-job identity is the
auditor's linkTo verdicts alone.
- contracts: NodeMapping dies; NotesSource.contentFp — sha256 of the text at
the last run, the re-import guard (unchanged re-import = TRUE no-op, no
model calls, no usage; changed text re-runs; legacy unstamped sources
re-run once)
- merger: job-local resolution map, in-submit duplicate ids skip once,
edges resolve extractor-local ids only
- ports/store: appendMapping/readMappings, LEDGER_FILES.mappings,
rewriteLedgers.mappings gone (stale data/graph/mappings.jsonl just stops
being read)
- session-clusters: birthSessionOf is provenance-only — the legacy
mapping-line fallback for provenance-empty notes nodes dies
- deletion: no mapping rows to strip; stats lose mappingsRemoved
- replay-layout batches nodes by first provenance point
- graph.test pins the new independence semantics (re-merge re-creates;
duplicate ids within one submit resolve once); extraction tests pin
per-chunk merges + the fingerprint guard
The graph becomes raw reasoning chains only. Quiz outcomes and
understanding observations are no longer formalized; assessment
moves to the future practice module.
Events ledger removed end-to-end: LedgerEvent/LedgerEventKind/
UnderstandingLevel/ExtractionAttribution/AuditResult.dropped and
snapshot nodeObservations deleted from contracts; quizEvent builder
and the events step of mergeFindings gone; appendEvent/readEvents
dropped from the port and store; LEDGER_FILES loses events.jsonl
with a one-time purge of a stale file next to the nodes ledger;
graph-deletion and the delete dialog lose event purge + quiz-answer
impact counts; GraphSessionProposition becomes a pure birth list.
Whole-session markdown source view: new core/session-markdown.ts is
the shared formatting engine (id-anchored speaker blocks, quiz/
recall/question blocks, tool one-liners) — the extractor's prompt
view and the future user-facing export render from one engine.
Coverage fingerprints deliberately stay on the old line format
(content-hash semantics; re-formatting never re-opens covered
history). The whole session goes to the extractor — no phase cut.
Auditor refocused as the identity linker: never sees the source;
explores the graph via new auditor-specific tools in
agent/tools/graph.ts (find — glob over name/statement/aliases;
list_sessions; list_propositions with full statement text; no
timestamps) instead of an injected full-graph dump; equivalence
bias — paraphrases merge, only genuinely distinct propositions stay
apart; no drop channel, the extractor is fully trusted; may draw
bridging edges (typed follows/supports) between existing nodes and
the batch, resolved by the merger with chat-scoped message-less
provenance. Agentic loop, maxTurns 16.
UI: the node detail card gains an edges section above the sessions
list (follows-from / supported-by / what-follows / what-it-supports,
rendered only when present, click-to-select neighbors); the
understanding-note chips are gone.
Prober list_propositions reworded to what a session covered —
names only, no levels. docs/architecture.md + README updated to
the as-built pipeline. Gates: typecheck, 1059/1059, lint, build.
Owner-locked extraction-UX redesign (dispatch 2026-09-24), part 1 (data
layer; the UI chunk follows).
THRESHOLD -> SETTING
- config.ts's EXTRACTION.chunkTokens dies; the default moves into
FACTORY_SETTINGS.graph.extractionChunkTokens (4096, the old constant's
value). GraphSettings gains the optional field, Required at resolution
via the standard ?? FACTORY pattern — pre-redesign records resolve the
default cleanly.
- FLOOR 500 (the implementer's call, documented on the field): every run
pays the graph-carrying prompt overhead out of the write channel's
money — a smaller budget fires penny-batches that cost more per
extracted token than the content is worth. The sanitizer CLAMPS up
(bd68486 clamp-never-reject discipline: fractions floor, junk drops);
the settings PUT REJECTS below-floor with the rationale in the message.
No ceiling: a large budget just delays extraction — the user's stated
cost/thoroughness tradeoff, not a misconfiguration to police.
- The trigger (turn-service.triggerExtraction) takes the budget as a
parameter — chat-handler passes effective.graph.extractionChunkTokens,
the fresh per-turn read, same as every other gate.
- The pipeline's BATCH side threads through the same number:
ExtractionPipelineDeps gains an OPTIONAL getChunkTokens getter with the
factory default (Teacher's 24 test fixtures stay untouched; the two
call-site lines inside extraction.ts are the only cross-band edits,
flagged). chat-handler + backfill/import-notes wire the live
effectiveSettings — trigger and batches share one budget, always.
PER-SESSION COVERAGE READ
- sessionCoverage(chatId) (server/chats.ts): the chat view's ring-meter
block — totalMessages/coveredCount/uncoveredCount/coveredTokens/
uncoveredTokens (the same coverageOf derivation /api/graph/coverage
serves, scoped to one chat), running (chat-kind extraction registered
for this id), and lastFoldedAt (the chat's newest coverage row's fold
— a cheap single pass; null before any fold). GET /api/chats/[id]
carries it as coverage: {...}; unknown chat -> null (the route 404s
first).
- /api/graph/coverage (the all-sessions list) compiles untouched — the
per-session read is a scoping of it, not a supersession; the UI's
sidebar/ring chunk decides which surface it consumes.
Tests: factory/sanitizer pins (clamp-up, fraction-floor, junk-drop),
resolution pins (absent -> 4096, stored 2000 wins), settings-body pins
(valid/absent/fraction/below-floor 400/junk 400), sessionCoverage pins
(derived split, newest-fold scoping, cross-session isolation, running via
the registry, null on unknown chat). 893/893.
Owner ruling (2026-09-23, amendment to 04e336d): no hardcoded fallback
exists anywhere. contextWindow(record, limits) resolves the denominator
ONLY from the live index (provider:id -> contextWindow straight from
pi's catalog); a model missing from it (historical session, package
update, failed fetch) reads limit null and the meter renders
usage-only — the honest-render precedent, never a fake window.
- src/core/prices.ts: the 1354-entry CONTEXT_LIMITS table (65KB) and its
generation apparatus deleted; the fold returns limit: number | null.
PRICES untouched (regen verified byte-identical via the reworked
sync-pricing script).
- scripts/sync-pricing.ts: generates PRICES only; contextWindow leaves
the priced map, the guard, and the template.
- The dock meter's owner-selected treatment: the activity feed carries
the limit server-side. catalogContextLimits() (server/model-catalog.ts)
builds the index over the same keyed providers the picker federates;
activityFeed() joins it onto every running job (ActivityJob.contextLimit,
null when the model is missing — the dock reads it off the feed, no
client-side table, no regression while the UI's catalog-join chunk lands).
- components/activity-dock.tsx: toBlobs reads job.contextLimit (import
of CONTEXT_LIMITS gone); stats-bar renders usage-only on null limit
(mechanical interim — UI stream's join chunk follows).
- tests: usage.test.ts pins the no-fallback contract (index hit / absent
/ empty / missing-model -> null); activity-feed.test.ts pins the
server join (catalog model -> 1048576, unknown -> null, no-usage ->
null); render pins updated to the interim usage-only render.
Part 1 — the ledger + remnants interlude (master-dispatched, owner ruling
2026-09-23: nothing configures the graph beyond chat models):
- The embedding ledger surface is gone: readEmbeddings/writeEmbeddings
ports, the EmbeddingsLedger contract, EMBEDDINGS_FILE (config, store
impls, graphRevision source list), and graph-deletion's vector-pruning
step. On-disk data files stay untouched (inert history).
- The extraction embedder seam dies wholesale: ExtractionPipelineDeps
drops the embedder dep (chat-handler + backfill/import-notes stubs go
with it), the 'embedding' phase leaves ExtractionPhase, the usage
capture/persist helpers die, and stages.embedding leaves
ExtractionRunUsage (historical records keep their shape — reads are
lenient). Extraction is two stages again, like the local-embedder era.
- Consumer-less remnants: latestQuizOutcomes + QUIZ_EVENT_KINDS,
clusterContentId, matchClusterLabelings + Community + ClusterLabeling,
the readClusterLabels/appendClusterLabel port seams + store impls +
LEDGER_FILES.clusterLabels. NamingRunUsage/readNamingRuns STAY
(standing ruling — the cost pill renders historical naming spend);
clusterHue + MIN_CLUSTER_SIZE stay (live).
- The 'labeler' role dies: BACKGROUND_MODEL_ROLES, ROLE_SLOT, ModelRole,
the catalog pin projections, and the UI roster strings. RolePins keys
shrink with the union.
The notes birth fallback (owner-approved 2026-09-23, surfaced by the
interlude's live smoke: production notes imports create provenance-EMPTY
nodes (importNotes passes messages: null — the file's anchor is the
mapping line), so the provenance-only birth rule left every imported
file unclustered and invisible to the prober's session list, against the
locked file-birth ruling. A provenance-empty node now births under its
FIRST mapping line's chatId (ledger order); only notes-born nodes ever
hit the fallback (chat nodes always carry provenance; a re-encountered
node's link-append re-opens the provenance rule). Deletion-safe: mapping
lines of deleted chats are stripped alongside their provenance.
Live-verified on port 3004: a fresh notes import clusters under its
src_ id with the file path as the label, end to end with no embedder.
845/845 tests, typecheck/lint/build green.
EOF
)
The owner's wave-7 ruling: embeddings die entirely (git history is the
stash); clustering is birth-session derived (chunk A, 950ddfd), labels
are session names, and the prober's graph tools replace injection.
- settings: GraphSettings collapses to { enabled } — embedding,
embeddingModel, clustering, labeling all die. Migration story: stale
record keys are UNREAD and dropped by the sanitizer's reconstruction;
the next graph-section PUT (replace-whole) persists the collapsed
shape. The settings-panel GraphSection renders one master row.
- deleted: src/server/embeddings.ts (adapters, resolveEmbeddingPortFor,
embedNewNodes, embeddingPort, embeddingModelCatalog), the
GET /api/models/embeddings route, scripts/embed-graph.ts, tests for
all three, and the /api/models/embeddings picker surfaces in the UI.
- prices: EMBEDDING_PRICES overlay + callCost fallback die. cost-summary:
the embedding stage fold dies — back to three stages (extraction/
audit/naming) per the owner ruling; historical stages.embedding spend
in old run records is simply not displayed.
- ports: EmbeddingPort/EmbeddingResult/EmbeddingTask die. The embeddings
LEDGER surface (readEmbeddings/writeEmbeddings + EmbeddingsLedger +
EMBEDDINGS_FILE + graph-deletion's vector pruning) STAYS — inert data,
kept only so graph-context.ts compiles until Teacher's pass removes
it; the seam dies in the follow-up interlude.
- chat-handler (approved cross-lane unwiring, wave-6 precedent): the
extraction pipeline's embedder becomes a documented no-op (nothing
embedded, nothing spent, ledger untouched) until Teacher's pass
removes the phase; the graph-context injection dispatch gate, the
graphContext deps wiring, the setContextRespawn seam, and
turn-service's triggerGraphContext are gone — injection is dead
code pending Teacher's file deletions.
- scripts: backfill-graph/import-notes pass the same no-op embedder;
replay-layout derives clusters from birthSessionPartition (the same
partition the real settle uses) instead of the embeddings ledger.
- settings-body: parseGraphSettingsBody reads the master switch only —
old-shape keys in a body are ignored, not rejected; bad shapes 400.
- next.config: the @huggingface tracing-exclusion guards die with the
last reference.
Cross-lane flags: extraction.ts/graph-context.ts keep stale embed
comments + the embed phase until Teacher's full pass (their parallel
interlude owns those files — zero overlap kept by design);
semantic-clusters.ts + its test stay one more round (extraction.test.ts,
Teacher's file, still imports it — they delete it with the namer cases);
ChatSettings.graphContextTokens + the graph-context-control UI stay
until Teacher+UI.
Gates: tsc clean, eslint clean, 885/885 tests, build compiles.
Live-verified on the port-3004 dev tree: PUT /api/settings/graph with
dead keys collapses on write, bad shapes 400, GET reads {enabled: true},
/api/models/embeddings 404s.
Wave 7 chunk A (950ddfd) replaced semantic clustering with birth-session
clusters labeled by session names — the LLM namer became dead code, but
extraction still called it, minting naming runs nobody reads on every
merge. Real model spend wasted per extraction.
Removed (agent band):
- src/agent/subagents/cluster-namer.ts — the naming sub-agent, deleted
whole (refreshClusterLabels, clusterNamingSettled, the naming loop).
- extraction.ts: both post-merge garnish blocks (chat + notes paths),
the labeler test seam on ExtractionModels, and the import. The labeler
role's only consumer is gone; the 'labeler' entry in
BACKGROUND_MODEL_ROLES/ROLE_SLOT (core band) is now consumer-less —
flagged to Platform, not crossed.
- scripts/import-notes.ts: the settle-before-exit line + the named-
clusters count in the summary.
- tests: the cluster-naming describe (naming + failure-non-fatal tests)
and the dead CLUSTER_* fixtures; the labeler-less BYOK test repurposed
as the kill pin — a merge mints NO naming runs and NO label records.
NOT touched (flagged, outside the agent band): matchClusterLabelings
(core/clusters.ts), clusterContentId (core/graph.ts — now consumer-
less), the appendClusterLabel/readClusterLabels port seams + store
impls + ClusterLabeling type (store/core band), tests/clusters.test.ts's
matchClusterLabelings describe. Cost-summary's naming stage untouched
per Platform's ruling (historical spend display reads readNamingRuns,
which stays live).
Gates: typecheck, 921/921, lint, build.
Teacher's Wave-6 review flagged a money-ledger hole: a mid-batch embed
throw discarded the spend of batches that had already landed (extraction
only sees success results). Owner ruling 2026-09-23 (option A, pay-once):
the landed vectors persist too — a retry embeds only what remains, so
each text is billed exactly once.
- embedNewNodes never throws on embed failures: per-batch try/catch,
result gains `failed` (error message), `embedded` counts what actually
landed, and usage carries the completed batches' steps either way
(usage-on-error).
- Zero-landed failures write nothing — the old ledger (its own
model/dim) survives untouched. Stale-wipe semantics unchanged.
- scripts/embed-graph.ts: honest failure output (lands count, not the
in-memory total), exit 1 on failure.
- tests: 4 new cases — mid-run failure (32 land, 8 remain; retry
resumes and re-bills only the rest), first-batch failure (no write, no
spend, old vectors survive), success carries no failed key.
Live-verified on port 3004: bogus-model 400 from OpenRouter → exit 1,
ledger untouched; healthy re-run embeds 0 (no re-spend); reset scratch
setting to null. Gates: typecheck, lint, 899/899 tests, build.
The owner needs exact new-user simulation as a one-command launch
(repeatable first-run BYOK testing). Option A, owner-approved:
env-hiding switch + launcher script.
Fix 1 — LH_IGNORE_ENV_KEYS=1 in the keys seam: when set, every env key
is invisible app-wide; only auth.json counts. The mechanism is boot-time
DELETION, not seam-local filtering: pi-ai resolves provider keys from
process.env NATIVELY at every request (verified — auth/context.js's
defaultProviderAuthContext reads live env, no boot snapshot), so
deleting the managed vars from process.env in initAuthEnv is the ONE
mechanism that hides keys from every consumer — model engines included,
plus the three direct process.env readers (visual-maker, pdf-reader,
openRouterApiKey). realEnv stays EMPTY under the switch (nothing counts
as env), and the fill-gap still reflects auth.json keys so a fresh user
can add keys mid-session and engines see them with no restart.
keyValue reads file-only under the switch as belt-and-suspenders (a
post-boot env reintroduction stays invisible to the seam). Unset
switch = today's behavior, byte-for-byte (server-keys.test.ts pins it).
Fix 2 — scripts/dev-fresh.sh (npm run dev:fresh): fresh throwaway
LH_DATA_DIR (.fresh-data/, wiped each launch, --keep preserves), and —
the subtle bit — LH_AUTH_FILE redirected INTO the fresh dir: dev's
AUTH_FILE default is the repo-root auth.json, which may symlink the
real keys; the fresh instance must never see it. Sets the switch,
dedicated port (default 3005, --port overrideable, standing assignments
3000-3004 refused), prints URL + data dir, safe to rerun. .fresh-data/
gitignored.
Test: tests/fresh-env-keys.test.ts (own process — the switch is
boot-time): the engine-path deletion asserted directly, the env/file
collision resolved to the file, status rows report file sources only,
mid-session saveKey/deleteKey round-trip, idempotent re-init, and the
unset-switch env-wins read seam live-checked.
Live-verified on 3005 via the launcher: zero federated models, all
providers keyPresent false, no present keys, no auth.json/settings.json
in the fresh dir; then mid-session ZAI save → 7 zai models federate
instantly, source 'file' — while .env.local's real keys stayed
invisible throughout.
Gates: typecheck, lint, 863/863, build.
Model slots wave (2026-09-20), chunk B:
- settings: new `models` section {strong, cheap, vision} slots (each
field independent, same shape as a role override); ROLE_SLOT maps all
nine roles to tiers (strong=prober+teacher, cheap=researcher/
demoBuilder/extractor/auditor/labeler, vision=visualMaker+pdfReader);
resolution chain per role: override → slot → factory pin.
- config: ModelSpec gains provider (all pins openrouter, the BYOK
default); MODEL_PINS in stores.ts now spans all nine roles (the strong
slot's factory rep IS the prober pin).
- engine: piModels() registers EVERY builtin provider catalog;
piEngineForSpec(resolved spec) is the new engine factory (pi-ai
resolves per-provider keys from env itself); piEngineFor stays as the
config-pin fallback. chat-handler resolves the conversation chain
(chat override → strong slot → pin) and injects engineFor into the job
runner, closing the old gap where per-role overrides never reached
job engines.
- /api/models: models federate ONLY keyed providers (providers surface
with keyPresent, env-wins, fresh per call); slots (resolved tier pins)
join rolePins; catalogDefaultsFor inherits the strong slot; the models
settings section gets the same catalog gate as chat.
- pricing: PRICES/CONTEXT_LIMITS uniformly keyed provider:id
(regenerated from ALL pi-ai catalogs, 1354 keys); usage stamps
(TurnUsage, JobUsage, extraction stages) switch to provider:id —
owner decision: no legacy compat, old stamps render unpriced.
The web app becomes a desktop app without changing shape: the Electron
shell spawns the Next standalone server on a loopback port and points a
sandboxed window at it. All app logic stays in the Next server.
Bands:
1. LH_DATA_DIR seam (config.ts): PATHS.data/chats/graph derive from
process.env.LH_DATA_DIR (else repo data/, unchanged in dev). New
PATHS.modelCache routes the transformers.js embedding-model cache
(~300MB) to <data>/models when the seam is set — the default cache
inside node_modules is read-only in a packaged bundle. All existing
fs writes flow through PATHS; teacherPrompt stays ROOT-anchored
(read-only asset, ships inside the bundle by design).
tests/data-dir-seam.test.ts covers set/unset via child process.
2. Standalone build (next.config.ts + scripts/build-server.mjs):
output 'standalone'; outputFileTracingExcludes data/**/* keeps the
repo's real session data out of the bundle (verified: an unexcluded
scratch build traced 34 files / 716K of real chats + ledgers).
Two empirically-verified tracer gaps required outputFileTracingIncludes:
pi-ai's provider catalog (JSON import attributes — 39 files, 500s on
/api/models without them) and onnxruntime-node's sibling dylib
(libonnxruntime.dlopen via @rpath). build-server.mjs copies public/
and .next/static per docs, refuses a leaked data/ dir, and asserts
the runtime-read prompt .md assets are traced.
3. Electron shell (electron/main.cjs, no preload):
- single-instance lock (two instances would fight over the data dir);
- free port via listen(0); spawn via process.execPath +
ELECTRON_RUN_AS_NODE=1 with PORT/HOSTNAME=127.0.0.1/LH_DATA_DIR;
- BYOK: userData/auth.json, flat {VAR: value} JSON, mode 0600,
merged into the child env only when absent from the real env
(auth UI is the settings milestone; file is hand-editable);
- poll until the server answers (no ready event), then open a
contextIsolation+sandbox window; deny-by-default permission
handler (media allowed only from the loopback origin — mic/STT
groundwork); will-navigate pinned to origin, window.open denied
with external links to the system browser; macOS Edit-menu roles;
- quit path SIGTERM→3s→SIGKILL + exit/SIGHUP nets; child stdout
mirrored to userData/logs/main.log;
- dev mode via LH_DEV_URL: no spawn, points at next dev (HMR).
4. Packaging (electron-builder.yml + build/entitlements.mac.plist):
files = electron shell only; the whole .next payload ships via
extraResources from .next (filter standalone/**/*) — NOT from
.next/standalone directly: electron-builder hard-excludes a copy's
root node_modules and the default pattern is a literal match, so
node_modules/ and .next/ were silently dropped and the server died
with 'Cannot find module next'. Asar holds no native code.
Unsigned (identity: null) until a Developer ID cert exists; hardened
runtime + audio-input entitlements laid now for the STT milestone.
desktop:dev / desktop:dist scripts (electron 44, electron-builder 26).
Live smoke of the packaged .app: server up on a random loopback port,
all API routes 200, a real GLM chat turn answered end-to-end via
auth.json BYOK, chat persisted under userData/data, second instance
exited via the lock, no orphaned server child after quit.
Gates: typecheck, eslint, 500/500 tests (+2 data-dir-seam), build.
Louvain only ever found the reasoning chains (the graph is disconnected
by nature — every connected component became its own cluster) and
edge-orphans could never join anything. Clusters are now RELATED TOPICS:
propositions grouped by meaning.
- Embedder (src/server/embeddings.ts): in-process transformers.js v4
running EmbeddingGemma-300M q8 (MTEB 68.4, ~300MB one-time download),
MRL-truncated to 512d, "title: none | text:" prefix on 100% of inputs,
L2-normalized so cosine = dot. Model/dim mismatch voids the ledger
(full re-embed). Injected at the composition root — the agent layer
never imports the server layer (architecture law).
- Ledger (data/graph/embeddings.json): model + dim + vectors + the
persisted cluster partition. Reclustering runs at the END of every
embedding batch (post-merge hook, both front doors) — never per read
(the agglomerative pass is ~1.5s at 338 nodes).
- Clustering (src/core/semantic-clusters.ts): hand-rolled average-linkage
agglomerative over cosine (npm HDBSCAN is an abandoned alpha). Stragglers
stay singletons by default; attachStragglers is the all-in toggle.
Threshold 0.60 swept on the real graph: 26 topics, 74/90 edge-orphans
land in real topics, 10 stragglers.
- Consumers (snapshot, settle, namer, replay) read the persisted
partition — one cluster concept everywhere; core-survival label
matching unchanged (20/26 topics kept labels).
- Louvain retired: detectCommunities + graphology* deps deleted
(graphology-louvain had leaked into the client bundle since the
clusters wave); scripts/bench-graph.ts (dead FA2 bench) removed.
- scripts/embed-graph.ts: backfill/cold-start (338 nodes in 150s
including the one-time model download).
The graph's follows/supports edges encode derivation (premise ->
conclusion); the layout can now express that rank spatially, as a
session-only playground toggle (nothing promoted yet):
- inferenceDepths: longest chain from a base node, CYCLE-SAFE by
SCC condensation (Tarjan, iterative) — the DAG property is typical,
not guaranteed; cycle members share their component's depth (mutual
support = equal footing). O(V+E), pure, deterministic.
- DepthAnchor force: per-node target scalar — vertical mode rows
(baseline at the anchor centroid, arrows up), radial mode rings
(foundations center, outer boundary = edge of knowledge). The rank
is computed, not simulated: pairwise up-forces can't give baselines.
- Mode projections: gravity x-only in vertical / dropped in radial;
cluster pull x-only in vertical / tangential-only in radial; the
anchor re-blend preserves user arrangement in the FREE dimension
only (x / angle) — meaning owns the constrained axis, memory owns
the free one.
- setPhysicsTuning reheats (alpha 1): toggling a mode re-forms the
graph live and commits the new rest state; drag release is springy
along the constrained axis (node returns toward its level/ring).
- Sweep on the real graph: strength 0.8 = 215/216 arrows correct with
organic sag; 1.5 = crisp rows. 174 base nodes, max chain depth 7,
zero cycles in current data. replay-layout --depth=<mode> bakes a
mode into the ledger for rest-state experiments.
The 'virtual node' realized as a centroid force: two allocation-free
O(n) passes per tick (membership resolved once at sim creation;
Float64Array sums). Measured indistinguishable from force-off at 338
nodes (271ms vs 272ms per 400-tick settle).
- createGraphSimulation: SimOptions.clusterPull + linkStrength +
velocityDecay (physics controls plumbing); degree-scaled link
stiffness so linkStrength=1 is exactly d3's default
- settleGraphLayout split into layout-settle.ts (it computes the
Louvain partition for the pull); layout.ts stays browser-barrel-pure
- barrel no longer re-exports clusters.ts: graphology + louvain are
fully out of the client bundle (verified against built chunks)
- replay passes the per-batch partition; CLUSTER_PULL_STRENGTH=0.15
tuned by sweep: cluster spread 63->36 world units, centroid
separation and total span unchanged (no blob merge, no collapse)
One engine owns the equilibrium: the merge settle and (next commit)
the live drag both run createGraphSimulation — d3-force link springs,
1/d charge, weak centroid gravity — so a drag's rest state IS the
server's rest state. The FA2-era anchor mechanism survives as a
post-tick POSITION-SPACE re-blend (retention 0.85): d3's forceLink is
a Hooke spring (strength*(d-L)*alpha), which overpowers any force-space
anchor at long range — the re-blend is alpha-independent and always
wins as the sim anneals (the 675-unit drift bug). Gravity retuned for
d3's 1/d many-body falloff (bootstrap 0.2 / incremental 0.12: cluster
spacing lands ~400 world units, the FA2-era density). Rim orphans seed
(and anchor) at 1.7x the CONNECTED core's mean radius — a max-span-based
rim fed back on itself and ran the layout away to 6000 units. Replay:
338 positions in 1.6s, old-node shift within a batch 4.8 (FA2: 23).
The layout now EVOLVES instead of reshuffling on every extraction. A pure
core module (src/core/layout.ts) warm-starts forceAtlas2 from the positions
ledger (data/graph/positions.json, read-modify-write like sources.json,
updatedAt stamp for future live-physics conflict handling), re-blends every
previously-positioned node toward its anchor each iteration (retention 0.85
- old placements stay recognizable, strong forces still win), seeds new
nodes near their positioned neighbors, and seeds orphans on the rim outside
the core so they read as unconnected knowledge.
- Port/store: GraphStore.readPositions/writePositions; snapshot carries
positions when the ledger exists (optional - fresh graphs unchanged).
- Extraction pipeline settles synchronously after every merge (chat and
notes paths), before usage persist - the next /api/graph fetch already
reflects new nodes.
- View: the ledger IS the layout; the local one-shot FA2 pass survives
only as a fallback when no ledger exists.
- scripts/replay-layout.ts: simulates organic growth by replaying the
current 338-node graph one imported file at a time (real import order,
edges activate when both endpoints exist). Real run: 338 positions in
0.3s, 90-degree-0 orphans collecting at mean r=338 vs connected r=194.
- Also restores the cluster region fills (hull background colors) that
6692b19 had removed on a misreading of "no glow".
Live-verified: fit zoom derives from the ledger bbox, pickAt aligns with
ledger positions at every probe, hover/click/detail-card work, 408/408
tests, typecheck/lint/build green.
Future live physics stays cheap by design: the same pure function is the
merge-time settler and the future client simulator; a pinned node is just
an anchor with full retention.
Core-survival labelings (vision section 1, DECIDED 2026-09-13):
ClusterLabeling records in a new append-only ledger
(data/graph/cluster-labels.jsonl), immutable, id = clu_ + sha256 of the
sorted core so re-naming races and re-runs dedupe at the store seam.
matchClusterLabelings (core/clusters.ts, pure): a community keeps a
record's label while at least half its core survives in it; both sides
single-successor (a split keeps one name; greedy by overlap, ties to
the newer record).
The namer (src/agent/subagents/cluster-namer.ts): ONE singular cheap LLM
call (new 'labeler' role, glm-flash/low) names every new/changed
community at once, given member names + statements (capped at 12
members per cluster). refreshClusterLabels runs POST-EXTRACTION
(user-decided) from both front doors, fire-and-forget — naming failure
never fails the run; refreshes serialize behind a chain so parallel
imports never mint duplicate labels. import-notes.ts awaits
clusterNamingSettled before exit.
Payload: GraphSnapshot.clusters (>= MIN_CLUSTER_SIZE 3) joins detection
with labelings at read time — claimed clusters carry the record's label
and id; unlabeled ones carry a member-hash id and null label. Detection
is deterministic so no memo in v1 (ms at current scale; revisit at
vault scale).
Tests (372/372): matcher half-rule/split/merge/greedy/tie cases; naming
end-to-end through importNotes (one call, one record, claim on
re-refresh), naming failure non-fatal; snapshot derivation with
labeling join; 11 test pipelines now pin an explicit labeler seam.
The same extractor→auditor→merger pipeline now runs over raw personal
notes in arbitrary formats (vision §1, DECIDED 2026-09-13):
- NotesSource records (data/graph/sources.json, deterministic id from
path hash → idempotent re-imports) play the chatId role in mappings,
checkpoints, and observation events; usage lands on the source record
(appendSourceUsage, idempotent per startedAt, kept across upserts).
- mergeFindings is chat-store-free: callers pass MergeSource
{messages, foldedAt} — sessions pass the chat record, notes pass
messages:null (no quiz-event rebuild; a deterministic rebuild needs
structured quiz tool parts).
- Second extractor prompt (NOTES_EXTRACTOR_INSTRUCTIONS) for general-
purpose extraction; understanding judges the AUTHOR's grasp. Auditor
wording input-agnostic. extract/audit share one generic runSubmitAgent.
- graphSnapshot.fileSources (srcId→filePath); notes sources render as
static path entries in the detail card (FileTextIcon, never a link).
- One shared pipeline instance (exported from chat-handler): same
single-flight registry, live status chips (label = file path), usage
accounting, and true abort for both front doors.
- Transport: POST /api/graph/import (multipart, 3-wide worker pool) +
Import notes… button in the graph view; scripts/import-notes.ts for
batch runs (concurrency 4).
- real-data-extraction/ gitignored (raw import sources never committed).
355/355 tests; typecheck, lint, build green.
sorokin.oleg.edu
amosblomqvist