Two follow-ups on the semantic-cluster wave, both user-reported:
1. Region coloring: edge-only dilation (the anti-double-counting rule
from the Louvain era) left 114 of 338 cluster members completely
uncolored — semantic clusters contain members that share no
intra-cluster edge with anyone (90 edge-orphans + isolated
micro-topics), and 20 of 45 clusters painted nothing at all. Fix:
a disc (radius = dilation, same as a round cap) for exactly the
members no intra-cluster edge stroke reached. Every member is
painted by its edges' caps OR one disc, never both — the
no-double-counting rule survives. Verified live: all 74
orphans-in-clusters sit on tinted region pixels.
2. Depth modes (ORPHAN RULE): a node with no edges has no dependency
structure, so depth-0 stamping put 90 unconnected propositions on
the vertical baseline / radial base ring — meaning they don't have.
Orphans are now exempt from the depth force and get the regular
view's unprojected forces instead (full center gravity + full
cluster pull). Structured-node projections unchanged (x-only
gravity in vertical, none in radial; tangential-only cluster pull
in radial). Gravity in depth modes is a custom force with EXACTLY
d3's forceX/forceY math (strength × alpha — the first draft
forgot the alpha and shifted the settle). Verified live: radial
orphans follow their topics instead of the base ring; vertical
orphans spread 1006 world units (6/90 near the baseline band vs
90/90 before).
The graph's follows/supports edges encode derivation (premise ->
conclusion); the layout can now express that rank spatially, as a
session-only playground toggle (nothing promoted yet):
- inferenceDepths: longest chain from a base node, CYCLE-SAFE by
SCC condensation (Tarjan, iterative) — the DAG property is typical,
not guaranteed; cycle members share their component's depth (mutual
support = equal footing). O(V+E), pure, deterministic.
- DepthAnchor force: per-node target scalar — vertical mode rows
(baseline at the anchor centroid, arrows up), radial mode rings
(foundations center, outer boundary = edge of knowledge). The rank
is computed, not simulated: pairwise up-forces can't give baselines.
- Mode projections: gravity x-only in vertical / dropped in radial;
cluster pull x-only in vertical / tangential-only in radial; the
anchor re-blend preserves user arrangement in the FREE dimension
only (x / angle) — meaning owns the constrained axis, memory owns
the free one.
- setPhysicsTuning reheats (alpha 1): toggling a mode re-forms the
graph live and commits the new rest state; drag release is springy
along the constrained axis (node returns toward its level/ring).
- Sweep on the real graph: strength 0.8 = 215/216 arrows correct with
organic sag; 1.5 = crisp rows. 174 base nodes, max chain depth 7,
zero cycles in current data. replay-layout --depth=<mode> bakes a
mode into the ledger for rest-state experiments.
User-facing live-sim controls (settings panel, Physics section):
node repulsion, edge attraction, edge length, central pull, cluster
pull, friction — session-only number inputs feeding createGraphSimulation
(the SAME engine the server settle runs, so the feel IS the physics).
Changing a value rebuilds the sim from current positions; drag pins
(fx/fy live on the node objects) survive; a mid-settle change re-anneals.
The graph-view FA2 bootstrap fallback is rewritten on incrementalLayout
(d3-force): the last graphology import in the shell is gone.
User feedback round 1 on the splat regions:
- Double counting: per-node gradient discs stacked with the edge-stroke
dilation wherever nodes sat on edges (hub hot spots). A cluster is its
CONNECTED skeleton — the region is now the dilated intra-cluster edge
network ONLY (round caps make line-end points discs), per the user's
definition. No per-node discs at all.
- The 'blur' was no blur: bilinear upscale of a 256px bitmap produced
tent-ramp chunkiness at zoom (the low-quality-JPEG look). Now a REAL
blur: 3 box-blur rounds on the premultiplied bitmap (separable running
sum, pure + unit-tested; premultiply keeps halos out of transparent
space). The upscale now has nothing left to make chunky.
- Live dev controls: number inputs (not sliders) in the settings panel —
dilation (screen px at fit), blur (bitmap px), bitmap resolution,
edge alpha. Session-only React state, never persisted; the renderer's
sanitizeRegionTuning makes bad/empty values safe. Verified end-to-end
through the real input (dilation 28 → 60 took effect).
- Premultiply un-recovery bug caught live: c = pm/a*255, not pm/a (the
first version rounded every color to black; regions rendered as
dark blobs with alpha 185).
- Region canvas: willReadFrequently (Chrome keeps it on the CPU for the
per-rebuild getImageData); defaults dilation 55 → 28 after measuring
screen coverage (63% → 55%).
- Perf: 120fps sustained through a drag with blur + rebuild-every-3rd-
frame (45 frames / 360ms measured), zero idle frames after settle.
The renderer grows a live d3 sim (createGraphSimulation, shared with
the server settle): pointerdown on a node pins fx/fy and pointer capture
(preventDefault kills the compat mousedown, so d3-zoom never pans); the
first CLICK_SLOP-exceeding move reheats to alphaTarget 0.3; release
un-pins and the sim anneals (release-and-relax) — alpha < 0.005 settles
it, a 250ms debounced onCommit PUTs the rest state. Verified live: 123fps
during a 320ms drag (regions rebuilding every 3rd frame), 85-tick settle,
then zero idle frames. A slop-free press stays a click (select, no reheat,
no commit). PUT /api/graph/positions sanitizes (finite coords, known ids)
and if the graph grew mid-drag runs ONE incremental settle with the
client's arrangement as anchors (50-id PUT test: anchored drift 0.15).
Cluster regions swap convex hulls for a low-res world-space SPLAT bitmap:
per-member radial-gradient discs (alpha 0.2) + intra-cluster edges as
round-cap strokes (Minkowski dilation — concave shapes finally render),
upscaled with bilinear smoothing (the upscale IS the blur, no ctx.filter:
Safari). One tiny blit per frame replaces 32 clusters x 4 offset
Path2D polygons re-rasterized at DPR 2 (the old GPU coil-whine path);
during the live sim the bitmap rebuilds every 3rd frame. cluster-geometry.ts
retires convexHull/offsetPolygon/traceRoundedPath (tests follow; clusterHue
stays). Region labels stay (centroid + clusterHue, inverse-gated on zoom).
The layout now EVOLVES instead of reshuffling on every extraction. A pure
core module (src/core/layout.ts) warm-starts forceAtlas2 from the positions
ledger (data/graph/positions.json, read-modify-write like sources.json,
updatedAt stamp for future live-physics conflict handling), re-blends every
previously-positioned node toward its anchor each iteration (retention 0.85
- old placements stay recognizable, strong forces still win), seeds new
nodes near their positioned neighbors, and seeds orphans on the rim outside
the core so they read as unconnected knowledge.
- Port/store: GraphStore.readPositions/writePositions; snapshot carries
positions when the ledger exists (optional - fresh graphs unchanged).
- Extraction pipeline settles synchronously after every merge (chat and
notes paths), before usage persist - the next /api/graph fetch already
reflects new nodes.
- View: the ledger IS the layout; the local one-shot FA2 pass survives
only as a fallback when no ledger exists.
- scripts/replay-layout.ts: simulates organic growth by replaying the
current 338-node graph one imported file at a time (real import order,
edges activate when both endpoints exist). Real run: 338 positions in
0.3s, 90-degree-0 orphans collecting at mean r=338 vs connected r=194.
- Also restores the cluster region fills (hull background colors) that
6692b19 had removed on a misreading of "no glow".
Live-verified: fit zoom derives from the ledger bbox, pickAt aligns with
ledger positions at every probe, hover/click/detail-card work, 408/408
tests, typecheck/lint/build green.
Future live physics stays cheap by design: the same pure function is the
merge-time settler and the future client simulator; a pinned node is just
an anchor with full retention.
- LABEL_GAP 6 -> 2 (user: labels sit way too far below their nodes).
- Hover glow is now PROPORTIONAL to the disc — outer radius = node radius
x 1.7 (GLOW_SCALE), matching the blur-a-copy-of-the-disc mental model,
instead of the old constant +6px pad that read huge on small nodes and
thin on big ones. GLOW_ALPHA 0.45 -> 0.3.
- Hover-OUT fade was broken by construction: draw() gated the glow on
hoverId, which nulls instantly on leave, so the 180ms fade-out ran
invisibly and the glow popped off in one frame. New hoverNode field
outlives hoverId through the outgoing fade (cleared when the fade
completes); verified live: 8 distinct frames over ~200ms then stable.
- Sliders rescaled per spec: ALL five are now log-scaled 0-100 with the
default dead-center at 50 (ranges chosen as default/4..default*4:
node/arrow/label size 0.5-8, edge width 0.25-4, labelZoom 0.0625-4),
coarse step 5 (21 stops; labels-until-zoom was "way too granular" at
201). Generic toSliderPosition/fromSliderPosition replace the
labelZoom-only pair; sanitize clamps follow the same range constants;
size sliders display compact values ("2", "1.6", "0.75").
EOF
- Defaults per user: nodeSize 2, edgeWidth 1, arrowLength 2, labelSize 2
(labelZoom 0.5 stays — reads as the "1x" they confirmed good).
- Label size slider floor 3 -> 1 so the default 2 is not the smallest
value; sanitize clamp widened to match.
- Cluster hull glow (layered low-alpha fills) removed for now: it read as
jagged up close (straight-line hull approximation at high zoom) and
does not belong until the layout settles. The solid viewer-relative
border and the inverse-gated region labels stay. HULL_FILL_LAYERS
constant deleted; hullShapes cache carries only the stroke path now.
- Saved render-settings.json reset so the new defaults apply on next load.
EOF
The defaults were calibrated in the screen-fixed era; world-scaling renders
sizes at setting x k/kFit, so at the default label gate (2x fit) everything
read ~2x too big ("labels way too large even at the minimum"). Fit-zoom
normalizes layout scale away, so the fix belongs in sizes, not layout:
spreading nodes further apart in world units changes nothing at fit.
- Defaults: nodeSize 6 -> 4, arrowLength 5 -> 4, labelSize 12 -> 7.
- Clamps/slider ranges widened at the low end: nodeSize 1-14 (step 0.5),
edgeWidth 0.25-4 (step 0.25), arrowLength 1-15 (step 0.5),
labelSize 3-24 (step 0.5).
- GLOW_PAD 9 -> 6 (halo dominated small discs), REGION_LABEL_SIZE 18 -> 14.
- Tests updated for the new clamp bounds plus a low-end pass-through case
(403/403). The saved render-settings.json still shadows defaults by
design; the local copy was reset so the new defaults showed immediately.
Fixes the zoom frame-rate collapse and rebuilds draw() around a single
camera transform, per the user's Obsidian-style sizing proposal.
Bug fix first: the eased zoom leaked a queued rAF callback per frame --
advanceZoom applies each step through d3-zoom, whose zoom event called
markDirty() from inside the tick loop, scheduling an extra draw while the
loop scheduled its own. Draws multiplied per displayed frame; sustained
wheel ran at ~20fps against a 120Hz display. markDirty now skips
scheduling while inTick (verified: 62 frames/500ms during a 28x zoom
burst, matching pan).
World-space model: every draw call now runs in graph coordinates under
one setTransform(k). Sizes are defined as screen-px-at-fit and converted
once per geometry build (/ kFit), so nodes, node labels, arrowheads and
the hover halo scale with the graph as you zoom (0.25x painted area at
half zoom, exactly k^2) while edge widths and hull borders stay
viewer-relative hairlines (/ k). Cluster hull geometry (convex hull +
three offset fills + solid boundary -- dashes dropped for now) is built
as cached Path2Ds per data/visibility/settings change; camera moves do
zero geometry math, making zoom per-frame cost identical to pan. Label
wrapping runs once per build, not per frame. Region labels stay
viewer-relative (18px at any zoom) as UI chrome. Picking radius scales
with the world-sized discs.
Glide feel: ZOOM_EASE_MS 120 -> 45, convergence 0.1% -> 0.2%; a notch now
settles in ~260ms instead of ~750ms. Resize still paints synchronously.
Also fixes barrel-law lint violations from the settings wave (import
from @/src/core and @/src/server, not deep module paths).
Five refinements to the hand-rolled renderer, all user-directed:
1. Settings are SERVER-persisted (data/render-settings.json via new
GET/PUT /api/graph/render-settings) - one source of truth for every
browser, replacing localStorage. The client loads once after mount,
saves debounced on user changes, and never lets a slow GET clobber a
faster user (or mounting PUT defaults over the server copy). The
shape/defaults/sanitizer moved to src/core/render-settings.ts - pure,
shared by client, server store, and tests; every disk/network value
passes through sanitizeRenderSettings.
2. New "Label size" control (default 12px - the previous hardcoded size).
Font, wrap width, and line height all scale; verified live: +76k px
with the label gate open.
3. "Labels until zoom" slider reworked: log-scaled over 0.0625-4 so the
useful band (0.25-0.5 camera-ratio units) sits at 33%/50% of the
track instead of crammed at the far left of the old linear 0.25-4
slider. Display units are normalized: the default 0.5 reads as "1x",
0.25 as "0.5x". The underlying labelZoom semantics are unchanged,
so existing values survive.
4. Smooth wheel zoom: d3-zoom instant per-notch jumps (x1.2, bang) are
replaced by an eased glide - wheel events accumulate a cursor-anchored
target scale and each tick eases the camera toward it (exponential,
120ms constant, frame-rate independent). Pan/pinch stay d3-owned; a
pan cancels a glide in progress. Verified live: 8-frame glide.
5. Resize blanking fixed (graph vanished for the duration of a window
drag, reappearing ~300ms after). Cause: every resize event
reallocated the canvas backing store - which CLEARS it - and the
redraw was deferred to rAF, which live resizes starve via relayout,
so the canvas sat wiped for the whole drag. Fix: the ResizeObserver
callback now redraws SYNCHRONOUSLY after reallocating (verified via
a probe RO firing in the same delivery batch: canvas fully painted).
kFit (the label gate 1x reference) is also recomputed on resize from
the now-cached world bbox - the camera itself still persists.
tests/render-settings.test.ts: 10 tests pinning the sanitizer (garbage/
partial/clamps, legacy no-labelSize records) and the slider mapping
(default dead-center at 500, monotonic, exact round-trips, display
units). 402/402 green; typecheck, lint, build pass.
Implements Option B from docs/research/graph-rendering.md. sigma.js v3
(WebGL) is replaced by one canvas, one rAF loop with a dirty flag, and
d3-zoom for the camera — eliminating the entire bug class from the
sigma-era bolt-on architecture:
- Hull/node desync during pan (two independent render loops) is now
structurally impossible: hull fills, dashed hull strokes, edges with
arrowheads, node discs, region labels, node labels, and hover glow
are all drawn in ONE frame under ONE camera.
- The settings-change canvas-sizing race (everything shifting up-left)
is gone: settings changes only redraw; the canvas and camera persist
(verified live: camera identical to 15 decimals across a slider drag).
- The GPU floor is gone: idle schedules NO frames at all (verified via
pixel-identity over 600ms idle); pan redraws exactly once per frame.
components/graph-canvas.ts: GraphCanvasRenderer class plus pure helpers
(zoomRatioOf, labelAlpha, fitTransform, arrowGeometry, pickNearest).
The sigma camera-ratio convention (1 = fit, smaller = zoomed in) is kept
verbatim so persisted labelZoom settings survive the swap; node labels
and cluster region labels cross-fade over the same LABELS_FADE_SPAN
(0.5, unchanged). Picking is a nearest-node distance check with the
10px pad that replaces the old padded-picking vertex shader. Hull math
stays in components/cluster-geometry.ts, reused as-is. No ctx.filter,
ever.
graph-view.tsx keeps every feature: fetch/refetch on runs-empty, FA2
run-then-freeze layout, localStorage render settings + sanitizers,
search filtering with edge drops and card-close, detail card with
flipped anchoring and per-source provenance, settings panel, import
button, pipeline status chips. Only the renderer plumbing changed.
tests/graph-canvas.test.ts pins the pure math: ratio direction, label
gate + inverse complement (alphas sum to 1), fit transform padding,
arrowhead tip-on-disc-edge and symmetry, picking pad/ties/null cases.
Live-verified end-to-end on the real 338-node graph (dev server
port 3002): single canvas, paint at fit, hover cursor+glow+fade, click
card open/anchored/close, stage click, search filter to zero with hint,
label gate opens via slider and zoom, cluster-labels toggle, zoom-to-
cursor 0.56px anchor drift, camera persistence across settings, zero
idle repaints. sigma dependency removed from package.json; d3-zoom and
d3-selection added.
Note: the settings feed effect needed isEmpty in its deps — without it
the renderer mounted with settings=null and every draw early-returned
(blank canvas); caught live, not by typecheck.
amosblomqvist