Files
ARCADIA-Studio/harness/release-text-service-audit.md

14 KiB
Raw Permalink Blame History

First-release text and service audit

Date: 2026-09-05. Scope: the new first-release objective in the goal attachment 81e84f53-2b51-4151-9e34-59414bda8b2d/goal-objective.md, current text editor/language, graphical field adapter, REST/storage and deployment. Read-only source audit; no native UI operation. No source files changed. This report assesses release suitability and does not inherit the V2 prototype exclusions as release acceptance decisions.

Material findings

TS1 — High: rejected oversized source monopolizes every hosted model

Observed on the actual Windows release server: a 638,689-byte source request containing 12,000 nested OA activities returned 422 model_limit after 26.163 seconds. A concurrent GET /api/v1/projects, issued 0.5 seconds later, took 25.655 seconds to return 200. The documented 5,000-object cap is applied only after this expensive work. The request is far below the 8 MiB body cap.

Evidence: crates/oa-server/src/routes.rs:271 imports source inside blocking; routes.rs:188 calls the unrestricted language::parse; crates/oa-core/src/language.rs:1026 performs full structural validation; only afterward does StoreInner::create call validate_model / check_model_size (store.rs:166, store.rs:280). crates/oa-server/src/lib.rs:203 holds the one store mutex throughout parsing and validation. Source replacement/candidate validation have the same ordering. JSON import calls persistence::decode, which also validates before the server size cap. Parent walks and several validation scans are quadratic. A longer request can exceed the 30-second response deadline while retaining its blocking worker and mutex; this latter consequence is source-derived, not separately timed.

Reproduction: launch the release binary with an isolated data directory and --bind 127.0.0.1:18787; POST text/plain to /api/v1/projects/import using:

n = 12000
source = 'arcadia 2; project oversize "Oversize" { oa_activity a0 "Root" {}\n'
source += ''.join(f'oa_activity a{i} "Activity {i}" {{ parent @a{i-1}; }}\n' for i in range(1, n))
source += '}'

Concurrently query the catalog. The isolated audit process (PID 27728) was stopped after its requests completed. Its logs and probe are under D:/CodexBuilds/Capella2-r1-text-service-audit; stderr contains only the expected startup line.

Fix before release: enforce declaration/record/reference budgets before expensive full validation, including source and legacy import; move request parsing outside the store lock, keeping snapshot/revision capture and final conditional commit atomic. Validate size at the shared parsing boundary or introduce an explicit bounded parse option rather than duplicating lexical heuristics. Add a real HTTP regression proving a rejected large candidate does not stall an unrelated read. A mutex refactor alone does not bound CPU cost; an early cap alone still leaves accepted large model validation holding the catalog lock.

TS2 — Medium: source lint executes expensive work on the GUI thread

crates/oa-desktop/src/app.rs:936 schedules a 350 ms sleep, but the actual language::lint runs synchronously in Message::Lint at line 945. Source undo/redo also lints synchronously. The version check prevents stale results, but does not make analysis asynchronous. Every source edit additionally clones full before/after buffers and retains up to 100 full-document versions (app.rs:923). Highlighting rebuilds tokens for the entire source on update (highlight.rs:19, highlight.rs:61).

A release-linked core probe measured:

Model Canonical/source bytes Full parse Canonical lint
201 graph objects, flat allocation 27,990 2 ms 2 ms
2,001 graph objects, flat allocation 281,790 60 ms 68 ms
4,999 graph objects, flat allocation 710,504 358 ms 303 ms
1,000 nested activities 49,687 88 ms not timed separately
5,000 nested activities 261,686 2,605 ms not timed separately

These are exact core timings, not claims that a GUI freeze was personally observed. The synchronous update call means equivalent analysis blocks native event handling. For a release aimed at working systems engineers, even an otherwise valid server-sized model introduces hundreds of milliseconds per pause in typing.

Fix: execute snapshot/versioned lint on a blocking worker, discard obsolete results, and keep source edit response independent from analysis completion. Bound/coalesce pending analysis; do not spawn an unbounded number of obsolete full parses. Cache model validation by committed revision where reused. Add a realistic model-size interaction check; do not silently reduce supported semantics to obtain a passing performance test.

TS3 — Medium: textual lint omits engineering warnings and offers no diagnostic navigation

crates/oa-core/src/language.rs:139 is parse(source).err().unwrap_or_default(). A valid draft arcadia 2; project demo "Needs" { sa_function assess "Assess" {} } produces zero lint diagnostics, while validation::validate returns both missing_system and unallocated_leaf. The parser intentionally returns only the first structural error; warnings are discarded. As a result, the source workspace does not provide the same immediate engineering feedback as Model checks, and warnings only become visible after applying the draft and changing pages.

crates/oa-desktop/src/views.rs:853 renders the first eight diagnostics as plain text. There are no diagnostic click actions, source search, go-to-line input or line-number gutter in the text workspace. app.rs:977 implements graphical selection → source declaration, but no general error navigation. An error at line 4,000 is located numerically without a practical way to reach it, undermining the explicit human/AI-editable modeling goal.

Fix: expose located syntax/semantic diagnostics, including warnings with severity and codes, and navigate/select the relevant source range on click. Add Ctrl+F / next/previous match or go-to-line; display cursor line/column. A line gutter is useful if it stays aligned during wrapping/scrolling; it is not a substitute for direct diagnostic navigation. Test draft warnings without applying, non-ASCII columns, stale analysis results and navigation after edits.

Product decisions requiring explicit release disposition

  • Source comments are erased on Apply and any save after open. Reproduced language::format(language::parse(source)) removing an explanatory comment. This is honest in views.rs:901 and README and does not currently violate the normative V2 contract. For first-release Git/AI use, preserving comment trivia or providing a deliberate canonicalize command would avoid surprising loss of engineering notes. At minimum, the guide should teach durable model description fields and make the actual loss behavior clear before a first destructive normalization. Do not describe whitespace/comment preservation as implemented.
  • No crash recovery/autosave exists. This is a declared V2 boundary; it is now worth a release decision for long engineering sessions. Ordinary save uses atomic replacement and external-byte conflict detection, but it does not recover unsaved source/property/model drafts after process failure. This audit did not simulate a crash.
  • Hosted models are accessed by explicit import/export, not a desktop connection. The objective explicitly requires a programmatic backend, so absence of direct connection is not automatically a violation. If the release promises editing shared stored models from the desktop, a connection/revision/conflict workflow must be implemented and verified; current shell transfer alone is insufficient for that stronger promise.
  • Deployment evidence is incomplete for the provided Docker builder. Dockerfile pins Rust 1.90; current verification records Rust 1.98.1 and no Docker execution. Native standalone Rust deployment is real; remote deployment is not required merely because a deployable service was requested. Before calling the provided container workflow verified, build it or independently compile the exact declared toolchain/dependencies and clearly retain the unrun container limitation.

Strong existing evidence and boundaries

  • Language tests exercise strict fields/enums/references/duplicates, all relation kinds, stable aliases/identities, ordered repeated occurrences, explicit V1 migration, UTF-8 diagnostics, exact permitted strings/float values, and full canonical round trips. complete_model is broad but every_demo_record_has_a_lossless_graphical_draft verifies only records in the demo; it is not an exhaustive proof that every closed enum/optional combination is easily authorable.
  • Graphical forms maintain temporary typed values, reject malformed edits atomically, expose optional/reference/ordered values and preserve current references even if temporarily invalid. Tests cover escaped strings, reference choices, identity remapping, draft preservation and modal command gates. This source audit found no new concrete field-loss counterexample.
  • crates/oa-server/tests/http.rs uses actual loopback listeners and filesystem stores, covering ETags, competing writes, stale deletes, no-ops, dry runs, exact source/JSON round-trip, revision-qualified pagination, bounded graph closure, authentication/browser/Host/media/body limits, restart/corrupt store/lock checks, forced pre-install storage failure, transitions/artifacts and store capacity. The audit inspected coverage and ran the additional release HTTP responsiveness probe; it did not rerun the entire suite.
  • Storage holds an OS directory lock, persists before successful return, keeps in-memory state consistent after post-install directory sync failures, and rejects corrupt startup envelopes. Timeout/503 uncertain-write behavior and lack of idempotency/history are documented accurately. No new data-corruption or authentication bypass was established in this bounded audit.
  • Request body caps and command limits exist, but do not currently solve TS1. Tests named capacity/bounds mostly check final rejection/status, not prompt rejection or concurrent service responsiveness.

Suggested release acceptance additions

  1. Under-limit and over-limit source/JSON candidates are bounded before expensive validation; an unrelated model/catalog read remains responsive during rejected import.
  2. A 5,000-object representative model remains editable while analysis runs; obsolete diagnostics cannot overwrite newer draft state.
  3. A text-only author sees engineering warnings and can jump directly to the affected declaration; search/navigation works on Unicode source and large documents.
  4. The release decision on comments, recovery, hosted editing and tested deployment platforms is reflected consistently in the guide, normative contract, README and acceptance record.

TS1 / TS3 implementation and re-verification — 2026-09-05

The previously observed TS1 catalog stall is fixed at both boundaries. Shared ParseLimits checks source declarations and decoded JSON record counts before structural validation; V1 checks include generated placements and process occurrences before migration. Source/model replacement and candidate checks use the same budgets. Store access now captures an owned snapshot under the mutex, prepares/validates outside it, then rechecks the exact ETag while atomically installing a privately constructed PreparedModel. Catalog reads copy only summaries. Capacity accounting, directory ownership and install-failure behavior remain under the commit lock.

TS3 core diagnostics now carry severity, stable engineering code, object/field identity and exact source location. Valid source drafts report engineering warnings without becoming unparseable; syntax and structural errors remain blocking. Unicode scalar columns are preserved. Semantic diagnostics index source line starts once, avoiding a complete prefix scan per warning. Native background analysis and diagnostic navigation are separately owned desktop verification gates, not claimed by this worker.

Observed current R2 verification: all 67 core tests, 14 HTTP tests and 1 deterministic prepared-candidate commit test pass on Windows. Strict core/server all-target Clippy and the optimized server build pass. The full language fixture covers every element/relation kind, new OA mission and SA mission links, conditional extension with quoted guards, and typed mission trace. OpenAPI exposes the same new vocabulary. These counts describe the R2 checkpoint; subsequent R1 topology integration has its own verification gate.

New actual HTTP cases exercise over-budget source, native JSON and legacy JSON imports, source/model replacement and non-mutating candidate checks. Every rejected candidate preserves the existing revision/model. A within-budget 2,000-deep candidate is prepared concurrently with catalog reads and a winning rename; the stale candidate returns 412 and cannot overwrite either memory or the installed envelope. A deterministic store unit test separately ensures two fully prepared candidates from one original head cannot both commit.

Optimized executable probe against an isolated Windows store, using the exact original source shape:

Candidate Bytes Candidate response Concurrent catalog
12,000 nested activities, source 638,689 422 model_limit, 0.050 s 0.016 s
12,000 nested activities, native JSON 2,737,965 422 model_limit, 0.142 s 0.0023 s
12,000 activities, legacy JSON 2,341,015 422 model_limit, 0.127 s 0.0013 s
2,000 nested activities, accepted-size source racing a rename 102,698 412 revision_conflict, 0.883 s 0.0014 s

The source rejection completed before the catalog read began 50 ms later; both JSON requests were confirmed still in flight during their catalog read. The accepted-size source was confirmed in flight at 100 ms; the competing rename committed in 0.024 s and remained revision 2 on disk and in memory. Measurements are one native run, not throughput guarantees. Evidence JSON and stderr are in D:/CodexBuilds/Capella2-r1-text-service-audit/fixed-20260905-095047. The probe process was terminated after all requests completed; no server is left running. Stderr contains only the expected startup line. Graceful restart and draining behavior remain covered by the transport suite and prior native platform verification, not this timing probe.

Remaining first-release work is not waived: comment/recovery/hosted editing decisions, exact deployment toolchain verification, native large-draft interaction, the R1–R6 engineering semantics and final independent review.