ARCADIA-Studio/harness/release-analysis-codec-implementation-20260911.md

Core JSON codec and Expr lifecycle candidate — 2026-09-11

Codex-956atf /root/analysis_core_resume. Isolated .local/analysis-core-worktree-20260911, branch codex/analysis-core-20260911, HEAD cdebb49d89. Source is frozen and uncompiled. No codec runtime, memory, Windows release or Linux acceptance is claimed. Root owns compilation, integration, dependency fetching/locking and Git. Linux execution/checking/cache work is explicitly deferred by the user (357); do not resume automatically.

The accepted cdebb49 static correction remains distinct: root's Windows depth128/129 and call32/33 test, analysis28/28, numeric26/26 and strict core all-target Clippy passed; independent review342 found no material new defect. Existing core WIP handoff and root iterative gate/review reports preserve the original STATUS_STACK_OVERFLOW failure at305 and immutable corrected274-input manifest/binary. This candidate changes lifecycle/persistence after that accepted source and needs fresh gates.

Lease 348. Design follows root harness/release-analysis-decoder-review-20260911.md / 257. Root maintains AnalysisCoreSchema820260911 Wiki; this worker does not concurrently rewrite it.

Result and boundary

Persistence scans original bytes before Serde allocation, probes schema_version without a current-schema header Value, then decodes the same bytes with the closed typed schema. No Value-to-typed normalization or parallel accepting JSON grammar. The scanner handles escapes solely to meter spans/container boundaries; Serde still validates JSON, decoded names, UUIDs, Exact values, unknown fields/variants, duplicates and trailing bytes. Schema1–7 migrations retain frozen readers. Pre6 presence checks use a bounded legacy Value, discard it iteratively before typed construction, and inspect it iteratively in original traversal order. Legacy Serde recursion rejection remains enabled.

Expr keeps its public boxed enum shape. Clone, equality, Debug and destruction use heap traversal; ordinary and partial rejection teardown do not recurse with original expression depth. Clone fills child slots from copied metadata; equality preserves variant metadata, list order and parameter keys. Debug streams a compact structural representation without invoking child Expr Debug recursively. Destruction detaches children; reentry through emptied nodes/sentinel leaves has constant nesting. The manual-depth test constructs100000 nodes. This is not a promise that arbitrary in-memory values cannot exhaust heap memory.

A private Serde remote derive preserves the original tagged Expr grammar without an unguarded public inherent decoder. Each Serialize/Deserialize edge enters a depth/node guard before the visitor and a stacker boundary. List seeds refuse item1025 before deserializing it. Call keys are checked for duplicate decoded ParameterId and width before their values are constructed. A scoped persistence meter aggregates disjoint bindings and restores state on failure/unwind; standalone Expr serde has its own root meter. Static admission, contextual empty-list checking and nominal/exact algebra are unchanged.

Native file reads use fixed chunks and check before allocation extension, including growth after metadata inspection. JSON save uses a capped geometrically growing writer and compact JSON to avoid indentation expansion. The newline is inside the cap. Finished bytes are parsed with the same profile and the reopened Project is compared before creating a sibling temporary file. Cap/reader refusal preserves destination bytes. External-content comparison streams fixed chunks instead of reading an unbounded second file. The existing documented non-CAS filesystem race remains.

DEPTH_128_VALIDATED is false. Production retains serde_json's default guard and now refuses to save JSON outside that reader profile. Prepared tests pass true through the same private codec/decode/save implementations. Do not flip the production switch from formatting success, a depth-only pass or the old static gate. Root must assess complete local lifecycle evidence and the explicitly deferred Linux obligation. Logical depth128 remains the acceptance target.

Explicit profile and allocation accounting

This private profile is separate from ParseLimits, the1,000,000 static-event meter and numeric work. Caller byte limits lower the64MiB document ceiling.

Resource	Document ceiling	Qualification
Original / saved JSON including newline	64MiB	Matches existing finite CLI/source-project envelope; replaces native unlimited read allocation.
Live raw containers	512	Fixed scanner storage; deep Calls use about three containers per logical Expr plus a finite envelope. Gate emits actual counts.
Lexical tokens including keys/scalars/punctuation	16,777,216	All original input, including rejected unknown/duplicate payloads, counted before typed allocation.
Aggregate container entries	8,388,608	Object-member/array-entry cardinality admission.
Entries in one ordinary container	500,000	Preserves declared500000 ordered-reference ceiling; no65536 cap on ordinary top-level maps.
Raw string span / aggregate raw strings	64MiB / 64MiB	No new1MiB metadata contraction. Valid encoded spans conservatively bound decoded UTF-8; caller byte cap applies first.
One stored Expr tree	65,536 nodes	Next Expr refuses before construction; dependency-closure admission remains separate.
All Expr nodes per codec operation	262,144	Disjoint definition/base/case/check bindings aggregate; CLI250000-record budget still applies independently.
Expr depth / List / Call arguments	128 / 1024 / 65,536	Ratified analysis bounds checked before child visitor/seed/value.

These are finite allocation cardinalities/text spans, not a measured allocator-byte or RSS ceiling. The scanner has fixed storage/no heap allocation. Serde's probe uses bounded scratch for container markers/escaped keys without materializing ignored subtrees. Schema fixes Rust slot types, while collection capacity, allocator overhead and scratch affect actual memory. The output writer makes checked finite geometric reserves before adding bytes. Root must measure peak memory rather than describe the table as an exact process-memory guarantee.

Compatibility reasoning uses actual CLI100000 graph objects,250000 records,500000 ordered references/64MiB and server5000/20000/50000/8MiB. Prepared tests encode/reopen100000 ordinary typed graph records, scan500000 UUID references and preserve metadata above1MiB. They check codec/count compatibility, not full large-model validation/UI speed. General ceilings remain provisional document policy, not a theorem that every previously unlimited native shape fits. Initial1MiB string/65536 ordinary-map proposals were discarded.

Direct dependency: stacker0.1.25; oa-core enables serde_json's unbounded_depth API feature, but only the private candidate switch disables its guard. Root fetched/locked successfully without compilation; log .local/logs/analysis-codec-fetch-20260911.log. Cargo.lock adds42 lines only: stacker0.1.25, psm0.1.32, ar_archive_writer0.5.3, object0.39.1 and oa-core dependency edge; no existing versions changed.

Expr instrumentation requests64KiB reserve /512KiB segments; fixed outer document/validation requests512KiB reserve /2MiB segments. At most128 admitted Expr guards plus the outer boundary are live on this path. Capacity products are not proved allocation/RSS ceilings: overhead, base stack, allocator and outer work matter. Persisted Project/model/semantic-fact schemas inspected have no other recursively self-containing Rust field; vectors/maps contain fixed record/reference types. Ordinary Command/Project Expr operations receive the new traits, but generic caller closure is not implied.

Primary basis: stacker maybe_grow, platform support, serde_json lifecycle warning, Serde adapter warning. Installed stacker/serde_json/serde_derive sources were read. Boundary instrumentation was chosen because a persistence-only serde_stacker wrapper does not protect generic callers' direct Expr serialization. The generic Value caveat below remains.

Frozen paths and hashes

Only these source/dependency paths and this report changed. The lock is root-owned, included to identify exact input; worker did not generate it.

Path	SHA256
Cargo.lock (root)	19499D93DDE6F2E2A00AB4B7D5F37689DD7A7D492B2B5E23063A761607839FC5
crates/oa-core/Cargo.toml	3377D3D9D43FE0023AC09728B0D20205F86A6755F15518DC0CD035780B47A95D
crates/oa-core/src/analysis.rs	845C3629098A13F3FFFAAC3E37D6A3CF8B7BDF33A8110465459990EB3FE70B11
crates/oa-core/src/analysis/types.rs	27363B1755F3C0943685A9837C69E010857811D4F67A0751B20EFDD9C1707AA3
crates/oa-core/src/analysis/expression_lifecycle.rs	5D66E4E8F33FC8F90EC364251079617A36226E3EA68AC321ABAE74096E1C7C8C
crates/oa-core/src/analysis/expression_serde.rs	581E4704FB4F2049E31C7731267BC9D52E3F17625B664C3D7C5F19CAA961C819
crates/oa-core/src/persistence.rs	EDA5E697ADB29A8BF38B0D367099BD23267455E4792F97607E13DC0B232A3476
crates/oa-core/src/persistence/json_codec.rs	2AA2C52EEA794ACB85744440192DF9EF5193160AD6EBD1FEBAA824E144AADD90
crates/oa-core/src/persistence/json_codec/tests.rs	7065CC9E177A5699432888110936529C621F13E7C4B164566AB1FD14B61D6B9C

Scoped installed rustfmt2024 parsing/formatting and git diff --check pass. Read-only textual check found no unmatched braces in authored raw format strings. These do not establish type/borrow/Clippy success. No worker Cargo/rustc/compiler/native/WSL/Git mutation.

Root gates prepared, not run

Run from isolated worktree with root's existing Windows toolchain/cache:

1. cargo test --locked -p oa-core --lib persistence::json_codec::tests::codec_lifecycle_subprocess -- --exact --test-threads=1 --nocapture
2. cargo test --locked -p oa-core --lib persistence::json_codec::tests -- --test-threads=1 --nocapture — 6 tests, including the subprocess driver.
3. cargo test --locked -p oa-core --lib analysis:: -- --test-threads=1 — 28 tests:25 schema8 plus3 topology.
4. cargo test --locked -p oa-core --lib analysis_numeric::tests -- --test-threads=1 — 26 tests.
5. cargo clippy --locked -p oa-core --all-targets -- -D warnings.
6. Cover modified legacy/save boundary with existing JSON/migration tests: language's malformed_json_never_ignores_duplicate_maps_or_nested_unknown_fields, review_regressions' checked_save_ filter, and frozen migration suites. Integration suites also contain source-format/parser obligations; distinguish prior source8 gaps from new regressions.
7. Repeat codec driver/filter with Windows --release after debug success; record child-phase peak memory/time and exact source identities. Linux debug/release remains explicitly deferred.

The exact driver invokes itself with a child-only phase environment variable. Each of five phases runs in another process on a 128KiB thread, smaller than ordinary test stack; no RUST_MIN_STACK override. A stack overflow is a failed child identified by phase. Each child has a120s deadline.

• accepted: actual Project version dispatch/structural/static validation at unary128 and Call128; ordinary Project Clone/Debug/equality/drop, compact output/reopen, real sibling-temp save/read.
• refused: logical129, List1025 with deliberately malformed rejected child, exact129/128 aggregate count, raw token cap, exact/lowered static events, stored-tree65536/65537 typed boundary, recovery after refusal.
• malformed: late duplicate/escaped operand, unknown field, second enum variant, duplicate decoded parameter key, late Project field, trailing root, lone surrogate, raw100000-container refusal; original-byte behavior/partial typed teardown.
• ownership: programmatic100000-node invalid-depth Expr Clone/equality/streamed Debug, safe serialization refusal and ordinary drop.
• atomic: exact bytes/newline, cap-minus-one preserves destination/no temp artifact, conflict refusal, finite read and no writer growth on refusal.

Other tests cover exact lowerable scanner resources, escapes/odd backslashes/encoded brackets and syntax rejection, all13 original variants, declared large-container compatibility, and production save/reader-profile symmetry. Gate output emits actual accepted bytes/tokens/entries/raw strings/container depth; these are not memory measurements.

Remaining obligations

• No compilation yet. Borrow/exhaustiveness/remote-derive/platform behavior and all prepared assertions need real root gates. Production deeper decoding stays disabled pending that decision.
• Profile is private to persistence. Server commands/requests/responses, CLI JSON ingress, hosted envelopes, error/output caps and full channel limits remain integration obligations.
• Source8 grammar/format/round-trip/conversion is not implemented. language.rs stays with9e8r6t's outline seam. The .arcadia formatting path remains separate; its allocation/recursion closure is not claimed.
• Generic to_value callers can still materialize a deep serde_json::Value and then invoke Value's own recursive Clone/Debug/equality/drop. Protecting Expr Serialize does not change that foreign lifecycle. Semantic catalog/source/response Value adapters require integration audit; persistence8 does not use that carrier.
• Fixed outer stack reasoning/requested segments are not runtime acceptance. Windows debug/release/memory remain required. Linux is deferred by user without automatic retry.
• Evaluator, quantities/nominal algebra, exact/numeric limits, static1m accounting, baseline/case/unused branches, scoped parameter identity and fixed KindHandle are unchanged. Full schema8 transport/source/product/release acceptance remains open.