Independent retained-text producer critique

Reviewed proposal and current 8d52dab97c92cc31baae9f20d8c324b55818aa5d producers, prior NEXT-PRODUCER-HOOKS/CRITIQUE, existing syntax text walk and carried-phase contract. Design-only; no source edits, compiler, network or parser execution.

Disposition: the explicit stronger mode, retained-text scope, named scanner result temporary and raw-payload bound form a useful bounded prerequisite. Do not expand it to U/M producers, duplicate-name sets or traversal. Before coding, make the successful-discard inventory and transfer semantics below concrete. It is not accepted-output verification or completion of broader before-growth requirements.

Required lifecycle clarification

The proposal explicitly identifies ordinary optional_id and typed identity, but there is a third successful UUID spelling consumer: language.rs::Parser::value_inner's uuid branch. It calls quoted(), parses an Id, returns Ast::HistoricalUuid and drops the decoded String. Hooking quoted without refunding this path leaks36 into final T and breaks independent reconciliation. Name this producer/consumer pair explicitly, including historical UUIDs nested in arrays/records. No projected text reservation applies to the copied Id.

For each charged shared producer, distinguish transfer, discard and duplication. Direct names/quoted results moved into retained AST/header/reference structures retain the charge. Optional/typed/historical UUID spellings remain charged until after their actual drop. Nested ordinary fields clone the key into a map while the local key remains alive; reserve the clone before creation, then refund the local original after drop on successful completion. Top-level field keys instead survive in two maps, so both remain charged. Projected keys/references create further live copies. Error-only abort paths need no reusable successful ledger, but scanner validation must release correctly before another token can be processed.

Avoid a generic rule that refunds every field-local name, counts every clone as retained forever, or replaces later mismatch with a corrective subtraction. Producer T must independently equal the established final inventory. Duplicate-field errors abort before success; they are not justification for ignoring ordinary successful temporary copies.

The ordinary declaration-kind String and existing typed parameter/call-label/override duplicate-set copies are not part of retained T. The proposal explicitly excludes coordination copies. Keep that limitation precise; shared name/quoted hooks do not automatically account downstream .clone() calls. This slice must not claim all parser String allocation has been covered.

Reservation and handoff shape

Scalar state on the shared Budget is a reasonable design because scanner, typed cursor and ordinary State already borrow it. Do not copy C authority or full policy into recursive frames. Initialize the scalar carried text only through the consumed legitimate guard and exact inherited profile; no independent C setter or public arbitrary budget entry.

Define whether an unclassified returned String counts provisionally as owned T until its consumer drops it, or is W until an explicit transfer. Either can implement correct live totals, but documentation/tests must agree: a method called retained-text observation must not count discarded UUIDs at successful syntax return. W slack must be zero and independently walked retained text must equal the final producer total. Moves are accounting-neutral; copies coexist; release follows actual ownership end.

Resolver should seed C once and transfer T once, then use the existing bounded syntax walk to independently sum/compare text while retaining its lower-node/occurrence computation and accounted work frontier. That walk must not reserve the same T again. Its legacy and carried-phase-only text actions stay unchanged. An inconsistent producer tally is a failure, never repaired into acceptance. No extra generic AST clone, independent traversal container or second decoder is needed.

Scanner and raw-payload policy

D<=P is sound for valid JSON UTF-8 string payloads and the selected escape/surrogate cases. Reserve P before the current decoder; reconcile successful output D, then release remaining live reservation after dropping validation/identity strings. This is a logical result-length bound, not serde scratch/capacity metering. Conservative rejection of escaped text whose D fits but P does not is a real new-mode policy and should be accepted explicitly.

Scanner validation happens before grammar consumes any of that file's strings. The scanner hook must run after finding the token extent but before its existing serde call. A closed invalid string can therefore receive a resource error before invalid-escape classification in stronger mode; an unclosed token never reaches that decoder. Preserve the unchanged Iterator/no-op scanner path and ordinary diagnostic precedence for legacy and phase-only modes. Do not accidentally latch stronger-mode failure into an unrelated reusable legacy scanner API.

Each successful scanner validation temporary ends before grammar decoding, so P must not accumulate across tokens. However retained T from earlier files remains live when scanning later files. Include a multi-file case that discriminates this overlap; releasing scanner W must not release previous-file T or C.

Independent finite oracles

These counts are source-derived, not obtained from the proposed implementation.

Existing empty E: exact existing78-byte main.arcadia source; actual owned C=(1,0,102), final T8. Scanner maximum validation P36 gives text138. Grammar retains alias1/name7 before decoding UUID36, producing maximum 146. Drop UUID and final C+T=110; resolver catalog maximum112. Therefore entries17 and bytes146 complete the whole path; bytes145 must refuse at ordinary project UUID decode, provided the selected hooks cover only the stated domain. Legacy phase-only complete cap112 and its original diagnostics must remain unchanged. The UUID token spans33..71; ordinary parser transport turns a start33 resource error into33..34, not the complete token span.
Escaped project fixture: use the E document constructor, replace display name Project with twenty literal source escapes \u0061, and set body to description " + twenty literal \u0061 escapes + ";. Each decoded value is20 ASCII a bytes, each raw payload120. Exact source bytes326; copied key12+entry12; C.text350, C.entries1. T is alias1+name20+description20=41. Scanner peakP120; grammar name peak1+120=121; UUID peak1+20+36=57; description peak1+20+120=141. Thus combined text peak 491, cap490 refuses at the description quoted token after earlier phases fit. Syntax has U2 and M14 (13 significant tokens+F1), so owned syntax entries17 and project catalog peak21. With entries21, bytes491 also admits complete resolution: retained text391, catalog peak393, no declarations/lower expressions or warnings. Expected Project has name and description each20 a characters. This proves slack refunds and scanner nonaccumulation without letting UUID36 preempt the intended escape bound. Ordinary description quoted token begins200 and ends322; normal ordinary-error transport at start200 is200..201.
T45 copy oracle caution: the existing activity description "é" final text45 remains correct: project8 + activity alias/name2 + field_spans key11 + AST key11 + literal2 + projected key11. But new-mode C+44 fails earlier at activity UUID, whose live subtotal is8+2+36=46. It cannot be reused as a projected-key producer refusal with the old field span. Keep the old phase-only test unchanged; use roomy new-mode T45 reconciliation, or design a later-copy-dominant fixture with independently counted earlier peaks before asserting that refusal site.
Add one historical UUID and one nested record to discriminate consumed spelling refund versus local-key clone refund; one ordinary @reference verifies AST and projected spelling count twice; one typed scoped-name/call-label case verifies retained move versus explicitly excluded coordination clone. Do not multiply permutations once each distinct ownership action is covered.
Direct arithmetic tests cover quoted P/D examples, overflow and invalid refund. Label arbitrary scalar headroom injections as helper tests. Actual pipeline positive/negative caps must include the real complete bundle C and earlier scanner/UUID/retained peaks. Malformed input controls should assert both legacy error stability and explicitly selected stronger-mode precedence.

Stack and scope

Retain the recent binding/prefix/continuation extraction and slim guard. Adding even a few Budget scalars changes inline type sizes and ancestor frames, so inspect actual compiler output and repeat established stack controls after implementation. Do not introduce a heavyweight per-String wrapper or stored callback merely for automatic refunds without reviewing its concrete frame/lifetime cost. A scoped scanner callback borrowing Budget avoids stored callback ownership; exact Rust signatures still require implementation review.

No new universal stack bound follows. The future verifier still owes actual-output semantic equality, original-E safety against both operands, output owner provenance and the remaining selected entry/scratch timing guarantees. This narrower text prerequisite advances one real gap without recasting every parser allocation as part of it.

Parent CONTRACT-DRAFT follow-up

Read the parent's CONTRACT-DRAFT.md and the implementer's reported lifecycle clarification. The draft now explicitly covers all three UUID consumers, kind-word borrow/accounting choice, nested-key overlap, duplicate-description rejection, separate fixture mode and legacy diagnostics. No further ownership/authority blocker found for this bounded prerequisite.

One scope sentence should be corrected before acceptance: grouping numeric reconstruction with “separate open domains” suggests new unresolved text-accounting work that the selected metric never required. State instead that internal Exact numeric reconstruction/formatting and BigInt storage remain controlled by the existing numeric policy and excluded from this logical syntax-text inventory, with work charges unchanged. That exclusion must not extend to retained AST String/Word/Reference text or copied syntax names merely because they occur near numeric parsing. Traversal/coordination and entry-timing limitations can remain the separately open domains.

The E146/145 and escaped fixture491/490 counts above assume the declaration-kind word is irrelevant (these are project-only sources), all named UUID Strings drop/refund on success, and the existing complete syntax inventory remains unchanged. They do not depend on the proposed implementation's output. T45 should remain a generous-cap reconciliation example, not an asserted projected-copy threshold. No draft approval should imply final PendingRename verification or a blanket parser-allocation guarantee.

Pre-execution coordinate correction: the exact 78-byte E document's UUID quoted token is 34..72 and its ordinary diagnostic is 34..35. The earlier 33..71 / 33..34 in this review was a source-character-count typo. Literal-only readback and the authored test use 34 correctly; the 146/145 storage caps are unchanged.

11 KiB Raw Blame History