Concrete peak-KV comparison across baselines
Table 1 reports 14.3 MiB peak KV for COA-PKV against 35.5 MiB for FLAT and 35.3 MiB for retrieval-matched RAG, while also disclosing higher cumulative traffic.
↳ §2, Table 1
Abstract
Decomposing long-context inference across cooperating agents bounds the active KV cache per call rather than total evidence, which matters when KV-cache memory binds. Many such systems add a persistent tier storing and recalling reasoning traces, usually validated by an ablation reporting an accuracy gain. We measure both on one three-tier agent architecture. Decomposition delivers: peak KV working set of 14.3 MiB per query against 35.5 and 35.3 MiB for single-pass and retrieval-augmented baselines. The persistent tier does not: across eight controlled dataset pairs at n=100 per arm it costs +0.368 MiB [+0.167, +0.590] of peak cache and produces no detectable accuracy change (+0.015, 95% CI [-0.011, +0.046]). We argue the null is structural: single-question benchmarks supply each item with its own evidence and score it independently, and correctness requires resetting stored traces between conditions, so recall has nothing informative to retrieve. Reaching it took four measurement corrections -- three inflating the apparent benefit, the fourth making an effect that size look resolvable -- none visible in the results table. We give the conditions an agent-memory ablation must satisfy and detection procedures that need no knowledge of the specific defect.
Key Claims
Primary findingSupported
Decomposition delivers: peak KV working set of 14.3 MiB per query against 35.5 and 35.3 MiB for single-pass and retrieval-augmented baselines.
matched within-system comparisons support the direction, though single-run FLAT and unisolated architectural differences limit attribution
Abstract; §2, Table 1
SecondaryTentative
The persistent tier does not: across eight controlled dataset pairs at n=100 per arm it costs +0.368 MiB [+0.167, +0.590] of peak cache and produces no detectable accuracy change (+0.015, 95% CI [−0.011, +0.046]).
paired estimates are reported, but fixed run order and asymmetric failures leave material systematic uncertainty
Abstract; §3, Table 2; Appendix A.2, Table 3
SecondaryTentative
We argue the null is structural: single-question benchmarks supply each item with its own evidence and score it independently, and correctness requires resetting stored traces between conditions, so recall has nothing informative to retrieve.
partial reachability coverage and an underpowered efficacy subgroup limit the proposed mechanism
Abstract; §4
SecondaryTentative
A non-zero memory effect reported on an independent-item benchmark is more likely an artifact than a finding.
one architecture and one team's reconstructed defects cannot establish comparative likelihood across the field
§4
Derived from the full evaluation — not a separate score.
Strengths · Limitations
Strengths
Mechanical isolation and paired analysis
The ablation admits cells only after five record-based checks, uses per-question pairing, and applies a two-stage cluster bootstrap across cells rather than pooling correlated questions.
↳ §3; Appendices A.1 and A.3
Measurement failures become reusable diagnostics
Conditions C1–C4 connect inert components, contamination, asymmetric execution, and measurement floors to concrete detection procedures such as live counters, store queries, request diffs, and counterbalanced order.
↳ §5 C1–C4
Limitations
Accuracy bound retains systematic confounding
Memory-enabled runs always occurred first, while memory-off recorded 20 orchestrator give-ups versus 1. The paper states that only a counterbalanced repeat can distinguish arm, order, and time, so the reported bootstrap interval is not a clean efficacy bound.
↳ §3; §5 C3; Appendix A.2
Decomposition effect is not isolated
The architecture comparison contrasts a multi-node controller with single-call FLAT and RAG pipelines without separately ablating critic loops, scoped retrieval, or per-node generation budgets. The measured difference is credible within the implementation, but attribution to decomposition alone is incomplete.
↳ §2, Table 1; Appendix A.1
Structural generalisation exceeds tested regime
Reachability covers six of the eight pooled cells, and the subgroup in which recall could act is underpowered. This limits the claim that non-zero effects on independent-item benchmarks are generally more likely artifacts than findings.
↳ §4; Appendix A.5
Nabu Commentary
The strongest empirical result is the within-system peak-KV comparison in Table 1, supported by matched reasoner and evidence settings but limited by a single-run FLAT arm and an architecture-level rather than decomposition-only contrast. The ablation protocol in §3 and Appendix A.1 is carefully specified, and the C1–C4 diagnostics convert the paper's correction history into a practical methodological contribution. The central difficulty is C3: memory-on always ran first, memory-off had substantially more orchestrator give-ups, and Appendix A.2 states that the record cannot separate arm, order, and time. That unresolved confounding makes the reported accuracy interval informative as an observed paired estimate but not a clean exclusion bound on persistent-memory efficacy.
Check another paper
Author Note / Field Feedback
Nabu’s assessment, alongside the field’s view.
Are you an author of this paper?
Dimension breakdown
Study Quality
Sound3.3
Confidence mediumContribution
3.6
The paper contributes a quantified peak-KV comparison and a concrete protocol for auditing persistent-memory ablations. Its broader structural and artifact-likelihood claims remain limited by one architecture, partial reachability analysis, and an underpowered efficacy subgroup.
Abstract; §§2–5; Appendix A.5
We do not claim persistent recall improves accuracy: that is untested, not disproven.
Methodological Rigour
3.4
The paired analysis, cell-aware bootstrap, live isolation probe, alternative failure convention, and contamination audit provide substantial methodological structure. Methodological Rigour is reduced because memory-on always ran first, failures were asymmetric, and the architecture comparison does not isolate decomposition from its other design components.
§§2–3; §5 C2–C3; Appendices A.1–A.3
With order and arm confounded, the record cannot say why.
Reporting
3.4
The progression from working-set measurement to memory ablation and diagnostic conditions is readily followable. The statement that effects above approximately 0.046 are excluded is nevertheless stronger than the unresolved systematic uncertainty permits.
§3; §5 C3; Appendix A.2
only a counterbalanced repeat can.
Positioning
2.8
The paper distinguishes decomposition, retrieval augmentation, cache management, and multi-session memory evaluation, and it states several scope limits directly. Positioning is capped because the acknowledged order confound is not carried through to the strength of the central exclusion conclusion.
§§3–7; Appendix A.2
the record cannot say whether the arm, the order or the time of day caused it
Lower confidence on Methodological Rigour — domain match limited.
Reliability
Concerns4 of 4 checks
Internal CoherenceConcern
The reported values are arithmetically consistent, but the principal accuracy bound does not incorporate a disclosed systematic confound. Memory-enabled runs always occurred first, while memory-disabled runs had many more orchestrator give-ups, leaving arm, order, and time inseparable.
Research ConductClean
No human participants or identifiable personal data are involved, and no ethics approval or preregistration is expected for this computational systems study. Code and per-sample materials are promised for future release but are not claimed to be presently available.
Flags: 1 declared / 5 total
Reference IntegrityNoted
12 references in manuscript 9 of 12 checkable references found in an index
Post-Publication RecordClean
No retraction notice found in Retraction Watch.
Last checked: 12 Oct 2026
Sources: Retraction Watch ✓
Readiness
Where this paper’s evidence sits on the path from initial observation to real-world use.
Demonstrated
The measurement and audit procedures have been exercised on two edge-device serving setups and could be reused by evaluation teams. Readiness remains proof-of-concept because the code is promised for later release and no serving integration, throughput characterisation, or scaling study is provided.
§7; Appendices A.1 and A.6
will be released as an open-source repository
AI-generated, human-governed. Something look off? Contact us to request a review.