# Recursa: Toward Recursively Self-Improving Foundation Model Systems

- **DOI:** [10.48550/arxiv.2611.04035](https://doi.org/10.48550/arxiv.2611.04035)
- **Rating:** Sound (3.0 out of 5)
- **Verdict:** Recursa combines corpus bootstrapping and parallel repository learning to improve legal-benchmark task success, but adaptive test evaluation limits the headline gains.
- **Framework version:** v5.10
- **Pipeline version:** v0.4
- **Evaluation date:** 2026-11-13
- **Evaluation page:** https://www.nabu.science/paper/10.48550%2Farxiv.2611.04035

## Key claims

1. On Harvey LAB's Firm Knowledge benchmark, Recursa raises the all-pass rate on held-out test tasks from 30.0% to 55.0% with DeepSeek-V4.1-Flash and from 35.0% to 46.0% with Qwen3.8-Flash.
   - Support: Tentative
   - Type: Primary finding
   - Basis: unspecified checkpoint selection across repeated test evaluations limits the reported gains as independent held-out estimates
   - Location: Abstract; Section 1; Table 3
2. With both solvers, gains concentrate in all-pass rate: learning converts partially correct answers into complete successes, and these gains carry over to test tasks that never informed learning.
   - Support: Tentative
   - Type: Secondary
   - Basis: post-hoc test-rubric revision and unspecified checkpoint selection weaken the claim of untouched held-out transfer
   - Location: Section 6.4, paragraph beginning 'With both solvers'
3. Bootstrapping reduces both interaction turns and input tokens by roughly a fifth to a quarter and raises the all-pass rate slightly, while the mean criteria pass rate remains unchanged.
   - Support: Supported
   - Type: Secondary
   - Basis: direct before-and-after comparison supports efficiency gains within one solver and benchmark, without repeated-run uncertainty
   - Location: Section 6.3, Figure 4 discussion
4. recursive self-improvement enables the system to complete substantially more tasks
   - Support: Tentative
   - Type: Secondary
   - Basis: non-monotone later-round performance does not isolate improvement from repeated recursion beyond the first merged repository
   - Location: Section 1

## Strengths

- **Persistent learning connects corpus and experience** Bootstrapping builds reusable domain skills, while parallel learners diagnose task failures and merge revisions into instructions, skills, and memory. The architecture makes the repository-level adaptation mechanism explicit.
  - Dimension: contribution
  - Location: Sections 2.2, 4–5; Algorithm 1
- **Complete tables expose every learning round** The appendix reports all learning rounds for both solvers under original and revised rubrics. No-learning, bootstrap, and same-model single-learner controls allow the reported gains and their instability to be inspected.
  - Dimension: methodological rigour
  - Location: Sections 6.1–6.4; Tables 3–4
- **Behavioral traces make learned changes concrete** The examples show changes in concept-based searching and evidence-based inclusion rules, connecting repository learning to specific differences in legal-task execution.
  - Dimension: reporting
  - Location: Section 6.5; Figures 5–6

## Limitations

- **Checkpoint selection weakens held-out gain estimates** The highlighted checkpoints coincide with test-set maxima, but no predefined selection rule or validation split is reported. Rubrics for 13 test tasks were also revised after failed runs were inspected, although original-rubric results preserve the direction of improvement.
  - Dimension: methodological rigour
  - Location: Sections 6.1 and 6.4; Tables 3–4; Appendix B
- **Uncertainty and mechanism attribution remain unresolved** Single runs lack confidence intervals or repeated-seed estimates, and the single-learner comparison does not isolate reflection, validation, or merging. Judge validation and learning-compute budgets are also unreported.
  - Dimension: methodological rigour
  - Location: Sections 5 and 6.1–6.4; Table 4
- **One synthetic workspace limits broader applicability** The experiment learns only the knowledge repository while holding models and harness fixed. A single synthetic legal environment does not establish general domain adaptation or readiness for operational legal work.
  - Dimension: application pathway
  - Location: Sections 2.2, 6, and 7

## Reliability checks

No concerns (4 of 4 checks).

- **Results check:** Clean. The reported numbers and statistics are consistent with the stated results.
- **Reporting and transparency:** Clean. The benchmark repository and commit are identified, and rubric revisions are disclosed alongside original-rubric results. No suspicious availability claim or ethics inconsistency is established.
  - Ethics approval: declared · Data availability: declared
- **Reference integrity:** Noted
  - 37 references in manuscript
  - 21 of 30 checkable references found in an index
  - 7 are books, websites or datasets — counted, but not index-checkable
  - Could not be verified automatically — queued for manual review:
    - Agrawal, L. A., Tan, S., Soylu, D., Ziems, N., Khare, R., Opsahl-Ong, K., Singhvi, A., Shandilya, H., Ryan, M. J., Jiang, M., Potts, C., Sen, K., Dimakis, A., Stoica, I., Klein, D., Zaharia, M., and Khattab, O. GEPA: Reflective prompt evolution can outperform reinforcement learning. In International Conference on Learning Representations, 2026. URL https://proceedings.iclr.cc/ paper_files/paper/2026/hash/0e9e708b 6f48e14fd0ac29e167413f76-Abstract-Con ference.html. (Ambiguous: title ratio 0.57 via crossref_bibliographic sits between the divergence (0.30) and match (0.70) thresholds; too close to call either way, so it is recorded as unresolved rather than judged.)
    - Choi, Y., Park, S., Kang, M., Baek, J., and Hwang, S. J. PREPING: Building agent memory without tasks, 2026. URL https://arxiv.org/abs/2605.13880. (Ambiguous: title ratio 0.46 via datacite sits between the divergence (0.30) and match (0.70) thresholds; too close to call either way, so it is recorded as unresolved rather than judged.)
    - Hu, S., Lu, C., and Clune, J. Automated design of agentic systems. In International Conference on Learning Representations, pp. 21344–21377, Singapore, 2025. ICLR. URL https://proceedings.iclr.cc/pape r_files/paper/2025/hash/36b7acf6f601 0652b3f2a433774a66fe-Abstract-Confere nce.html. (Crossref bibliographic top candidate ratio 0.37 — no plausible match)
    - Khattab, O., Singhvi, A., Maheshwari, P., Zhang, Z., Santhanam, K., A, S. V., Haq, S., Sharma, A., Joshi, T. T., Moazam, H., Miller, H., Zaharia, M., and Potts, C. DSPy: Compiling declarative language model calls into state-of-the-art pipelines. In International Conference on Learning Representations, 2024. URL https: //proceedings.iclr.cc/paper_files/pa per/2024/hash/f1cf02ce09757f57c3b93c 0db83181e0-Abstract-Conference.html. (Crossref bibliographic top candidate ratio 0.42 — no plausible match)
    - Li, J. X., Battle, R., and Khattab, O. Machine studying: A system-level reframing of continual adaptation from declarative corpora. In ICML 2026 Workshop on Continual Adaptation at Scale: Towards Sustainable AI, 2026a. URL https://openreview.net/forum?id= e7duOnFgL7. (Crossref bibliographic top candidate ratio 0.48 — no plausible match)
    - Sarthi, P., Abdullah, S., Tuli, A., Khanna, S., Goldie, A., and Manning, C. RAPTOR: Recursive abstractive processing for tree-organized retrieval. In International Conference on Learning Representations, pp. 32628–32649, Vienna, Austria, 2024. ICLR. URL https://proceeding s.iclr.cc/paper_files/paper/2024/has h/8a2acd174940dbca361a6398a4f9df91-A bstract-Conference.html. (Crossref bibliographic top candidate ratio 0.44 — no plausible match)
    - Wang, X., Chen, Y., Yuan, L., Zhang, Y., Li, Y., Peng, H., and Ji, H. Executable code actions elicit better LLM agents. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 50208–50232, Vienna, Austria, 2024. PMLR. URL https://proceeding s.mlr.press/v235/wang24h.html. (Crossref bibliographic top candidate ratio 0.36 — no plausible match)
    - Ye, H., He, X., Arak, V., Dong, H., and Song, G. Meta context engineering via agentic skill evolution. In Proceedings of the 43rd International Conference on Machine Learning, volume 306 of Proceedings of Machine Learning Research, pp. 148325–148368, 2026. URL https://proceedings.mlr.press/v306/y e26g.html. (Crossref bibliographic top candidate ratio 0.36 — no plausible match)
    - Zhang, J., Xiang, J., Yu, Z., Teng, F., Chen, X., Chen, J., Zhuge, M., Cheng, X., Hong, S., Wang, J., Zheng, B., Liu, B., Luo, Y., and Wu, C. AFlow: Automating agentic workflow generation. In International Conference on Learning Representations, 2025b. URL https://pr oceedings.iclr.cc/paper_files/paper/ 2025/hash/5492ecbce4439401798dcd2c90 be94cd-Abstract-Conference.html. (Crossref bibliographic top candidate ratio 0.42 — no plausible match)
  - References not verified automatically are resolved manually within 24 hours.

## Post-publication record

- **Status:** Clean
- **Summary:** No retraction or expression of concern found in Crossref (includes Retraction Watch).
- **Last checked:** 2026-11-13
