Assembling the evidence…
Scientific discovery is driven by scientists generating hypotheses for complex problems that undergo rigorous experimental validation. To augment this process, we introduce Co-Scientist, a multi-agent artificial intelligence (AI) system built on Gemini for structured scientific thinking and hypothesis generation. Co-Scientist aims to help scientists discover new original knowledge. Conditioned on their research objectives and previous scientific evidence, it formulates demonstrably novel research hypotheses for experimental verification. The system's design involves agents continuously generating, critiquing and refining hypotheses accelerated by scaling test-time compute. Key contributions include (1) a multi-agent architecture with an asynchronous task execution framework for flexible compute scaling, and (2) a tournament evolution process for self-improving hypotheses generation. Automated evaluations show continued benefits of test-time compute scaling, improving hypothesis quality over time. Although this is a general-purpose system, we focus the validation in three biomedical applications: drug repurposing; novel-target discovery1; and explaining mechanisms of antimicrobial resistance2. Specifically, Co-Scientist helped to identify new drug-repurposing candidates and synergistic combination therapies for acute myeloid leukaemia that were validated through in vitro experiments. These real-world validations demonstrate the potential of Co-Scientist to accelerate scientific discovery and usher in an era of AI-empowered scientists.
Co-Scientist helped to identify new drug-repurposing candidates and synergistic combination therapies for acute myeloid leukaemia that were validated through in vitro experiments
selected cell-line findings support activity, but expert curation and preliminary assays limit the broader validation claim
Co-Scientist eventually substantially surpassed the other frontier LLMs and reasoning models in Elo rating with iterative improvement
the internal Elo comparison lacks compute-equivalent baselines and independent objective measurement
Co-Scientist consistently achieved the highest expert ratings in novelty and impact and was selected as the preferred AI system by experts
blinded ratings directly support the comparison, though only across 11 curated goals
KIRA6 exhibits highly selective cytotoxicity against the KG-1a AML cell line compared with the non-malignant TK6 control
triplicate dose-response assays show an 18-fold IC50 separation, within a preliminary cell-line design
Co-Scientist independently and accurately proposed the top-ranked hypothesis that cf-PICIs interact with diverse phage tails to expand their host range
the matching hypothesis is documented, but informed researchers supplied the task and background
Derived from the full evaluation — not a separate score.
Strengths
The evaluation reaches beyond textual benchmarks to AML cell-line assays, liver-organoid work, and comparison with a co-timed antimicrobial-resistance finding. This provides concrete tests of selected hypotheses across distinct biomedical problems.
↳ Fig. 1c; Real-World Validations; Figs. 3–4
The ablation programme tests search access, scientific-debate prompting, evolutionary refinement, and meta-review, with reported changes in novelty scores, GPQA precision, and correctness AUC.
↳ Methods, Ablation Analysis
The Discussion traces open-access dependence and variable source quality to omitted evidence and propagation risk, then identifies provenance, verification, and broader evaluation as needed safeguards.
↳ Discussion, limitations and expanded-evaluations paragraphs
Limitations
Fig. 2b compares an iteratively scaled multi-agent system with individual frontier models without an equivalent compute budget. The design therefore cannot isolate whether architecture, tools, or additional inference drives the reported advantage.
↳ Scaling Test-Time Compute Improves Scientific Reasoning; Fig. 2b
The comparative Elo outcome is produced through Co-Scientist's own Ranking agent and tournament framework. Small blinded expert and cross-model judge studies provide corroboration, but the article states that larger studies are required for reliable conclusions.
↳ Methods, Ranking Agent; Fig. 2b–c
The AML findings rely on preliminary cell-line assays, while in vivo efficacy, pharmacokinetics, tumour microenvironment, heterogeneity, and clinical outcomes remain untested. The article explicitly states that these experiments are not substitutes for preclinical and clinical assessment.
↳ Drug Repurposing with Co-Scientist; Guiding Clinical Translation Design
The multi-domain experimental follow-up and quantitative component ablations support a meaningful contribution. Methodological Rigour is constrained by Fig. 2b's comparison of a compute-scaled multi-agent system against baselines without equivalent inference resources, compounded by reliance on the system's own Elo tournament. The article's organization and limitations discussion are generally clear, but its strongest superiority wording exceeds what the small expert studies and internally generated metric independently establish. Impact is most credible as a hypothesis-prioritization aid, with clinical translation still requiring substantial preclinical and in vivo work.
Nabu’s assessment, alongside the field’s view.
Are you an author of this paper?
Sound3.2
The paper combines a multi-agent hypothesis-generation system with experimental follow-up in AML, liver fibrosis, and antimicrobial resistance. The contribution is meaningful, although the architecture's incremental advantage over compute-equivalent alternatives is not isolated.
These findings provide preliminary evidence that Co-Scientist can contribute meaningfully to scientific discovery by amplifying scientists.
Component ablations and biological triplicates provide useful execution evidence, but Fig. 2b compares an iteratively scaled system against baselines without equivalent compute and evaluates them through Co-Scientist's Elo framework. This prevents clean attribution of the reported comparative advantage.
Performance was assessed using the Co-Scientist Elo rating metric.
The article is structured coherently and repeatedly describes the experimental findings as preliminary. However, the claim that Co-Scientist “surpass[es]” experts and frontier models communicates more certainty than the asymmetric Elo comparison and small expert evaluation support.
Owing to the small scale of these evaluations, further studies are necessary for any reliable conclusions.
The Discussion connects literature-access bias, source unreliability, hallucination, and preliminary validation to concrete interpretive consequences and future safeguards. It also recognizes the need for objective metrics and evaluation across more disciplines.
Expanded evaluations are also necessary to assess Co-Scientist’s generalizability across a wider range of scientific disciplines.
Lower confidence on Contribution, Positioning — domain match limited.
Caveats4 of 4 checks
Reported numerical results are internally consistent, but the strongest comparative and validation language is not fully aligned with the evaluation's acknowledged limitations. The internally generated Elo metric also limits the independence of the principal automated superiority result.
No supplied-text inconsistency involving ethics, conflicts, availability, or experimental conduct was identified. Disclosures and supplementary materials not visible in the supplied article remain assessability gaps rather than reliability findings.
Flags: 0 declared / 5 total
8 of 8 checkable references verified
10 references in manuscript 2 have no canonical index record — counted, but not index-checkable 2 references confirmed by manual review
No retraction notice found in Retraction Watch.
Sources: Retraction Watch ✓
Medium3.2
The system addresses identifiable needs in oncology, fibrosis, and antimicrobial-resistance research, with clinicians and domain experts participating in candidate selection and experimental follow-up. This establishes a visible user group and workflow, though not deployment readiness.
all experimental validations were conducted in collaboration with expert scientists
The AML cell-line and liver-organoid studies constitute laboratory proof of concept rather than clinical validation. The paper identifies pharmacokinetics, heterogeneity, microenvironment effects, in vivo efficacy, and trials as unresolved requirements.
this does not guarantee in vivo efficacy or clinical success
The system is demonstrated across three distinct biomedical applications and a broader computational goal set. Experimental evidence remains narrow and biomedical, with systematic validation outside those settings still required.
Expanded evaluations are also necessary to assess Co-Scientist’s generalizability across a wider range of scientific disciplines.
The work extends active research on test-time compute, multi-agent systems, drug repurposing, and AI-assisted scientific discovery. Its cumulative value is constrained by the absence of compute-matched comparisons and independently standardized measures of hypothesis quality.
developing more objective and automated evaluation metrics that move beyond current ranking systems
Lower confidence on Generalizability, Trajectory — domain match limited.
AI-generated, human-governed. Something look off? Contact us to request a review.