nabu.science
MethodologyAboutContact

 

  • Snapshot
  • Key Claims
  • Strengths & Limitations
  • Commentary
  • From the Authors
  • Author Note / Field Feedback
  • Dimension breakdown
  • Impact Profile
Free · About 5 minutes
Report #NBU-159D2475·Framework v5.5·Pipeline v0.4·Evaluated 22 Jul 2026·OECD: 5.1 Psychology·Subfield: Experimental and Cognitive Psychology·Basis: Full text·Type: Randomized controlled trial

Does Testosterone Affect Cognitive Reflection? Evidence From a Double-Blind, Randomized Controlled Study of 1,000 Participants

Erik L. Knight et al.Erik L. Knight, Gideon Nave, Steven D. Shaw et al.·2026·10.1177/09567976261458325 ↗

Primary claim

We found an insignificant treatment effect on the CRT, with the point estimate in the opposite direction of the original hypothesis

Well-supported · large randomized trial with consistent primary estimate and tight confidence interval
In brief

A strong adversarial replication narrows acute-testosterone claims on cognitive reflection, with single-dose male sampling as its main constraint.

ReliabilityCaveats4 of 4 checks completeRead the 2 entries ↓
Likely to hold — the claim's weakest requirement is that the effect exists at all. Experimental. Every point below is traceable to a quoted passage, but this forecast has not yet been validated against known replication outcomes.
Abstract

The cognitive reflection test (CRT) measures reliance on intuitive thinking versus deliberate reasoning and predicts important real-world outcomes. Prior research has suggested that testosterone administration impaired CRT performance, but follow-up studies produced null results. To provide a rigorous test, we conducted a large, preregistered, double-blind, placebo-controlled experiment, unprecedented in size, with 1,000 adult men, as part of an adversarial collaboration. Participants received a single dose of intranasal testosterone or placebo, completed the CRT, and rated their confidence level. We found an insignificant treatment effect on the CRT, with the point estimate in the opposite direction of the original hypothesis (β LOGIT = 0.118, 95% confidence interval (CI) = [−0.099, 0.335]). In a second primary test, we found a significant negative treatment effect on confidence (β LOGIT = −0.329, 95% CI = [−0.558, −0.100]), which is also the opposite of our prediction. Our findings challenge earlier claims about testosterone’s cognitive effects and highlight the importance of high-powered replications. Long-term or developmental testosterone effects remain potentially important but difficult to study.

Key Claims

Primary finding · Well-supported

We found an insignificant treatment effect on the CRT, with the point estimate in the opposite direction of the original hypothesis

Likely to hold — the claim's weakest requirement is that the effect exists at all. Experimental. Every point below is traceable to a quoted passage, but this forecast has not yet been validated against known replication outcomes.

large randomized trial with consistent primary estimate and tight confidence interval

Abstract; Results, Table 2; Discussion, p. 473
Secondary · Supported

we found a significant negative treatment effect on confidence

primary model supports the effect, while multiplicity and robustness checks temper certainty

Abstract; Results, p. 468; Table 2
Secondary · Supported

Our findings challenge earlier claims about testosterone’s cognitive effects

directly addresses the prior acute-testosterone CRT claim, but only within this paradigm

Abstract; Discussion, p. 473
Exploratory · Supported

we failed to conceptually replicate the interaction between testosterone treatment and trait impulsivity

preregistered exploratory analysis found no evidence for the reported interaction

Introduction, p. 465; Results, Table 3; Discussion, p. 473
Secondary · Supported

Long-term or developmental testosterone effects remain potentially important but difficult to study

appropriately framed as a limitation rather than an empirical conclusion

Abstract; Discussion, pp. 473–474

Derived from the full evaluation — not a separate score.

Strengths · Limitations

Strengths

#contribution

Large adversarial randomized replication

The study directly addresses a contested prior finding through an adversarial collaboration between teams that had previously reported conflicting results. Its N = 1,000 sample is far larger than the prior studies discussed in the Introduction.

↳ Introduction, pp. 464–465; Abstract

#methodological rigour

Strong trial design and manipulation check

Participants were randomized 500/500 to testosterone or placebo in a double-blind design, and post-administration saliva data confirmed that treatment increased testosterone relative to placebo. The primary CRT outcome was analyzed with ordered logistic regression consistent with the stated analysis plan.

↳ Methods, pp. 466–468; Results, Table 2

#reporting

Careful calibration of unexpected findings

The confidence-score effect is reported as statistically significant in the opposite direction of the hypothesis, but the article explicitly notes its low prior probability, multiplicity considerations, and need for replication. Exploratory hormone and interaction findings are likewise described as requiring confirmation.

↳ Results, pp. 468–470; Discussion, p. 473

Limitations

#methodological rigour

Registration was not fully prospective

The article describes the study as preregistered but discloses that the PAP was posted in March 2018 after data collection began in February 2018. This is transparent, but it weakens strict claims of fully prospective protocol registration.

↳ Study 1 disclosures; Preregistration and statistical testing, p. 467

#methodological rigour

Harms reporting is not visible

The study administered intranasal testosterone, but the supplied main text does not include a systematic adverse-events or tolerability section. For a pharmacological randomized trial, that omission limits reporting completeness even if the single-dose paradigm is established.

↳ Methods, pp. 466–467; Results/Discussion

#application pathway

External validity remains bounded

The sample consisted of adult men aged 18 to 45 in three Canadian locations, using one intranasal single-dose paradigm and one modified CRT implementation. The Discussion explicitly notes that women, chronic exposure, developmental exposure, and delayed effects remain unresolved.

↳ Method—Study location and sample, p. 466; Discussion, pp. 473–474

Does this match your read? Add your assessment ↓

Nabu Commentary

The quality score is driven by the match between the research question and the design: a large double-blind randomized trial with balanced allocation, a manipulation check, and preregistered primary analyses directly tests the disputed CRT claim. The main methodological deductions come from delayed PAP posting, nonpreregistered interpretation of some confidence responses, and missing adverse-event reporting in the visible main text. The paper’s conclusions are generally proportionate: the null CRT result is presented strongly, while the unexpected confidence finding and exploratory analyses are explicitly hedged. Impact potential is lower than quality because the main payoff is cumulative scientific clarification rather than deployment-ready evidence for a specific decision-maker.

Evaluate another paper

Free · About 5 minutes · Research preview

Author Note / Field Feedback

Nabu’s assessment, alongside the field’s view.

Are you an author of this paper?

Dimension breakdown

Evidence Score

Out of 5
012345
4.1±0.1

Contribution
4.2

The score reflects a substantial cumulative-science contribution: a large adversarial collaboration directly tests a contested prior finding and materially narrows uncertainty around the acute testosterone–CRT claim. It is not scored higher because the advance is primarily adjudicative rather than a broad new discovery.

Abstract; Introduction, pp. 464–465; Discussion, p. 473

unprecedented in size, with 1,000 adult men

Methodological Rigour
4.0

The randomized, double-blind, placebo-controlled design, balanced allocation, biological manipulation check, appropriate ordered-logistic analyses, and public reproducibility claims support a high Methodological Rigour score. Deductions reflect the PAP posting after data collection began, nonpreregistered coding of some confidence responses, and lack of visible adverse-event reporting for a pharmacological RCT.

Methods, pp. 466–468; Study 1 disclosures; Results, Table 2

This interpretation process was not preregistered.

Reporting
4.2

The article clearly separates primary hypotheses, robustness tests, exploratory analyses, and nonpreregistered robustness tests, and it consistently hedges the unexpected confidence result. The main clarity limitation is the repeated preregistration framing despite the disclosed post-start PAP timing.

Abstract; Preregistration and statistical testing, p. 467; Results, pp. 468–471

we interpret it cautiously until it is confirmed

Positioning
4.2

The article is well situated in the directly relevant literature, including the original positive study, follow-up null studies, and the broader testosterone-behavior framework. It also explicitly limits interpretation to acute, single-dose effects in men and discusses sex, timing, chronic exposure, and developmental effects.

Introduction, pp. 464–465; Discussion, pp. 473–474

should not be interpreted as a direct replication

Lower confidence on Methodological Rigour — domain match limited.

Reliability

Caveats4 of 4 checks


Internal CoherenceNoted

The main numerical results, sample flow, and interpretation are internally coherent across the abstract, Results, and Table 2. The notable internal framing issue is that the article describes the study as preregistered while also disclosing that registration occurred after data collection had started.

Research ConductNoted

Ethics approval, informed consent, data/material/script availability, and computational reproducibility are declared. The conduct watch-out is that registration was not fully prospective before data collection, although the article states it occurred before data access or analysis.

Flags: 4 declared / 5 total

Reference IntegrityClean

38 references verified via PubMed, Crossref, or OpenAlex.

References not verified automatically are resolved manually within 24 hours.

Post-Publication RecordClean

No retraction or expression of concern found in Crossref (includes Retraction Watch).

Last checked: 17 Oct 2026

Sources: Crossref (incl. Retraction Watch) ✓

Impact Profile

Impact areaHealth · Generic health relevance
Type of advanceEmpirical
Route to useNot stated
AdvanceAdvances
CitationsToo early
Policy · patentNot yet traced
ReachedNot reachedNot assessable or unverified
ReadinessValidated
Foundational
Controlled
Piloted
Validated
Implementable
FoundationalImplementable

The primary claim was tested directly in a large preregistered double-blind randomized trial of 1,000 men across several collection sites, a confirmatory trial far exceeding prior studies' power.

The paper states no real-world use of its own findings; its only stated next steps are further replication and research on long-term or developmental testosterone effects.

Pathway

Behavioral endocrinology researchers and clinicians interpreting testosterone's short-term cognitive effects would use these null findings to overturn earlier claims and redirect research toward chronic or developmental exposures.

Barrier

The single-dose design cannot address long-term or developmental testosterone effects, and the unexpected confidence finding requires independent replication before firm conclusions or applications.

Evaluated 13 Nov 2026 · GBD 2023 · glm_v1_3_3 · impact_profile_v1_3_3

Report an issue →

AI-generated, human-governed. Something look off? Contact us to request a review.

Keep reading

Related evaluations

Other papers Nabu has evaluated blind, starting with the same field.

  1. A Critical Review of the Literature Regarding the Selection of Long-Term Romantic Partners

    Devenport, Davis-McCabe, Winter · 2023 · Psychology

    Evidence Score
    3.2±0.2
    Reliability
    No concerns
  2. Diverse Gender and Sexual Identity in Romantic Partner Selection Experiences: An Exploration of Similarities, Differences, and Potential Explanations

    Devenport, Davis-McCabe, Mullan, et al. · 2025 · Psychology

    Primary claim

    Factor analysis replicated established trait preference dimensions in this large, heterogeneous Australian sample, with dimensions remaining stable in majority and minority subsamples.

    Tentative
    Evidence Score
    3.2±0.2
    Reliability
    Caveats
  3. Facial appearance affects science communication

    Gheorghiu, Callan, Skylark · 2017 · Psychology

    Primary claim

    Apparent competence and morality were positively related to both interest and quality judgments, whereas attractiveness boosted interest but decreased perceived quality.

    Well-supported
    Evidence Score
    3.9±0.1
    Reliability
    No concerns

How the rubric works →