# Does Testosterone Affect Cognitive Reflection? Evidence From a Double-Blind, Randomized Controlled Study of 1,000 Participants

- **DOI:** [10.1177/09567976261458325](https://doi.org/10.1177/09567976261458325)
- **Rating:** Strong (4.1 out of 5)
- **Verdict:** A strong adversarial replication narrows acute-testosterone claims on cognitive reflection, with single-dose male sampling as its main constraint.
- **Framework version:** v5.5
- **Pipeline version:** v0.4
- **Evaluation date:** 2026-07-22
- **Evaluation page:** https://www.nabu.science/paper/10.1177%2F09567976261458325

## Key claims

1. We found an insignificant treatment effect on the CRT, with the point estimate in the opposite direction of the original hypothesis
   - Support: Well-supported
   - Type: Primary finding
   - Basis: large randomized trial with consistent primary estimate and tight confidence interval
   - Location: Abstract; Results, Table 2; Discussion, p. 473
2. we found a significant negative treatment effect on confidence
   - Support: Supported
   - Type: Secondary
   - Basis: primary model supports the effect, while multiplicity and robustness checks temper certainty
   - Location: Abstract; Results, p. 468; Table 2
3. Our findings challenge earlier claims about testosterone’s cognitive effects
   - Support: Supported
   - Type: Secondary
   - Basis: directly addresses the prior acute-testosterone CRT claim, but only within this paradigm
   - Location: Abstract; Discussion, p. 473
4. we failed to conceptually replicate the interaction between testosterone treatment and trait impulsivity
   - Support: Supported
   - Type: Exploratory
   - Basis: preregistered exploratory analysis found no evidence for the reported interaction
   - Location: Introduction, p. 465; Results, Table 3; Discussion, p. 473
5. Long-term or developmental testosterone effects remain potentially important but difficult to study
   - Support: Supported
   - Type: Secondary
   - Basis: appropriately framed as a limitation rather than an empirical conclusion
   - Location: Abstract; Discussion, pp. 473–474

## Strengths

- **Large adversarial randomized replication** The study directly addresses a contested prior finding through an adversarial collaboration between teams that had previously reported conflicting results. Its N = 1,000 sample is far larger than the prior studies discussed in the Introduction.
  - Dimension: contribution
  - Location: Introduction, pp. 464–465; Abstract
- **Strong trial design and manipulation check** Participants were randomized 500/500 to testosterone or placebo in a double-blind design, and post-administration saliva data confirmed that treatment increased testosterone relative to placebo. The primary CRT outcome was analyzed with ordered logistic regression consistent with the stated analysis plan.
  - Dimension: methodological rigour
  - Location: Methods, pp. 466–468; Results, Table 2
- **Careful calibration of unexpected findings** The confidence-score effect is reported as statistically significant in the opposite direction of the hypothesis, but the article explicitly notes its low prior probability, multiplicity considerations, and need for replication. Exploratory hormone and interaction findings are likewise described as requiring confirmation.
  - Dimension: reporting
  - Location: Results, pp. 468–470; Discussion, p. 473

## Limitations

- **Registration was not fully prospective** The article describes the study as preregistered but discloses that the PAP was posted in March 2018 after data collection began in February 2018. This is transparent, but it weakens strict claims of fully prospective protocol registration.
  - Dimension: methodological rigour
  - Location: Study 1 disclosures; Preregistration and statistical testing, p. 467
- **Harms reporting is not visible** The study administered intranasal testosterone, but the supplied main text does not include a systematic adverse-events or tolerability section. For a pharmacological randomized trial, that omission limits reporting completeness even if the single-dose paradigm is established.
  - Dimension: methodological rigour
  - Location: Methods, pp. 466–467; Results/Discussion
- **External validity remains bounded** The sample consisted of adult men aged 18 to 45 in three Canadian locations, using one intranasal single-dose paradigm and one modified CRT implementation. The Discussion explicitly notes that women, chronic exposure, developmental exposure, and delayed effects remain unresolved.
  - Dimension: application pathway
  - Location: Method—Study location and sample, p. 466; Discussion, pp. 473–474

## Reliability checks

Caveats (4 of 4 checks).

- **Internal coherence:** Noted. The main numerical results, sample flow, and interpretation are internally coherent across the abstract, Results, and Table 2. The notable internal framing issue is that the article describes the study as preregistered while also disclosing that registration occurred after data collection had started.
- **Research conduct:** Noted. Ethics approval, informed consent, data/material/script availability, and computational reproducibility are declared. The conduct watch-out is that registration was not fully prospective before data collection, although the article states it occurred before data access or analysis.
- **Reference integrity:** Clean. 38 references verified via PubMed, Crossref, or OpenAlex.
  - References not verified automatically are resolved manually within 24 hours.

## Post-publication record

- **Status:** Clean
- **Summary:** No retraction or expression of concern found in Crossref (includes Retraction Watch).
- **Last checked:** 2026-10-17
