# Open access research outputs receive more diverse citations

- **DOI:** [10.1007/s11192-023-04894-0](https://doi.org/10.1007/s11192-023-04894-0)
- **Rating:** Sound (3.6 out of 5)
- **Verdict:** A large-scale citation-diversity analysis meaningfully extends open-access evidence, but observational confounding limits claims that access itself widens research use.
- **Framework version:** v5.9
- **Pipeline version:** v0.4
- **Evaluation date:** 2026-09-28
- **Evaluation page:** https://www.nabu.science/paper/10.1007%2Fs11192-023-04894-0

## Key claims

1. we found a robust association between open access and increased diversity of citation sources by institutions, countries, subregions, regions, and fields of research, across outputs with both high and medium–low citation counts
   - Support: Supported
   - Type: Primary finding
   - Basis: large observational corpus and extensive robustness checks support direction, but residual selection and coverage limitations remain
   - Location: Abstract; Results
2. Open access through disciplinary or institutional repositories showed a stronger effect than open access via publisher platforms.
   - Support: Supported
   - Type: Secondary
   - Basis: GREEN generally exceeds GOLD across reported comparisons, though overlapping categories and residual selection limit interpretation
   - Location: Abstract; Results, Comparing OA categories
3. It also provides new evidence at global scale of the benefits of open access as a mechanism for widening the use of research and increasing the diversity of the communities that benefit from it.
   - Support: Tentative
   - Type: Secondary
   - Basis: observational citation diversity cannot identify OA-caused wider use or the full communities benefiting
   - Location: Abstract; Conclusion
4. the citation diversity advantage also accrues preferentially to traditionally prestigious centres of research.
   - Support: Supported
   - Type: Secondary
   - Basis: regional citation-flow patterns support the direction, with incomplete bibliographic coverage limiting geographic inference
   - Location: Results, Comparing geographies; Discussion

## Strengths

- **Global scale across multiple dimensions** The analysis covers 19 million outputs and 420 million citation links, measuring diversity by institutions, countries, subregions, regions, and fields of research.
  - Dimension: contribution
  - Location: Contribution and Method, pp. 827–833
- **Extensive internal robustness analyses** The reported pattern is checked across publication years, two diversity indices, OA categories, equal-sized samples, fields, and 14 citation-count bins.
  - Dimension: methodological rigour
  - Location: Robustness of results, pp. 837–840; Supplementary Figures A–N
- **Substantive engagement with competing explanations** The paper discusses randomized null results, author self-selection, prestige, discipline, and resource access, then explicitly limits its conclusion to an observational association.
  - Dimension: positioning
  - Location: Introduction, p. 826; Results, p. 836; Discussion, pp. 840–841

## Limitations

- **Residual OA selection remains unresolved** Citation-count stratification reduces one source of confounding but does not independently adjust for discipline, institutional prestige, author characteristics, or research-resource context.
  - Dimension: methodological rigour
  - Location: Method, p. 827; Results, pp. 834–836; Discussion, pp. 840–841
- **Coverage bias is not quantified** Crossref-only DOI coverage, MAG inclusion, and the exclusion of outputs with fewer than two citations constrain regional and disciplinary generalization without quantitative bias bounds.
  - Dimension: methodological rigour
  - Location: Input data, pp. 832–833; Data limitations, pp. 841–842
- **Benefit language exceeds the design** The Abstract describes OA as a mechanism widening use, and the Conclusion treats diverse citing outputs as wider use, although the study identifies an observational association in a narrow proxy.
  - Dimension: reporting
  - Location: Abstract, final sentence; Conclusion, pp. 842–843

## Reliability checks

Caveats (4 of 4 checks).

- **Internal coherence:** Noted. The reported corpus, methods, and descriptive findings are internally consistent, and the paper explicitly disclaims causal identification. Some Abstract, Results, and Conclusion wording nevertheless shifts from association in citation-based proxies toward causal benefits and wider use.
- **Research conduct:** Noted. Funding and competing interests are disclosed, the study uses bibliographic metadata rather than human-subject data, and processed data and analysis code are claimed available through Zenodo and GitHub. One or more identifiers named in the paper's availability or registration statements did not resolve when checked on 2026-09-28T16:08:08.243051+00:00: https://​github.​com/​Curtin-​Open-​Knowl​edge-​Initi​ative/​citat​ion-​diver​sity. Recorded as a discrepancy between the paper's claims and the cited sources; not an assessment of the underlying research.
- **Reference integrity:** Clean. 21 of 21 checkable references verified
  - 25 references in manuscript
  - 4 are books, websites or datasets — counted, but not index-checkable
  - References not verified automatically are resolved manually within 24 hours.

## Post-publication record

- **Status:** Clean
- **Summary:** No retraction or expression of concern found in Crossref (includes Retraction Watch).
- **Last checked:** 2026-10-17
