Crunching the numbers. Responsibly.
We model open access as facilitating full‐text acquisition, which, while often increasing cites, can reduce cites from readers who refrain from citing superficially after realizing the article is not worth citing. We test the theory with data on over 200,000 science articles binned by cites in the pre‐study period. Consistent with theory, we find that opening access to an article on the journal's website has a “Matthew effect” on citations: negative for the least‐cited articles, positive for the most cited, and monotonic for quality levels in between. Estimates for broader open‐access platforms and for cites coming from insiders versus outsiders also follow patterns consistent with theory.
the marginal effect of open access is negative for the least-cited articles, positive for the most cited, and generally monotonic for quality levels in between
preferred and alternative bins preserve the broad gradient, despite low-bin non-monotonicity and mixed significance
PubMed access rotates the open-access effect, exaggerating the citation losses for low-quality content and the gains for high-quality content
extreme-bin interactions differ significantly, conditional on PubMed representing lower non-pecuniary access costs
the effect of open access on outsider cites is negative across virtually the entire quality spectrum
preferred-bin estimates broadly agree, but alternative binning produces opposing significant outsider effects
Derived from the full evaluation — not a separate score.
Strengths
The model separates citing from reading and derives quality-gradient, platform-convenience, and insider/outsider predictions in Propositions 1–5. This moves the argument beyond an average open-access effect.
↳ Theory §§3.1–3.5; Propositions 1–5
Equations (1)–(6), condition (5), Figure 1, and Appendix A make the maintained assumptions and proof cases explicit. The derivations cover both open-access cost regimes and endpoint behavior.
↳ Theory §§3.1–3.4; Appendix A
Tables 4, 5, and 7 separately examine the quality gradient, PubMed convenience, and insider/outsider citation sources. Figure 5 presents these patterns in a common visual format.
↳ Results §§7.1–7.3; Tables 4, 5, and 7; Figure 5
Limitations
Section 5 states that access timing is plausibly orthogonal after controls and illustrates this with selected journal histories. It does not provide a direct test that access changes are unrelated to unobserved citation shocks.
↳ Methodology §5; Figure 4
Table B4 reports a positive and significant outsider estimate in the 2–5-citation bin when all citations form the bins. This sensitivity is not reconciled with the broad outsider interpretation in §8.
↳ Appendix B, Table B4, Panel B; Conclusions §8
The data cover 100 journals centered on ecology and botany during 1996–2005. The study does not test other disciplines or contemporary repository and access practices.
↳ Data §4; Table 1; Appendix B, Table B1
The formal contribution is substantial because the model generates distinct predictions about article quality, platform convenience, and citation source, with complete proofs in Appendix A. The article-level panel design and pre-period bin construction support a broad quality gradient, while Tables 5 and 7 extend the test beyond the baseline specification. The quality score remains below the airtight band because conditional exogeneity is defended through plausibility rather than directly established, and §3.6 concedes that other mechanisms could produce similar patterns. Appendix B, Table B4 also weakens the breadth of the outsider-citation conclusion without overturning the main quality-gradient result.
Nabu’s assessment, alongside the field’s view.
Are you an author of this paper?
Strong4.1
Confidence mediumThe score reflects a clearly delineated mechanism separating citation from reading decisions and several empirically testable comparative statics. The observed patterns are not uniquely diagnostic because the paper identifies alternative mechanisms with similar implications.
“This is one possible mechanism behind a negative open-access effect, but other mechanisms might be possible having similar empirical implications.”
The formal assumptions, propositions, boundary cases, and proofs are explicit, while the empirical analysis uses article fixed effects, pre-period bins, and alternative bin structures. Causal interpretation still depends on conditional exogeneity argued from institutional timing rather than directly tested.
“Given the wealth of controls included in the conditional expectation, the nature of access to an article is plausibly orthogonal to the remaining error.”
The progression from model to predictions and corresponding empirical tests is readily followable, and departures from monotonicity are explicitly acknowledged. Claims that the results align closely with theory are tempered by mixed significance and the unreconciled outsider sensitivity.
“The one place where the results depart from the theory is in the non-monotonicity.”
The review distinguishes cross-sectional, experimental, instrumental-variable, panel, and theoretical antecedents and states the paper's extension relative to them. Limitations and alternative mechanisms are discussed, although the conclusion does not integrate the alternative outsider-binning result.
“The papers cited in this paragraph employ convincing empirical strategies for causal identification.”
Caveats4 of 4 checks
The principal estimates are reported consistently, and departures from predicted monotonicity are disclosed. However, an alternative binning analysis produces an outsider-citation result that is not reconciled with the broad conclusion about outsider effects.
The study uses observational bibliometric data for which human-subject ethics approval is not required, and its funding source is disclosed. The absence of preregistration and public data or code statements is era-typical and does not constitute a conduct concern.
Flags: 0 declared / 5 total
39 of 42 checkable references verified
44 references in manuscript 2 are books, websites or datasets — counted, but not index-checkable
No retraction notice found in Retraction Watch.
Sources: Retraction Watch ✓
Where this paper’s evidence sits on the path from initial observation to real-world use.
The mechanism is formalized and examined using a large real-world article panel, placing it beyond a purely conceptual stage. Contemporary application requires broader validation and clarification of whether citation effects are the relevant policy outcome.
“The resulting dataset from these two sources includes observations for over 230,000 individual cited articles.”
AI-generated, human-governed. Something look off? Contact us to request a review.