Assembling the evidence…
Using an instrumental variable approach, we find no significant effect of open access.
reported 2SLS and GMM estimates are insignificant, though exclusion restrictions and precision limit causal certainty
Self-selection of higher quality articles into open access explains at least part of the observed open access citation advantage.
quality proxies predict open-access selection, but the causal decomposition remains conditional on instrument validity
regressing citations on open access and other controls yields upward-biased estimates.
the model predicts selection bias and the adjusted OLS coefficient substantially exceeds the IV estimates
the diffusion effect of open access is smaller than previously thought.
the comparison is directionally consistent but rests on imprecise, setting-specific IV estimates
Derived from the full evaluation — not a separate score.
Strengths
The model in §2 explains why higher-quality papers may disproportionately select into open access, giving a mechanism for the gap between raw citation advantages and randomized null findings.
↳ Introduction, pp. 2–3; §2
Table 2 presents OLS, first-stage, 2SLS, and GMM results together with an excluded-instrument F statistic and Hansen J test. Appendix B reports the instability and imprecision of single-instrument estimates rather than concealing them.
↳ §4, Table 2; Appendix B, Table 3
The introduction discusses both influential observational studies and contemporary randomized evidence, while Appendix A presents an additional randomized comparison from the Journal of Medical Genetics.
↳ Introduction, pp. 2–3; Appendix A
Limitations
Fiscal-year timing, NIH policy exposure, and HHMI affiliation are treated as affecting citations only through open access after controls, but the paper provides no placebo or negative-control analysis to probe those assumptions.
↳ §3.4
The author-productivity matching procedure removes 590 observations, mainly papers with Asian last-author names, without directly examining the resulting selection or generalizability consequences.
↳ §3.3, footnote 4
The principal estimate covers PNAS biology articles from May 2004 through March 2006, where subscription barriers and the optional fee may differ from other publishing environments.
↳ §3.1
Figure 1 reports 5,498 papers while the main analysis reports 4,388, and Appendix B contains malformed or duplicated table entries. These defects do not overturn Table 2 but reduce confidence in secondary checks.
↳ Figure 1 caption; §3.1; Appendix B, Table 3
The contribution score reflects a clear theoretical account of endogenous open-access choice and a meaningful comparison between OLS and IV estimates. Methodological Rigour is lower because the three instruments have plausible first stages but exclusion restrictions that are mainly asserted, while the common-surname exclusion is not analyzed for selection effects. Reporting remains strong because the paper distinguishes statistical non-significance from proof of no effect and explicitly states that a small effect cannot be ruled out. Positioning is also strong but stops short of fully tracing instrument sensitivity and the single-journal setting into the breadth of its policy advice.
Nabu’s assessment, alongside the field’s view.
Are you an author of this paper?
Sound3.6
Confidence mediumThe paper adds a formal selection mechanism and an IV-based attempt to distinguish diffusion from author self-selection, meaningfully extending earlier correlational and randomized evidence. The advance remains conditional on the instruments identifying the intended causal contrast.
“regressions of the number of citations on open access capture both a diffusion effect and a self-selection effect”
The analysis reports OLS, 2SLS, GMM, first-stage strength, an overidentification test, and alternative instrument specifications. The exclusion restrictions are defended mainly through plausibility, while the consequences of excluding 590 common-surname observations are not examined.
“we expect that the timing of publication within the year to have no relationship with article quality”
The paper progresses coherently from the selection model through data, instruments, estimates, and qualified conclusions. Its statement that a small effect cannot be ruled out is appropriately calibrated, although the stronger causal decomposition remains conditional on instrument validity.
“a small effect cannot be ruled out”
The discussion engages both prior observational findings and contemporaneous randomized counterevidence, while distinguishing citations from other possible welfare benefits. It gives less attention to how exclusion-restriction violations and the single-journal setting would alter the central inference.
“they do not quite settle the debate”
Caveats4 of 4 checks
The headline estimates in Table 2 are internally consistent, but a sample-size discrepancy in Figure 1 and malformed entries in Appendix B reduce auditability of secondary results. These issues do not appear to alter the principal multi-instrument estimates.
The bibliometric design does not require an ethics declaration, and the supplied text contains no identified conflict, registration, availability, or other conduct contradiction.
Flags: 0 declared / 5 total
15 of 15 checkable references verified
22 references in manuscript 7 have no canonical index record — counted, but not index-checkable 1 reference confirmed by manual review
No retraction notice found in Retraction Watch.
Sources: Retraction Watch ✓
Where this paper’s evidence sits on the path from initial observation to real-world use.
The findings arise from actual publication choices, citation records, and funder policies, making them relevant to real decisions. Readiness remains preliminary because the causal estimate is imprecise and depends on setting-specific instruments whose exclusion restrictions are not directly probed.
“the coefficient on open access decreases considerably and is and no longer significant”
AI-generated, human-governed. Something look off? Contact us to request a review.