Assembling the evidence…
We analyze the effect of open access (OA) status of journal articles on citations. Using cross‐sectional and panel data from mathematics and economics, we perform negative binomial, Poisson, and generalized method of moments/instrumental variable methods regressions. We benefit from a natural experiment via hybrid OA pilot agreements. Citations to pre‐prints allow us to identify the intrinsic quality of articles prior to journal publication. Overall, our analysis suggests that there is no hybrid OA citation benefit. However, for the subpopulation of articles without OA pre‐ or post‐prints, we find positive hybrid OA effects for the full sample and each discipline separately. (JELL17, O33, A11)
Overall, our analysis suggests that there is no hybrid OA citation benefit.
consistent insignificant estimates across model families, tempered by imprecision and residual institution-level selection
For the subpopulation of articles without OA pre- or post-prints, we find positive hybrid OA effects for the full sample and each discipline separately.
positive subgroup estimates recur across disciplines, though two results are significant only at the 10% level
We provide empirical evidence for a positive and statistically significant correlation between cites to OA pre-prints and net cites to journal articles.
the explicitly correlational result is significant across model and source robustness specifications
Our analysis of the full sample suggests that a $3,000 investment in HOA can never be profitable for a researcher who values an additional cite less than $3,278
the illustrative threshold inherits coefficient uncertainty and residual institution-level selection from the underlying models
OA pre-prints and/or post-prints that are typically free of charge for authors and readers are good substitutes for paid journal OA.
subgroup coefficient differences support substitution, but the claim derives from observational comparisons
Derived from the full evaluation — not a separate score.
Strengths
Cross-sectional count models are complemented by a 403-article panel, quality controls, same-issue comparisons, and IV/GMM analyses. This triangulation tests whether the main pattern persists under materially different specifications.
↳ Sections III–V; Tables 2, 9, and 10
The study directly examines whether OA preprints and postprints reduce the citation value of paid journal OA, a question positioned as unresolved in the closest prior literature.
↳ Introduction, pp. 3–4; Tables 3 and 6
Table 7 translates citation estimates into probabilities and value thresholds tied to a $3,000 fee, making the application pathway visible to authors and funders.
↳ Table 7; Conclusion, p. 23
Limitations
The first-stage IV results indicate that better-ranked institutions were more likely to participate in the pilot. The paper addresses this through IV and panel analyses, but the preferred results do not eliminate uncertainty from this selection mechanism.
↳ Section V.C, p. 23; Table S6
Only 50 of 268 contacted authors responded to the survey used to argue that pilot agreements were not common knowledge. The 18.7% response rate limits how firmly author-driven journal-choice selection can be ruled out.
↳ Section III.A, pp. 7–8
The evidence concerns economics and interdisciplinary mathematics during 2006–2012 and estimates the citation benefit for Springer journals. The conclusion itself anticipates different effects under other publishers' access arrangements.
↳ Section II.B; Conclusion, p. 23
The study's contribution rests on combining a hybrid-OA pilot setting with an explicit test of substitution by preprints and postprints. Tables 2, 9, and 10 show that the overall HOA estimate becomes small and statistically insignificant in more fully specified models, while Tables 3 and 6 identify a qualified positive pattern where no alternative OA version exists. The methods are strengthened by panel and IV/GMM analyses, but Section V.C documents selection of higher-ranked institutions into the pilot and Section III.A relies on a survey with an 18.7% response rate. Those limitations support a competent overall assessment while keeping the causal-isolation and investment claims below the strongest quality band.
Nabu’s assessment, alongside the field’s view.
Are you an author of this paper?
Sound3.6
Confidence highThe paper extends the OA-citation literature by examining hybrid OA alongside the availability of preprints and postprints in a pilot-agreement setting. The advance is meaningful but bounded by two disciplines and predominantly Springer journals.
“The present paper examines these substitution effects.”
The analysis combines cross-sectional count models, a 403-article panel, article and institution controls, and IV/GMM robustness analyses. Institution-level selection, an 18.7% survey response rate, and an exclusion restriction supported mainly by argument leave meaningful uncertainty around causal isolation.
“there might be a concern that institutions self-select into the HOA pilot program.”
The hypotheses, specifications, and corresponding tables are sequenced clearly, and the abstract preserves the no-preprint subgroup qualification. The conclusion's claim to isolate both selection channels and its categorical investment language are less calibrated than the surrounding methodological caveats.
“Our approach allows us to isolate the OA effect”
The paper substantively contrasts early large OA-advantage estimates with later selection-focused studies and positions its analysis against the closest panel work. It also identifies publisher specificity and institution-level selection as boundaries, although the final causal framing does not fully absorb the latter.
“This calls for further research on publisher-specific OA citation effects.”
Lower confidence on Methodological Rigour, Reporting — domain match limited.
Caveats4 of 4 checks
Sample counts, estimates, significance descriptions, and conclusions reconcile across the text and tables. The disclosed survey and selection limitations affect inferential strength but do not create an internal contradiction.
The funder's non-involvement is disclosed, and no conduct issue material to this bibliometric observational study is evident. Supporting information is referenced, while the absence of an explicit data or code availability statement is treated as an assessability gap rather than a concern.
Flags: 0 declared / 5 total
32 of 34 checkable references verified
40 references in manuscript 6 are books, websites or datasets — counted, but not index-checkable
No retraction notice found in Retraction Watch.
Sources: Retraction Watch ✓
Where this paper’s evidence sits on the path from initial observation to real-world use.
Actual pilot agreements and observed citation outcomes place the evidence in real-world conditions, and Table 7 offers an illustrative decision aid. It is not deployment-ready guidance because the causal estimates and scope remain conditional.
“This analysis is intended to be illustrative only.”
AI-generated, human-governed. Something look off? Contact us to request a review.