Loading evaluation data…
articles deposited in the arXiv received 35% more citations on average than non-deposited articles (an advantage of about 1.1 citations per article)
the unadjusted difference is consistent across yearly comparisons, but deposited and non-deposited articles differ through unresolved self-selection
this difference was most pronounced for highly-cited articles
Table 3 shows greater arXiv representation above five citations, though article age and selection limit interpretation
There was little support for a universal Open Access explanation, and no empirical support for Early View. There was some inferential support for a Quality Differential
one regression and exploratory download-citation analyses cannot comprehensively distinguish the three explanatory mechanisms
arXiv-deposited articles received 23% fewer downloads from the publisher’s website (about 10 fewer downloads per article) in all but the most recent two years after publication
Table 2 supports the observed difference, but historical download coverage begins only in 2003
the arXiv and the publisher’s website may be fulfilling distinct functional needs of the reader
the functional-needs mechanism is a tentative interpretation without direct reader-level evidence
Derived from the full evaluation — not a separate score.
Strengths
The study examines citation advantage and publisher-site usage together across 2,765 articles, providing a distinctive view of how arXiv availability relates to two different outcomes.
↳ Abstract; Results, Tables 1–3
The Early View postulate is tested through regression of citations on prepublication days while controlling for publication age and journal, with the null coefficient and p-value reported.
↳ Discussion, Early View Postulate, p. 6; Figure 5
The paper compares its findings with studies by Lawrence, Kurtz, Harnad and Brody, Moed, Wren, and others rather than treating the open-access explanation as settled.
↳ Introduction, pp. 2–3; Discussion, pp. 5–7
Limitations
Deposited and non-deposited articles may differ in author prominence, article quality, topic, and readership. The Quality Differential discussion infers selection from converging patterns but does not adjust for those differences.
↳ Discussion, Quality Differential Postulate, pp. 6–7
The paper calls the arXiv copy clearly the most plausible cause of reduced publisher downloads, although the group comparison cannot exclude unmeasured readership or access differences.
↳ Discussion, Explanation for the reduction in downloads, pp. 7–8
The analysis covers four journals from one mathematics society, with incomplete historical download coverage and no multi-publisher comparison. The Generalizability section appropriately calls for broader confirmation.
↳ Data and Methods, p. 3; Generalizability and further study, p. 8
The descriptive citation and publisher-download differences are numerically consistent across the Abstract, Results, and Tables 1–3. The methodology includes several suitable exploratory analyses, particularly the regression test of prepublication timing, but it does not control the self-selection that distinguishes deposited from non-deposited articles. The article’s generally careful postulate-based structure supports its clarity and contextual scores, while the emphatic causal explanation for reduced publisher downloads lowers confidence in its interpretation. Its relevance to scholarly-communication policy is recognizable, but the four-journal sample and lack of an operational intervention limit translation and transferability.
Nabu’s assessment, alongside the field’s view.
Are you an author of this paper?
Sound3.0
Confidence highThe study extends established open-access citation research to 2,765 articles in four mathematics journals and jointly examines citations and publisher downloads. The advance is meaningful but incremental and narrowly scoped.
articles deposited in the arXiv received 35% more citations on average than non-deposited articles
The paper transparently uses nonparametric comparisons, regression, correlation, and LOESS, but the deposited and non-deposited groups are not credibly adjusted for quality, readership, or author-level self-selection. The strong substitution explanation for publisher downloads adds causal overreach to the uncontrolled confounding.
Clearly, the most plausible explanation for the decrease is the presence a nearly-identical article in the arXiv.
The progression from descriptive findings through the three explanatory postulates is easy to follow, and much of the abstract is carefully qualified. Reporting is reduced by the shift from tentative framing to emphatic causal language in the publisher-download discussion.
There was some inferential support for a Quality Differential
The paper engages directly with the principal open-access, Early View, and self-selection literature and contrasts its results with prior astrophysics findings. Generalizability and measurement limitations are integrated, although their implications are not consistently carried into the download explanation.
Because of the limited sample size, care should be taken not to generalize very broadly.
Lower confidence on Reporting, Positioning — domain match limited.
Caveats4 of 4 checks
The reported totals, percentages, yearly comparisons, and methods-results links reconcile across the Abstract, Tables 1–3, and Results. Concerns about causal interpretation affect methodological quality rather than internal coherence.
The authors’ arXiv and Cornell roles are visible, but the paper does not explicitly frame the associated institutional interest as a conflict of interest. Proprietary data and absent code availability are assessability gaps and do not independently elevate reliability severity.
Flags: 0 declared / 5 total
9 of 14 checkable references verified
15 references in manuscript 1 are books, websites or datasets — counted, but not index-checkable
No retraction notice found in Retraction Watch.
Sources: Retraction Watch ✓
Where this paper’s evidence sits on the path from initial observation to real-world use.
The study uses real publisher and citation data, but it evaluates observational associations rather than an intervention or operational policy. It offers hypotheses and follow-up studies rather than implementation guidance.
Qualitative studies of readers and their rationale for choosing either the arXiv or the publisher’s website may help
AI-generated, human-governed. Something look off? Contact us to request a review.