Crunching the numbers. Responsibly.
Abstract This article statistically analyzes how the citation impact of articles deposited in the Condensed Matter section of the preprint server ArXiv (hosted by Cornell University), and subsequently published in a scientific journal, compares to that of articles in the same journal that were not deposited in the archive. Its principal aim is to further illustrate and roughly estimate the effect of two factors, “early view” and “quality bias,” on differences in citation impact between these two sets of papers, using citation data from Thomson Scientific's Web of Science. It presents estimates for a number of journals in the field of condensed matter physics. To discriminate between an “open access” effect and an early view effect, longitudinal citation data were analyzed covering a time period as long as 7 years. Quality bias was measured by calculating ArXiv citation impact differentials at the level of individual authors publishing in a journal, taking into account coauthorship. The analysis provided evidence of a strong quality bias and early view effect. Correcting for these effects, there is in a sample of six condensed matter physics journals studied in detail no sign of a general “open access advantage” of papers deposited in ArXiv. The study does provide evidence that ArXiv accelerates citation due to the fact that ArXiv makes papers available earlier rather than makes them freely available.
Correcting for these effects, there is in a sample of six condensed matter physics journals studied in detail no sign of a general “open access advantage” of papers deposited in ArXiv.
within-author and later-window results reduce the differential, though paper-level selective deposition remains unmeasured
The analysis provided evidence of a strong quality bias and early view effect.
author-level CID reductions and lag-adjusted citation curves support both patterns with residual selection limitations
The study does provide evidence that ArXiv accelerates citation due to the fact that ArXiv makes papers available earlier rather than makes them freely available.
causal attribution to earlier availability is not isolated by the observational curve-shift analysis
Top authors in terms of average citation impact per nondeposited paper are overrepresented in ArXiv.
journal-level quartile comparisons consistently show greater ArXiv representation among highly cited authors
Derived from the full evaluation — not a separate score.
Strengths
CID is calculated for authors publishing both deposited and nondeposited papers, and Table 2 shows substantial reductions from journal-level differentials. This directly tests whether prominent authors are overrepresented among ArXiv papers.
↳ Quality Bias section; Figures 4–5; Table 2
Figures 2 and 3 compare monthly citation-age distributions and shift the deposited-paper curve by the six-month median publication lag. Fixed first-to-third-year and fourth-to-sixth-year windows then show declining citation differentials.
↳ Early View Effect section; Figures 2–3; Table 1
The paper compares its metric and interpretation with Harnad and Brody, Kurtz and colleagues, Davis and Fromerth, and Eysenbach rather than presenting the open-access literature one-sidedly.
↳ Introduction, pp. 2047–2048; Discussion and Conclusions, pp. 2053–2054
Limitations
The author-level analysis addresses who deposits papers but not whether authors selectively deposit their better work. This limits the claim that quality bias has been controlled.
↳ Discussion and Conclusions, p. 2054
The shifted citation curves are consistent with earlier exposure, but they do not isolate earlier availability as the cause of acceleration or distinguish it conclusively from remaining selection mechanisms.
↳ Abstract; Figures 2–3; Discussion and Conclusions, pp. 2053–2054
Only about 75% of ArXiv-CM records were linked to Web of Science using author names, title words, and incomplete journal-reference fields. The possible error rate and direction of resulting bias are not quantified.
↳ Data and Methods, pp. 2048–2049
The CID construction, seven-year windows, and author-level comparisons provide a substantive methodological refinement over simple aggregate citation ratios. Figures 2–3 and Tables 1–2 support early-view patterns and reductions in citation differentials after accounting for author prominence. The strongest conclusions remain constrained because authors’ selective deposition of better papers was not examined, and the ArXiv-to-WoS matching procedure was not validated through sensitivity analysis. The causal wording about why ArXiv accelerates citations therefore receives less weight than the descriptive evidence of shifted citation timing.
Nabu’s assessment, alongside the field’s view.
Are you an author of this paper?
Sound3.1
Confidence mediumThe CID metric, seven-year citation windows, and author-level comparisons meaningfully extend earlier analyses of ArXiv citation differentials. The advance remains a bounded condensed-matter case study and does not resolve paper-level selective deposition.
“a measure that is similar to Harnad and Brody’s ‘open access to non-OA impact ratios’ but more appropriate”
Fixed citation windows and within-author CID comparisons address temporality and author prominence, but the design does not isolate the claimed causal mechanism or authors’ selection of their better papers. The 75% database-linkage rate also lacks validation or directional sensitivity analysis.
“tend to deposit their ‘better’ papers in ArXiv had not been examined”
The paper progresses coherently from data linkage and CID definition through early-view and quality-bias analyses. The assertion that ArXiv accelerates citation because of earlier availability is more causal than the observational comparisons establish.
“this acceleration is due to the fact that ArXiv makes papers available earlier”
The analysis directly engages Harnad and Brody, Kurtz and colleagues, Davis and Fromerth, and Eysenbach, including findings that complicate its interpretation. However, claiming to control quality bias conflicts with the later acknowledgment that paper-level quality selection was not examined.
“A second dimension... had not been examined and awaits further research.”
Caveats4 of 4 checks
Reported counts and analyses are coherent, but the specific mechanism claimed for citation acceleration is stronger than the observational design directly identifies.
The named bibliometric data sources raise no participant-ethics requirement, and no suspicious availability or conflict claim appears in the text.
Flags: 1 declared / 5 total
6 of 8 checkable references verified
8 references in manuscript
No retraction notice found in Retraction Watch.
Sources: Retraction Watch ✓
Where this paper’s evidence sits on the path from initial observation to real-world use.
The evidence comes from actual deposit, publication, and citation behavior, making it relevant to scholarly communication practice. Its observational mechanism claims and lack of implementation guidance keep it from being directly actionable.
AI-generated, human-governed. Something look off? Contact us to request a review.