Crunching the numbers. Responsibly.
Abstract A potential motivation for scientists to deposit their scientific work as preprints is to enhance its citation or social impact. In this study we assessed the citation and altmetric advantage of bioRxiv, a preprint server for the biological sciences. We retrieved metadata of all bioRxiv preprints deposited between November 2013 and December 2017, and matched them to articles that were subsequently published in peer-reviewed journals. Citation data from Scopus and altmetric data from Altmetric.com were used to compare citation and online sharing behavior of bioRxiv preprints, their related journal articles, and nondeposited articles published in the same journals. We found that bioRxiv-deposited journal articles had sizably higher citation and altmetric counts compared to nondeposited articles. Regression analysis reveals that this advantage is not explained by multiple explanatory variables related to the articles’ publication venues and authorship. Further research will be required to establish whether such an effect is causal in nature. bioRxiv preprints themselves are being directly cited in journal articles, regardless of whether the preprint has subsequently been published in a journal. bioRxiv preprints are also shared widely on Twitter and in blogs, but remain relatively scarce in mainstream media and Wikipedia articles, in comparison to peer-reviewed journal articles.
bioRxiv-deposited journal articles had sizably higher citation and altmetric counts compared to nondeposited articles.
matched observational comparisons support the association, but unmeasured self-selection prevents stronger attribution
the strength of the citation advantage increases over time: At 12 months postpublication, articles deposited to bioRxiv cumulatively receive 1.62 times more citations than those not deposited, which increases to 1.74 times more citations by 36 months postpublication.
monthly trajectories and the citation-interval interaction directly support the observed 36-month pattern
bioRxiv preprints themselves are being directly cited in journal articles, regardless of whether the preprint has subsequently been published in a journal.
direct citation records include both subsequently published and unpublished preprints
bioRxiv preprints are also shared widely on Twitter and in blogs, but remain relatively scarce in mainstream media and Wikipedia articles, in comparison to peer-reviewed journal articles.
indicator-specific Altmetric.com comparisons directly show contrasting coverage across sharing channels
this advantage is not explained by multiple explanatory variables related to the articles’ publication venues and authorship.
adjusted models retain the association, though measured covariates do not exhaust plausible confounding
Derived from the full evaluation — not a separate score.
Strengths
The Introduction delineates four additions to earlier studies: longitudinal citation analysis, direct preprint citations, indicator-specific altmetrics, and regression adjustment. This establishes a concrete incremental contribution rather than a first-study claim.
↳ Introduction, final two paragraphs
Articles are paired within journal and publication month, with subject-category matching for multidisciplinary journals. Negative-binomial models include matched-pair random effects and observed venue and authorship covariates.
↳ Methods §§2.2 and 2.6
The Abstract, Introduction, Results §3.4, and Conclusions repeatedly distinguish the observed advantage from a demonstrated causal effect. This qualification is linked to acknowledged self-selection and unmeasured quality.
↳ Abstract; Introduction; Results §3.4; Conclusions
Limitations
Matching and regression address observed differences but cannot determine whether authors preferentially deposit stronger or more visible work. The impact-factor interaction is an indirect proxy rather than a direct sensitivity analysis for this threat.
↳ Methods §2.6; Results §3.4; Conclusions
Only 12,755 of 18,841 preprints were linked to journal articles, and the unmatched fraction is not examined for systematic differences. This limits interpretation of analyses centered on subsequently published articles.
↳ Methods §2.1; Results §3.1
The analysis treats bioRxiv as a whole, does not decompose individual biological subject areas, and follows a 2013–2017 cohort for no more than 36 months. Transfer to other repositories or mature preprint environments therefore remains uncertain.
↳ Methods §§2.1–2.4; Results §3.4
The score reflects a clear incremental advance over earlier bioRxiv studies, especially the longitudinal citation analysis and separate treatment of altmetric channels. Journal-month matching, matched-pair random effects, and negative-binomial models support the reported associations, while §3.4 appropriately acknowledges unmeasured quality and author selection. The paper's causal language is generally disciplined, although the Conclusions move toward practical encouragement and suggest access as a possible driver without identifying that mechanism. Impact potential is therefore moderate: the findings are relevant to preprint policy and author behavior but are not causal evidence of the return to depositing a preprint.
Nabu’s assessment, alongside the field’s view.
Are you an author of this paper?
Strong3.8
Confidence highThe study meaningfully extends earlier bioRxiv comparisons through longitudinal citation trajectories, direct preprint citations, separate altmetric indicators, and adjusted matched-pair models. The advance is incremental and associational rather than causal.
Our study builds on the initial work of Serghiou and Ioannidis (2018) and Fu and Hughey (2019) in several ways
Journal-month matching, multidisciplinary category matching, negative-binomial mixed models, and interaction tests provide a credible association-focused design. Unmeasured article quality and author selection remain unresolved despite adjustment for observed venue and authorship factors.
cannot account for as-yet-unmeasurable variables, such as an article’s exact quality or novelty
The argument proceeds from matched descriptive comparisons to adjusted models and consistently distinguishes association from causation. The closing encouragement to deposit preprints is somewhat more practically suggestive than the design warrants.
Further research will be required to establish whether such an effect is causal in nature.
The paper distinguishes its contribution from the closest bioRxiv studies and directly discusses conflicting longitudinal evidence from arXiv. Limitations concerning self-selection, subject-area generalizability, and unmeasured quality are carried into the causal qualification, although access is still suggested as a possible driver.
More research is needed to establish the exact cause of the citation and altmetric advantage.
Lower confidence on Methodological Rigour, Positioning — domain match limited.
Caveats4 of 4 checks
The reported sample construction and principal findings are internally consistent, but main-text presentation of the longest citation windows is incomplete. The issue does not materially change interpretation of the central association.
Competing interests and funding are disclosed, and data and code are claimed available through a persistent Zenodo DOI. Ethics approval is not applicable to the bibliographic and platform metadata design.
Flags: 3 declared / 5 total
37 of 38 checkable references verified
43 references in manuscript 4 are books, websites or datasets — counted, but not index-checkable 1 could not be checked — not the same as no match found
No retraction notice found in Retraction Watch.
Sources: Retraction Watch ✓
Where this paper’s evidence sits on the path from initial observation to real-world use.
The evidence comes from actual publication, citation, and online-attention systems, making it relevant to current practice. Readiness remains limited because the study cannot identify whether preprint deposition itself causes the observed advantage.
More research is needed to establish the exact cause
AI-generated, human-governed. Something look off? Contact us to request a review.