nabu.science
MethodologyAboutContact

 

  • Snapshot
  • Key Claims
  • Strengths & Limitations
  • Commentary
  • From the Authors
  • Author Note / Field Feedback
  • Dimension breakdown
  • Impact Profile
Free · About 5 minutes
Report #NBU-8623D397·Framework v5.9·Pipeline v0.4·Evaluated 28 Sept 2026·OECD: 5.8 Other social sciences·Subfield: Information Systems and Management·Basis: Full text·Type: Cross-sectional study

The relationship between bioRxiv preprints, citations and altmetrics

Nicholas Fraser et al.Nicholas Fraser, Fakhri Momeni, Philipp Mayr et al.·2020·Quantitative Science Studies·10.1162/qss_a_00043 ↗

Primary claim

bioRxiv-deposited journal articles had sizably higher citation and altmetric counts compared to nondeposited articles.

Supported · matched observational comparisons support the association, but unmeasured self-selection prevents stronger attribution
In brief

A careful matched bibliometric study establishes a persistent bioRxiv visibility advantage, but unresolved author self-selection prevents causal attribution.

ReliabilityCaveats4 of 4 checks completeRead the entry ↓
Possible — the claim's weakest requirement is that the effect exists at all. Based on how the study was designed and conducted, not on statistics extracted from the paper. Experimental. Every point below is traceable to a quoted passage, but this forecast has not yet been validated against known replication outcomes.
Abstract

Abstract A potential motivation for scientists to deposit their scientific work as preprints is to enhance its citation or social impact. In this study we assessed the citation and altmetric advantage of bioRxiv, a preprint server for the biological sciences. We retrieved metadata of all bioRxiv preprints deposited between November 2013 and December 2017, and matched them to articles that were subsequently published in peer-reviewed journals. Citation data from Scopus and altmetric data from Altmetric.com were used to compare citation and online sharing behavior of bioRxiv preprints, their related journal articles, and nondeposited articles published in the same journals. We found that bioRxiv-deposited journal articles had sizably higher citation and altmetric counts compared to nondeposited articles. Regression analysis reveals that this advantage is not explained by multiple explanatory variables related to the articles’ publication venues and authorship. Further research will be required to establish whether such an effect is causal in nature. bioRxiv preprints themselves are being directly cited in journal articles, regardless of whether the preprint has subsequently been published in a journal. bioRxiv preprints are also shared widely on Twitter and in blogs, but remain relatively scarce in mainstream media and Wikipedia articles, in comparison to peer-reviewed journal articles.

Key Claims

Primary finding · Supported

bioRxiv-deposited journal articles had sizably higher citation and altmetric counts compared to nondeposited articles.

Possible — the claim's weakest requirement is that the effect exists at all. Based on how the study was designed and conducted, not on statistics extracted from the paper. Experimental. Every point below is traceable to a quoted passage, but this forecast has not yet been validated against known replication outcomes.

matched observational comparisons support the association, but unmeasured self-selection prevents stronger attribution

Abstract; Results §§3.2.1, 3.3–3.4
Quote verified
Secondary · Well-supported

the strength of the citation advantage increases over time: At 12 months postpublication, articles deposited to bioRxiv cumulatively receive 1.62 times more citations than those not deposited, which increases to 1.74 times more citations by 36 months postpublication.

monthly trajectories and the citation-interval interaction directly support the observed 36-month pattern

Results §3.4
Quote verified
Secondary · Well-supported

bioRxiv preprints themselves are being directly cited in journal articles, regardless of whether the preprint has subsequently been published in a journal.

direct citation records include both subsequently published and unpublished preprints

Abstract; Results §3.2.2
Quote verified
Secondary · Well-supported

bioRxiv preprints are also shared widely on Twitter and in blogs, but remain relatively scarce in mainstream media and Wikipedia articles, in comparison to peer-reviewed journal articles.

indicator-specific Altmetric.com comparisons directly show contrasting coverage across sharing channels

Abstract; Results §3.3, Figure 6
Quote verified
Secondary · Supported

this advantage is not explained by multiple explanatory variables related to the articles’ publication venues and authorship.

adjusted models retain the association, though measured covariates do not exhaust plausible confounding

Abstract; Results §3.4, Table 3
Quote verified

Derived from the full evaluation — not a separate score.

Strengths · Limitations

Strengths

#contribution

Clear extension of prior bioRxiv evidence

The Introduction delineates four additions to earlier studies: longitudinal citation analysis, direct preprint citations, indicator-specific altmetrics, and regression adjustment. This establishes a concrete incremental contribution rather than a first-study claim.

↳ Introduction, final two paragraphs

#methodological rigour

Matched modeling suits the association question

Articles are paired within journal and publication month, with subject-category matching for multidisciplinary journals. Negative-binomial models include matched-pair random effects and observed venue and authorship covariates.

↳ Methods §§2.2 and 2.6

#reporting

Causal limits remain explicit throughout

The Abstract, Introduction, Results §3.4, and Conclusions repeatedly distinguish the observed advantage from a demonstrated causal effect. This qualification is linked to acknowledged self-selection and unmeasured quality.

↳ Abstract; Introduction; Results §3.4; Conclusions

Limitations

#methodological rigour

Self-selection remains causally unresolved

Matching and regression address observed differences but cannot determine whether authors preferentially deposit stronger or more visible work. The impact-factor interaction is an indirect proxy rather than a direct sensitivity analysis for this threat.

↳ Methods §2.6; Results §3.4; Conclusions

#methodological rigour

Publication linkage lacks bias analysis

Only 12,755 of 18,841 preprints were linked to journal articles, and the unmatched fraction is not examined for systematic differences. This limits interpretation of analyses centered on subsequently published articles.

↳ Methods §2.1; Results §3.1

#application pathway

Generalization is platform and period bound

The analysis treats bioRxiv as a whole, does not decompose individual biological subject areas, and follows a 2013–2017 cohort for no more than 36 months. Transfer to other repositories or mature preprint environments therefore remains uncertain.

↳ Methods §§2.1–2.4; Results §3.4

Does this match your read? Add your assessment ↓

Nabu Commentary

The score reflects a clear incremental advance over earlier bioRxiv studies, especially the longitudinal citation analysis and separate treatment of altmetric channels. Journal-month matching, matched-pair random effects, and negative-binomial models support the reported associations, while §3.4 appropriately acknowledges unmeasured quality and author selection. The paper's causal language is generally disciplined, although the Conclusions move toward practical encouragement and suggest access as a possible driver without identifying that mechanism. Impact potential is therefore moderate: the findings are relevant to preprint policy and author behavior but are not causal evidence of the return to depositing a preprint.

Evaluate another paper

Free · About 5 minutes · Research preview

Author Note / Field Feedback

Nabu’s assessment, alongside the field’s view.

Are you an author of this paper?

Dimension breakdown

Evidence Score

Out of 5
012345
3.8±0.1

Contribution
3.7

The study meaningfully extends earlier bioRxiv comparisons through longitudinal citation trajectories, direct preprint citations, separate altmetric indicators, and adjusted matched-pair models. The advance is incremental and associational rather than causal.

Introduction, final two paragraphs; Results §§3.2–3.4

Our study builds on the initial work of Serghiou and Ioannidis (2018) and Fu and Hughey (2019) in several ways

Methodological Rigour
3.6

Journal-month matching, multidisciplinary category matching, negative-binomial mixed models, and interaction tests provide a credible association-focused design. Unmeasured article quality and author selection remain unresolved despite adjustment for observed venue and authorship factors.

Methods §§2.2 and 2.6; Results §3.4

cannot account for as-yet-unmeasurable variables, such as an article’s exact quality or novelty

Reporting
4.3

The argument proceeds from matched descriptive comparisons to adjusted models and consistently distinguishes association from causation. The closing encouragement to deposit preprints is somewhat more practically suggestive than the design warrants.

Abstract; Results §3.4; Conclusions

Further research will be required to establish whether such an effect is causal in nature.

Positioning
4.0

The paper distinguishes its contribution from the closest bioRxiv studies and directly discusses conflicting longitudinal evidence from arXiv. Limitations concerning self-selection, subject-area generalizability, and unmeasured quality are carried into the causal qualification, although access is still suggested as a possible driver.

Introduction; Results §§3.2.1 and 3.4; Conclusions

More research is needed to establish the exact cause of the citation and altmetric advantage.

Lower confidence on Methodological Rigour, Positioning — domain match limited.

Reliability

Caveats4 of 4 checks


Internal CoherenceNoted

The reported sample construction and principal findings are internally consistent, but main-text presentation of the longest citation windows is incomplete. The issue does not materially change interpretation of the central association.

Research ConductClean

Competing interests and funding are disclosed, and data and code are claimed available through a persistent Zenodo DOI. Ethics approval is not applicable to the bibliographic and platform metadata design.

Flags: 3 declared / 5 total

Reference IntegrityClean

37 of 37 checkable references verified

43 references in manuscript 5 have no canonical index record — counted, but not index-checkable 1 could not be checked — not the same as no match found 1 reference confirmed by manual review

References not verified automatically are resolved manually within 24 hours.

Post-Publication RecordClean

No retraction or expression of concern found in Crossref (includes Retraction Watch).

Last checked: 17 Oct 2026

Sources: Crossref (incl. Retraction Watch) ✓

Impact Profile

Impact areaPolicy and practice · SDG 9
Type of advanceEmpirical
Route to useNamed
AdvanceExtends
CitationsFWCI 37.3
Policy · patentNot yet traced
ReachedNot reachedNot assessable or unverified
ReadinessValidated
Foundational
Controlled
Piloted
Validated
Implementable
FoundationalImplementable

The primary claim is tested on comprehensive real-world population data covering all bioRxiv preprints over four years with matched controls and regression, independently confirming prior studies that used different citation data sources.

From the paper
“This may help to motivate and encourage authors, some of whom remain skeptical of preprint servers, to publish their work earlier”

The conclusion names authors as the actors who would use these findings to decide to deposit preprints earlier, though without specifying a concrete decision setting.

Pathway

Researchers, funders, and journal policy makers could cite this evidence to encourage preprint deposition, shifting scholarly communication practice toward earlier open sharing.

Barrier

The authors explicitly state causality is unestablished, since unmeasured author self-selection of highest-quality articles cannot be ruled out as the driver of the advantage.

Evaluated 13 Nov 2026 · GBD 2023 · glm_v1_3_3 · impact_profile_v1_3_3

Report an issue →

AI-generated, human-governed. Something look off? Contact us to request a review.

Keep reading

Related evaluations

Other papers Nabu has evaluated blind, starting with the same field.

  1. Discovery Stack Pilot demonstrates the feasibility and outcomes of a scientist-designed peer-review model that separates quality and impact

    McGargill, Liu, Kuhns, et al. · 2026 · Other social sciences

    Primary claim

    Reviewers effectively evaluated Quality and Impact as separate dimensions, with Quality scores being more consistent across reviewers than Impact scores.

    Tentative
    Evidence Score
    3.4±0.2
    Reliability
    Concerns
  2. Perceived Risk and Barriers to Open and Responsible Research Across Fifteen UK Universities

    Noémie Aubert Bonn, Andrew J. Stewart, Lukas Hughes-Noehrer · 2026 · Other social sciences

    Primary claim

    Lack of time and resources was the most frequently reported barrier.

    Supported
    Evidence Score
    3.4±0.2
    Reliability
    Caveats
  3. Pediatric Orthopaedic Studies Published Open Access are Associated with Increased Citation Rates

    Sidharthan, Wang, Nichols, et al. · 2024 · Surgery

    Primary claim

    Open access publication in pediatric orthopaedics confers an advantage of 3 extra citations over a decade, on average.

    Tentative
    Evidence Score
    2.9±0.1
    Reliability
    Caveats

How the rubric works →