nabu.science
MethodologyAboutContact

 

  • Snapshot
  • Key Claims
  • Strengths & Limitations
  • Commentary
  • From the Authors
  • Author Note / Field Feedback
  • Dimension breakdown
  • Impact Profile
Free · About 5 minutes
Report #NBU-4D074D12·Framework v5.9·Pipeline v0.4·Evaluated 28 Sept 2026·OECD: 5.8 Other social sciences·Subfield: Statistics, Probability and Uncertainty·Basis: Full text·Type: Cross-sectional study

Citing and reading behaviours in high-energy physics

Anne Gentil-Beccot et al.Anne Gentil-Beccot, Salvatore Mele, Travis C. Brooks·2009·Scientometrics·10.1007/s11192-009-0111-1 ↗

Primary claim

Articles submitted to an Open Access subject repository, arXiv, receive 5 times more citations than articles which are not.

Tentative · unadjusted comparison leaves acknowledged author and article selection as competing explanations
In brief

A valuable HEP-wide account of preprint citation and readership patterns whose causal conclusions exceed its uncontrolled observational design.

ReliabilityCaveats4 of 4 checks completeRead the entry ↓
Fragile — the claim's weakest requirement is that the effect exists at all. Based on how the study was designed and conducted, not on statistics extracted from the paper. Experimental. Every point below is traceable to a quoted passage, but this forecast has not yet been validated against known replication outcomes.
Abstract not available — view at DOI ↗

Key Claims

Primary finding · Tentative

Articles submitted to an Open Access subject repository, arXiv, receive 5 times more citations than articles which are not.

Fragile — the claim's weakest requirement is that the effect exists at all. Based on how the study was designed and conducted, not on statistics extracted from the paper. Experimental. Every point below is traceable to a quoted passage, but this forecast has not yet been validated against known replication outcomes.

unadjusted comparison leaves acknowledged author and article selection as competing explanations

Section 7, finding 1; Section 3, Figure 2
Quote verified
Secondary · Supported

The citation advantage of articles appearing in a repository is connected to their dissemination prior to publication, 20% of citations of HEP articles over a two-year period occur before publication.

time-aligned curves establish pre-publication accumulation but do not isolate dissemination as the causal mechanism

Section 7, finding 2; Section 4, Figure 4
Quote verified
Secondary · Supported

No discernable citation advantage can yet be observed in the statistically-limited sample of articles published in “gold” Open Access journals.

controlled institutional comparison supports the qualified null within a short and statistically limited sample

Section 7, finding 3; Section 5, Figure 5
Quote verified
Secondary · Supported

HEP scientists are between four and eight times more likely to download an article in its preprint form from arXiv rather than its final published version on a journal web site.

single-month SPIRES clickstream has unequal access paths and only approximates community-wide downloads

Section 7, finding 4; Section 6, Figure 6
Quote verified

Derived from the full evaluation — not a separate score.

Strengths · Limitations

Strengths

#contribution

Multiple evidence streams address one debate

Sections 3–6 combine citation comparisons, temporal accumulation curves, a gold-OA analysis, and clickstream behavior rather than relying on a single bibliometric outcome.

↳ Sections 3–6; Figures 2–6

#methodological rigour

Citation timing is directly characterized

Figures 3 and 4 align citations to publication time and show that arXiv papers already hold 20% of their two-year citations at publication.

↳ Section 4; Figures 3–4

#application pathway

Specific policy users are identified

The Introduction names librarians, policymakers, publishers, and scientists, while Section 7 connects the findings to SCOAP3.

↳ Introduction; Section 7

Limitations

#methodological rigour

Headline comparison leaves selection uncontrolled

Section 3 identifies influential adopters and selective posting of higher-quality work as alternative explanations but does not match, stratify, or adjust for them.

↳ Section 3, discussion after Figure 2

#reporting

Conclusions exceed observational identification

The Abstract says dissemination “creates” the citation advantage, and Section 7 attributes faster discourse to free circulation despite the unresolved selection explanation.

↳ Abstract; Section 7, general conclusion 1

#methodological rigour

Click paths complicate reading inference

SPIRES links directly to arXiv full text but commonly sends publisher clicks to a splash page, so Figure 6 does not isolate format preference from access friction.

↳ Section 6, Figure 6 and following discussion

Does this match your read? Add your assessment ↓

Nabu Commentary

The linked SPIRES and arXiv records support a broad descriptive account of how HEP papers accumulated citations and how SPIRES users selected access links. Figures 3 and 4 provide the strongest design fit by locating citations relative to publication and documenting substantial pre-publication accumulation. Figure 2 does not control the selection mechanisms explicitly discussed in Section 3, so the Abstract's claim that dissemination “creates” the advantage is not identified by the analysis. Section 5's institutional comparison and the balanced literature discussion add useful context, while Figure 6 remains sensitive to direct-full-text versus splash-page access paths.

Evaluate another paper

Free · About 5 minutes · Research preview

Author Note / Field Feedback

Nabu’s assessment, alongside the field’s view.

Are you an author of this paper?

Dimension breakdown

Evidence Score

Out of 5
012345
2.8±0.1

Contribution
3.8

The paper combines field-wide citation data, citation timing, a gold-OA comparison, and clickstream evidence to provide a meaningfully extended account of scholarly communication in HEP. The advance remains below the highest band because the gold-OA sample is statistically limited and the central citation comparison cannot identify causation.

Sections 3–6; Figures 2–6

The analysis of almost two decades of use of preprints and repositories in the HEP community provides unique evidence

Methodological Rigour
2.0

The temporal citation analyses and within-institution gold-OA comparison fit their descriptive questions, but the headline arXiv comparison does not adjust for the author and article selection mechanisms acknowledged by the paper. Causal conclusions and the clickstream reading inference consequently exceed what the observational design directly identifies.

Section 3 after Figure 2; Sections 4–6; Section 7

A more detailed analysis would be needed to ascertain these historical and sociological effects

Reporting
3.5

The question-to-analysis structure and four numbered findings make the argument easy to follow. Precision is reduced when the Abstract says dissemination “creates” the advantage and Section 7 says journals have lost their discourse role, although the evidence is associational and click-based.

Abstract; Sections 3–7

free and immediate online dissemination of preprints creates an immense citation advantage

Positioning
2.8

The paper engages both supportive and null Open Access studies across several fields and explicitly identifies selection as an alternative explanation. It then presents causal general conclusions without retaining that qualification, producing a material contradiction in limitation handling.

Section 3 after Figure 2; Section 5; Section 7

Several explanations for this observation are possible

Lower confidence on Positioning — domain match limited.

Reliability

Caveats4 of 4 checks


Internal CoherenceNoted

The numerical findings are mutually consistent, but the causal synthesis is stronger than the acknowledged uncertainty permits. The unresolved selection explanation in Section 3 is not carried into the Abstract or general conclusions.

Research ConductClean

No ethics, conflict, protocol, or suspicious availability concern is evident for the aggregate bibliometric and clickstream analyses. The absence of a replication package is consistent with the study's era and was not treated as a reliability defect.

Flags: 0 declared / 5 total

Reference IntegrityClean

17 of 17 checkable references verified

25 references in manuscript 8 have no canonical index record — counted, but not index-checkable 2 references confirmed by manual review

References not verified automatically are resolved manually within 24 hours.

Post-Publication RecordClean

No retraction or expression of concern found in Crossref (includes Retraction Watch).

Last checked: 17 Oct 2026

Sources: Crossref (incl. Retraction Watch) ✓

Impact Profile

Impact areaPolicy and practice · SDG 9
Type of advanceEmpirical
Route to useSuggested
AdvanceExtends
CitationsFWCI 7.6
Policy · patentNot yet traced
ReachedNot reachedNot assessable or unverified
ReadinessValidated
Foundational
Controlled
Piloted
Validated
Implementable
FoundationalImplementable

The primary claim rests on comprehensive real-world citation data covering the entire HEP literature across almost two decades, not a controlled experiment.

From the paper
“provides unique evidence to inform the Open Access debate”

The paper names a real-world use of its findings, informing the Open Access debate, but does not specify who would act on them or in what decision setting.

Pathway

Funders, university libraries and scientific publishers would use this evidence to shape Open Access mandates, repository policies and journal business models.

Barrier

The observational design cannot fully exclude selection effects, since authors choosing arXiv may differ systematically from those who do not.

Evaluated 13 Nov 2026 · GBD 2023 · glm_v1_3_3 · impact_profile_v1_3_3

Report an issue →

AI-generated, human-governed. Something look off? Contact us to request a review.

Keep reading

Related evaluations

Other papers Nabu has evaluated blind, starting with the same field.

  1. Suspected distortion of citations in high-impact cancer journals

    Baptiste Scancar, Jennifer Anne Byrne, David Causeur, et al. · 2026 · Media and communications

    Primary claim

    These findings suggest that paper mills inflate the citation metrics of supported publications and affected journals at scale.

    Tentative
    Evidence Score
    3.5±0.1
    Reliability
    Caveats
  2. The citation trap: Papers published at year-end receive systematically fewer citations

    Chao Ma, Yiwei Li, Feng Guo, et al. · 2019 · Economics

    Primary claim

    papers published between October and December on average get as much as 18.5% fewer citations than those published in the other months in the year

    Tentative
    Evidence Score
    3.0±0.2
    Reliability
    Caveats
  3. Quantitative Analysis of the Open Access Citation Advantage

    Masashi Kawai, Koichi Ojiro, Makoto Asaoka, et al. · 2022 · Other social sciences

    Primary claim

    日本の機関リポジトリによるオープンアクセス論文の数は限られるものの,その被引用数と様々なタイプのオープンアクセス論文の被引用数の比較からは,日本の機関リポジトリによるオープンアクセスのメリットは確認できなかった.

    Tentative
    Evidence Score
    2.9±0.2
    Reliability
    Caveats

How the rubric works →