nabu.science
MethodologyAboutContact

 

  • Snapshot
  • Key Claims
  • Strengths & Limitations
  • Commentary
  • From the Authors
  • Author Note / Field Feedback
  • Dimension breakdown
  • Readiness
Report #NBU-F3B8EC4A·Framework v5.9·Pipeline v0.4·Evaluated 11 Sept 2026·OECD: 5.8 Media and communications·Subfield: Statistics, Probability and Uncertainty·Basis: Full text·Type: Cross-sectional study

On the culture of open access: the Sci-hub paradox

Maddi et al.Maddi, Sapinho·2023·Scientometrics·10.1007/s11192-023-04792-5 ↗

Primary claim

Open Access Citation Advantage is altered when non-OA articles available in Sci-hub are stratified separately from non-OA articles not in Sci-hub.

Tentative · descriptive Sci-Hub stratification without demonstrated confounding control or visible classification methods
In brief

Sci-Hub stratification adds a useful dimension to open-access citation analysis, but uncontrolled confounding limits interpretation of the reported advantages.

ReliabilityNo concerns3 of 4 checks completeSee what was checked ↓
Fragile — the claim's weakest requirement is that the effect exists at all. Based on how the study was designed and conducted, not on statistics extracted from the paper. Experimental. Every point below is traceable to a quoted passage, but this forecast has not yet been validated against known replication outcomes.
Abstract not available — view at DOI ↗

Key Claims

Primary finding · Tentative

Open Access Citation Advantage is altered when non-OA articles available in Sci-hub are stratified separately from non-OA articles not in Sci-hub.

Fragile — the claim's weakest requirement is that the effect exists at all. Based on how the study was designed and conducted, not on statistics extracted from the paper. Experimental. Every point below is traceable to a quoted passage, but this forecast has not yet been validated against known replication outcomes.

descriptive Sci-Hub stratification without demonstrated confounding control or visible classification methods

Figure on page 8
Secondary · Supported

The Open Access Citation Advantage (OACA) ranges from approximately 17% to 85% across scientific disciplines.

the displayed field estimates substantiate the range, although the underlying analytical controls remain unclear

Figure on page 7

Derived from the full evaluation — not a separate score.

Strengths · Limitations

Strengths

#contribution

Sci-Hub adds a distinctive comparison

Figure content on page 8 distinguishes non-OA articles available through Sci-Hub from those not available there, adding an access channel often omitted from OA citation comparisons.

↳ Page 8

#methodological rigour

Broad disciplinary comparisons are presented

The page 7 ranking reports OACA across 27 life-science, physical-science, and social-science categories, making disciplinary heterogeneity visible.

↳ Page 7

#reporting

Metric and subgroup structure are recoverable

The OACA formula on page 5 and the fully OA, hybrid, and Sci-Hub labels on page 8 communicate the basic comparison despite limited narrative text.

↳ Pages 5 and 8

Limitations

#methodological rigour

Major citation confounders remain uncontrolled

The visible comparisons do not show an identification strategy for article age, journal prestige, self-selection into OA, or related determinants of citation outcomes. This prevents the reported advantages from being interpreted as effects of access status.

↳ Pages 5–8

#methodological rigour

Core classification methods remain unavailable

The available text does not explain how OA and Sci-Hub status were determined or timestamped. This is an assessability gap for the central stratification and limits reproducibility.

↳ Pages 4–8

#application pathway

Policy pathway is not operationalized

The topic is relevant to libraries, publishers, and research policy, but the visible text supplies no decision rule, tested intervention, or implementation pathway.

↳ Title and pages 5–8

Does this match your read? Add your assessment ↓

Nabu Commentary

The OACA formula and field-level ranking establish a coherent descriptive analysis, while the Sci-Hub split adds a distinctive comparison. The central methodological constraint is that pages 5–8 do not show control for major determinants of citation outcomes, so the comparisons cannot identify effects of access status. The cross-disciplinary scope supports moderate transferability, but operational details for OA and Sci-Hub classification are unavailable in the visible text. The resulting contribution is useful for bibliometric debate rather than ready for direct policy implementation.

Check another paper

Free · About 5 minutes · Research preview

Author Note / Field Feedback

Nabu’s assessment, alongside the field’s view.

Are you an author of this paper?

Dimension breakdown

Study Quality

Limited2.7

Confidence medium
Contribution
3.1

The study differentiates open-access citation advantage by discipline, journal type, and Sci-Hub availability, providing a targeted extension of established OA citation research. The broader novelty and interpretation are not clearly delineated in the visible narrative.

Pages 5–8, OACA formula and field/Sci-Hub figures

"OACA (non-OA not in Sci-hub)"

Methodological Rigour
2.5

The OACA metric and descriptive subgroup comparisons are intelligible, but the visible analysis does not establish control for major determinants of citation outcomes. The serious uncontrolled-confounding finding binds the score at 2.5.

Pages 5–8, OACA formula and comparative figures

"OACA (non-OA in Sci-hub)"

Reporting
2.9

The sequence from the OACA formula to field-level and journal-type comparisons is partially recoverable. Missing connective prose prevents verification of the complete inferential chain but does not demonstrate that the paper itself is incoherent.

Pages 5–8

"a) Fully OA journals b) Hybrid journals"

Positioning
2.7

The paper cites relevant open-access and bibliometric literature, supporting baseline field awareness. The available introduction and discussion are too incomplete to establish substantive engagement with competing explanations or limitation tracing.

Page 3 and references, pages 10–12

"On a large sample of the Web of Science database"

Lower confidence on Reporting, Positioning — domain match limited.

Reliability

No concerns3 of 4 checks


Internal CoherenceClean

The visible formula and figures are mutually intelligible, and no internal numerical contradiction is demonstrated. Missing narrative and methods material is recorded as an assessability gap rather than an internal-coherence defect.

Research ConductClean

No research-conduct contradiction or design-required ethics omission is demonstrated for this bibliometric study. Funding, competing-interest, data, and code disclosures cannot be evaluated from the truncated text.

Flags: 0 declared / 5 total

Reference IntegrityNot checked

Reference integrity check could not run: no references section detected in extracted text.

0 references in manuscript

Post-Publication RecordClean

No retraction or expression of concern found in Crossref (includes Retraction Watch).

Last checked: 17 Oct 2026

Sources: Crossref (incl. Retraction Watch) ✓

Readiness

Where this paper’s evidence sits on the path from initial observation to real-world use.

Mechanistic

The work supplies a descriptive bibliometric comparison rather than an intervention, implementation framework, or decision tool. Readiness therefore remains several steps from operational use.

Pages 5–8, OACA formula and descriptive figures
Report an issue →

AI-generated, human-governed. Something look off? Contact us to request a review.

Keep reading

Related evaluations

Other papers Nabu has evaluated blind, starting with the same field.

  1. Suspected distortion of citations in high-impact cancer journals

    Baptiste Scancar, Jennifer Anne Byrne, David Causeur, et al. · 2026 · Media and communications

    Primary claim

    These findings suggest that paper mills inflate the citation metrics of supported publications and affected journals at scale.

    Tentative
    Study quality
    Sound3.5
    Reliability
    Caveats
  2. The citation trap: Papers published at year-end receive systematically fewer citations

    Chao Ma, Yiwei Li, Feng Guo, et al. · 2019 · Economics

    Primary claim

    papers published between October and December on average get as much as 18.5% fewer citations than those published in the other months in the year

    Tentative
    Study quality
    Sound3.0
    Reliability
    Caveats
  3. Quantitative Analysis of the Open Access Citation Advantage

    Masashi Kawai, Koichi Ojiro, Makoto Asaoka, et al. · 2022 · Other social sciences

    Primary claim

    日本の機関リポジトリによるオープンアクセス論文の数は限られるものの,その被引用数と様々なタイプのオープンアクセス論文の被引用数の比較からは,日本の機関リポジトリによるオープンアクセスのメリットは確認できなかった.

    Tentative
    Study quality
    Limited2.9
    Reliability
    Caveats

How the rubric works →