Novel Sci-Hub citation association
The study moves beyond descriptive accounts of Sci-Hub use by estimating its association with citations across a sizeable article sample.
↳ Introduction, Related works; Materials and method, Data acquisition
Abstract not available — view at DOI
Key Claims
Primary findingTentative
articles downloaded from Sci-Hub were cited 1.72 times more than papers not downloaded from Sci-Hub
Fragile — the claim's weakest requirement is that the effect exists at all. Based on how the study was designed and conducted, not on statistics extracted from the paper. Experimental. Every point below is traceable to a quoted passage, but this forecast has not yet been validated against known replication outcomes.
the coefficient is framed multiplicatively and causally despite unresolved demand confounding in an observational design
Abstract; Results, Appendix Table 9
SecondarySupported
the number of downloads from Sci-Hub was a robust predictor of future citations
the positive association persists across specifications, although citation timing and causal direction remain incompletely established
Abstract; Results, Appendix Tables 2–11
SecondarySupported
the number of figures in a manuscript consistently predicts its future citations
figure count remains associated with citations across reported models, though automated extraction was not independently validated
Abstract; Results, Appendix Tables 7–11
SecondaryTentative
limited access to publications may limit some scientific research from achieving its full impact
the access interpretation requires a causal pathway that the selected observational comparison cannot identify
Abstract
Derived from the full evaluation — not a separate score.
Strengths · Limitations
Strengths
Extensive specification and diagnostic checks
The paper reports OLS, robust regression, heteroscedasticity-consistent estimates, generated-instrument models, GAMLSS, outlier diagnostics, and blockwise sensitivity analyses.
↳ Statistical analyses; Appendix, Tables 2–11 and Figs. 3–8
Transparent data and code declaration
The datasets and analytical code are declared available through a named OSF repository, supporting reproducibility and follow-up work.
↳ Data Availability statement
Limitations
Causal identification remains unresolved
The observational comparison does not rule out that already popular or highly cited papers attract more Sci-Hub downloads. Generated instruments based on heteroscedasticity do not provide a clear exogenous source of download variation, and two specifications fail the Hansen test.
↳ Introduction, final paragraphs; Appendix, Tables 6 and 9
Impossible descriptive statistics remain unexplained
Appendix Table 1 reports a mean of 1488 for Tablessup despite a maximum of 39, as well as implausible maxima for pages and authors. Because these characteristics enter adjusted models, the unresolved errors create a material reliability concern.
↳ Appendix, Table 1
Headline language overstates the evidence
The abstract presents a regression coefficient as “1.72 times more,” while the Methods and Discussion label the design quasi-experimental and say Sci-Hub almost doubles citations. These formulations imply multiplicative and causal conclusions not established by the design.
↳ Abstract; Materials and method, Data acquisition; Discussion, final paragraph
Nabu Commentary
The paper makes a genuine contribution by estimating a previously underexplored Sci-Hub download and citation association across 12 journals. Its multiverse-style analyses consistently recover a positive coefficient, but the observational comparison and heteroscedasticity-generated instruments do not resolve whether pre-existing article demand drives both downloads and citations. The causal framing therefore carries more weight than the design can support, particularly in the Abstract and Discussion. Appendix Table 1 further lowers confidence because several descriptive values are impossible or implausible and their relationship to the analytical data is unexplained.
Check another paper
Author Note / Field Feedback
Nabu’s assessment, alongside the field’s view.
Are you an author of this paper?
Dimension breakdown
Study Quality
Limited2.7
Confidence highContribution
3.6
The study supplies an early empirical estimate of the association between Sci-Hub downloads and citations across 12 journals and four disciplinary groupings. This meaningfully extends descriptive Sci-Hub research, although it does not identify the causal effect of access.
Introduction, Related works; Materials and method, Data acquisition
“Sci-Hub downloads’ role as a relevant predictor of citations remains neglected.”
Methodological Rigour
2.0
The analysis uses multiple estimators, diagnostics, journal and discipline controls, and sensitivity specifications, but generated instruments do not resolve the principal demand and reverse-causality problem. Methodological Rigour is also affected by unresolved impossible descriptive statistics for analytical covariates.
Statistical analyses; Appendix, Tables 1 and 7–9
“We were able to verify the validity of the instruments, except for models 1 and 3.”
Reporting
3.2
The paper follows a clear progression from motivation and data construction to diagnostics, robustness analyses, and implications. However, “1.72 times more,” “quasi-experimental,” and “almost doubles” communicate a multiplicative or causal conclusion that the reported coefficient and observational design do not establish.
Abstract; Materials and method, Data acquisition; Discussion
“Sci-Hub almost doubles the citations of articles accessed through it.”
Positioning
2.8
The paper engages relevant Sci-Hub usage and citation-prediction literature and explicitly identifies reverse causality as an alternative explanation. It then dismisses that alternative as no formal impediment and proceeds to causal-strength conclusions without tracing the limitation through.
Introduction, final paragraphs; Discussion, paragraphs 1 and 4
“the most popular and most cited articles are more downloaded from Sci-Hub because of their previous citations”
Lower confidence on Contribution, Methodological Rigour — domain match limited.
Reliability
Concerns4 of 4 checks
Internal CoherenceConcern
Appendix Table 1 contains arithmetically impossible and highly implausible descriptive statistics for covariates used in the analyses. The paper does not establish whether these are presentation errors or errors in the analytical data.
Research ConductClean
The study uses bibliometric data for which human-subject ethics approval is not expected and declares public repositories for its data and code. No conduct concern grounded in the supplied text was identified.
Flags: 2 declared / 5 total
Reference IntegrityClean
57 of 57 checkable references verified
59 references in manuscript 2 have no canonical index record — counted, but not index-checkable 3 references confirmed by manual review
Post-Publication RecordClean
No retraction notice found in Retraction Watch.
Last checked: 01 Oct 2026
Sources: Retraction Watch ✓
Readiness
Where this paper’s evidence sits on the path from initial observation to real-world use.
Mechanistic
The study uses real download and citation data, but the findings remain an observational signal rather than tested guidance for policy or operational use. The proposed use of Sci-Hub downloads as an impact indicator is therefore preliminary.
Discussion, paragraph 1
“could be potentially used for practical purposes instead of other indicators of quality and impact”
AI-generated, human-governed. Something look off? Contact us to request a review.