# The Sci-Hub effect on papers’ citations

- **DOI:** [10.1007/s11192-020-03806-w](https://doi.org/10.1007/s11192-020-03806-w)
- **Rating:** Limited (2.7 out of 5)
- **Verdict:** A novel, broad-sample association between Sci-Hub downloads and citations is undermined by unresolved confounding and causal interpretation.
- **Framework version:** v5.9
- **Pipeline version:** v0.4
- **Evaluation date:** 2026-10-01
- **Evaluation page:** https://www.nabu.science/paper/10.1007%2Fs11192-020-03806-w

## Key claims

1. articles downloaded from Sci-Hub were cited 1.72 times more than papers not downloaded from Sci-Hub
   - Support: Tentative
   - Type: Primary finding
   - Basis: the coefficient is framed multiplicatively and causally despite unresolved demand confounding in an observational design
   - Location: Abstract; Results, Appendix Table 9
2. the number of downloads from Sci-Hub was a robust predictor of future citations
   - Support: Supported
   - Type: Secondary
   - Basis: the positive association persists across specifications, although citation timing and causal direction remain incompletely established
   - Location: Abstract; Results, Appendix Tables 2–11
3. the number of figures in a manuscript consistently predicts its future citations
   - Support: Supported
   - Type: Secondary
   - Basis: figure count remains associated with citations across reported models, though automated extraction was not independently validated
   - Location: Abstract; Results, Appendix Tables 7–11
4. limited access to publications may limit some scientific research from achieving its full impact
   - Support: Tentative
   - Type: Secondary
   - Basis: the access interpretation requires a causal pathway that the selected observational comparison cannot identify
   - Location: Abstract

## Strengths

- **Novel Sci-Hub citation association** The study moves beyond descriptive accounts of Sci-Hub use by estimating its association with citations across a sizeable article sample.
  - Dimension: contribution
  - Location: Introduction, Related works; Materials and method, Data acquisition
- **Extensive specification and diagnostic checks** The paper reports OLS, robust regression, heteroscedasticity-consistent estimates, generated-instrument models, GAMLSS, outlier diagnostics, and blockwise sensitivity analyses.
  - Dimension: methodological rigour
  - Location: Statistical analyses; Appendix, Tables 2–11 and Figs. 3–8
- **Transparent data and code declaration** The datasets and analytical code are declared available through a named OSF repository, supporting reproducibility and follow-up work.
  - Dimension: methodological rigour
  - Location: Data Availability statement

## Limitations

- **Causal identification remains unresolved** The observational comparison does not rule out that already popular or highly cited papers attract more Sci-Hub downloads. Generated instruments based on heteroscedasticity do not provide a clear exogenous source of download variation, and two specifications fail the Hansen test.
  - Dimension: methodological rigour
  - Location: Introduction, final paragraphs; Appendix, Tables 6 and 9
- **Impossible descriptive statistics remain unexplained** Appendix Table 1 reports a mean of 1488 for Tablessup despite a maximum of 39, as well as implausible maxima for pages and authors. Because these characteristics enter adjusted models, the unresolved errors create a material reliability concern.
  - Dimension: methodological rigour
  - Location: Appendix, Table 1
- **Headline language overstates the evidence** The abstract presents a regression coefficient as “1.72 times more,” while the Methods and Discussion label the design quasi-experimental and say Sci-Hub almost doubles citations. These formulations imply multiplicative and causal conclusions not established by the design.
  - Dimension: reporting
  - Location: Abstract; Materials and method, Data acquisition; Discussion, final paragraph

## Reliability checks

Concerns (4 of 4 checks).

- **Internal coherence:** Concern. Appendix Table 1 contains arithmetically impossible and highly implausible descriptive statistics for covariates used in the analyses. The paper does not establish whether these are presentation errors or errors in the analytical data.
- **Research conduct:** Clean. The study uses bibliometric data for which human-subject ethics approval is not expected and declares public repositories for its data and code. No conduct concern grounded in the supplied text was identified.
- **Reference integrity:** Clean. 57 of 57 checkable references verified
  - 59 references in manuscript
  - 2 have no canonical index record — counted, but not index-checkable
  - 3 references confirmed by manual review
  - References not verified automatically are resolved manually within 24 hours.

## Post-publication record

- **Status:** Clean
- **Summary:** No retraction or expression of concern found in Crossref (includes Retraction Watch).
- **Last checked:** 2026-10-17
