Large, explicitly structured citation corpus
The analysis covers 33,159 index papers, approximately 2.4 million citing papers, and 1.5 million referenced papers, enabling article-, network-, and journal-level analyses.
↳ Results, Fig. 1 and pp. 4–14
Abstract
Abstract Research and scholarship are shaped by article citations, which underpin the communication of ideas, assignment of credit, journal impact factors, and author career progression. Given their key influence on author and journal metrics, citations can be intentionally manipulated to inflate the reputation of journals and researchers. Paper mills, unethical organisations that produce and sell manuscripts and publishing services, may also be manipulating citations, but the extent of this manipulation is unknown. Here, we show that molecular cancer articles sharing features with retracted papers from paper mills display citation patterns that suggest systematic inflation. These articles were published in journals in the top decile of journal rankings. Suspected paper mill articles received 50 to 100% more citations than other papers 1 to 3 years after publication, while paradoxically attracting fewer readers and online accesses. Suspected paper mill articles also cited – and were cited by – other suspected paper mill articles, which frequently appeared in journals previously reported as paper mill targets. Citations from suspected paper mill articles measurably inflated journal citation metrics. These findings suggest that paper mills inflate the citation metrics of supported publications and affected journals at scale. This widespread manipulation of citation metrics may amplify unreliable findings, slowing scientific progress and spurring unreasonable citation benchmarks for research articles, journals and authors. Our findings point to measurable signatures of citation manipulation that could support article- and journal-level screening. Significance statement The central role of citations in research evaluation makes them an attractive target for gaming. It has been suggested that fraudulent organisations, known as research paper mills , may manipulate citations for a price. This research uncovers the impact of potential paper mill manipulation on citation metrics. Across leading cancer journals, clusters of interconnected papers bearing the hallmarks of paper mill products accumulate far more citations than other papers despite having fewer readers. In some cases, this effect is large enough to inflate journal rankings, highlighting the need for safeguards against paper mills. This study provides evidence that citation manipulation leaves measurable traces in citation patterns, offering a practical basis for future screening and intervention.
Key Claims
Primary findingTentative
These findings suggest that paper mills inflate the citation metrics of supported publications and affected journals at scale.
Possible — the claim's weakest requirement is that it works through the path described. Experimental. Every point below is traceable to a quoted passage, but this forecast has not yet been validated against known replication outcomes.
classifier-based observational patterns cannot directly establish paper-mill causation or intent
Abstract
SecondarySupported
Suspected paper mill articles received 50 to 100% more citations than other papers 1 to 3 years after publication, while paradoxically attracting fewer readers and online accesses.
adjusted associations are consistent across thresholds, with geographic limitations affecting the reader proxy
Abstract; Results, Fig. 3
SecondarySupported
Suspected paper mill articles also cited – and were cited by – other suspected paper mill articles, which frequently appeared in journals previously reported as paper mill targets.
large adjusted associations persist, though the same text classifier labels both citation endpoints
Abstract; Results, Figs. 4–5
SecondaryTentative
Citations from suspected paper mill articles measurably inflated journal citation metrics.
counterfactual removal quantifies flagged-paper contributions but does not identify manipulation as their cause
Abstract; Results, Fig. 6
Derived from the full evaluation — not a separate score.
Strengths · Limitations
Strengths
Multiple robustness checks across analyses
Results are examined at two classifier thresholds and through alternative citation data, Chinese-affiliation adjustment, non-coding-RNA restriction, in-domain restriction, and a topic covariate.
↳ Results, Figs. 3–5; Methods, p. 20
Concrete institutional screening pathway
The paper identifies publishers, indexing services, funders, and researchers as potential users and proposes several measurable indicators for prioritizing integrity investigations.
↳ Perspectives and future work, p. 17
Limitations
Proxy cannot establish manipulation or intent
Flagged status measures textual similarity to retracted paper-mill papers rather than verified provenance. The Limitations acknowledge no direct evidence of misconduct and no ability to determine citation motivations.
↳ Limitations, p. 16
Conclusions contradict stated inferential limits
The Results characterize the pattern as deliberate exchange and an organised system, while the Limitations state that paper-mill intent cannot be directly demonstrated.
↳ Results, p. 11; Limitations, p. 16
Transferability and screening validation remain narrow
The corpus is limited to top-decile molecular-oncology journals, and the proposed screening indicators have not been formally validated against confirmed manipulation or in other fields.
↳ Limitations, p. 16; Perspectives and future work, p. 17
Nabu Commentary
The study’s scale, explicit corpus construction, and multiple sensitivity analyses support a strong contribution score and a competent craft assessment. The central associations—citation premiums, attention differences, and clustering among flagged papers—are consistently quantified across thresholds and several alternative analyses. The main score constraint is inferential: textual similarity to retracted paper-mill papers does not verify provenance, and applying the same classifier across citation endpoints leaves residual template and topic explanations. The strongest downward adjustment arises because claims of deliberate exchange and an organised system conflict with the Limitations section’s acknowledgement that misconduct and intent cannot be directly demonstrated.
Check another paper
Author Note / Field Feedback
Nabu’s assessment, alongside the field’s view.
Are you an author of this paper?
Dimension breakdown
Study Quality
Sound3.5
Confidence highContribution
3.9
The study meaningfully extends prior detection work by quantifying citation premiums, attention differences, citation-network clustering, and journal-metric contributions across 33,159 papers. Its advance remains forensic and associational because flagged status is an indirect proxy for paper-mill provenance.
Results, pp. 4–14; Introduction, pp. 2–3
“Suspected paper mill articles received 50 to 100% more citations than other papers”
Methodological Rigour
3.5
The analysis combines two classification thresholds, adjusted count and mixed-effects models, alternative citation data, topic and affiliation sensitivity analyses, and a negative-control journal. Confidence is tempered because the same text classifier labels both ends of citation links and the topic proxy does not exclude homophily.
Methods, pp. 18–20; Results, Figs. 3–5; Limitations, pp. 16–17
“Our embedding-based classification therefore does not exclude a topic homophily hypothesis.”
Reporting
3.5
The paper is logically organized and generally distinguishes flagged papers from confirmed misconduct. However, the Abstract and Fig. 5 discussion use causal and intentional language that is stronger than the indirect observational evidence permits.
Abstract; Results, p. 11; Limitations, p. 16
“Citation motivations cannot be determined, precluding direct demonstration of paper mill intent.”
Positioning
2.8
The paper engages paper-mill, citation-cartel, home-bias, topic-homophily, and journal-metric literature and explicitly discusses relevant alternatives. Its score is capped because the Results assert deliberate exchange and an organised system after the Limitations concede that intent cannot be demonstrated.
Introduction, pp. 2–3; Results, p. 11; Limitations, pp. 16–17
“This reciprocal pattern supports the existence of a deliberate, non-random citation exchange”
Lower confidence on Methodological Rigour — domain match limited.
Reliability
Caveats4 of 4 checks
Internal CoherenceNoted
The reported sample sizes, models, and numerical results are internally consistent, but some central interpretations extend beyond what the classifier-based observational design establishes. The concern affects attribution and intent rather than whether the reported associations were calculated as described.
Research ConductClean
The paper declares funding and competing interests, identifies its data sources and analysis-code repository, and explains why the prior detection model is withheld. No human-subjects ethics requirement is evident for this bibliometric study.
Flags: 3 declared / 5 total
Reference IntegrityClean
56 references in manuscript 52 of 52 checkable references found in an index 4 are books, websites or datasets — counted, but not index-checkable
Post-Publication RecordClean
No retraction notice found in Retraction Watch.
Last checked: 16 Oct 2026
Sources: Retraction Watch ✓
Readiness
Where this paper’s evidence sits on the path from initial observation to real-world use.
Demonstrated
The paper proposes citation-to-reader decoupling, an early-citation inflation index, journal-normalised citation surplus, and reference-list analysis as screening indicators. These remain candidate heuristics that the authors state require formal testing before operational use.
Perspectives and future work, p. 17
“all of which could be formally tested in further research”
AI-generated, human-governed. Something look off? Contact us to request a review.