# Fake publications in biomedical science: red-flagging method indicates mass production.

- **DOI:** [10.1007/s00210-025-04275-9](https://doi.org/10.1007/s00210-025-04275-9)
- **Rating:** Limited (2.5 out of 5)
- **Verdict:** A practical screening study exposes a serious integrity problem, but nonrepresentative validation prevents its precise prevalence estimate from being trusted.
- **Framework version:** v5.9
- **Pipeline version:** v0.4
- **Evaluation date:** 2026-10-06
- **Evaluation page:** https://www.nabu.science/paper/10.1007%2Fs00210-025-04275-9

## Key claims

1. Applying Bayes' rule resulted in an estimate of 5.8% actual fakes in the biomedical literature.
   - Support: Tentative
   - Type: Primary finding
   - Basis: historical presumed controls and a potentially mismatched flag rate cannot identify contemporary prevalence precisely
   - Location: Abstract; Results, “Estimating the incidence of true fake publications”
2. The multifactorial tallying rule was the best red-flagging method, with a 94% sensitivity and only a 11.5% false-alarm rate.
   - Support: Tentative
   - Type: Secondary
   - Basis: performance was selected and measured on assembled positives and historical unverified controls without held-out validation
   - Location: Abstract; Results, “Validation of fake indicators”
3. The rate of red-flagged articles increased during the last decade, reaching an estimated 14.9% in 2020 and 16.3% in 2023.
   - Support: Tentative
   - Type: Secondary
   - Basis: different sampling frames and inconsistent rule identification prevent a secure like-for-like trend comparison
   - Location: Abstract; Results, Table 4
4. Given 1.86 million Scimago-listed biomedical publications in 2023, we estimate the actual number of true fakes at 107.800 articles per year, growing steadily.
   - Support: Tentative
   - Type: Secondary
   - Basis: the global extrapolation inherits the nonrepresentative validation and unresolved uncertainty of the 5.8% estimate
   - Location: Abstract; Results, “Estimating the incidence of true fake publications”
5. Countries with the highest proportion of read-flagged publications were China, India, Iran, Russia, and Turkey, with China and India the largest absolute contributors globally.
   - Support: Tentative
   - Type: Secondary
   - Basis: country is embedded in the scoring rule and country shares vary sharply across heterogeneous samples
   - Location: Abstract; Table 4

## Strengths

- **Important and operationally relevant question** The study targets the poorly measured scale of paper-mill publishing and identifies editors, publishers, and integrity teams as direct users of screening results.
  - Dimension: contribution
  - Location: Introduction, final paragraphs; Discussion, recommendations
- **Transparent inferential calculation** The prevalence equation, its sensitivity and false-alarm inputs, and the resulting positive predictive value are presented explicitly, making the assumptions inspectable.
  - Dimension: reporting
  - Location: Results, “Estimating the incidence of true fake publications”
- **Simple pathway to editorial triage** The proposed indicators can be checked from publication metadata without specialized forensic tools, supporting a plausible triage workflow rather than a final fraud determination.
  - Dimension: application pathway
  - Location: Methods, “Validation of detection algorithm”; Discussion, concluding paragraphs

## Limitations

- **Rule mismatch threatens headline estimate** The Methods and Fig. 1 identify the multifactorial rule for Study 9, but the incidence Results attribute the 14.9% and 16.3% rates to the email-hospital rule. Combining that rate with the tallying rule’s operating characteristics may invalidate the 5.8% estimate.
  - Dimension: methodological rigour
  - Location: Methods, “Estimating RFP and true fake rates”; Fig. 1 legend; Results, incidence sections
- **Validation does not represent target population** Operating characteristics are estimated from assembled known fakes and presumed non-fakes from 2000 and 2010, then transported to contemporary biomedical publications. This leaves the false-alarm rate and prevalence estimate highly assumption-dependent.
  - Dimension: methodological rigour
  - Location: Methods, “Validation of detection algorithm”
- **Certainty exceeds reported evidence** The abstract and Discussion describe 5.8% and 107,800 annual papers as actual or true fakes without uncertainty intervals. The language obscures how strongly the result depends on the control set, sampling frame, and selected rule.
  - Dimension: reporting
  - Location: Abstract; Discussion, opening and limitations paragraphs

## Reliability checks

Caveats (4 of 4 checks).

- **Results check:** Noted
  - Applying Bayes' rule resulted in an estimate of 5.8% actual fakes in the biomedical literature: "the validated multifactorial tallying rule was applied to published studies from 10 journals for the year 2020 (n = 1,000) and a random key word search (selecting the first reference listed in PubMed) for the year 2023 (n = 1,000)." (Methods, Estimating RFP and true fake rates) vs "When estimating the global RFP incidence with the email-hospital rule (Table 4), most red-flagged articles came from China (42.3%) and India (33%). Within-country percentages of RFPs vary considerably and are highest for Russia (66.7%), India (54.2%), China (38.7%), Iran (29.6%), and Turkey (20.8%). Overall, the estimated RFP rates are 14.9% in 2020 and 16.3% in 2023, close to the 20.4% of our author questionnaire in Study 1 and the 15.4% private/institutional ratio of all email addresses in PubMed (see Fig. 2 legend)." (Results, Estimating the incidence of potentially fake publications). Methods attributes the contemporary flag rates to the multifactorial tallying rule whereas Results attributes them to the email-hospital rule, leaving it unresolved whether the 16.3% rate can be combined with the tallying-rule sensitivity and false-alarm rate. → Effect on result unclear
  - Minor reporting notes (2):
    - Descriptive: "this was topped only by open access (OA) Frontiers journals in Study 6 (112/300 = 40.3%)." (Results, Exploration of fake indicators, Hypothesis 3) vs "In Study 5, we counted RFPs in 10 randomly chosen biomedical journals, where the 2020 rate was 23.8% in journals with lower IF;" (Results, Exploration of fake indicators, Hypothesis 3). 112 divided by 300 multiplied by 100 equals 37.3% to one decimal place, not 40.3%, but remains above the comparison rate of 23.8%. → Conclusion holds
    - Descriptive: "Given the 61 million publications in all science disciplines and all years (https://www.scimagojr.com/worldreport.php), the percentage of roughly 50,000 known fakes (retractions) is 0.00082%." (Introduction, Table 1 explanatory text) vs "The 5.8% estimate of true fakes is between the 2% retractions (proven fakes) that accumulated for >10 years and the 30% range of the number of image manipulations, respectively." (Discussion, limitations). 50,000 divided by 61,000,000 multiplied by 100 equals approximately 0.08197%, about 100 times the reported 0.00082%, while the separate 2% comparison has no stated denominator permitting reconciliation. → Changes the size materially
- **Reporting and transparency:** Noted. The supplied text describes author questionnaires with retraction warnings and a recorded paper-mill interview without stating ethics-review or consent arrangements. An author's editorial role is disclosed narratively, but no formal conflict-of-interest statement is visible.
  - Conflict of interest: not stated · Ethics approval: not stated · Data availability: not stated · Code availability: not stated
- **Reference integrity:** Noted
  - 53 references in manuscript
  - 49 of 49 checkable references found in an index
  - 4 have no canonical index record — counted, but not index-checkable
  - the reference list may be incompletely parsed; counts are indicative
  - 2 references confirmed by manual review
  - Citation details diverge for 2 references:
    - #25 Fang FC Grant Steen R Casadevall A Misconduct accounts for the majority of retracted scientific publications Proc Natl Acad Sci USA 2012 109 17028 107033 10.1073/pnas.1212247109 23027971 PMC3479492 Fang FC, Grant Steen R, Casadevall A (2012) Misconduct accounts for the majority of retracted scientific publications. Proc Natl Acad Sci USA 109:17028–17033 23027971 10.1073/pnas.1212247109 PMC3479492 (DOI resolves to: “Misconduct accounts for the majority of retracted scientific publications” — Fang, 2012 (Proceedings of the National Academy of Sciences). Differs from the DOI’s record on: container, authors) https://doi.org/10.1073/pnas.1212247109
    - #50 van Noorden R (2023) How big is science’s fake-paper problem? An unpublished analysis suggests that there are hundreds of thousands of bogus ‘paper-mill’ articles lurking in the literature. Nature 623 | (News):466–467. https://www.nature.com/articles/d41586-023-03464-x . Accessed 14 May 2025 10.1038/d41586-023-03464-x 37949983 (DOI resolves to: “How big is science’s fake-paper problem?” — Van Noorden, 2023 (Nature). Differs from the DOI’s record on: title, authors) https://doi.org/10.1038/d41586-023-03464-x
  - References not verified automatically are resolved manually within 24 hours.

## Post-publication record

- **Status:** Clean
- **Summary:** No retraction or expression of concern found in Crossref (includes Retraction Watch).
- **Last checked:** 2026-10-17
