nabu.science
MethodologyAboutContact

 

  • Snapshot
  • Key Claims
  • Strengths & Limitations
  • Commentary
  • From the Authors
  • Author Note / Field Feedback
  • Dimension breakdown
  • Impact Profile
Free · About 5 minutes
Report #NBU-6592BAA4·Framework v5.9·Pipeline v0.4·Evaluated 06 Oct 2026·OECD: 3.2 Clinical medicine·Subfield: Safety Research·Basis: Full text·Type: Diagnostic accuracy study

Fake publications in biomedical science: red-flagging method indicates mass production.

Sabel et al.Sabel, Knaack, Gigerenzer et al.·2025·Naunyn-Schmiedeberg's archives of pharmacology·10.1007/s00210-025-04275-9 ↗

Primary claim

Applying Bayes' rule resulted in an estimate of 5.8% actual fakes in the biomedical literature.

Tentative · historical presumed controls and a potentially mismatched flag rate cannot identify contemporary prevalence precisely
In brief

A practical screening study exposes a serious integrity problem, but nonrepresentative validation prevents its precise prevalence estimate from being trusted.

ReliabilityCaveats4 of 4 checks completeRead the 3 entries ↓
Unlikely to hold — the claim's weakest requirement is that the effect exists at all. Based on how the study was designed and conducted, not on statistics extracted from the paper. Experimental. Every point below is traceable to a quoted passage, but this forecast has not yet been validated against known replication outcomes.
Abstract

Integrity of academic publishing is increasingly undermined by fake publications massively produced by commercial "editing services" (so-called "paper mills"). These services use AI-supported production techniques at scale and sell fake publications to students, scientists, and physicians under pressure to advance their careers. Because the scale of fake publications in biomedicine is unknown, we developed an easy-to-apply rule to red-flag potentially fake publications and estimate their number. After analyzing questionnaires sent to authors of published papers, we developed simple classification rules and tested them in a 9-step bibliometric analysis in a sample of 17,120 publications listed in PubMed®. We first validated various simple rules and finally applied a multifactorial tallying rule comparing 400 known fakes with 400 random (presumed) non-fakes. This rule was then applied to 1,000 random publications each from 2020 and 2023. The multifactorial tallying rule was the best red-flagging method, with a 94% sensitivity and only a 11.5% false-alarm rate. The rate of red-flagged articles increased during the last decade, reaching an estimated 14.9% in 2020 and 16.3% in 2023. Countries with the highest proportion of read-flagged publications were China, India, Iran, Russia, and Turkey, with China and India the largest absolute contributors globally. Applying Bayes' rule resulted in an estimate of 5.8% actual fakes in the biomedical literature. Given 1.86 million Scimago-listed biomedical publications in 2023, we estimate the actual number of true fakes at 107.800 articles per year, growing steadily. Scientific publications in biomedicine can be red-flagged as potentially fake using fast-and-frugal classification rules to earmark them for subsequent scrutiny. When applying Bayes´rule, the annual true scale of fake publishing in biomedicine is about 19 times that of the 5.671 biomedicine retractions in 2023. This scale of fraudulent publishing is concerning as it can damage trust in science, endanger public health, and impact economic spending. But fake detection tools can enable retractions of fake publications at scale and help prevent further damage to the permanent scientific record.

Key Claims

Primary finding · Tentative

Applying Bayes' rule resulted in an estimate of 5.8% actual fakes in the biomedical literature.

Unlikely to hold — the claim's weakest requirement is that the effect exists at all. Based on how the study was designed and conducted, not on statistics extracted from the paper. Experimental. Every point below is traceable to a quoted passage, but this forecast has not yet been validated against known replication outcomes.

historical presumed controls and a potentially mismatched flag rate cannot identify contemporary prevalence precisely

Abstract; Results, “Estimating the incidence of true fake publications”
Quote verified
Secondary · Tentative

The multifactorial tallying rule was the best red-flagging method, with a 94% sensitivity and only a 11.5% false-alarm rate.

performance was selected and measured on assembled positives and historical unverified controls without held-out validation

Abstract; Results, “Validation of fake indicators”
Quote verified
Secondary · Tentative

The rate of red-flagged articles increased during the last decade, reaching an estimated 14.9% in 2020 and 16.3% in 2023.

different sampling frames and inconsistent rule identification prevent a secure like-for-like trend comparison

Abstract; Results, Table 4
Quote verified
Secondary · Tentative

Given 1.86 million Scimago-listed biomedical publications in 2023, we estimate the actual number of true fakes at 107.800 articles per year, growing steadily.

the global extrapolation inherits the nonrepresentative validation and unresolved uncertainty of the 5.8% estimate

Abstract; Results, “Estimating the incidence of true fake publications”
Quote verified
Secondary · Tentative

Countries with the highest proportion of read-flagged publications were China, India, Iran, Russia, and Turkey, with China and India the largest absolute contributors globally.

country is embedded in the scoring rule and country shares vary sharply across heterogeneous samples

Abstract; Table 4
Quote verified

Derived from the full evaluation — not a separate score.

Strengths · Limitations

Strengths

#contribution

Important and operationally relevant question

The study targets the poorly measured scale of paper-mill publishing and identifies editors, publishers, and integrity teams as direct users of screening results.

↳ Introduction, final paragraphs; Discussion, recommendations

#reporting

Transparent inferential calculation

The prevalence equation, its sensitivity and false-alarm inputs, and the resulting positive predictive value are presented explicitly, making the assumptions inspectable.

↳ Results, “Estimating the incidence of true fake publications”

#application pathway

Simple pathway to editorial triage

The proposed indicators can be checked from publication metadata without specialized forensic tools, supporting a plausible triage workflow rather than a final fraud determination.

↳ Methods, “Validation of detection algorithm”; Discussion, concluding paragraphs

Limitations

#methodological rigour

Rule mismatch threatens headline estimate

The Methods and Fig. 1 identify the multifactorial rule for Study 9, but the incidence Results attribute the 14.9% and 16.3% rates to the email-hospital rule. Combining that rate with the tallying rule’s operating characteristics may invalidate the 5.8% estimate.

↳ Methods, “Estimating RFP and true fake rates”; Fig. 1 legend; Results, incidence sections

#methodological rigour

Validation does not represent target population

Operating characteristics are estimated from assembled known fakes and presumed non-fakes from 2000 and 2010, then transported to contemporary biomedical publications. This leaves the false-alarm rate and prevalence estimate highly assumption-dependent.

↳ Methods, “Validation of detection algorithm”

#reporting

Certainty exceeds reported evidence

The abstract and Discussion describe 5.8% and 107,800 annual papers as actual or true fakes without uncertainty intervals. The language obscures how strongly the result depends on the control set, sampling frame, and selected rule.

↳ Abstract; Discussion, opening and limitations paragraphs

Does this match your read? Add your assessment ↓

Nabu Commentary

The paper’s central practical contribution is an inexpensive metadata screen paired with an explicit Bayes calculation. Its principal methodological limitation is that performance is estimated using heterogeneous known fakes and presumed non-fakes from 2000 and 2010, then transported to contemporary samples drawn through different procedures. The Results also attribute the 2020 and 2023 rates to the email-hospital rule even though the Methods and Fig. 1 identify the multifactorial tallying rule, leaving the headline calculation internally unresolved. The method may support triage, but the 5.8% prevalence and 107,800-paper extrapolation require representative contemporary validation and uncertainty analysis.

Evaluate another paper

Free · About 5 minutes · Research preview

Author Note / Field Feedback

Nabu’s assessment, alongside the field’s view.

Are you an author of this paper?

Dimension breakdown

Evidence Score

Out of 5
012345
2.5±0.1

Contribution
3.2

The study addresses an important unresolved question and offers an inspectable screening rule, temporal estimates, and a prevalence calculation. Its advance is limited because the exact scale estimate is not established by representative contemporary validation.

Abstract; Introduction, final paragraph; Methods, “Validation of detection algorithm”

“Applying Bayes' rule resulted in an estimate of 5.8% actual fakes in the biomedical literature.”

Methodological Rigour
2.0

The exploration-validation-estimation sequence and Bayes arithmetic are explicit, but operating characteristics come from heterogeneous known positives and historical presumed negatives. Transporting those characteristics to contemporary nonrandom samples triggers serious reference-standard and spectrum concerns.

Methods, “Validation of detection algorithm” and “Estimating RFP and true fake rates”; Results, “Estimating the incidence of true fake publications”

“one cannot be certain that each control publication is indeed not fake”

Reporting
2.7

The study sequence and prevalence equation are followable, but the abstract and Discussion describe assumption-dependent outputs as actual or true fakes. The unresolved rule-label inconsistency and numerical slips further reduce reporting precision.

Abstract; Results, “Estimating the incidence of potentially fake publications”; Discussion, opening and limitations paragraphs

“we estimate the actual number of true fakes at 107.800 articles per year”

Positioning
2.8

The paper surveys prior estimates and identifies several scope limitations. However, its acknowledged uncertainty about presumed controls is not carried through to the conclusion that 5.8% is the most reasonable approximation and may be an underestimate.

Introduction, Table 1; Methods, “Validation of detection algorithm”; Discussion, limitations paragraphs

“it is therefore the most reasonable approximation at this time”

Lower confidence on Contribution, Positioning — domain match limited.

Reliability

Caveats4 of 4 checks


Results checkNoted
  • Applying Bayes' rule resulted in an estimate of 5.8% actual fakes in the biomedical literature: "the validated multifactorial tallying rule was applied to published studies from 10 journals for the year 2020 (n = 1,000) and a random key word search (selecting the first reference listed in PubMed) for the year 2023 (n = 1,000)." (Methods, Estimating RFP and true fake rates) vs "When estimating the global RFP incidence with the email-hospital rule (Table 4), most red-flagged articles came from China (42.3%) and India (33%). Within-country percentages of RFPs vary considerably and are highest for Russia (66.7%), India (54.2%), China (38.7%), Iran (29.6%), and Turkey (20.8%). Overall, the estimated RFP rates are 14.9% in 2020 and 16.3% in 2023, close to the 20.4% of our author questionnaire in Study 1 and the 15.4% private/institutional ratio of all email addresses in PubMed (see Fig. 2 legend)." (Results, Estimating the incidence of potentially fake publications). Methods attributes the contemporary flag rates to the multifactorial tallying rule whereas Results attributes them to the email-hospital rule, leaving it unresolved whether the 16.3% rate can be combined with the tallying-rule sensitivity and false-alarm rate. → Effect on result unclear
Reporting and transparencyNoted

The supplied text describes author questionnaires with retraction warnings and a recorded paper-mill interview without stating ethics-review or consent arrangements. An author's editorial role is disclosed narratively, but no formal conflict-of-interest statement is visible.

Conflict of interest: not stated · Ethics approval: not stated · Data availability: not stated · Code availability: not stated

Reference IntegrityNoted

53 references in manuscript 49 of 49 checkable references found in an index 4 have no canonical index record — counted, but not index-checkable the reference list may be incompletely parsed; counts are indicative 2 references confirmed by manual review

References not verified automatically are resolved manually within 24 hours.

Post-Publication RecordClean

No retraction or expression of concern found in Crossref (includes Retraction Watch).

Last checked: 17 Oct 2026

Sources: Crossref (incl. Retraction Watch) ✓

Impact Profile

Impact areaPolicy and practice · SDG 9
Type of advanceMethodological
Route to useNamed
AdvanceAdvances
CitationsToo early
Policy · patentNot yet traced
ReachedNot reachedNot assessable or unverified
ReadinessPiloted
Foundational
Controlled
Piloted
Validated
Implementable
FoundationalImplementable

The tallying rule was validated on 400 known fakes versus 400 controls and applied to random PubMed samples, but the 5.8% true-fake estimate has not been independently confirmed in another setting.

From the paper
“we developed a method to red-flag potential fake publications (RFPs) with indicators that are easy to use by reviewers, editors, and publishers”

The paper names reviewers, editors, and publishers as the actors who would apply its red-flagging rules to earmark suspicious publications for scrutiny and retraction.

Pathway

Journal editors, reviewers, and publishers would apply the red-flagging tallying rule to screen submitted manuscripts and published articles for scrutiny and retraction at scale.

Barrier

A red flag has only 33.5% positive predictive value and is not legal proof a paper is fake, so publishers cannot retract flagged articles without further case-by-case investigation.

Evaluated 13 Nov 2026 · GBD 2023 · glm_v1_3_3 · impact_profile_v1_3_3

Report an issue →

AI-generated, human-governed. Something look off? Contact us to request a review.

Keep reading

Related evaluations

Other papers Nabu has evaluated blind, starting with the same field.

  1. Performance of AI Tools in Citing Retracted Literature : Content Analysis

    Sebastian Labenbacher, Maximilian Niederer, Sascha Hammer, et al. · 2026 · Media and communications

    Primary claim

    None of the evaluated AI tools consistently handled retracted articles correctly. No model achieved perfect accuracy across all question sets.

    Supported
    Evidence Score
    2.9±0.2
    Reliability
    Caveats
  2. Scientific misconduct, questionable research practices, and research climate: empirical approaches to integrity problems

    Nazarovets · 2026 · Other social sciences

    Primary claim

    aggregate harm is rarely defined explicitly and is instead inferred from perceived prevalence and severity of practices

    Tentative
    Evidence Score
    2.9±0.2
    Reliability
    Caveats
  3. Neighborly social pressure and collective action: Evidence from a field experiment in Tunisia

    Jöst · 2024 · Political science

    Primary claim

    I do not find evidence to suggest that neighborly social pressure encourages participation in neighborhood cleanups, with low participation rates both for the neighbor and outsider contact conditions.

    Tentative
    Evidence Score
    3.2±0.2
    Reliability
    Caveats

How the rubric works →