nabu.science
MethodologyAboutContact

 

  • Snapshot
  • Key Claims
  • Strengths & Limitations
  • Commentary
  • From the Authors
  • Author Note / Field Feedback
  • Dimension breakdown
  • Readiness
Report #NBU-063EE566·Framework v5.9·Pipeline v0.4·Evaluated 11 Sept 2026·OECD: 3.2 Clinical medicine·Subfield: Statistics, Probability and Uncertainty·Basis: Full text·Type: Cross-sectional study

Do open-access dermatology articles have higher citation counts than those with subscription-based access?

Xie et al.Xie, Ghozy, Kallmes et al.·2022·PloS one·10.1371/journal.pone.0279265 ↗

Primary claim

OA articles had significantly higher mean citation counts (mean 13.2, SD 17.0) compared to SA articles (mean 7.9, SD 8.8) (OR 1.04; 95% CI 1.02-1.05; P < .001).

Tentative · observational comparison with unresolved confounding and an unreconciled OR-versus-IRR primary estimate
In brief

A useful dermatology-specific citation study adds incremental evidence, but unresolved self-selection and confounding prevent causal interpretation.

ReliabilityConcerns4 of 4 checks completeRead the entry ↓
Fragile — the claim's weakest requirement is that the effect exists at all. Experimental. Every point below is traceable to a quoted passage, but this forecast has not yet been validated against known replication outcomes.
Abstract

BACKGROUND: Open-access (OA) publishing is increasingly prevalent in dermatology, and many journals now offer hybrid options, including conventional (subscription-based access [SA]) publishing or OA (with an author publishing charge) in a subscription journal. OA publishing has been noted in many disciplines, but this has been rarely studied in dermatology.

METHODS: Using the Clarivate Journal Citation Report, we compiled a list of English-language dermatology hybrid OA journals containing more than 5% OA articles. We sampled any OA review or original research article in 4 issues from 2018 to 2019 and matched an equal number of SA articles. Citation count, citation count excluding self-citations and view counts found using Scopus and Altmetrics score were recorded for each article. Statistical analyses were performed using logistic and negative binomial models using R software.

RESULTS: Twenty-seven hybrid dermatology journals were found, and 538 articles were sampled (269 OA, 269 SA). For both original research and review articles, OA articles had significantly higher mean citation counts (mean 13.2, standard deviation [SD] 17.0) compared to SA articles (mean 7.9, SD 8.8) (odds ratio [OR] 1.04; 95% CI 1.02-1.05; P < .001) including when adjusted for time from publication. Original research OA articles had significantly higher citation counts than original research SA articles (excluding self-citations; OR, 1.03; 95% CI, 1.01-1.05; P = .003), and review articles also had OA citation advantage than review SA articles (OR, 1.06; 95% CI, 1.02-1.11; P = .008). There was, however, no significant difference in citation counts between review articles and original research articles (OR, 1.00; 95% CI, 0.19-5.31; P = 1.000). There was no significant difference seen in view counts (OA: mean±SD 17.7±10.8; SA: mean±SD 17.1±12.4) and Altmetric score (OA: mean±SD 13.2±47.8; SA: mean±SD 6.3±25.0) between OA and SA articles. Potential confounders included the fact that more OA articles were published in Europe than in Asia, and pharmaceutical-funded articles were more likely to be published OA.

CONCLUSIONS: We noted a higher citation count for OA articles than SA articles in dermatology hybrid journals. However, dermatology researchers should take into account confounding factors when deciding whether to increase the impact of their work by selecting OA over SA publishing.

Key Claims

Primary finding · Tentative

OA articles had significantly higher mean citation counts (mean 13.2, SD 17.0) compared to SA articles (mean 7.9, SD 8.8) (OR 1.04; 95% CI 1.02-1.05; P < .001).

Fragile — the claim's weakest requirement is that the effect exists at all. Experimental. Every point below is traceable to a quoted passage, but this forecast has not yet been validated against known replication outcomes.

observational comparison with unresolved confounding and an unreconciled OR-versus-IRR primary estimate

Abstract, Results
Secondary · Supported

Original research OA articles had significantly higher citation counts than original research SA articles (excluding self-citations; OR, 1.03; 95% CI, 1.01-1.05; P = .003).

direction persists after excluding self-citations, though OA self-selection remains incompletely controlled

Abstract, Results
Quote verified
Secondary · Supported

Review articles also had OA citation advantage than review SA articles (OR, 1.06; 95% CI, 1.02-1.11; P = .008).

reported subgroup association from a smaller review-article sample with residual confounding

Abstract, Results
Quote verified
Secondary · Supported

There was no significant difference seen in view counts and Altmetric score between OA and SA articles.

reported null comparisons use directly collected bibliometric measures, though practical measurement limits remain

Abstract, Results; Results, paragraph 1

Derived from the full evaluation — not a separate score.

Strengths · Limitations

Strengths

#methodological rigour

Within-journal comparison reduces broad confounding

OA and SA articles were compared within the same hybrid journals and were matched by article category and proximity, reducing differences that arise from comparing separate OA and subscription journals.

↳ Materials and Methods, article-sampling paragraph; Introduction, paragraph 2

#positioning

Conflicting evidence receives substantive attention

The Discussion addresses systematic-review findings, small or absent effects in other fields, and randomized trials that found more downloads but no citation advantage at later follow-up.

↳ Discussion, paragraphs 2–3

#contribution

Clear specialty-specific research contribution

The study examines article-level OACA in dermatology, a specialty the Introduction identifies as previously unstudied, using 538 articles from 27 hybrid journals.

↳ Introduction, paragraph 2; Results, paragraph 1

Limitations

#reporting

Primary effect estimate is unreconciled

The Abstract reports OR 1.04 for the headline citation comparison, whereas Results reports IRR 1.44 from a negative-binomial model. These estimates imply materially different effect magnitudes and require correction or explanation.

↳ Abstract, Results; Materials and Methods, statistical-analysis paragraph; Results, paragraph 1 and Table 3

#methodological rigour

Self-selection confounding remains unresolved

Article topic, author prestige, geography, and industry funding can influence both OA selection and citations. The paper acknowledges these factors, including a 43% versus 15% industry-funding imbalance, but cannot isolate an access effect.

↳ Results, Figure 2D and Table 4; Discussion, confounding and limitations paragraphs

#application pathway

Sampling limits broader applicability

The sample covers selected 2018–2019 issues from English-language dermatology hybrid journals with more than 5% OA content, excluding issues without OA articles. The findings therefore may not transfer to pure OA journals, other specialties, or newer publishing conditions.

↳ Materials and Methods, eligibility and sampling paragraphs; Discussion, limitations paragraph

Does this match your read? Add your assessment ↓

Nabu Commentary

The within-hybrid-journal comparison and negative-binomial approach are appropriate for estimating citation-count associations, and the sensitivity analysis shows that the direction persists after excluding several freely accessible SA categories. The principal methodological constraint is that choosing OA is associated with funding, geography, article topic, and author prestige, so the analysis cannot identify a causal access effect. The Discussion handles this limitation responsibly by engaging conflicting randomized evidence and qualifying the conclusion. The Abstract's OR 1.04 and Results' IRR 1.44 nevertheless require reconciliation because they give materially different numerical interpretations of the headline finding.

Check another paper

Free · About 5 minutes · Research preview

Author Note / Field Feedback

Nabu’s assessment, alongside the field’s view.

Are you an author of this paper?

Dimension breakdown

Study Quality

Sound3.0

Confidence high
Contribution
3.1

The study adds a dermatology-specific, article-level estimate to an established and mixed OACA literature. Its advance is genuine but incremental because it does not identify the mechanism or causal effect of selecting OA.

Introduction, paragraph 2; Discussion, paragraphs 1–3

“To our knowledge, OACA has not been studied in the field of dermatology.”

Methodological Rigour
2.5

The matched hybrid-journal design, power calculation, count-data model, and access-status sensitivity analysis provide a credible association study. Unmeasured article quality, topic, and author prestige, together with the OA-selection mechanism and adjacent-SA sampling, materially limit identification.

Materials and Methods, sampling and statistical-analysis paragraphs; Results, paragraph 2; Discussion, limitations paragraph

“limitations and biases that exist in our study include unavoidable confounders, such as article topic and author prestige.”

Reporting
3.3

The paper follows a coherent question-to-methods-to-conclusion structure and explicitly qualifies interpretation. However, OR 1.04 in the Abstract and IRR 1.44 in Results give materially different accounts of the headline effect magnitude.

Abstract, Results and Conclusions; Materials and Methods, statistical-analysis paragraph; Results, paragraph 1

“Results of the negative binomial regression were expressed as incidence rate ratio (IRR) and 95% CI.”

Positioning
3.7

The Discussion presents supportive studies alongside randomized trials and systematic-review evidence showing small or absent citation advantages. It also carries residual-confounding concerns into a qualified conclusion, although it does not quantify their likely bias.

Discussion, paragraphs 2–3 and limitations paragraph; Conclusions

“results should be interpreted bearing in mind confounders”

Lower confidence on Methodological Rigour, Reporting — domain match limited.

Reliability

Concerns4 of 4 checks


Internal CoherenceConcern

The primary OA citation effect is directionally consistent but numerically and terminologically unreconciled across the Abstract, Methods, and Results. The discrepancy affects which model estimate and effect magnitude should be treated as primary.

Research ConductClean

The study states that institutional ethics approval was not required because it analyzed publicly available bibliometric data. No text-grounded research-conduct concern was identified.

Flags: 1 declared / 5 total

Reference IntegrityClean

23 of 23 checkable references verified

36 references in manuscript 13 are books, websites or datasets — counted, but not index-checkable

References not verified automatically are resolved manually within 24 hours.

Post-Publication RecordClean

No retraction or expression of concern found in Crossref (includes Retraction Watch).

Last checked: 17 Oct 2026

Sources: Crossref (incl. Retraction Watch) ✓

Readiness

Where this paper’s evidence sits on the path from initial observation to real-world use.

Mechanistic

The study provides an observational bibliometric association rather than evidence that paying for or selecting OA causes additional citations. Residual self-selection and the inconsistent primary estimate leave it short of decision-ready guidance.

Materials and Methods, study design and statistical-analysis paragraphs; Discussion, confounding and limitations paragraphs

“These are undoubtedly confounders that may affect interpretation of our results”

Report an issue →

AI-generated, human-governed. Something look off? Contact us to request a review.

Keep reading

Related evaluations

Other papers Nabu has evaluated blind, starting with the same field.

  1. Suspected distortion of citations in high-impact cancer journals

    Baptiste Scancar, Jennifer Anne Byrne, David Causeur, et al. · 2026 · Media and communications

    Primary claim

    These findings suggest that paper mills inflate the citation metrics of supported publications and affected journals at scale.

    Tentative
    Study quality
    Sound3.5
    Reliability
    Caveats
  2. The citation trap: Papers published at year-end receive systematically fewer citations

    Chao Ma, Yiwei Li, Feng Guo, et al. · 2019 · Economics

    Primary claim

    papers published between October and December on average get as much as 18.5% fewer citations than those published in the other months in the year

    Tentative
    Study quality
    Sound3.0
    Reliability
    Caveats
  3. Quantitative Analysis of the Open Access Citation Advantage

    Masashi Kawai, Koichi Ojiro, Makoto Asaoka, et al. · 2022 · Other social sciences

    Primary claim

    日本の機関リポジトリによるオープンアクセス論文の数は限られるものの,その被引用数と様々なタイプのオープンアクセス論文の被引用数の比較からは,日本の機関リポジトリによるオープンアクセスのメリットは確認できなかった.

    Tentative
    Study quality
    Limited2.9
    Reliability
    Caveats

How the rubric works →