nabu.science
MethodologyAboutContact

 

  • Snapshot
  • Key Claims
  • Strengths & Limitations
  • Commentary
  • From the Authors
  • Author Note / Field Feedback
  • Dimension breakdown
  • Impact Profile
Free · About 5 minutes
Report #NBU-55B073E7·Framework v5.9·Pipeline v0.4·Evaluated 29 Sept 2026·OECD: 5.8 Other social sciences·Subfield: Statistics, Probability and Uncertainty·Basis: Full text·Type: Cross-sectional study

Does open access citation advantage depend on paper topics?

Hajar SotudehHajar Sotudeh·2019·Journal of Information Science·10.1177/0165551519865489 ↗

Primary claim

the OA citation advantage that is observed in the whole sample still holds even for the highly similar OA and NOA papers

Tentative · repeated-document dependence may make the matched-pair significance anti-conservative
In brief

Article-level topic matching advances the open-access citation debate, but unmodelled dependence among repeated matches weakens its central inference.

ReliabilityConcerns4 of 4 checks completeRead the entry ↓
Fragile — the claim's weakest requirement is that the effect exists at all. Based on how the study was designed and conducted, not on statistics extracted from the paper. Experimental. Every point below is traceable to a quoted passage, but this forecast has not yet been validated against known replication outcomes.
Abstract

Research topics vary in their citation potential. In a metric-wise scientific milieu, it would be probable that authors tend to select citation-attractive topics especially when choosing open access (OA) outlets that are more likely to attract citations. Applying a matched-pairs study design, this research aims to examine the role of research topics in the citation advantage of OA papers. Using a comparative citation analysis method, it investigates a sample of papers published in 47 Elsevier article processing charges (APC)-funded journals in different access models including non-open access (NOA), APC, Green and mixed Green-APC. The contents of the papers are analysed using natural language processing techniques at the title and abstract level and served as a basis to match the NOA papers to their peers in the OA models. The publication years and journals are controlled for in order to avoid their impacts on the citation numbers. According to the results, the OA citation advantage that is observed in the whole sample still holds even for the highly similar OA and NOA papers. This implies that the OA citation surplus is not an artefact of the OA and NOA papers’ differences in their topics and, therefore, in their citation potential. This leads to the conclusion that OA authors’ self-selectivity, if it exists at all, is not responsible for the OA citation advantage, at least as far as selection of topics with probably higher citation potentials is concerned.

Key Claims

Primary finding · Tentative

the OA citation advantage that is observed in the whole sample still holds even for the highly similar OA and NOA papers

Fragile — the claim's weakest requirement is that the effect exists at all. Based on how the study was designed and conducted, not on statistics extracted from the paper. Experimental. Every point below is traceable to a quoted passage, but this forecast has not yet been validated against known replication outcomes.

repeated-document dependence may make the matched-pair significance anti-conservative

Abstract; §4.2, Table 5
Quote verified
Secondary · Tentative

the OA citation surplus is not an artefact of the OA and NOA papers’ differences in their topics

exclusionary inference rests on an observational lexical topic proxy with dependent matches

Abstract; §6 Conclusion
Quote verified
Secondary · Tentative

OA authors’ self-selectivity, if it exists at all, is not responsible for the OA citation advantage, at least as far as selection of topics with probably higher citation potentials is concerned

topic matching cannot securely exclude the proposed selection mechanism under the unresolved design limitations

Abstract; §6 Conclusion
Quote verified
Secondary · Supported

the OA papers significantly outperform their highly similar NOA peers no matter if they are published in the same or different years, journals or document types

Table 6 reports consistent stratified directions, though repeated matches temper inferential certainty

§4.3, Table 6
Quote verified

Derived from the full evaluation — not a separate score.

Strengths · Limitations

Strengths

#contribution

Finer-grained topic matching design

The study pairs OA and NOA papers using TF-IDF cosine similarity on titles and abstracts, addressing topic at a finer level than journal- or discipline-level controls.

↳ Introduction §1; Methods §3.1.3

#methodological rigour

Transparent multi-stage analytical workflow

The paper describes the KNIME preprocessing and matching nodes, citation normalisation, similarity clustering, and Wilcoxon comparisons sufficiently to expose the major analytical choices.

↳ Methods §§3.1.3–3.2

#positioning

Substantive engagement with competing explanations

The introduction discusses visibility, early access, quality advantage, quality bias, journal prestige, collaboration, and contrary findings rather than presenting OACA causation as settled.

↳ Introduction §1

Limitations

#methodological rigour

Repeated matches undermine inferential independence

Methods §3.1.3 allows papers to appear in multiple matches, but Tables 4–6 do not adjust the Wilcoxon inference for repeated-document dependence. Reported significance may therefore be anti-conservative.

↳ Methods §3.1.3; Tables 4–6

#methodological rigour

Topic proxy lacks direct validation

TF-IDF cosine similarity over titles and abstracts is treated as equalising topic and citation potential without comparison to alternative semantic measures or quantified measurement error.

↳ Methods §§3.1.3–3.2

#reporting

Conclusion overstates observational exclusion

The abstract and conclusion say topic self-selection is not responsible, although the analysis uses an observational lexical-matching design and leaves material inferential limitations unresolved.

↳ Abstract; Conclusion §6

Does this match your read? Add your assessment ↓

Nabu Commentary

The article makes a distinct contribution by matching OA and NOA papers at the title-and-abstract level and then examining citation differences across similarity strata. Its methods are reported in useful detail, and Table 6 extends the analysis across year, journal, and document-type groupings. The principal constraint is that §3.1.3 permits documents to enter multiple matches while Tables 4–6 do not adjust for the resulting dependence. The abstract and conclusion also convert this observational result into an exclusionary claim about topic-based self-selection, while the limitations section does not address the dependence problem or fully examine the validity of the topic proxy.

Evaluate another paper

Free · About 5 minutes · Research preview

Author Note / Field Feedback

Nabu’s assessment, alongside the field’s view.

Are you an author of this paper?

Dimension breakdown

Evidence Score

Out of 5
012345
2.9±0.2

Contribution
3.4

The study provides a genuine but incremental advance by matching OA and NOA papers at the title-and-abstract topic level rather than relying only on journal or discipline categories. Its contribution is bounded by one publisher, 47 hybrid journals, and a narrow lexical operationalisation of topic.

Introduction §1; Methods §3.1.3; Conclusion §6

papers’ topics are more specific and therefore more prone to reflect authors’ interests and preferences

Methodological Rigour
2.6

The workflow reports sampling, TF-IDF processing, cosine matching, normalisation, Wilcoxon tests, and stratification by publication factors. Methodological Rigour is limited by repeated documents across matches without dependence adjustment, an unvalidated lexical topic proxy, unexplained seed selection, and causal-exclusion language.

Methods §§3.1.3–3.2; Tables 4–6; Conclusion §6

some NOA papers matched more than one document in the OA models and vice versa

Reporting
3.2

The research aims, workflow, and progression from overall to similarity-stratified results are readily followable. The abstract and conclusion nevertheless express the central observational result as exclusion of topic-based self-selection, creating a material mismatch between wording and evidence.

Research Aims §2; Results §§4.1–4.3; Abstract; Conclusion §6

the OACA does not result from OA papers’ differences in topics

Positioning
2.8

The introduction engages competing OACA explanations and both confirming and disconfirming studies, while the conclusion acknowledges publisher scope and subgroup-size limits. It does not acknowledge repeated-match dependence or adequately trace limitations of the topic proxy through the exclusionary conclusion.

Introduction §1; Conclusion §6

there is not yet any certainty about the causation of the phenomenon

Lower confidence on Methodological Rigour — domain match limited.

Reliability

Concerns4 of 4 checks


Internal CoherenceConcern

The reported counts and rank statistics are arithmetically plausible, but the inferential analysis does not account for dependence created when documents participate in multiple matched pairs. That issue could affect the statistical support for the central results.

Research ConductClean

Conflicts and funding are disclosed, and human-participant ethics approval is not expected for this bibliometric study. The absence of shared data, matching outputs, and code limits independent inspection but does not constitute a suspicious availability claim.

Flags: 1 declared / 5 total

Reference IntegrityClean

77 of 77 checkable references verified

86 references in manuscript 9 have no canonical index record — counted, but not index-checkable 8 references confirmed by manual review

References not verified automatically are resolved manually within 24 hours.

Post-Publication RecordClean

No retraction or expression of concern found in Crossref (includes Retraction Watch).

Last checked: 17 Oct 2026

Sources: Crossref (incl. Retraction Watch) ✓

Impact Profile

Impact areaPolicy and practice · SDG 4
Type of advanceEmpirical
Route to useSuggested
AdvanceExtends
CitationsFWCI 2.2
Policy · patentNot yet traced
ReachedNot reachedNot assessable or unverified
ReadinessFoundational
Foundational
Controlled
Piloted
Validated
Implementable
FoundationalImplementable

The claim rests on a single observational matched-pairs analysis of one publisher's journals, with the authors themselves calling for replication on larger, more diverse samples before generalising.

From the paper
“the citation superiority of OA papers may serve as a leverage to stimulate their survival motivations”

The paper suggests its finding could motivate authors to adopt open access, but names no specific organisation, profession or decision-setting that would act on it.

Pathway

Universities, funders and publishers designing open-access mandates and incentives could cite this evidence that OA citation advantage is genuine, encouraging author compliance.

Barrier

The evidence covers only 47 Elsevier hybrid journals with small APC samples, and the authors state replication across publishers and larger samples is required to generalise.

Evaluated 13 Nov 2026 · GBD 2023 · glm_v1_3_3 · impact_profile_v1_3_3

Report an issue →

AI-generated, human-governed. Something look off? Contact us to request a review.

Keep reading

Related evaluations

Other papers Nabu has evaluated blind, starting with the same field.

  1. Suspected distortion of citations in high-impact cancer journals

    Baptiste Scancar, Jennifer Anne Byrne, David Causeur, et al. · 2026 · Media and communications

    Primary claim

    These findings suggest that paper mills inflate the citation metrics of supported publications and affected journals at scale.

    Tentative
    Evidence Score
    3.5±0.1
    Reliability
    Caveats
  2. The citation trap: Papers published at year-end receive systematically fewer citations

    Chao Ma, Yiwei Li, Feng Guo, et al. · 2019 · Economics

    Primary claim

    papers published between October and December on average get as much as 18.5% fewer citations than those published in the other months in the year

    Tentative
    Evidence Score
    3.0±0.2
    Reliability
    Caveats
  3. Quantitative Analysis of the Open Access Citation Advantage

    Masashi Kawai, Koichi Ojiro, Makoto Asaoka, et al. · 2022 · Other social sciences

    Primary claim

    日本の機関リポジトリによるオープンアクセス論文の数は限られるものの,その被引用数と様々なタイプのオープンアクセス論文の被引用数の比較からは,日本の機関リポジトリによるオープンアクセスのメリットは確認できなかった.

    Tentative
    Evidence Score
    2.9±0.2
    Reliability
    Caveats

How the rubric works →