# Does open access citation advantage depend on paper topics?

- **DOI:** [10.1177/0165551519865489](https://doi.org/10.1177/0165551519865489)
- **Rating:** Limited (2.9 out of 5)
- **Verdict:** Article-level topic matching advances the open-access citation debate, but unmodelled dependence among repeated matches weakens its central inference.
- **Framework version:** v5.9
- **Pipeline version:** v0.4
- **Evaluation date:** 2026-09-29
- **Evaluation page:** https://www.nabu.science/paper/10.1177%2F0165551519865489

## Key claims

1. the OA citation advantage that is observed in the whole sample still holds even for the highly similar OA and NOA papers
   - Support: Tentative
   - Type: Primary finding
   - Basis: repeated-document dependence may make the matched-pair significance anti-conservative
   - Location: Abstract; §4.2, Table 5
2. the OA citation surplus is not an artefact of the OA and NOA papers’ differences in their topics
   - Support: Tentative
   - Type: Secondary
   - Basis: exclusionary inference rests on an observational lexical topic proxy with dependent matches
   - Location: Abstract; §6 Conclusion
3. OA authors’ self-selectivity, if it exists at all, is not responsible for the OA citation advantage, at least as far as selection of topics with probably higher citation potentials is concerned
   - Support: Tentative
   - Type: Secondary
   - Basis: topic matching cannot securely exclude the proposed selection mechanism under the unresolved design limitations
   - Location: Abstract; §6 Conclusion
4. the OA papers significantly outperform their highly similar NOA peers no matter if they are published in the same or different years, journals or document types
   - Support: Supported
   - Type: Secondary
   - Basis: Table 6 reports consistent stratified directions, though repeated matches temper inferential certainty
   - Location: §4.3, Table 6

## Strengths

- **Finer-grained topic matching design** The study pairs OA and NOA papers using TF-IDF cosine similarity on titles and abstracts, addressing topic at a finer level than journal- or discipline-level controls.
  - Dimension: contribution
  - Location: Introduction §1; Methods §3.1.3
- **Transparent multi-stage analytical workflow** The paper describes the KNIME preprocessing and matching nodes, citation normalisation, similarity clustering, and Wilcoxon comparisons sufficiently to expose the major analytical choices.
  - Dimension: methodological rigour
  - Location: Methods §§3.1.3–3.2
- **Substantive engagement with competing explanations** The introduction discusses visibility, early access, quality advantage, quality bias, journal prestige, collaboration, and contrary findings rather than presenting OACA causation as settled.
  - Dimension: positioning
  - Location: Introduction §1

## Limitations

- **Repeated matches undermine inferential independence** Methods §3.1.3 allows papers to appear in multiple matches, but Tables 4–6 do not adjust the Wilcoxon inference for repeated-document dependence. Reported significance may therefore be anti-conservative.
  - Dimension: methodological rigour
  - Location: Methods §3.1.3; Tables 4–6
- **Topic proxy lacks direct validation** TF-IDF cosine similarity over titles and abstracts is treated as equalising topic and citation potential without comparison to alternative semantic measures or quantified measurement error.
  - Dimension: methodological rigour
  - Location: Methods §§3.1.3–3.2
- **Conclusion overstates observational exclusion** The abstract and conclusion say topic self-selection is not responsible, although the analysis uses an observational lexical-matching design and leaves material inferential limitations unresolved.
  - Dimension: reporting
  - Location: Abstract; Conclusion §6

## Reliability checks

Concerns (4 of 4 checks).

- **Internal coherence:** Concern. The reported counts and rank statistics are arithmetically plausible, but the inferential analysis does not account for dependence created when documents participate in multiple matched pairs. That issue could affect the statistical support for the central results.
- **Research conduct:** Clean. Conflicts and funding are disclosed, and human-participant ethics approval is not expected for this bibliometric study. The absence of shared data, matching outputs, and code limits independent inspection but does not constitute a suspicious availability claim.
- **Reference integrity:** Clean. 77 of 77 checkable references verified
  - 86 references in manuscript
  - 9 have no canonical index record — counted, but not index-checkable
  - 8 references confirmed by manual review
  - References not verified automatically are resolved manually within 24 hours.

## Post-publication record

- **Status:** Clean
- **Summary:** No retraction or expression of concern found in Crossref (includes Retraction Watch).
- **Last checked:** 2026-10-17
