nabu.science
MethodologyAboutContact

 

  • Snapshot
  • Key Claims
  • Strengths & Limitations
  • Commentary
  • From the Authors
  • Author Note / Field Feedback
  • Dimension breakdown
  • Impact Profile
Free · About 5 minutes
Report #NBU-EFAEA80E·Framework v5.9·Pipeline v0.4·Evaluated 28 Sept 2026·OECD: 5.8 Other social sciences·Subfield: Statistics, Probability and Uncertainty·Basis: Full text·Type: Cross-sectional study

The open access citation premium may depend on the openness and inclusiveness of the indexing database, but the relationship is controversial because it is ambiguous where the open access boundary lies

Sergio CopielloSergio Copiello·2019·Scientometrics·10.1007/s11192-019-03221-w ↗

Primary claim

The relationship takes place in two out of the three analyzed case studies, while it is essentially absent in the other case study.

Supported · the descriptive pattern appears across three cases, though purposive selection limits inference beyond them
In brief

A focused bibliometric test reveals database-dependent OA citation patterns, but non-random case selection prevents broad inference.

ReliabilityCaveats4 of 4 checks completeRead the entry ↓
Fragile — the claim's weakest requirement is that the effect exists at all. Experimental. Every point below is traceable to a quoted passage, but this forecast has not yet been validated against known replication outcomes.
Abstract not available — view at DOI ↗

Key Claims

Primary finding · Supported

The relationship takes place in two out of the three analyzed case studies, while it is essentially absent in the other case study.

Fragile — the claim's weakest requirement is that the effect exists at all. Experimental. Every point below is traceable to a quoted passage, but this forecast has not yet been validated against known replication outcomes.

the descriptive pattern appears across three cases, though purposive selection limits inference beyond them

Conclusions
Quote verified
Secondary · Supported

Statistically significant evidence can be achieved only by setting low to medium citation thresholds, thus the relationship fades away when looking at the documents that accrue many cites.

the threshold tables show this pattern, though shrinking samples and repeated testing temper certainty

Conclusions
Quote verified
Secondary · Tentative

While focusing on the Gold OA route leads to inconclusive evidence, the joint consideration of Gold and Green OA provides some confirmation of the tested hypothesis.

the three-case design is limited and confirmation relies on multiple unadjusted threshold-specific tests

Conclusions
Quote verified

Derived from the full evaluation — not a separate score.

Strengths · Limitations

Strengths

#contribution

Specific and identifiable research question

The paper distinguishes database dependence from the broader OA citation-advantage debate and operationalises it through the Google Scholar-to-Scopus citation ratio.

↳ Abstract; Methodological setting

#methodological rigour

Transparent reporting across all thresholds

Search queries, matching procedures, OA classification, and the complete threshold series are reported, including null and conflicting results rather than only favourable comparisons.

↳ Data gathering process; Appendix Tables 4–9

#positioning

Balanced engagement with contested evidence

The background separately organises research supporting and questioning the OA citation advantage, then connects self-selection, early-view effects, and database quality to the study's interpretation.

↳ Background studies; Tables 1–2; Conclusions

Limitations

#methodological rigour

Confounding remains analytically uncontrolled

The OA and non-OA comparisons are not matched or adjusted for known citation drivers such as self-selection, field, article age, or author prominence, limiting interpretation of the observed ratio differences.

↳ Related issues of the citation advantage: open access definition; Methodological setting

#contribution

Non-random cases restrict generalisation

The analysis covers one journal, one topic in one publication year, and one highly prolific author. The data-gathering section acknowledges that the limited, non-random sample reduces generalisability.

↳ Case studies; Data gathering process

#methodological rigour

Repeated tests lack multiplicity strategy

Tables 4–9 report many nested threshold-specific p-values without a stated multiplicity approach, reducing confidence in isolated significant thresholds and Gold-versus-Green contrasts.

↳ Results; Appendix Tables 4–9

Does this match your read? Add your assessment ↓

Nabu Commentary

The paper earns its strongest scores for a clear inferential chain and balanced treatment of the contested OA citation literature. Its contribution is narrower: the GS-to-Scopus ratio offers a useful framing, but findings vary across the journal, topic, and author cases and between Gold-only and combined OA definitions. Methodological Rigour is constrained by the lack of adjustment for known citation drivers, the purposive sample, and unquantified database errors. The complete reporting of Tables 4–9 supports transparency, although the many nested significance tests lack a stated multiplicity strategy.

Evaluate another paper

Free · About 5 minutes · Research preview

Author Note / Field Feedback

Nabu’s assessment, alongside the field’s view.

Are you an author of this paper?

Dimension breakdown

Evidence Score

Out of 5
012345
3.1±0.1

Contribution
3.0

The paper isolates a specific database-dependence hypothesis within the broader OA citation debate and compares Gold-only with Gold-plus-Green definitions. The advance remains incremental because confirmation is mixed across three purposively selected cases.

Abstract; Methodological setting; Conclusions

The relationship takes place in two out of the three analyzed case studies

Methodological Rigour
2.5

Log-transformed citation ratios, explicit threshold sweeps, detailed queries, and full result tables provide a transparent descriptive analysis. The OA-group comparison lacks adjustment for self-selection and other citation drivers, while non-random selection and database measurement errors remain analytically unresolved.

Methodological setting; Data gathering process; Appendix Tables 4–9

it is not randomly chosen and it has a limited size

Reporting
4.2

The paper visibly connects its hypothesis, ratio definition, threshold analyses, and mixed conclusions, with language calibrated to the reported uncertainty. Dense exposition and repeated threshold inference modestly reduce precision without obscuring the central argument.

Abstract; Methodological setting; Results; Conclusions

the hypothesized relationship is neither so strong as to be generally accepted nor so weak as to be neglected

Positioning
4.1

The background presents both supportive and skeptical OA-citation studies and discusses self-selection, early-view effects, OA subtypes, and database quality. The conclusion carries these limitations into the scope and certainty of its interpretation.

Background studies; Tables 1–2; Data gathering process; Conclusions

The results presented in the above-referenced studies do not converge.

Lower confidence on Contribution, Methodological Rigour — domain match limited.

Reliability

Caveats4 of 4 checks


Internal CoherenceNoted

The reported numerical series is internally coherent and includes all examined thresholds, but inferential emphasis across numerous nested tests lacks a stated multiplicity strategy. This affects the weight placed on threshold-specific significance rather than indicating fabricated or contradictory results.

Research ConductClean

The study uses public bibliometric records, describes manual matching and OA classification, and acknowledges database errors and missing raw-data availability. No ethics requirement, availability misrepresentation, or other conduct concern is established.

Flags: 0 declared / 5 total

Reference IntegrityClean

102 of 102 checkable references verified

113 references in manuscript 11 have no canonical index record — counted, but not index-checkable 9 references confirmed by manual review

References not verified automatically are resolved manually within 24 hours.

Post-Publication RecordClean

No retraction or expression of concern found in Crossref (includes Retraction Watch).

Last checked: 17 Oct 2026

Sources: Crossref (incl. Retraction Watch) ✓

Impact Profile

Impact areaPolicy and practice · SDG 17
Type of advanceEmpirical
Route to useNot stated
AdvanceExtends
CitationsFWCI 2.4
Policy · patentNot yet traced
ReachedNot reachedNot assessable or unverified
ReadinessPiloted
Foundational
Controlled
Piloted
Validated
Implementable
FoundationalImplementable

The authors themselves demonstrate the relationship with ANOVA on three small, non-random samples, with no independent replication of the finding.

The conclusion frames the findings as open questions deserving further consideration rather than naming any concrete real-world use.

Pathway

Research evaluators and bibliometric analysts could weigh database openness and open access definitions when using citation metrics in assessment and publishing policy.

Barrier

The relationship appeared in only two non-random case studies and may stem from Google Scholar's duplicate records and citation errors.

Evaluated 13 Nov 2026 · GBD 2023 · glm_v1_3_3 · impact_profile_v1_3_3

Report an issue →

AI-generated, human-governed. Something look off? Contact us to request a review.

Keep reading

Related evaluations

Other papers Nabu has evaluated blind, starting with the same field.

  1. Suspected distortion of citations in high-impact cancer journals

    Baptiste Scancar, Jennifer Anne Byrne, David Causeur, et al. · 2026 · Media and communications

    Primary claim

    These findings suggest that paper mills inflate the citation metrics of supported publications and affected journals at scale.

    Tentative
    Evidence Score
    3.5±0.1
    Reliability
    Caveats
  2. The citation trap: Papers published at year-end receive systematically fewer citations

    Chao Ma, Yiwei Li, Feng Guo, et al. · 2019 · Economics

    Primary claim

    papers published between October and December on average get as much as 18.5% fewer citations than those published in the other months in the year

    Tentative
    Evidence Score
    3.0±0.2
    Reliability
    Caveats
  3. Quantitative Analysis of the Open Access Citation Advantage

    Masashi Kawai, Koichi Ojiro, Makoto Asaoka, et al. · 2022 · Other social sciences

    Primary claim

    日本の機関リポジトリによるオープンアクセス論文の数は限られるものの,その被引用数と様々なタイプのオープンアクセス論文の被引用数の比較からは,日本の機関リポジトリによるオープンアクセスのメリットは確認できなかった.

    Tentative
    Evidence Score
    2.9±0.2
    Reliability
    Caveats

How the rubric works →