Loading evaluation data…
To examine changes in the open access landscape over time, this study partially replicated Kristin Antelman’s 2004 study of open access citation advantage. Results indicated open access articles still have a citation advantage. For three of the four disciplines examined, the most common sites hosting freely available articles were independent sites, such as academic social networks or article sharing sites. For the same three disciplines, more than 70% of the open access copies were publishers’ PDFs. The major difference from Antelman’s is the increase in the number of freely available articles that appear to be in violation of publisher policies.
Results indicated open access articles still have a citation advantage.
significant median differences across four disciplines, but uncontrolled confounding prevents causal interpretation
For three of the four disciplines examined, the most common sites hosting freely available articles were independent sites, such as academic social networks or article-sharing sites.
Table 3 supports the pattern, though first-result Google coding may favor independent sites
For the same three disciplines, more than 70 percent of the open access copies were publishers’ PDFs.
Table 4 directly reports the percentages, though document categorization relies on visible publisher-imprint signals
The major difference from Antelman’s is the increase in the number of freely available articles that appear to be in violation of publisher policies.
permission status was not assessed directly, so policy noncompliance remains an inference from site and imprint patterns
more articles are freely available across all four disciplines
Table 1 shows higher free-access shares, though changed samples and searches limit exact historical comparison
Derived from the full evaluation — not a separate score.
Strengths
The Methodology identifies the intended correspondence with the earlier study and explains changes to sample construction, title searching, hosting categories, and version coding. This makes the limits of longitudinal comparison visible.
↳ Methodology, pp. 307–310
Tables 1–4 update free availability, citation medians, hosting locations, and publisher-imprint prevalence for mathematics, engineering, political science, and philosophy. The discipline-specific reporting also exposes substantial heterogeneity rather than imposing a universal pattern.
↳ Results, Tables 1–4, pp. 310–313
The Discussion explains how journal policies, unknown access timing, Google ranking, and reverse causation constrain interpretation of the citation association. It explicitly declines claims about changes in author behavior.
↳ Discussion, pp. 313–315
Limitations
The free-versus-toll comparison does not adjust for journal citation baselines, delayed-access policies, article citability, author selection, or reverse causation. Consequently, the reported association cannot establish that free access caused additional citations.
↳ Discussion, pp. 313–315
The sample sizes differ from the original study, searches use an escalation procedure, hosting categories were redefined, and publisher imprint replaced the preprint/postprint distinction. These changes make the replication informative but not directly equivalent.
↳ Methodology, pp. 307–310; Discussion, p. 313
Each article was coded using the first freely available copy in Google results rather than systematically cataloguing all copies or preferring repositories. Independent sites may therefore be overrepresented in the hosting distribution.
↳ Methodology, pp. 309–310; Discussion, p. 314
The study earns credit for a preregistered partial replication, transparent methodological departures, and discipline-specific results across Tables 1–4. Its strongest methodological constraint is the unadjusted free-versus-toll citation comparison, which cannot distinguish an access effect from journal characteristics, selection, or reverse causation. The hosting-site analysis is informative but depends on whichever free copy appeared first in Google, limiting direct comparison with the earlier repository-preference rule. Clear noncausal language and substantive engagement with contrary evidence strengthen the interpretation despite these design limits.
Nabu’s assessment, alongside the field’s view.
Are you an author of this paper?
Sound3.3
Confidence highThe study meaningfully extends a foundational citation-advantage study by repeating its four-discipline comparison and documenting changes in hosting sites and document versions. Its advance remains bounded by partial replication and changed categories and procedures.
“This study revisits a prominent early open access citation advantage study in the context of this changing environment.”
The preregistered methods, search escalation rules, sampling choices, and nonparametric analyses are described transparently. The primary citation association lacks adjustment for journal characteristics, article citability, self-selection, or reverse causation, and first-result Google coding introduces additional selection uncertainty.
“These journal practices pose a problem for attributing the higher citations of open access articles to open access.”
The paper follows a clear progression from replication rationale through methods, discipline-specific results, and interpretation. It repeatedly distinguishes citation association from causation and states the consequences of methodological departures, although the abstract compresses these qualifications.
“This lens, however, cannot clarify everything, both because of limitations inherent in the study design.”
The literature review includes observational support, attenuated and null findings, and randomized experimental evidence. The Discussion connects sampling, access-duration, journal-policy, and Google-ranking limitations to the scope and certainty of the conclusions.
“The results of this study cannot support claims about changes in author behavior.”
Lower confidence on Positioning — domain match limited.
Caveats4 of 4 checks
The discipline totals, free-access counts, percentages, and statistical results are internally consistent, and departures from the original study are disclosed throughout the Methodology. The paper does not present the observed citation association as a demonstrated causal effect.
The analysis plan is identified as preregistered on OSF, and the bibliometric/web-search design does not require human-participant ethics approval. No conduct concern is evident in the supplied text.
Flags: 1 declared / 5 total
41 of 49 checkable references verified
65 references in manuscript 8 are books, websites or datasets — counted, but not index-checkable 8 could not be checked — not the same as no match found citation details diverge from the cited records for 1 reference
No retraction notice found in Retraction Watch.
Sources: Retraction Watch ✓
Where this paper’s evidence sits on the path from initial observation to real-world use.
The study observes citation and web-availability patterns rather than testing an intervention or implementation strategy. Its results can inform discussion, but they do not provide a deployment-ready practice recommendation.
AI-generated, human-governed. Something look off? Contact us to request a review.