# Proportion of Open Access Papers Published in Peer-Reviewed Journals at the European and World Levels, 1996-2013

- **DOI:** [10.5281/zenodo.10030609](https://doi.org/10.5281/zenodo.10030609)
- **Rating:** Sound (3.4 out of 5)
- **Verdict:** A large-scale open-access benchmark with unusually explicit calibration, constrained by flawed uncertainty calculations and overinterpreted citation associations.
- **Framework version:** v5.9
- **Pipeline version:** v0.4
- **Evaluation date:** 2026-09-28
- **Evaluation page:** https://www.nabu.science/paper/10.5281%2Fzenodo.10030609

## Key claims

1. As of April 2014, more than 50% of the scientific papers published in 2007, 2008, 2009, 2010, 2011, and 2012 can be downloaded for free on the Internet.
   - Support: Tentative
   - Type: Primary finding
   - Basis: the adjusted year-specific threshold depends on uncertainty calculations that may misstate whether each estimate clears 50%
   - Location: Executive Summary; §3.5; Figure 9
2. On average, the citation advantage of OA papers is 40.3% while the citation disadvantage is 27% for non-OA papers.
   - Support: Supported
   - Type: Secondary
   - Basis: a large field-normalized sample supports the association, but the observational comparison does not identify an access effect
   - Location: Executive Summary; §3.6; Figure 10
3. about 47% of the papers indexed in Scopus between 1996 and 2013 can be downloaded for free as of April 2014.
   - Support: Supported
   - Type: Secondary
   - Basis: the large adjusted sample supports the estimate's direction, though calibration and uncertainty calculations limit its precision
   - Location: Executive Summary; §3.1; Figure 5

## Strengths

- **Unusually broad empirical coverage** The analysis combines a roughly 245,000-record historical sample with a one-million-record sample spanning 22 fields and 44 countries, supporting temporal and cross-sectional policy comparisons.
  - Dimension: contribution
  - Location: Executive Summary; Methods §2.3; Tables II–XIV
- **Explicit instrument calibration and limitations** The report estimates retrieval precision, recall, and an adjustment factor using a 500-record calibration, while disclosing that calibration is small, limited to 2008, and unavailable for individual OA categories.
  - Dimension: methodological rigour
  - Location: Methods §2.2; Table I
- **Direct policy-monitoring pathway** The report was produced under a European Commission contract to develop open-access indicators and provides country, field, and temporal results suited to monitoring and policy deliberation.
  - Dimension: application pathway
  - Location: Front matter; §§5–6

## Limitations

- **Uncertainty calculations threaten threshold claims** Methods §2.1 specifies a critical score of 0.95, while §2.2 combines unsquared margin components under a square root. As written, these formulas may misstate uncertainty around Adjusted OA estimates and whether each annual estimate truly exceeds 50%.
  - Dimension: methodological rigour
  - Location: Methods §§2.1–2.2; §3.5
- **Citation associations become causal guidance** The field-normalized comparisons are observational, yet §4.2 and Table V recommend Green OA as the rational strategy for maximizing citations without addressing self-selection or related article, author, and journal differences.
  - Dimension: positioning
  - Location: §4.2; Tables IV–V
- **Largest category remains heterogeneous** Other OA combines delayed, hybrid, aggregator-hosted, transient, and potentially copyright-infringing copies. Because it contributes the largest share of measured OA, its lack of disaggregation restricts interpretation and transferability.
  - Dimension: methodological rigour
  - Location: §§1.2 and 3.5

## Reliability checks

Concerns (4 of 4 checks).

- **Internal coherence:** Concern. The descriptive figures are generally internally consistent, but the stated uncertainty calculations are mathematically problematic and bear directly on threshold claims based on Adjusted OA estimates.
- **Research conduct:** Clean. No ethics, conflict, protocol, or suspicious availability issue is evident for this non-human-subject bibliometric study. The proprietary harvester and absent code and raw classifications are reproducibility gaps rather than conduct concerns.
- **Reference integrity:** Clean. 37 of 37 checkable references verified
  - 46 references in manuscript
  - 9 have no canonical index record — counted, but not index-checkable
  - 9 references confirmed by manual review
  - References not verified automatically are resolved manually within 24 hours.

## Post-publication record

- **Status:** Clean
- **Summary:** No retraction or expression of concern found in Crossref (includes Retraction Watch).
- **Last checked:** 2026-10-17
