nabu.science
MethodologyAboutContact

 

  • Snapshot
  • Key Claims
  • Strengths & Limitations
  • Commentary
  • From the Authors
  • Author Note / Field Feedback
  • Dimension breakdown
  • Impact Profile
Free · About 5 minutes
Report #NBU-82B9DDF5·Framework v5.9·Pipeline v0.4·Evaluated 28 Sept 2026·OECD: 5.8 Other Social Sciences·Subfield: Statistics, Probability and Uncertainty·Basis: Full text·Type: Dataset or benchmark paper

Proportion of Open Access Papers Published in Peer-Reviewed Journals at the European and World Levels, 1996-2013

Éric Archambault et al.Éric Archambault, Didier Amyot, Philippe Deschamps et al.·2014·Lincoln (University of Nebraska)·10.5281/zenodo.10030609 ↗

Primary claim

As of April 2014, more than 50% of the scientific papers published in 2007, 2008, 2009, 2010, 2011, and 2012 can be downloaded for free on the Internet.

Tentative · the adjusted year-specific threshold depends on uncertainty calculations that may misstate whether each estimate clears 50%
In brief

A large-scale open-access benchmark with unusually explicit calibration, constrained by flawed uncertainty calculations and overinterpreted citation associations.

ReliabilityConcerns4 of 4 checks completeRead the entry ↓
Abstract

This study report assesses the free availability of scholarly publications during the 1996 to 2013 period. It is the largest scale measurement of open access availability performed to date: a sample of one-quarter of a million records was used to study the historical evolution of open access (OA) between 1996 and 2013 and a larger, one million records sample was used to perform an in-depth assessment of the proportion and scientific impact of OA between 2008 and 2013 in different types of OA, for different scientific fields of knowledge, and for 44 countries, the EU28, ERA, and the world. Compared to previous studies done on the availability of OA, the present study presents the following characteristics: (1) it used the Scopus database, which currently covers a broader range of journals from various countries and scientific disciplines than other comprehensive databases; (2) it uses a simple definition of OA—freely available online to all (no money had to be paid, no registration to a service or website had to be made); (3) it used huge samples to maximise statistical precision; (4) it made careful and extensive efforts to harvest papers wherever they could be downloaded for free, without restriction, rather than restricting the approach to a search engine (in order to obtain a high 'recall' rate, that is, the capacity to retrieve a large part of the relevant records, while, in addition, carefully minimising the number of false records collected (that is, the approach maximised retrieval precision); and (5) it carefully characterised the strengths and weaknesses of the measurement instrument in order to apply a correction that would provide a truer measure based on an Adjusted OA score. This study also provided a series of rational definitions of access, open access, and ideal open access. The definitions provided examine aspects such as restrictions, payment, delay, transiency, and legitimacy. Because of the limited means (time and budget) available for this project, it was necessary to use operational definitions of OA which do not provide all the details one may wish to obtain. Though it was easy to obtain a clear and easily operational definition of Gold OA by stating that it referred to papers published in Gold OA journals listed in the Directory of Open Access Journals, defining and measuring Green OA was more challenging. The operational definition restricted Green OA to researchers' self-archived papers in institutional and some thematic repositories listed in OpenDOAR and ROAR. This left a sizeable residual number of papers that could still be downloaded for free; these were classified as Other OA. This comprises, for example, Gold OA papers from subscription-based journals, which are made available through article processing charges (APC). Other OA also include papers available in large repositories such as PubMed Central and aggregator sites such as CiteSeerX. There are also Robin Hood OA or Rogue OA papers, that is, papers that infringe on copyrights by making them accessible to the public despite licenses that restrict them to being behind pay walls.

Key Claims

Primary finding · Tentative

As of April 2014, more than 50% of the scientific papers published in 2007, 2008, 2009, 2010, 2011, and 2012 can be downloaded for free on the Internet.

the adjusted year-specific threshold depends on uncertainty calculations that may misstate whether each estimate clears 50%

Executive Summary; §3.5; Figure 9
Quote verified
Secondary · Supported

On average, the citation advantage of OA papers is 40.3% while the citation disadvantage is 27% for non-OA papers.

a large field-normalized sample supports the association, but the observational comparison does not identify an access effect

Executive Summary; §3.6; Figure 10
Quote verified
Secondary · Supported

about 47% of the papers indexed in Scopus between 1996 and 2013 can be downloaded for free as of April 2014.

the large adjusted sample supports the estimate's direction, though calibration and uncertainty calculations limit its precision

Executive Summary; §3.1; Figure 5
Quote verified

Derived from the full evaluation — not a separate score.

Strengths · Limitations

Strengths

#contribution

Unusually broad empirical coverage

The analysis combines a roughly 245,000-record historical sample with a one-million-record sample spanning 22 fields and 44 countries, supporting temporal and cross-sectional policy comparisons.

↳ Executive Summary; Methods §2.3; Tables II–XIV

#methodological rigour

Explicit instrument calibration and limitations

The report estimates retrieval precision, recall, and an adjustment factor using a 500-record calibration, while disclosing that calibration is small, limited to 2008, and unavailable for individual OA categories.

↳ Methods §2.2; Table I

#application pathway

Direct policy-monitoring pathway

The report was produced under a European Commission contract to develop open-access indicators and provides country, field, and temporal results suited to monitoring and policy deliberation.

↳ Front matter; §§5–6

Limitations

#methodological rigour

Uncertainty calculations threaten threshold claims

Methods §2.1 specifies a critical score of 0.95, while §2.2 combines unsquared margin components under a square root. As written, these formulas may misstate uncertainty around Adjusted OA estimates and whether each annual estimate truly exceeds 50%.

↳ Methods §§2.1–2.2; §3.5

#positioning

Citation associations become causal guidance

The field-normalized comparisons are observational, yet §4.2 and Table V recommend Green OA as the rational strategy for maximizing citations without addressing self-selection or related article, author, and journal differences.

↳ §4.2; Tables IV–V

#methodological rigour

Largest category remains heterogeneous

Other OA combines delayed, hybrid, aggregator-hosted, transient, and potentially copyright-infringing copies. Because it contributes the largest share of measured OA, its lack of disaggregation restricts interpretation and transferability.

↳ §§1.2 and 3.5

Does this match your read? Add your assessment ↓

Nabu Commentary

The contribution score reflects the report's large samples, broad field and country coverage, and explicit calibration against a 500-record set. Methodological Rigour is held near the boundary between score bands because the detailed measurement design coexists with a 2008-only calibration, a proprietary pipeline, and uncertainty formulas that may not support the reported precision. Reporting and Positioning are reduced by the confident 50% threshold language and by Table V's conversion of observational citation associations into publishing advice without addressing self-selection. Impact potential remains high because the study was commissioned for a named European Commission indicator programme and supplies directly policy-relevant outputs.

Evaluate another paper

Free · About 5 minutes · Research preview

Author Note / Field Feedback

Nabu’s assessment, alongside the field’s view.

Are you an author of this paper?

Dimension breakdown

Evidence Score

Out of 5
012345
3.4±0.2

Contribution
3.8

The report establishes a large descriptive benchmark using approximately 245,000 historical records and one million records for country-by-field analyses. Its scale, OA taxonomy, and calibration extend earlier availability studies, although the heterogeneous Other OA category limits interpretation.

Executive Summary; Introduction; Methods §2.3; §3.5

Some 1.25 million papers were sampled to carry out the present study.

Methodological Rigour
3.4

Random sampling, operational definitions, precision and recall estimates, and explicit calibration limitations support the design. Methodological Rigour is constrained by the small 2008-only calibration, problematic uncertainty formulas, proprietary harvesting system, and unavailable code and raw classifications.

Methods §§1.2 and 2.1–2.3; Table I

Only records from 2008 were calibrated.

Reporting
3.3

The report follows a coherent definitions-to-methods-to-results structure and usually distinguishes measured from Adjusted OA. Its confidence in the 50% threshold and prescriptive language about citation advantage are stronger than the uncertainty treatment and observational citation comparison warrant.

§3.5; §4.2; Table V

we can confidently assert that... more than 50% of the scientific papers... can be downloaded for free

Positioning
2.7

The report engages major OA definitions, earlier availability studies, and several measurement limitations. It does not address self-selection when converting observational citation differences into publishing advice, and stated calibration limitations are not carried through to the certainty of the threshold conclusion.

Introduction; §§1.1, 2.2, 3.5, 4.2; Table V

rational scientists would consider this choice the most obvious choice

Reliability

Concerns4 of 4 checks


Internal CoherenceConcern

The descriptive figures are generally internally consistent, but the stated uncertainty calculations are mathematically problematic and bear directly on threshold claims based on Adjusted OA estimates.

Research ConductClean

No ethics, conflict, protocol, or suspicious availability issue is evident for this non-human-subject bibliometric study. The proprietary harvester and absent code and raw classifications are reproducibility gaps rather than conduct concerns.

Flags: 1 declared / 5 total

Reference IntegrityClean

37 of 37 checkable references verified

46 references in manuscript 9 have no canonical index record — counted, but not index-checkable 9 references confirmed by manual review

References not verified automatically are resolved manually within 24 hours.

Post-Publication RecordClean

No retraction or expression of concern found in Crossref (includes Retraction Watch).

Last checked: 17 Oct 2026

Sources: Crossref (incl. Retraction Watch) ✓

Impact Profile

Impact areaPolicy and practice
Type of advanceEmpirical
Route to useNamed
AdvanceExtends
CitationsFWCI 7.2
Policy · patentNot yet traced
ReachedNot reachedNot assessable or unverified
ReadinessValidated
Foundational
Controlled
Piloted
Validated
Implementable
FoundationalImplementable

The claim is based on large samples (245,000 and 1,000,000 records) across many countries and fields, with instrument calibration, constituting real-world population data though from a single measurement effort.

From the paper
“it is worthwhile using institutional repositories and immediate Green OA to cut the delays associated with what many consider to be weak OA mandates”

The report names who would act (researchers, institutions via repositories) and for what purpose, and discusses policy implications for OA mandates and monitoring by bodies like the European Commission.

Pathway

European Commission and national research funders would use these indicators to monitor and refine open access mandates and policies for scholarly publishing.

Barrier

The measurement relies on a single harvester calibrated on only 500 records with transient web availability, so replication and finer disaggregation of OA types are needed before policy decisions.

Evaluated 13 Nov 2026 · GBD 2023 · glm_v1_3_3 · impact_profile_v1_3_3

Report an issue →

AI-generated, human-governed. Something look off? Contact us to request a review.

Keep reading

Related evaluations

Other papers Nabu has evaluated blind, starting with the same field.

  1. Suspected distortion of citations in high-impact cancer journals

    Baptiste Scancar, Jennifer Anne Byrne, David Causeur, et al. · 2026 · Media and communications

    Primary claim

    These findings suggest that paper mills inflate the citation metrics of supported publications and affected journals at scale.

    Tentative
    Evidence Score
    3.5±0.1
    Reliability
    Caveats
  2. The citation trap: Papers published at year-end receive systematically fewer citations

    Chao Ma, Yiwei Li, Feng Guo, et al. · 2019 · Economics

    Primary claim

    papers published between October and December on average get as much as 18.5% fewer citations than those published in the other months in the year

    Tentative
    Evidence Score
    3.0±0.2
    Reliability
    Caveats
  3. Quantitative Analysis of the Open Access Citation Advantage

    Masashi Kawai, Koichi Ojiro, Makoto Asaoka, et al. · 2022 · Other social sciences

    Primary claim

    日本の機関リポジトリによるオープンアクセス論文の数は限られるものの,その被引用数と様々なタイプのオープンアクセス論文の被引用数の比較からは,日本の機関リポジトリによるオープンアクセスのメリットは確認できなかった.

    Tentative
    Evidence Score
    2.9±0.2
    Reliability
    Caveats

How the rubric works →