# Who's Aligned to Whom? How AI Agents Shape Experts' Research Judgment Calls

- **DOI:** [10.0000/anthropic-whos-aligned-to-whom-ai-agents](https://doi.org/10.0000/anthropic-whos-aligned-to-whom-ai-agents)
- **Rating:** Strong (4.0 out of 5)
- **Verdict:** Randomized agent preferences reveal substantial influence on practitioners' analytical choices, but one synthetic task limits conclusions about real expert workflows.
- **Framework version:** v5.10
- **Pipeline version:** v0.4
- **Evaluation date:** 2026-11-04
- **Evaluation page:** https://www.nabu.science/paper/10.0000%2Fanthropic-whos-aligned-to-whom-ai-agents

## Key claims

1. On the analytical decisions, participants went with the agent's lean 86% of the time (50% by chance)
   - Support: Well-supported
   - Type: Primary finding
   - Basis: randomized preferences and consistent primary estimates support the observed alignment rate within this task
   - Location: Abstract
2. the four intermediate leans alone correctly predict 82% of delivered recommendations among participants who did not choose an extension (61% majority baseline; AUC 0.84)
   - Support: Supported
   - Type: Secondary
   - Basis: exploratory in-sample regression in a small single-task sample rather than independent predictive validation
   - Location: Section 1.1, Summary of Our Results; Section 4.1
3. Yet when asked to justify their choices, 77% repeated the agent's talking points, sometimes nearly verbatim, and only 15% mentioned the agent in any answer.
   - Support: Supported
   - Type: Secondary
   - Basis: descriptive coding supports repetition and attribution rates, but the shared briefing prevents identifying unrecognized deferral
   - Location: Abstract; Section 4.2.2
4. more than half [of participants] rated the recommendation as more their own than the agent's
   - Support: Supported
   - Type: Secondary
   - Basis: direct ownership self-reports describe perceived authorship rather than independently measured control
   - Location: Abstract; Section 4.2.2, Figure 3
5. The shift on the policy recommendation was smaller and not statistically significant under the registered primary estimator: 17% of the distance from discontinuing to rolling out (22% unadjusted).
   - Support: Supported
   - Type: Secondary
   - Basis: randomized comparison supports the qualified estimate, but the small sample leaves a wide confidence interval
   - Location: Section 1.1, Summary of Our Results; Table 2

## Strengths

- **Randomized preferences identify influence on analytical choices** Hidden, randomly assigned agent preferences allow a causal comparison without labeling analytical options right or wrong. The primary pooled arm contrast is 0.72 with a 95% confidence interval of 0.60–0.83.
  - Dimension: contribution
  - Location: Sections 3.3, 3.5, and 4.1; Table 2
- **Inference and hypothesis handling match the design** The primary analysis uses participant-clustered CR2 inference for repeated decisions. H3 is explicitly downgraded after its falsification check fails, and the non-significant registered policy estimate is distinguished from the significant unadjusted result.
  - Dimension: methodological rigour
  - Location: Section 3.5; Table 2
- **Limitations are connected to their interpretive consequences** The discussion explains how amplified preferences, a single setting, and average-only identification constrain the findings. It also acknowledges that agents can curb analytical error rather than treating all expert discretion as beneficial.
  - Dimension: positioning
  - Location: Sections 2 and 5

## Limitations

- **Nonuser exclusions leave selection uncertainty after assignment** Eleven of 61 valid completers never used the agent and were excluded from the main analysis. The absence of arm-wise take-up comparisons leaves uncertainty about whether the included users remain comparable across assigned configurations.
  - Dimension: methodological rigour
  - Location: Sections 3.1 and 3.3
- **Exploratory measures do not establish unrecognized deferral** Talking-point repetition can reflect the shared task briefing, and participants were asked to justify choices rather than attribute their sources. The 82% recommendation-prediction result is also in-sample rather than an independently validated predictive result.
  - Dimension: reporting
  - Location: Abstract; Sections 4.1 and 4.2.2; Appendix A.5
- **One amplified setting limits real workflow generalization** A single synthetic task and pinned agent cannot establish how frequently comparable influence occurs in professional settings with stronger domain knowledge and stakeholder input. No proposed countermeasure is tested.
  - Dimension: application pathway
  - Location: Sections 5–6

## Reliability checks

Caveats (4 of 4 checks).

- **Results check:** Clean. The reported numbers and statistics are consistent with the stated results.
- **Reporting and transparency:** Noted. The pre-analysis plan was registered after data collection began, with the timing and exceptions to the analysis freeze disclosed. Ethics review and participant consent are declared.
  - Ethics approval: declared · Preregistration: declared
- **Reference integrity:** Clean
  - 47 references in manuscript
  - 43 of 43 checkable references found in an index
  - 4 have no canonical index record — counted, but not index-checkable
  - 1 reference confirmed by manual review
  - References not verified automatically are resolved manually within 24 hours.

## Post-publication record

- **Status:** Clean
- **Summary:** No retraction or expression of concern found in Crossref (includes Retraction Watch).
- **Last checked:** 2026-11-04
