# A Heart Means Recursion: Exploring Visual Meaning and Communication in VLMs

- **DOI:** [10.48550/arxiv.2610.36449](https://doi.org/10.48550/arxiv.2610.36449)
- **Rating:** Sound (3.5 out of 5)
- **Verdict:** Simple symbols shift VLM coding choices across a broad experimental suite, but prompt-dependent effects constrain the reach of its visual-meaning conclusions.
- **Framework version:** v5.10
- **Pipeline version:** v0.4
- **Evaluation date:** 2026-11-04
- **Evaluation page:** https://www.nabu.science/paper/10.48550%2Farxiv.2610.36449

## Key claims

1. simple visual symbols like a red heart can strongly influence model behavior on seemingly unrelated tasks, such as steering models towards recursive solutions on simple coding problems
   - Support: Supported
   - Type: Primary finding
   - Basis: selected symbols shift coding choices, with small per-condition samples and prompt-dependent effects limiting generality
   - Location: Abstract; Sec. 3.1
2. models' direct interpretations and thinking traces about visual symbols diverge substantially from their observed behavior
   - Support: Tentative
   - Type: Secondary
   - Basis: unvalidated trace classification and symbol-level attribution limit inference about implicit influence
   - Location: Abstract; Sec. 3.4
3. VLMs generate creative visualizations to communicate abstract concepts and stories, but do not always interpret their own visualizations in the intended way.
   - Support: Supported
   - Type: Secondary
   - Basis: controlled games demonstrate communication successes and failures, with same-model sender–receiver testing limiting transfer
   - Location: Introduction, main finding 2; Section 4.2
4. Models' visual meaning spaces are only partially aligned with human interpretations.
   - Support: Tentative
   - Type: Secondary
   - Basis: seven participants and outcome-conditioned visualization selection provide only exploratory evidence of human–model alignment
   - Location: Introduction, main finding 3; Section 5
5. Beyond coding, the same 288 colored shapes reliably steer both writing style and responses to binary preference questions.
   - Support: Tentative
   - Type: Secondary
   - Basis: binary preferences were tested with five selected symbols rather than the claimed 288
   - Location: Section 3.3, paragraph beginning 'Visual steering influences behavior on non-coding tasks'

## Strengths

- **Controls distinguish symbols from generic images** Token-matched captions, Gaussian noise, natural images, and direct text instructions test competing explanations for coding-strategy shifts. Perturbations further probe the role of shape geometry and orientation.
  - Dimension: methodological rigour
  - Location: Section 3.2; Appendices B.2.2–B.2.3
- **Integrated experiments connect steering and communication** A 288-symbol assay is linked to sender–receiver games and human interpretation, bringing behavioral influence and visual communication into one empirical framework.
  - Dimension: contribution
  - Location: Sections 3–5
- **Literature framing separates related visual phenomena** The related-work section distinguishes simple-symbol steering from adversarial manipulation, visual distractors, visual reasoning, and emergent communication. The discussion explicitly treats deployment implications as prospective.
  - Dimension: positioning
  - Location: Sections 2 and 6

## Limitations

- **Preference experiment covers five symbols only** Section 3.3 attributes both writing-style and binary-preference steering to the same 288 shapes, but Appendix B.3.1 lists only five preference symbols. The preference evidence therefore has substantially narrower coverage than the Results statement.
  - Dimension: reporting
  - Location: Section 3.3; Appendix B.3.1, Binary preference choices
- **Trace analysis lacks independent classifier validation** The regex classifier is not validated like the separate code-strategy judge. Its assignment of symbol-level steering effects to individual traces limits conclusions about whether models recognize an image's influence.
  - Dimension: methodological rigour
  - Location: Section 3.4; Appendix B.8
- **Selected maxima obscure prompt dependent effects** Headline symbol effects are selected from 288 conditions with 20 generations each and no held-out rerun. Prompt wording and image placement substantially change steering, limiting extrapolation from the strongest examples.
  - Dimension: methodological rigour
  - Location: Section 3.1; Appendix B.2.4

## Reliability checks

Concerns (4 of 4 checks).

- **Results check:** Concern
  - Beyond coding, the same 288 colored shapes reliably steer both writing style and responses to binary preference questions: "Beyond coding, the same 288 colored shapes reliably steer both writing style and responses to binary preference questions." (Section 3.3, paragraph beginning 'Visual steering influences behavior on non-coding tasks') vs "We use a small subset of visual symbols to steer preferences: red circle, pink heart, red square, green triangle, and blue triangle." (Appendix B.3.1, 'Binary preference choices'). Counting the five named preference symbols contradicts the Results attribution of the 288-symbol library to both writing-style and preference experiments. → Changes the size materially
  - 97.8% of experiments eventually converge on the correct target, with 91.2% succeeding zero-shot: "97.8% of experiments eventually converge on the correct target, with 91.2% succeeding zero-shot (i.e. on the first communication turn; Fig. 2B)." (Section 4.2, paragraph beginning 'Models construct creative visualizations') vs "| GPT 5.4 | Free-form PIL | 78.2% | 6.4% |" (Appendix C.3, Table 13, Concept Communication, GPT 5.4 Free-form PIL row). Approximately weighting the six rounded free-form rows by the stated 30 concept and 20 story sets gives 91.7% zero-shot and 98.4% eventual success, assuming these set sizes are the aggregation denominators. → Effect on result unclear
  - Minor reporting notes (1):
    - Descriptive: "binary search, maximum depth of a binary tree, postorder binary tree traversal, and preorder binary tree traversal" (Appendix B.3.1) vs "Graph depth-first search (DFS)" (Table 2). The prose includes maximum tree depth among the recursion-versus-iteration tasks, whereas Table 2 substitutes graph DFS. → Effect on result unclear
- **Reporting and transparency:** Clean. Ethics approval and informed consent are declared. No contradictory registration, disclosure, or availability claim is established in the supplied text.
  - Ethics approval: declared
- **Reference integrity:** Noted
  - 56 references in manuscript
  - 41 of 54 checkable references found in an index
  - 2 are books, websites or datasets — counted, but not index-checkable
  - Could not be verified automatically — queued for manual review:
    - L. Bailey, E. Ong, S. Russell, and S. Emmons. Image hijacks: Adversarial images can control generative models at runtime. In International Conference on Machine Learning (ICML), 2024. (Crossref bibliographic top candidate ratio 0.40 — no plausible match)
    - J. Betley, X. Bao, M. Soto, A. Sztyber-Betley, J. Chua, and O. Evans. Tell me about yourself: LLMs are aware of their learned behaviors. In International Conference on Learning Representations (ICLR), 2025. (Crossref bibliographic top candidate ratio 0.38 — no plausible match)
    - F. J. Binder, J. Chua, T. Korbak, H. Sleight, J. Hughes, R. Long, E. Perez, M. Turpin, and O. Evans. Looking inward: Language models can learn about themselves by introspection. In International Conference on Learning Representations (ICLR), 2025. (Ambiguous: title ratio 0.55 via crossref_bibliographic sits between the divergence (0.30) and match (0.70) thresholds; too close to call either way, so it is recorded as unresolved rather than judged.)
    - S. Havrylov and I. Titov. Emergence of language with multi-agent games: Learning to communicate with sequences of symbols. In Advances in Neural Information Processing Systems (NeurIPS), volume 30, 2017. (Crossref bibliographic top candidate ratio 0.39 — no plausible match)
    - T. Kouwenhoven, M. Peeperkorn, and T. Verhoef. Searching for structure: Investigating emergent communication with large language models. In Proceedings of the 31st International Conference on Computational Linguistics (COLING), 2025a. (Crossref bibliographic top candidate ratio 0.47 — no plausible match)
    - A. Lazaridou, A. Peysakhovich, and M. Baroni. Multi-agent cooperation and the emergence of (natural) language. In International Conference on Learning Representations (ICLR), 2017. (Crossref bibliographic top candidate ratio 0.47 — no plausible match)
    - D. Lewis. Convention: A Philosophical Study. Harvard University Press, Cambridge, MA, 1969. (Crossref bibliographic top candidate ratio 0.22 — no plausible match)
    - X. L. Li, V. Shrivastava, S. Li, T. Hashimoto, and P. Liang. Benchmarking and improving generatorvalidator consistency of language models. In International Conference on Learning Representations (ICLR), 2024. (Ambiguous: title ratio 0.55 via crossref_bibliographic sits between the divergence (0.30) and match (0.70) thresholds; too close to call either way, so it is recorded as unresolved rather than judged.)
    - P. Lu, H. Bansal, T. Xia, J. Liu, C. Li, H. Hajishirzi, H. Cheng, K.-W. Chang, M. Galley, and J. Gao. Math-Vista: Evaluating mathematical reasoning of foundation models in visual contexts. In International Conference on Learning Representations (ICLR), 2024. (Ambiguous: title ratio 0.69 via crossref_bibliographic sits between the divergence (0.30) and match (0.70) thresholds; too close to call either way, so it is recorded as unresolved rather than judged.)
    - C. Shi, C. Yang, Y. Wu, L. Jin, B. Shui, T. Berg-Kirkpatrick, and X. Ma. Are VLMs seeing or just saying? uncovering the illusion of visual re-examination. In International Conference on Machine Learning (ICML), 2026. (Crossref bibliographic top candidate ratio 0.46 — no plausible match)
    - A. Vo, K.-N. Nguyen, M. R. Taesiri, V. T. Dang, A. T. Nguyen, and D. Kim. Vision language models are biased. In International Conference on Learning Representations (ICLR), 2026. (Ambiguous: title ratio 0.54 via crossref_bibliographic sits between the divergence (0.30) and match (0.70) thresholds; too close to call either way, so it is recorded as unresolved rather than judged.)
    - J. Zeller, T. Wiedemer, F. Li, T. Klein, P. Mayilvahanan, M. Bethge, F. Wichmann, R. Cotterell, and W. Brendel. MENTISOCULI: Revealing the limits of reasoning with mental imagery. In Proceedings of the 43rd International Conference on Machine Learning (ICML), 2026. URL https://jana-z.github.io/mentis-oculi. (Crossref bibliographic top candidate ratio 0.41 — no plausible match)
    - J. R. Zeller, A. Shtedritski, and C. Rupprecht. Highlight: Learning visual prompts for vision-language models. In International Conference on Learning Representations (ICLR), 2025. (Ambiguous: title ratio 0.56 via crossref_bibliographic sits between the divergence (0.30) and match (0.70) thresholds; too close to call either way, so it is recorded as unresolved rather than judged.)
  - References not verified automatically are resolved manually within 24 hours.

## Post-publication record

- **Status:** Clean
- **Summary:** No retraction or expression of concern found in Crossref (includes Retraction Watch).
- **Last checked:** 2026-11-04