Building Multilingual Bridges: Data Mixing as the Pillar of Generalization for In-Language Reasoning

Analysis

Why This Paper Matters

Reasoning language models have become increasingly capable, yet their reasoning processes remain overwhelmingly English-centric. Even when prompted in another language, models tend to reason in English, which can distort user intent, exclude non-English speakers, and discard knowledge that is more naturally expressed in the target language. This paper tackles this accessibility gap head-on by formalizing L2 reasoning—the ability to reason consistently in the language of the user's prompt—and demonstrating that it can be achieved at scale through careful data mixing.

The significance lies in the data-centric approach. Rather than requiring reasoning supervision in every target language, which is prohibitively expensive for low-resource languages, the authors show that reasoning is a language-agnostic behavior that can be transferred across typologically diverse languages. This finding has broad implications for multilingual AI development, suggesting that strategic data composition can unlock in-language reasoning for dozens of languages without per-language reasoning annotations.

By releasing model weights and multilingual reasoning data, the work lowers barriers for researchers and practitioners aiming to build more inclusive reasoning systems. It also challenges the assumption that high-quality reasoning must be English-first, paving the way for models that respect linguistic diversity.

Technical Contributions

• Formalization of L2 reasoning: Defines and measures the ability of a model to reason in the prompt language, providing a clear target for multilingual reasoning research.
• Data mixing and scheduling for SFT: Investigates how to optimize data composition and scheduling during supervised fine-tuning to generalize reasoning across languages.
• Tiny Aya L2-Thinker: A 3.35B parameter model trained with a mix of multilingual non-reasoning data and an English reasoning backbone, achieving high L2 reasoning rates.
• Generalization recipe: Identifies three key ingredients for generalizing to held-out languages: broader language coverage, readily available multilingual non-reasoning data, and a sufficient English reasoning backbone.
• Open resources: Releases model weights and multilingual reasoning data to support further research.

Results

Tiny Aya L2-Thinker achieves an L2 reasoning rate above 93% across 60 languages on 6 benchmarks spanning math, commonsense reasoning, instruction following, open-ended generation, and cultural reasoning. This is accomplished while keeping overall performance strong. The results demonstrate that reasoning supervision in every target language is not necessary; instead, a well-designed data mix can transfer reasoning behavior across languages. The paper does not provide direct comparisons to other models in the abstract, but the high L2 rate across a wide language set is a notable achievement.

Significance

This work has the potential to significantly advance multilingual AI by making reasoning models more accessible and effective for non-English speakers. It provides a practical, data-centric methodology that can be adopted by others to extend reasoning capabilities to many more languages. The finding that reasoning is language-agnostic and transferable challenges the English-centric paradigm and encourages the development of models that better serve global users. By releasing resources, the authors enable the community to build upon this foundation, fostering more inclusive and culturally aware AI systems.