Why LLMs Form Geometric Representations

Analysis

Why This Paper Matters

This paper addresses a fundamental question in AI: why do large language models (LLMs) develop geometric representations of concepts like months, years, and cities? While such structures have been observed empirically, a rigorous theoretical explanation has been lacking. The authors provide a compelling answer by linking these geometries to statistical symmetries in natural language, specifically translation symmetry. This is a significant step toward a principled understanding of LLM internals, moving beyond descriptive observations to predictive theory.

The work is particularly important because it bridges the gap between high-dimensional word embedding models and the complex representations in modern LLMs. By proving that translation symmetry in co-occurrence statistics leads to geometric manifolds, the authors offer a unifying explanation that applies across model scales. This could have profound implications for interpretability, as it suggests that geometric structures are not arbitrary but are direct reflections of statistical properties of the training data.

Technical Contributions

• Translation Symmetry Identification: The paper identifies that language statistics exhibit translation symmetry, such as the co-occurrence frequency of months depending only on the time interval between them, not absolute positions.
• Theoretical Proof: It provides a mathematical proof that this symmetry forces geometric structures in high-dimensional word embedding models, analytically deriving the manifold geometry.
• Empirical Validation: The predictions are validated against large text embedding models and LLMs, showing that the derived geometries match actual representations.
• Robustness Analysis: The paper proves that geometric representations persist even when co-occurrence statistics are perturbed (e.g., removing sentences with month co-occurrences), and shows this robustness emerges when statistics are governed by a latent variable.

Results

The paper reports that the analytically derived manifold geometries match empirical representations in large text embedding models and LLMs. For instance, calendar months form a circle, historical years form a one-dimensional manifold, and cities' latitudes and longitudes are linearly decodable. The robustness experiments show that even when all sentences with month co-occurrences are removed, the circular structure remains, confirming the theoretical prediction. These results are consistent across different model architectures and sizes, suggesting a universal principle.

Significance

This work provides a foundational theoretical framework for understanding representational geometry in LLMs. It shifts the narrative from empirical observation to predictive theory, enabling researchers to anticipate what structures might emerge from given data statistics. This could guide model design, such as choosing training data to encourage desired representations, and enhance interpretability by linking geometric structures to interpretable statistical properties. The findings also open avenues for further research into other symmetries and their effects on representations, potentially leading to a more comprehensive theory of neural code in language models.