What do Vision Transformers Learn

Analysis

Why This Paper Matters

Vision Transformers (ViTs) have become a cornerstone of modern computer vision, yet their internal workings remain less understood than those of convolutional neural networks (CNNs). This paper systematically peels back the layers of ViTs to answer a fundamental question: what exactly are they learning? By visualizing features and comparing across architectures, the authors reveal surprising similarities and critical differences between ViTs and CNNs. These insights are crucial for practitioners who need to choose the right architecture, debug model behavior, or design new training paradigms—especially in domains where spatial reasoning or texture robustness is key.

How It Works

The researchers employ activation maximization to visualize what individual neurons and layers in ViTs respond to. Starting from random noise, they iteratively adjust the input image to maximize the activation of a specific feature, using total variation regularization and augmentations (jitter, color shift, ensemble) to produce interpretable images. The optimization objective is:

They augment inputs and optimize over these versions, leading to the final formulation:

To validate the visualizations, each is paired with the most activating ImageNet images. The study focuses on ViT-B16 and visualizes features from multi-headed attention layers (keys, queries, values) and feedforward layers, finding that feedforward features are more interpretable. The GELU layer outputs are also visualized:

A key experiment isolates the CLS token: by removing it from early layers and reintroducing it only in the last layer, the network still classifies well, indicating that the CLS token primarily gathers global information at the final stage. Applying the classification head to individual patches without fine-tuning also works, showing that last-layer token mixing globalizes information across all patches.

Results

The paper reveals that ViTs, like CNNs, progress from learning low-level features (colors, edges) in early layers to high-level objects in deeper layers:

When comparing background vs. foreground reliance, ViTs use background information more effectively than CNNs:

Low-pass filtering experiments show ViTs are more robust to texture removal than CNNs:

ViTs trained with language supervision (CLIP) learn more abstract, concept-driven features, such as those responding to prepositions or repetition of objects:

Significance

This paper demystifies Vision Transformers by showing they are not just "CNNs without convolutions" but have unique properties: they preserve spatial information throughout most layers, use background cues more effectively, and are less texture-dependent. The finding that the last layer acts as a global pooling operation has practical implications for architecture design—for instance, tasks requiring fine-grained spatial information might benefit from skipping or modifying that final layer. Additionally, the discovery that language-supervised ViTs learn semantic concepts opens doors for more interpretable and transferable representations. For AI practitioners, this work provides a toolkit for visualizing and understanding ViT behavior, enabling better debugging and model selection in real-world applications.