Transformer Explainer
FreeInteractive visualization of how transformer-based LLMs work, running a live GPT-2 model in the browser. [#opensource](https://github.com/poloclub/transformer-explainer)
FreeFree tier
Inputs: textOutputs: text
About Transformer Explainer
An interactive visualization tool that demystifies the inner workings of transformer-based large language models. It runs a live GPT-2 (small) model in the browser, allowing users to see how embedding, attention mechanisms, MLP layers, and output probabilities work step by step. The tool explains the architecture's key components — tokenization, token embeddings, positional encoding, and the Transformer block — with expandable views and real-time computation. Built for education and exploration, it is open source and freely available.
Key Features
Runs a live GPT-2 (small) model in the browser with 124 million parameters
Step-by-step visualization of embedding: tokenization, token embedding, positional encoding, final embedding
Expandable views of Transformer block components: Attention Mechanism and MLP Layer
Demonstrates next-token prediction process from input to output probabilities
Open source, available on GitHub for customization and learning
Explains self-attention mechanism and its role in capturing long-range dependencies
Pros & Cons
Pros
- Runs entirely in the browser with no installation or API key required
- Provides an interactive, step-by-step walkthrough of the entire transformer pipeline
- Uses a real, pre-trained GPT-2 model (124M parameters) for authentic demonstrations
- Open source, allowing customization and learning from the code
- Visualizes complex concepts like attention and positional encoding in an intuitive manner
Cons
- Only uses the relatively small GPT-2 (small) model, not state-of-the-art LLMs like GPT-4 or Llama
- Limited to text generation; does not demonstrate other transformer applications (image, audio)
- May be resource-intensive in the browser due to live model execution
- Requires a modern browser with good JavaScript performance
Best For
Learning how transformer-based large language models work under the hoodTeaching AI and deep learning concepts in classrooms or workshopsVisual debugging of GPT-2's token-level processing for researchersDemonstrating the architecture of GPT-2 for technical audiencesExploring the effect of different prompts on token embeddings and attention patterns
FAQ
What is Transformer Explainer?
It is an interactive visualization tool that explains how transformer-based LLMs work, running a live GPT-2 model in the browser.
What model powers Transformer Explainer?
It uses the GPT-2 (small) model with 124 million parameters.
What are the key components of a transformer shown?
Embedding, Transformer block (Attention and MLP), and Output Probabilities.
How does tokenization work?
Input text is broken into tokens (words or subwords). For example, 'Data visualization empowers users to' is tokenized into distinct tokens; 'empowers' is split into two tokens.
Is this tool open source?
Yes, it is open source and available on GitHub.