TinyZero
FreeClean, minimal, accessible reproduction of DeepSeek R1-Zero
About TinyZero
TinyZero is a minimal reproduction of DeepSeek R1-Zero focused on countdown and multiplication tasks. Built upon the veRL (versatile reinforcement learning) library, it demonstrates how a 3B base language model can autonomously develop self-verification and search abilities through reinforcement learning — often called the "aha moment" of reasoning emergence. The project provides installation instructions, data preprocessing scripts, and training configurations for both single-GPU (1.5B model) and multi-GPU (3B+ model) setups, using the Qwen2.5 base model series. It is designed to be accessible and costs approximately $30 to run. Note: the repository is deprecated; users are directed to the latest veRL library for ongoing experiments.
Key Features
Pros & Cons
- Clean, minimal codebase with low cost (~$30) to replicate results
- Demonstrates emergence of self-verification and search in a 3B model
- Built on the well-established veRL library for RL training
- Open source with permissive license (MIT)
- Supports multiple model sizes and GPU configurations
- Repository deprecated; recommended to use latest veRL directly
- Requires specific dependencies (vllm, flash-attn, Ray, etc.) and GPU memory
- 0.5B base model fails to learn reasoning (as noted in docs)
- Only two tasks (countdown, multiplication) are demonstrated
- Not actively maintained; may not receive updates or bug fixes