Local LLM Hardware Calculator
Calculate the exact GPU, VRAM, and system requirements to run Llama, Qwen, Mistral, DeepSeek, and 20+ other open-source models locally.
Model
Quantization
Runtime Settings
VRAM Breakdown
Compatible GPU Options
Estimates assume inference only (not training). KV cache grows linearly with context and batch size. For training, multiply VRAM by ~4-8× for gradients, optimizer state, and activations.
Frequently asked questions
What is quantization and why does it matter?+
Quantization reduces the precision of model weights — for example, from 16-bit floats (FP16) down to 4-bit integers (Q4). This cuts VRAM requirements by 2-4× with modest quality loss. Q4 is the most popular choice for running large models on consumer GPUs.
How is the VRAM estimate calculated?+
The calculator sums three components: model weights (params × bits / 8), KV cache (grows with context length and batch size), and activation overhead (~10% of model size). For training, you'd need 4-8× more for gradients and optimizer state.
Can I split a model across multiple GPUs?+
Yes. Tools like vLLM, Ollama, and llama.cpp support tensor parallelism across multiple GPUs. VRAM adds together (2× 24GB = 48GB effective), but you get bandwidth penalties. For inference, multi-GPU works well; for training, NVLink or InfiniBand is important.
Why does context length affect VRAM?+
Each token in the context requires storage for keys and values in the attention mechanism (the KV cache). KV cache grows linearly with context length, so a 128K context can easily use as much VRAM as the model weights themselves for large models.
What about Mac / Apple Silicon?+
Apple Silicon uses unified memory, so the GB requirement roughly equals your total RAM (minus overhead for macOS). An M3 Max with 128GB can run 70B models at Q4. For rough conversion, halve the VRAM estimate for Apple Silicon since there's no KV cache separation penalty.