DeepSeek Ollama Local Deployment Rules

## DeepSeek Local Deployment with Ollama

### Hardware Requirements
- DeepSeek R1 1.5B: 4GB RAM, any modern GPU or CPU-only
- DeepSeek R1 7B: 8GB VRAM (RTX 3060+) or 16GB RAM (CPU)
- DeepSeek R1 14B: 12GB VRAM (RTX 4070+) or 32GB RAM (CPU)
- DeepSeek R1 32B: 24GB VRAM (RTX 4090) or 64GB RAM (CPU)
- DeepSeek R1 70B: 48GB VRAM (2x RTX 4090) or 128GB RAM (CPU)
- Full DeepSeek R1 671B: 8x A100 80GB or equivalent

### Installation
```bash
# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh

# Pull DeepSeek models
ollama pull deepseek-r1:7b          # 7B parameter distilled model
ollama pull deepseek-r1:14b         # 14B parameter distilled model
ollama pull deepseek-r1:32b         # 32B parameter distilled model
ollama pull claude-coder-v2:16b   # Coding-focused model
```

### Configuration Rules
- Set OLLAMA_NUM_PARALLEL=2 for concurrent requests (adjust based on VRAM)
- Set OLLAMA_MAX_LOADED_MODELS=1 to prevent OOM with large models
- Use OLLAMA_KEEP_ALIVE=5m to free VRAM after inactivity
- For CPU-only: set OLLAMA_NUM_THREAD to physical core count

### Performance Tuning
- Use quantized models (Q4_K_M) for best speed/quality tradeoff
- Enable GPU offloading: set num_gpu layers based on available VRAM
- For batch inference: use the /api/generate endpoint with keep_alive
- Monitor GPU utilization with nvidia-smi
- Expect 10-30 tokens/second on RTX 4090 for 14B model

### Integration
- Ollama exposes OpenAI-compatible API at http://localhost:11434/v1
- Use the same OpenAI SDK with baseURL: 'http://localhost:11434/v1'
- Set apiKey to 'ollama' (required but not validated)

### Security
- Bind Ollama to localhost only (default) in production
- If exposing via network, use a reverse proxy with authentication
- Do not run Ollama as root