IBM Granite 4.1 LLMs: Full Training Breakdown

IBM Unveils Granite 4.1 LLM Training Process

Granite 4.1 consists of dense, decoder-only large language models at 3B, 8B, and 30B parameters. Developers trained these from scratch on roughly 15 trillion tokens via a five-stage pre-training approach that includes long-context extension up to 512K tokens. Post pre-training, supervised fine-tuning uses around 4.1 million high-quality samples. Reinforcement learning applies on-policy GRPO with DAPO loss. The 8B instruct model performs as well as or better than Granite 4.0-H-Small, a 32B-A9B Mixture-of-Experts model, despite its simpler dense design and smaller size. IBM releases all Granite 4.1 models under Apache 2.0 license.

The Granite series from IBM focuses on open-source models suited for enterprise tasks. This latest version stresses data quality in every step, from curation to final alignment.

Model Architecture Details

Granite 4.1 employs a decoder-only transformer with grouped query attention, rotary position embeddings, SwiGLU activations, RMSNorm, and shared input-output embeddings.

| Component | 3B Dense | 8B Dense | 30B Dense | |, , , , , , , , , , |, , , , , |, , , , , |, , , , , -| | Embedding size | 2560 | 4096 | 4096 | | Number of layers | 40 | 40 | 64 | | Attention head size| 64 | 128 | 128 | | Number of attention heads | 40 | 32 | 32 | | Number of KV heads | 8 | 8 | 8 | | MLP hidden size | 8192 | 12800 | 32768 | | MLP activation | SwiGLU | SwiGLU | SwiGLU | | Position embedding | RoPE | RoPE | RoPE |

All sizes follow identical training and data plans, varying only in dimensions.

Five-Phase Pre-Training Pipeline

Pre-training spans about 15 trillion tokens in five phases. Early phases build basic skills. Later ones use higher-quality data and extend context length.

Phase 1: General Pre-Training (10T tokens) This stage uses a power learning rate schedule with warmup for wide language grasp. Data mix: CommonCrawl 59%, Code 20%, Math 7%, Technical 10.5%, Multilingual 2%, Domain Specific 1.5%.

Phase 2: Math/Code Pre-Training (2T tokens) Focus shifts to reasoning with more code and math data. Data mix: Math 35%, Code 30%, CommonCrawl-HQ 12%, Synthetic 9%, Technical 10%, Multilingual 3%, Domain 1%.

Phase 3: High-Quality Data Annealing (2T tokens) Exponential decay learning rate introduces chain-of-thought and instructions. Data mix: CommonCrawl-HQ 16.67%, Math 16.67%, Code 16.67%, Synthetic 8.5%, Technical 12.5%, Multilingual 4.5%, Long Chain-of-Thought 12.5%, Language Instructions 7.5%, Code Instructions 4.5%.

Phase 4: High-Quality Data Annealing, Refinement (0.5T tokens) Linear decay to zero learning rate targets top data. Data mix: CommonCrawl-HQ 40%, Code 20%, Math 20%, Long Chain-of-Thought 6%, Code Instructions 5%, Language Instructions 9%.

Phase 5: Long Context Training (LCE) Context grows from 4K to 512K in steps: 32K, 128K, then 512K (for 8B and 30B with 80% books + 20% code). Exponential schedule from 1e-4 to 0. Model merges preserve short-context ability. RULER scores for base models:

| Model name | 32K | 64K | 128K | |, , , , , , , , , , -|, , , |, , , |, , , | | granite-4.1-3b-base | 75.0 | 66.6 | 58.0 | | granite-4.1-8b-base | 83.6 | 79.1 | 73.0 | | granite-4.1-30b-base| 85.2 | 84.6 | 76.7 |

Supervised Fine-Tuning Process

SFT data preparation relies on LLM-as-Judge and rules to filter 4.1 million samples. Judges score assistant replies on instruction following, correctness, completeness, conciseness, naturalness, calibration. Hard rejects for hallucinations, false premises, wrong math. Rules handle structure, length, dedup.

Training setup (all variants): 16 nodes with 4x GB200 each, 3 epochs, 5e-6 learning rate (warmup 3%, decay over 25K steps), 16,384 tokens sequence, batch 256 (~4.2M tokens/iter).

Multi-Stage Reinforcement Learning

RL uses on-policy GRPO with DAPO loss via SkyRL stack. Samples per prompt: 16, batch 1024, context 8,192. Stages:

Multi-domain RL: Joint training on math, science, logic, instructions, structured output, Text2SQL, temporal, chat, in-context. 45,504 prompts, LR 5e-7, β 0.05.

RLHF: Chat prompts with multilingual reward model. +18.9 Alpaca-Eval avg. 17,920 prompts, LR 3e-7, β 0.09.

Identity & Knowledge-Calibration RL: Short run on calibration prompts. 1728 prompts, LR 3e-7, β 0.09.

Math RL: Recovers math drops. +3.8 GSM8K, +23.48 DeepMind-Math avg. 13,504 prompts, LR 5e-7, β 0.05.

Benchmark Performance

Base Model Benchmarks

| Benchmark | Metric | 3B | 8B | 30B | |, , , , , , , -|, , , , , , , |, , , -|, , , -|, , , -| | MMLU | 5-shot | 66.47 | 73.60 | 78.44 | | MMLU-Pro | 5-shot, CoT | 37.16 | 44.58 | 49.51 | | BBH | 3-shot, CoT | 63.84 | 73.83 | 80.66 | | AGI EVAL | 3-shot | 54.32 | 61.68 | 69.20 | | DROP | 5-shot | 66.04 | 72.36 | 78.57 | | GSM8K | 8-shot | 72.93 | 73.54 | 83.78 | | Minerva Math | 4-shot | 38.00 | 43.42 | 45.66 | | HumanEval | pass@1 (StarCoder) | 76.19 | 79.24 | 81.52 | | HumanEval | pass@1 | 59.76 | 68.29 | 69.50 | | HumanEval+ | pass@1 | 54.27 | 62.20 | 61.60 | | Eval+ Avg | | 65.94 | 62.05 | 63.90 | | MMMLU | 5-shot | 56.59 | 64.73 | 73.36 | | INCLUDE | 5-shot | 51.77 | 57.60 | 67.07 | | MGSM | 8-shot | 58.48 | 63.68 | 74.40 |

Instruct Model Benchmarks

| Benchmark | Metric | 3B | 8B | 30B | |, , , , , , , -|, , , , , , -|, , , -|, , , -|, , , -| | MMLU | 5-shot | 67.02 | 73.84 | 80.16 | | MMLU-Pro | 5-shot, CoT | 49.83 | 55.99 | 64.09 | | BBH | 3-shot, CoT | 75.83 | 80.51 | 83.74 | | AGI EVAL | 0-shot, CoT | 65.16 | 72.43 | 77.80 | | GPQA | 0-shot, CoT | 31.70 | 41.96 | 45.76 | | SimpleQA | | 3.68 | 4.82 | 6.81 | | AlpacaEval 2.0| | 38.57 | 50.08 | 56.16 | | IFEval Avg | | 82.30 | 87.06 | 89.65 | | ArenaHard | | 37.80 | 68.98 | 71.02 | | MTBench Avg | | 7.53 | 8.50 | 8.53 | | GSM8K | 8-shot | 86.88 | 92.49 | 94.16 | | GSM Symbolic | 8-shot | 81.32 | 83.70 | 75.70 | | Minerva Math | 0-shot, CoT | 67.94 | 80.10 | 81.32 | | DeepMind Math | 0-shot, CoT | 64.64 | 80.07 | 81.93 | | HumanEval | pass@1 | 79.27 | 87.20 | 89.63 | | HumanEval+ | pass@1 | 74.39 | 80.49 | 85.98 | | MBPP | pass@1 | 61.64 | 82.54 | 83.33 | | MBPP+ | pass@1 | 52.91 | 70.64 | 71.69 | | CRUXEval-O | pass@1 | 40.75 | 47.63 | 55.75 | | BigCodeBench | pass@1 | 32.19 | 35.00 | 38.77 | | MULTIPLE | pass@1 | 52.54 | 60.26 | 62.31 | | Eval+ Avg | pass@1 | 67.05 | 80.21 | 82.66 | | BFCL v3 | | 60.80 | 68.27 | 73.68 | | MMMLU | 5-shot | 57.61 | 64.84 | 73.71 | | INCLUDE | 5-shot | 52.05 | 58.89 | 67.26 | | MGSM | 8-shot | 70.00 | 82.32 | 71.12 | | SALAD-Bench | | 93.95 | 95.80 | 96.41 | | AttaQ | | 81.88 | 81.19 | 85.76 | | Tulu3 Safety Eval Avg | | 66.84 | 75.57 | 78.19 |

Supported languages include English, German, Spanish, French, Japanese, Portuguese, Arabic, Czech, Italian, Korean, Dutch, Chinese.

Granite 4.1-8B beats or ties Granite 4.0-H-Small (32B-A9B) on IFEval, AlpacaEval, MMLU-Pro, BBH, GSM8K, DeepMind-Math, Evalplus, ArenaHard, BFCL V3, MBPP(+). Scores rise with size across the family.

Quantization and Infrastructure

FP8 versions cut disk and GPU memory by 50% using LLM Compressor on linear layers.

Training ran on NVIDIA GB200 NVL72 at CoreWeave: 72-GPU NVLink intra-rack, 400 Gb/s InfiniBand inter-rack, thousands of GPUs.

Getting Started Example

Granite 4.1 models carry Apache 2.0 license. Code for 30B instruct with tool calling:

pip install torch torchvision torchaudio
pip install accelerate
pip install transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

device = "cuda"
model_path = "ibm-granite/granite-4.1-30b"
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForCausalLM.from_pretrained(model_path, device_map=device)
model.eval()

tools = [
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Get the current weather for a specified city.",
"parameters": {
"type": "object",
"properties": {
"city": {
"type": "string",
"description": "Name of the city"
}
},
"required": ["city"]
}
}
]

chat = [
{"role": "user", "content": "What's the weather like in London right now?"},
]
chat = tokenizer.apply_chat_template(chat, tokenize=False, tools=tools, add_generation_prompt=True)
input_tokens = tokenizer(chat, return_tensors="pt").to(device)
output = model.generate(**input_tokens, max_new_tokens=100)
output = tokenizer.batch_decode(output)
print(output[0])

Expected output shows tool call for London weather.

Granite 4.1 advances open-source models through strict data handling, SFT, and RL stages. Dense 8B models compete with larger MoEs in instructions, tools, chat.

Related on Neura Market

• AI Models Directory
• LLM Frameworks
• Open Source AI Tools