Neura Intelligence Index
Category Comparison
Benchmark-by-Benchmark
coding
| Benchmark | Devstral Medium | GPT-4o mini |
|---|---|---|
| SWE-bench Verified | 61.6 | — |
| HumanEval | — | 87.2 |
language
| Benchmark | Devstral Medium | GPT-4o mini |
|---|---|---|
| HellaSwag | — | 91.2 |
| WinoGrande | — | 82.3 |
reasoning
| Benchmark | Devstral Medium | GPT-4o mini |
|---|---|---|
| BigBench-Hard | — | 78.5 |
| GPQA Diamond | — | 40.2 |
| ARC-Challenge | — | 93.1 |
general
| Benchmark | Devstral Medium | GPT-4o mini |
|---|---|---|
| Chatbot Arena Elo | — | 1219 |
| IFEval | — | 80.1 |
| MMLU-Pro | — | 63.2 |
safety
| Benchmark | Devstral Medium | GPT-4o mini |
|---|---|---|
| TruthfulQA | — | 65.8 |
Category Winners
coding
61.6
math
48.4
reasoning
18.9
general
34.6
language
35.0
safety
62.9