Neura Intelligence Index
Category Comparison
Benchmark-by-Benchmark
general
| Benchmark | Claude 3.5 Haiku | Devstral Medium |
|---|---|---|
| IFEval | 78.4 | — |
| MMLU-Pro | 65 | — |
| Chatbot Arena Elo | 1200 | — |
reasoning
| Benchmark | Claude 3.5 Haiku | Devstral Medium |
|---|---|---|
| GPQA Diamond | 41.6 | — |
| ARC-Challenge | 91.8 | — |
coding
| Benchmark | Claude 3.5 Haiku | Devstral Medium |
|---|---|---|
| HumanEval | 88.1 | — |
| SWE-bench Verified | — | 61.6 |
safety
| Benchmark | Claude 3.5 Haiku | Devstral Medium |
|---|---|---|
| TruthfulQA | 63.5 | — |
Category Winners
coding
65.6
math
42.6
reasoning
20.8
general
30.1
safety
48.3