Neura Intelligence Index
Category Comparison
Benchmark-by-Benchmark
general
| Benchmark | Llama 3.3 70B | Nemotron 3 Ultra (550B A55B) |
|---|---|---|
| MMLU-Pro | 68.9 | — |
| Chatbot Arena Elo | 1247 | — |
| IFEval | 85.3 | — |
reasoning
| Benchmark | Llama 3.3 70B | Nemotron 3 Ultra (550B A55B) |
|---|---|---|
| GPQA Diamond | 49.5 | 87 |
| BigBench-Hard | 82.9 | — |
| ARC-Challenge | 95.2 | — |
| Humanity's Last Exam | — | 37.4 |
coding
| Benchmark | Llama 3.3 70B | Nemotron 3 Ultra (550B A55B) |
|---|---|---|
| HumanEval | 88.4 | — |
| SWE-bench Verified | — | 70.7 |
| SciCode | — | 44.6 |
safety
| Benchmark | Llama 3.3 70B | Nemotron 3 Ultra (550B A55B) |
|---|---|---|
| TruthfulQA | 62.8 | — |
language
| Benchmark | Llama 3.3 70B | Nemotron 3 Ultra (550B A55B) |
|---|---|---|
| WinoGrande | 86.1 | — |
| HellaSwag | 94.8 | — |
agent
| Benchmark | Llama 3.3 70B | Nemotron 3 Ultra (550B A55B) |
|---|---|---|
| BrowseComp | — | 44.4 |
Category Winners
coding
66.8
math
71.9
reasoning
72.8
general
60.5
language
77.3
safety
43.7
agent
13.1