Neura Intelligence Index
Category Comparison
Benchmark-by-Benchmark
reasoning
| Benchmark | GPT-4 Turbo | Nemotron 3 Ultra (550B A55B) |
|---|---|---|
| ARC-Challenge | 96.3 | — |
| GPQA Diamond | 49.3 | 87 |
| BigBench-Hard | 85.1 | — |
| Humanity's Last Exam | — | 37.4 |
language
| Benchmark | GPT-4 Turbo | Nemotron 3 Ultra (550B A55B) |
|---|---|---|
| WinoGrande | 87.5 | — |
| HellaSwag | 95.3 | — |
coding
| Benchmark | GPT-4 Turbo | Nemotron 3 Ultra (550B A55B) |
|---|---|---|
| HumanEval | 87.6 | — |
| SWE-bench Verified | — | 70.7 |
| SciCode | — | 44.6 |
safety
| Benchmark | GPT-4 Turbo | Nemotron 3 Ultra (550B A55B) |
|---|---|---|
| TruthfulQA | 70.1 | — |
general
| Benchmark | GPT-4 Turbo | Nemotron 3 Ultra (550B A55B) |
|---|---|---|
| Chatbot Arena Elo | 1256 | — |
| MMLU-Pro | 64.1 | — |
| IFEval | 82.1 | — |
agent
| Benchmark | GPT-4 Turbo | Nemotron 3 Ultra (550B A55B) |
|---|---|---|
| BrowseComp | — | 44.4 |
Category Winners
coding
63.4
math
60.3
reasoning
74.9
general
47.9
language
85.2
safety
84.8
agent
15.9