Gopher
FreeGopher by DeepMind is a 280 billion parameter language model.
About Gopher
Gopher is a 280 billion parameter transformer language model developed by DeepMind, released alongside three papers covering language modelling at scale, ethical and social risks, and a new architecture. It demonstrates strong performance on reading comprehension, fact-checking, toxic language identification, and the Massive Multitask Language Understanding (MMLU) benchmark, surpassing prior language models. Gopher also exhibits surprising coherence in dialogue interactions, even without dialogue-specific fine-tuning. However, the model has documented failure modes including repetition, stereotypical biases, and confident propagation of incorrect information. Scale does not significantly improve performance on logical reasoning and common-sense tasks. The research emphasizes the need for interdisciplinary collaboration to anticipate and mitigate risks associated with large language models.
Key Features
Pros & Cons
- Achieves state-of-the-art results on several benchmarks including MMLU
- Shows strong reading comprehension and fact-checking capabilities
- Demonstrates coherent dialogue without specific fine-tuning
- Research includes thorough documentation of failure modes and ethical risks
- Tendency toward repetition in outputs
- Reflects stereotypical biases present in training data
- Confidently propagates incorrect information
- Scale does not significantly improve logical reasoning and common-sense tasks