OPT-1.3|6.7|13|30|66B
FreeOpen Pre-trained Transformer language models for research
FreeFree tier
Inputs: textOutputs: text
About OPT-1.3|6.7|13|30|66B
OPT (Open Pre-trained Transformers) is a suite of decoder-only pre-trained transformer language models released by Meta AI, ranging in size from 125 million to 175 billion parameters. The models are designed to be fully and responsibly shared with researchers to enable study of large language models. OPT-175B achieves performance comparable to GPT-3 while requiring only one-seventh the carbon footprint. The release includes a logbook documenting infrastructure challenges and code for experimenting with all models. The paper was submitted to arXiv on May 2, 2022.
Key Features
Suite of decoder-only pre-trained transformers from 125M to 175B parameters
Fully and responsibly shared with researchers, including model weights
OPT-175B comparable to GPT-3 with 1/7th the carbon footprint
Includes logbook detailing infrastructure challenges and code for experimentation
Pros & Cons
Pros
- Openly available model weights and code enable full reproducibility
- Comparable performance to GPT-3 with significantly lower environmental impact
- Range of model sizes allows study across different scales
- Includes detailed logbook of development challenges
Cons
- Requires substantial computational resources for larger models (175B)
- Limited to text generation tasks (decoder-only architecture)
Best For
Zero-shot and few-shot learning researchStudying large language model behavior and capabilitiesReproducing and extending results from GPT-3 style modelsOpen-source benchmarking and comparison
FAQ
What is OPT?
OPT (Open Pre-trained Transformers) is a suite of decoder-only pre-trained transformer language models ranging from 125 million to 175 billion parameters, openly shared with researchers.
How does OPT compare to GPT-3?
OPT-175B achieves performance comparable to GPT-3 while requiring only one-seventh the carbon footprint to develop.
Who developed OPT?
OPT was developed by researchers at Meta AI, including Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, and others.
Is OPT freely available?
Yes, OPT is released as open source with model weights and code freely available to researchers.