DBRX-132B-MoE
FreeA New State-of-the-Art Open LLM from Databricks
FreeFree tier
Inputs: textOutputs: text
About DBRX-132B-MoE
DBRX is an open, general-purpose large language model (LLM) created by Databricks, setting a new state-of-the-art among established open LLMs. It uses a fine-grained mixture-of-experts (MoE) architecture with 132 billion total parameters (36B active), 16 experts (4 selected per input), and is pretrained on 12 trillion tokens of text and code. DBRX surpasses GPT-3.5 on standard benchmarks, is competitive with Gemini 1.0 Pro, and excels at code generation, outperforming specialized models like CodeLLaMA-70B. It features up to 2x faster inference than LLaMA2-70B, supports a 32k token context, and uses RoPE, GLU, GQA, and the GPT-4 tokenizer. Available as DBRX Base and DBRX Instruct under an open license on Hugging Face, and accessible via Databricks APIs.
Key Features
Fine-grained mixture-of-experts (MoE) architecture with 132B total parameters (36B active)
16 experts, selects top 4, offering 65x more expert combinations than Mixtral/Grok-1
Surpasses GPT-3.5 and matches Gemini 1.0 Pro on standard benchmarks
Outperforms specialized code models like CodeLLaMA-70B on programming tasks
Inference up to 2x faster than LLaMA2-70B; up to 150 tok/s/user on Databricks Model Serving
Pretrained on 12 trillion tokens of curated text and code data with 32K token context
Uses RoPE, gated linear units (GLU), grouped query attention (GQA), and GPT-4 tokenizer
Available as DBRX Base and DBRX Instruct under an open license on Hugging Face
Databricks customers can use via APIs, pretrain from scratch, or fine-tune on checkpoints
Excels in RAG tasks, surpassing GPT-3.5 Turbo and other open models
Pros & Cons
Pros
- Open source with weights freely available on Hugging Face
- State-of-the-art performance among open LLMs, surpassing GPT-3.5
- Highly efficient MoE design reduces training and inference costs compared to dense models
- Strong coding capabilities, outperforming specialized code models
- Fast text generation (up to 150 tok/s/user) when hosted on Databricks
- Flexible architecture allows enterprises to pretrain or fine-tune their own models
- Improves over previous MPT models with nearly 4x less compute for equivalent quality
- Well-suited for RAG tasks and production deployment
Cons
- Large total parameter count (132B) still requires significant compute resources
- Context length limited to 32K tokens, which may be insufficient for very long documents
- MoE architecture can be complex to train and optimize for external users
- Only available via Databricks APIs for hosted inference, potentially locking into Databricks ecosystem for production use
- Competing models (e.g., GPT-4, Claude) may still outperform in some benchmarks
Best For
General language understanding and generationCode generation and programming assistanceMathematical reasoning and problem solvingRetrieval-augmented generation (RAG) applicationsBuilding enterprise-grade LLMs via pretraining or fine-tuningIntegration into AI-powered products (e.g., SQL generation)
FAQ
What is DBRX?
DBRX is an open, general-purpose large language model created by Databricks. It uses a fine-grained mixture-of-experts architecture with 132B total parameters (36B active) and is pretrained on 12 trillion tokens. It sets a new state-of-the-art for open LLMs, outperforming GPT-3.5 and matching Gemini 1.0 Pro.
Is DBRX free to use?
Yes, DBRX Base and DBRX Instruct are released under an open license and can be downloaded for free from Hugging Face. Databricks also offers hosted APIs for DBRX, which may incur costs based on usage.
How does DBRX compare to GPT-3.5?
According to Databricks, DBRX surpasses GPT-3.5 on a range of standard benchmarks and is competitive with Gemini 1.0 Pro. It also outperforms GPT-3.5 Turbo on RAG tasks and code generation.
What architecture does DBRX use?
DBRX is a transformer-based decoder-only LLM with a fine-grained mixture-of-experts (MoE) architecture. It has 16 experts and selects 4 per input, uses rotary position encodings (RoPE), gated linear units (GLU), grouped query attention (GQA), and the GPT-4 tokenizer.
Where can I download DBRX?
DBRX Base and DBRX Instruct are available on Hugging Face. The model repository and code are also on GitHub at databricks/dbrx.