Neura Market
Nemotron-4-340B logo

Nemotron-4-340B

Free

NVIDIA's 340B model for synthetic data generation

FreeFree tier
Inputs: textOutputs: text
Type
Open Source
Company
NVIDIA
Visit Website GitHub (38 stars)

About Nemotron-4-340B

Nemotron-4-340B-Instruct is a large language model (LLM) developed by NVIDIA, designed primarily as part of a synthetic data generation pipeline to help researchers and developers create training data for their own LLMs. It is a fine-tuned version of the Nemotron-4-340B-Base model, optimized for English-based single and multi-turn chat. The model has 340 billion parameters and supports a context length of 4,096 tokens. It was pre-trained on 9 trillion tokens covering English, 50+ natural languages, and 40+ coding languages. Alignment was achieved through supervised fine-tuning (SFT), direct preference optimization (DPO), and reward-aware preference optimization (RPO), using only approximately 20,000 human-annotated samples while over 98% of the training data was synthesized. The model is released under the NVIDIA Open Model License, allowing commercial use and derivative models. It requires substantial hardware for inference, such as 8x H200 or 16x H100 or 16x A100 80GB nodes.

Key Features

340 billion parameters
Context length of 4,096 tokens
Pre-trained on 9 trillion tokens including English, 50+ languages, and 40+ coding languages
Alignment: Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), Reward-aware Preference Optimization (RPO)
Only approximately 20,000 human-annotated data points used; over 98% of training data synthesized
Released under NVIDIA Open Model License – commercially usable, allows derivative models
Requires 8x H200 or 16x H100 or 16x A100 80GB for BF16 inference
Decoder-only Transformer architecture with Grouped-Query Attention (GQA) and Rotary Position Embeddings (RoPE)
Optimized for English single and multi-turn chat

Pros & Cons

Pros
  • Commercially usable under a permissive open model license
  • Generates high-quality synthetic data that reduces reliance on human annotation
  • Advanced alignment techniques improve reasoning, coding, and instruction-following
  • Supports multi-turn conversations and few-shot prompting
  • Well-documented and integrated with NVIDIA NeMo Framework
Cons
  • Very large model (340B parameters) requires substantial hardware (multiple high-end GPUs)
  • Optimized only for English chat; limited to 4,096 token context length
  • High inference cost due to hardware requirements
  • Relatively new with limited community adaptation compared to smaller models

Best For

Synthetic data generation for training and customizing large language modelsEnglish chat applications (single and multi-turn)Further customization with NeMo Framework (Parameter-Efficient Fine-Tuning, LoRA, etc.)

FAQ

What is the license for Nemotron-4-340B-Instruct?
It is licensed under the NVIDIA Open Model License, which allows commercial use, derivative models, and does not claim ownership of outputs.
What hardware is required to run this model?
For BF16 inference, it requires either 8x H200 (1 node), 16x H100 (2 nodes), or 16x A100 80GB (2 nodes).
What is the primary intended use of this model?
The model is designed for synthetic data generation to help developers and enterprises build and customize their own large language models and LLM applications.