NVIDIA has released Nemotron 3 Embed, a collection of open and commercially available embedding models designed to improve retrieval quality in multi-step agentic workflows. The collection includes three models led by an 8B flagship that ranks first overall on the RTEB leaderboard. The models aim to give developers practical deployment options for production-scale RAG, agentic retrieval, code retrieval, and agent memory.
Retrieval plays a critical role in multi-step agentic workflows. Poor retrieval can cause agents to fetch irrelevant context, re-query, waste token budget, and carry noise into later reasoning steps. The Nemotron 3 Embed collection is built to address these challenges.
The collection includes three open models that achieve state-of-the-art retrieval across the accuracy-efficiency curve. The flagship model is Nemotron-3-Embed-8B-BF16, which ranks first on RTEB and is designed for precision-critical retrieval and high-stakes enterprise RAG. The Nemotron-3-Embed-1B-BF16 is a high-efficiency model for production retrieval where latency and cost matter. The Nemotron-3-Embed-1B-NVFP4 is a Blackwell-optimized variant for high-throughput retrieval with a smaller memory footprint.
Key Features
Beyond the RTEB result, Nemotron 3 Embed introduces a production-ready feature set for enterprise retrieval deployments. The models come with open weights, datasets, and recipes, giving teams control to inspect, tune, fine-tune, and deploy retrieval models on their own infrastructure. They support a 32k context window for retrieval over long documents, large code contexts, and multi-turn agent histories while reducing truncation.
The models also support multilingual and code retrieval across global enterprise data, technical documentation, and multi-file code repositories. The NVFP4 efficiency provides a Blackwell-optimized 4-bit deployment path for high-throughput retrieval with a smaller memory footprint. Fine-tuning and distillation recipes are available through NVIDIA NeMo AutoModel recipes for domain adaptation and model compression. The models are available immediately on Hugging Face, deployable as NVIDIA NIM microservice, supported by vLLM, and accessible through leading AI Cloud and inference partners.
Evaluation: Retrieval Quality, Agentic Efficiency, and Deployment Tradeoffs
NVIDIA evaluated Nemotron 3 Embed across three dimensions: retrieval quality, downstream agentic efficiency, and deployment tradeoffs. The 8B model establishes the model collection's quality ceiling, while the 1B BF16 and NVFP4 variants bring the same retrieval-focused design to lower-cost and higher-throughput deployment settings.
RTEB Leadership and Strong Gains Across Retrieval Benchmarks
The models were first evaluated on RTEB, where Nemotron-3-Embed-8B-BF16 ranks first. They were also tested across ViDoRe V3 Text, MMTEB Retrieval, and LongEmbed using average NDCG@10.
Nemotron-3-Embed-8B-BF16 ranks first on RTEB, scoring 78.5% on RTEB and 75.5% on MMTEB Retrieval. Nemotron-3-Embed-1B-BF16 brings much of the 8B model's retrieval quality into a smaller deployment footprint. It scores 72.4% on RTEB, reducing error rate by 27% over its 1B predecessor (llama-nemotron-embed-vl-1b-v2), and scores 71.0% on MMTEB Retrieval, reducing error rate by 28%.
Why Better Retrieval Matters for Agents
To evaluate retrieval in an agentic setting, NVIDIA used a search agent powered by Nemotron 3 Ultra and varied the embedding model used by the retrieval system. Better retrieval can return relevant evidence earlier, helping the agent avoid repeated searches, unnecessary reasoning turns, and extra context inspection. The team compared average retrieval accuracy with estimated downstream agentic token cost per query across ViDoRe V3, BRIGHT, and BrowseComp-Plus.
The evaluation showed that stronger retrieval reduces downstream agentic token cost. More accurate retrievers return relevant evidence earlier, which helps agents complete tasks with fewer repeated searches and fewer reasoning turns. In these evaluations, the Nemotron 3 Embed models improve the agentic retrieval frontier, with the 8B model delivering both the highest average retrieval accuracy and the lowest estimated downstream token cost across ViDoRe V3, BRIGHT, and BrowseComp-Plus.
Scaling Retrieval with NVFP4 on Blackwell
For high-throughput deployments, teams often choose smaller embedding models to meet latency and cost targets. Nemotron-3-Embed-1B-NVFP4 is designed to narrow the gap between serving efficiency and retrieval quality by using native NVFP4 acceleration on NVIDIA Blackwell architectures. The model quantizes the weights and activations of linear layers to NVFP4 for efficient inference, and uses Quantization-Aware Distillation (QAD) to help recover accuracy for long input sequences.
NVFP4 on Blackwell delivers up to 2x higher throughput than BF16 for high-throughput, low-latency retrieval serving. The NVFP4 variant retains 99%+ of BF16 retrieval accuracy while reducing memory footprint.
Day 0 Performant NIM
For production-scale retrieval systems, the serving stack also needs to preserve that efficiency under real request loads, across different input sequence lengths and hardware targets. To make Nemotron 3 Embed performant at enterprise scale today, NVIDIA is also releasing an optimized NVIDIA NIM microservice for the 1B model. The Rust-based Nemotron 3 Embed NIM matches or outperforms the vLLM checkpoint on NVIDIA GB200 and RTX PRO 6000 GPUs across ISLs of 256 and 1024.
How We Built the Nemotron 3 Embed Models
Nemotron-3-Embed-8B-BF16 adapts the Ministral-3-8B-Instruct-2512 backbone by converting its causal decoder into a bidirectional encoder for full-sequence retrieval. The model is trained with contrastive pre-training on a blend of web-sourced and synthetic text pairs, then fine-tuned on curated multilingual retrieval datasets across domains such as legal, finance, medical, business, and education. This 8B model serves as the flagship embedding model, while earlier 8B teacher checkpoints from the same development line were used to distill the efficient 1B variants.
Scaling Down to 1B
The 1B model is not a small retriever trained from scratch. NVIDIA first applied the bidirectional adaptation recipe to the Ministral-3-3B-Instruct-2512 backbone to establish a 3B retriever base, then compressed it through two rounds of structured pruning and distillation.
First, the 3B parent model was compressed to a 2B intermediate footprint using NVIDIA ModelOpt's mcore_minitron Neural Architecture Search engine. The NAS pipeline searched across hidden width, FFN size, attention heads, and depth under a strict parameter budget to identify an efficient architecture for retrieval workloads.
Stay updated
Get the day's AI and automation news in your inbox. No spam, unsubscribe anytime.
The resulting 2B intermediate model was then distilled from an 8B teacher checkpoint to recover ranking accuracy. NVIDIA used a combined cosine distance loss and mean squared error loss on a multilingual, in-domain retrieval data blend to align the student's embeddings with the teacher.
This same sequence, ModelOpt structured pruning followed by 8B teacher distillation, was repeated a second time to compress the 2B intermediate model down to the final 1.14B embedding model. Final training used a progressive two-stage context-scaling schedule. Stage 1 focused on broad multilingual alignment at 1024-token context length to reconstruct the core retrieval behavior of the parent model. Stage 2 expanded context length to 4096 tokens and added long-context synthetic and reasoning datasets, helping the 1B model retain discriminative recall across longer inputs.
The following table summarizes the core technical specifications and deployment targets for the Nemotron 3 Embed models:
- Nemotron-3-Embed-8B-BF16: 8.0B parameters, 4096 embedding dimension, 32k context window, mean pooling, query: / document: input prefix, target hardware general GPU inference.
- Nemotron-3-Embed-1B-BF16: 1.14B parameters, 2048 embedding dimension, 32k context window, mean pooling, query: / document: input prefix, target hardware low-latency CPU/GPU.
- Nemotron-3-Embed-1B-NVFP4: 1.14B parameters, 2048 embedding dimension, 32k context window, mean pooling, query: / document: input prefix, target hardware NVIDIA Blackwell/GB200.
Enterprise Partner Evaluations
Enterprise ISVs, AI-native companies, and memory providers are already evaluating Nemotron 3 Embed across agentic retrieval, agent memory, code retrieval, and production inference workflows.
Adi Kuruganti, Chief AI and Development Officer at Automation Anywhere, said: "Context is the key to agentic accuracy. Our Context Intelligence Graph uses embeddings and semantic similarity to deliver the most relevant enterprise context to agents like EnterpriseClaw, which we launched with NVIDIA in May. Early results from NVIDIA's new Nemotron 3 Embed models are promising, particularly for question answering, where they show improvements over our current model. We're excited about their potential to further improve the accuracy and reliability of our enterprise agents."
Mani Gill, Senior Vice President of Product Management at Boomi, said: "Our initial evaluation of the NVIDIA Nemotron 3 Embed models shows strong retrieval performance for our agentic retrieval use cases. The availability of both 1B and 8B variants gives teams the flexibility to balance quality, latency, and deployment requirements across different environments. We're excited to continue evaluating the models and exploring how they can support high-performance retrieval for production AI applications."
IBM has seen promising early results evaluating the new NVIDIA Nemotron Embed model in a proof-of-concept built on watsonx.data.
Mem0 is evaluating NVIDIA Nemotron 3 Embed for its AI agent memory solution, where embeddings help retrieve relevant memories, link related context, and track relationships across an agent's interactions.
Palantir is collaborating with NVIDIA to evaluate the deployment of Nemotron 3 Embed for end customers conducting edge retrieval workloads, building on the prior work making NVIDIA embedding models available in AIP's Model Catalog.
ServiceNow is evaluating NVIDIA Nemotron Embed to retrieve through their documentation, with a continued interest in in-domain fine-tuning to improve their retrieval accuracy.
turbopuffer is bringing Nemotron 3 Embed to its native embeddings service, enabling developers to use the models within its semantic search engine.
Rahul Mohan, Senior AI Engineer at You.com, said: "Swapping in NVIDIA Nemotron 3 Embed model within our re-ranking stack delivered a significant leap in performance, letting us select query-relevant snippets from web pages with far greater accuracy than any model we'd used before."
Zep evaluated NVIDIA Nemotron 3 Embed 1B for agent memory and context retrieval. In early internal benchmarks against several models used by Zep, including some much larger than Nemotron 3 Embed 1B, the model ranked first across all memory retrieval tasks. Zep believes the availability of an FP4 checkpoint also makes the model very storage and latency competitive for their scale.
Zoom is evaluating NVIDIA Nemotron 3 Embed for the retrieval layer behind the enterprise agentic search that powers Zoom's contextual layer, which helps agents retrieve relevant context across meetings and workplace knowledge sources, including docs, chat, tickets, and other internal systems.
Getting Started
NVIDIA Nemotron 3 Embed is released with open weights and open-source training recipes, giving organizations full control over how retrieval models are customized and deployed for production AI applications.
Developers can get started using the deployment option that best fits their workflow. On Hugging Face, they can access the model weights, model cards, and example code for SentenceTransformers, Transformers, and vLLM. On NVIDIA NIM, they can access optimized microservice on build.nvidia.com for production-ready inference. Through AI Cloud and Inference Partners, they can deploy Nemotron 3 Embed through leading ecosystem partners including Baseten, Bitdeer AI, DeepInfra, Friendli AI, and OpenRouter.
For workloads requiring domain adaptation or footprint reduction, NVIDIA has also open-sourced NVIDIA NeMo AutoModel training recipes. The fine-tuning recipe adapts Nemotron 3 Embed to enterprise corpus and retrieval tasks. The distillation recipe compresses larger retrieval models while retaining ranking quality for production deployment.
For example, on the NV Docs evaluation, fine-tuning Nemotron-3-Embed-1B-BF16 improved NDCG@10 from 56.7% to 63.3% (+11.6%) and Recall@5 from 56.1% to 62.8% (+11.9%).
Whether you are building enterprise search, production RAG, agent memory, code retrieval, or agentic AI systems, NVIDIA Nemotron 3 Embed provides flexible deployment options from open-source models on Hugging Face to fully managed AI Cloud platforms and NVIDIA NIM microservices.