Baseten has added NVIDIA Nemotron 3 Embed 8B and Nemotron 3 Embed 1B to its model library for dedicated inference. Both models turn text or code into numerical embeddings that can be stored in a vector index and retrieved when an AI agent, search system or coding assistant needs relevant context.
The two sizes give developers a choice between retrieval quality and operational efficiency. Baseten positions the 8-billion-parameter model for accuracy-sensitive workloads and the smaller 1-billion-parameter version for systems that must index large volumes of frequently changing information.
Two models for different retrieval needs
Baseten says Nemotron 3 Embed 8B leads the Retrieval Embedding Benchmark leaderboard across open and closed models. That result supports its use for enterprise knowledge bases and code search where missing a relevant passage has a high cost. As with any benchmark, teams should confirm the result on their own languages, document types and query patterns.
Nemotron 3 Embed 1B uses pruning and distillation to reduce size while retaining what NVIDIA and Baseten report as 95 per cent of the larger model's retrieval accuracy. For NVIDIA Blackwell hardware, an NVFP4 version is claimed to deliver up to twice the throughput while keeping 99 per cent of BF16 accuracy. These figures depend on the specified hardware and evaluation setup.
Index freshness is a practical reason to consider the smaller model. When source documents, repositories or runbooks change frequently, slow embedding generation can leave a retrieval system searching an outdated index. Faster indexing may therefore improve the final answer even if the smaller model has slightly lower standalone benchmark accuracy.
Managed deployment on Baseten
Both models are available through Baseten's Dedicated Inference service, giving customers a managed route to production without setting up the serving infrastructure themselves. Baseten has not published a single price in the announcement, so teams will need to compare deployment cost, expected utilisation and service requirements directly.
Baseten also worked with turbopuffer, a vector and full-text search service, to make Nemotron 3 Embed available through its native embeddings. That integration can reduce the work required to connect model serving and vector indexing, although customers should still examine data location, failure handling and how easily they can re-embed a collection when models change.
Fine-tuning and evaluation
For specialised terminology or domain-specific relevance, NVIDIA provides a fine-tuning recipe that can be run with Baseten Training. Baseten reports that in-domain fine-tuning produced about a 10 per cent accuracy improvement within five hours in its example. That is a useful starting point, not a guaranteed result for every dataset.
A sensible evaluation should measure retrieval recall, ranking quality, indexing time, inference latency and total cost using representative documents. Teams should also test whether poor retrieval comes from the embedding model, chunking choices, metadata filters or the generation model that consumes the results.
The release gives Baseten customers prompt access to NVIDIA's new retrieval models and a clear choice between a larger accuracy-oriented option and a smaller throughput-oriented one. The operational benefit will depend on how each model performs within the complete retrieval pipeline, rather than on model benchmarks alone.