NVIDIA says its Groq 3 LPX rack-scale inference system has entered full production, extending the Vera Rubin NVL72 platform with specialised hardware for the token-generation stage of AI inference. The announcement, made on 24 August, matters because NVIDIA is framing agent workloads as a distinct infrastructure problem: systems must not only process large prompts, but also return many small, sequential outputs quickly while tools, agents and sub-agents continue to interact.
The company describes Groq 3 LPX as an LPU-based component designed to work alongside Vera Rubin GPUs rather than replace them. In NVIDIA’s account, Rubin GPUs handle large-scale context processing while LPX accelerates latency-sensitive decoding. That division is intended to address a familiar bottleneck in long-running agent workflows, where a delay per generated token can compound over a chain of tool calls, planning steps and follow-up responses.
A production claim tied to agent workloads
NVIDIA’s central product claim is that Groq 3 LPX is now in full production. The company says a rack-scale deployment can include 256 LP30 accelerators connected through direct chip-to-chip links, and that the system has been codesigned with Vera Rubin NVL72. Its positioning is focused on low latency, high throughput and more predictable token generation for real-time, agentic applications.
The announcement uses a benchmark based on Gemma 4 31B, where NVIDIA says the system achieved 3,400 output tokens per second for 100,000-token long-context use cases. NVIDIA characterises that result as four times faster than the nearest alternative platform. As with any vendor benchmark, prospective users should treat the result as a company-reported measurement rather than an independent comparison; model choice, prompt shape, concurrency and deployment configuration can materially affect realised performance.
Still, the practical distinction NVIDIA is drawing is useful. Conventional inference discussions often focus on a single response or a static batch. An agentic application may instead alternate between model output, retrieval, code execution, calls to other services and additional model turns. It can repeatedly grow its context while users wait for the next action. For those workloads, decode speed and latency consistency can be as relevant as headline training or prefill throughput.
How the platform is meant to fit together
NVIDIA presents the launch as part of a wider AI-factory architecture. Vera Rubin NVL72 is the rack-scale base, Spectrum-X networking is intended to move data across the environment, and Groq 3 LPX addresses the interactive generation portion of inference. The company calls this approach “extreme codesign”: engineering compute, networking and inference acceleration as a unified system rather than treating each layer as a separate purchase.
That integrated approach can simplify evaluation for organisations already standardising on NVIDIA infrastructure, but it also makes the announcement more than a claim about one accelerator. Buyers will need to understand the operational boundaries between GPUs and LPUs, the supported models and runtimes, network requirements, power and cooling assumptions, and how observability works across a mixed serving stack. NVIDIA’s post gives a product direction but does not set out full availability, pricing or service-level terms for every deployment path.
Nebius is named as the first AI cloud to adopt Groq 3 LPX, while CoreWeave is described as deploying Spectrum-X Multiplane in production. NVIDIA also says SpaceXAI plans to use NVIDIA Vera CPUs for work behind agentic AI, including orchestration, tool use, code execution, data processing and simulation. These ecosystem references show the company’s strategy: position the platform as infrastructure for a whole agent lifecycle, not simply a faster endpoint for one model request.
Why token economics are becoming a product decision
For model providers and enterprise operators, inference costs increasingly depend on the workload rather than a nominal model size alone. Long prompts, repeated retrieval, multi-step reasoning and tool outputs can all increase the number of tokens a system has to process. A platform that improves the responsiveness of decoding may help preserve a usable experience, while better utilisation can change the economics of serving large volumes of interactive requests.
NVIDIA’s language about “token factories” reflects that commercial framing. The company argues that AI infrastructure should be assessed through delivered output, latency, context capacity and utilisation as well as raw hardware specifications. That is a reasonable lens for production teams, though the relevant metric will vary: a coding agent, a voice assistant and a batch document-processing workflow can have very different latency tolerance and cost profiles.
The launch also lands amid growing competition around inference hardware and software. The benefit NVIDIA is claiming comes from joining specialised decode acceleration to a broad GPU and networking platform. Whether that combination is compelling will depend on deployment scale, model support, regional availability and the total cost of operating the complete rack, not only on the peak performance figure highlighted in the announcement.
What teams can verify before adopting it
Organisations evaluating the platform should ask for workload-specific evidence. Useful tests would include their actual context lengths, tool-call patterns, concurrency and service targets, measured across both prefill and decode phases. They should also compare the end-to-end effect on queueing, failover, monitoring and cost per successful task, rather than relying only on tokens per second.
NVIDIA’s update is nevertheless a concrete signal that the company sees agent inference as a major design target for Vera Rubin. By moving Groq 3 LPX into production and naming early cloud and ecosystem deployments, it is attempting to turn that design into an available infrastructure option. The next evidence to watch will be detailed availability, independent performance results and how broadly software teams can use the combined GPU-and-LPU system in practice.