llm-d 1.0 coordinates distributed inference for agent workloads
Google and the llm-d community have released version 1.0 of the open Kubernetes-native inference stack. It adds stable APIs, cache-aware routing, multi-tenant flow control, disaggregated upgrades and asynchronous queues across TPU and GPU fleets. Published tests report large time-to-first-token, throughput and cache-hit improvements for long-context and bursty agent workloads.