AI infrastructure has a storage utilisation problem

Google Cloud has made the Storage-optimised Z4M machine series generally available in Compute Engine. The systems are offered as virtual machines and bare-metal instances and are designed for workloads that move enormous volumes of data in parallel. Google specifically calls out AI training, inference, checkpointing, distributed key-value caches and parallel file systems as primary use cases.

Accelerators can only deliver their headline throughput when data reaches them quickly enough. Large training jobs repeatedly save checkpoints, while inference clusters maintain growing KV caches for long contexts and agent sessions. If those operations wait on storage or networking, expensive GPUs and TPUs sit idle. Z4M addresses that bottleneck with dense local flash and high-bandwidth remote access.

The headline configuration is unusually dense

A Z4M instance can provide up to 168,000 GiB of local SSD storage and as much as 400 Gbps of Cloud RDMA networking. The machines use fifth-generation Intel Xeon Scalable processors and Google’s Titanium infrastructure. That combination is aimed at data-intensive systems that need both high aggregate throughput and predictable low-latency access across a cluster.

Local SSD changes the failure model compared with durable managed storage. It is excellent for caches, scratch data and replicated file systems, but applications must assume that data tied to an instance can disappear. Architects need an explicit durability tier, automated rebuilds and tested recovery. Large capacity is useful only when the software can distribute data and recover placement without lengthy accelerator downtime.

Checkpointing is becoming a first-order design constraint

Frontier training runs save increasingly large model and optimiser states. A slow checkpoint can interrupt thousands of accelerators, while infrequent checkpoints increase the amount of work lost after a failure. Storage bandwidth therefore affects both utilisation and recovery risk. Z4M gives platform teams another way to stage checkpoints near compute before moving durable copies into object or parallel storage.

Benchmarking should use the actual checkpoint format, number of writers and recovery process. Sequential throughput figures rarely capture metadata contention, small-file behaviour or the burst created when many workers save at once. Teams should measure pause time at the training framework, not just disk bandwidth, and confirm that background transfer does not starve the next compute phase.

Distributed KV caches create a different workload

Inference systems use KV caches to avoid recalculating previous tokens. With long conversations and agentic workflows, cache capacity and placement can shape time to first token and cluster efficiency. Fast local storage may extend the effective cache hierarchy below GPU memory, while RDMA can help workers exchange state without sending every operation through a slower general-purpose network path.

The trade-off is complexity. A cache tier must understand eviction, model versions, tenant isolation and prompt sensitivity. Persisting cache material can expose user data beyond the expected request lifetime. Operators need encryption, deletion guarantees and workload-aware routing, along with evidence that a storage-backed cache improves latency after accounting for lookup and transfer overhead.

Who should evaluate Z4M

Z4M is most relevant to organisations operating large training clusters, high-throughput inference or specialist analytics with parallel file-system requirements. It is less likely to change the economics of a modest application that is limited by model cost or network calls. The right comparison is an end-to-end job on the intended accelerator fleet, not a storage microbenchmark in isolation.

Teams should compare completed training steps or served tokens per dollar and per watt, including replication and durable-storage costs. They should also test instance loss, replacement capacity and rebuild time. General availability makes Z4M a supported production option; it does not remove the architecture work needed to turn local flash and RDMA into reliable AI infrastructure.

Storage design now affects model-serving quality

For retrieval and agent systems, storage latency can appear to users as slow reasoning even when model inference is fast. Vector indexes, session state, tool results and cache entries all compete for I/O. A platform team evaluating Z4M should trace a request across the complete path and identify where faster local access changes time to first token, tail latency or the number of concurrent sessions the service can sustain.

Capacity planning should include failure domains. Packing a large working set onto one storage-heavy machine may look efficient but can create a disruptive rebuild after instance loss. Sharding, replication and placement need to balance recovery time against write amplification and cost. RDMA networks can move data quickly, yet congestion control and topology awareness still matter when many nodes attempt a rebuild or checkpoint at once.

Google’s GA announcement gives buyers a defined product and support boundary, which is important for production procurement. The next useful evidence will be customer measurements that compare Z4M with general-purpose compute plus managed storage on representative AI workloads. Those results should include utilisation, recovery and total cost. Storage-optimised infrastructure earns its place when it keeps accelerators productive through ordinary failures, not just peak benchmark runs.