SambaNova has made prompt caching live on SambaCloud, beginning with the MiniMax M2.7 model. The feature automatically reuses long, unchanged prompt prefixes instead of processing the same tokens on every request, reducing both input cost and the time before a model starts returning output.
The launch is aimed at workloads that repeatedly send the same large block of context. Examples include coding agents with a stable system prompt, support applications that attach the same product documentation and analysis tools that ask multiple questions about one report.
Automatic prefix caching
SambaCloud's implementation uses automatic prefix caching. When a request begins with a prefix that matches one processed recently, the platform can serve those tokens from its cache. Developers do not need to enable a new request parameter or change their client configuration.
There are still structural requirements. The shared prefix must contain at least 4,096 tokens, stable content should appear first in the prompt and the reused section must remain identical. A small change inside the prefix can prevent a cache match, so applications that insert volatile data near the beginning of every request may see less benefit.
Lower input pricing
SambaNova says cached MiniMax M2.7 input is billed at US$0.06 per million tokens, compared with US$0.60 per million for standard input. That is a 90 per cent discount on the part of the prompt served from cache. Output remains priced separately at the model's normal rate.
The actual saving will depend on cache-hit frequency and how much of each request is reusable. Applications with a long reference document and many follow-up questions stand to save more than workloads where every prompt is substantially different.
Responses now include a prompt_tokens_details object with cached_tokens and cache_creation_tokens fields. These counters allow developers to see which tokens received the cached rate and which were written into the cache for potential reuse.
Latency gains grow with context
In SambaNova's tests across prompts containing 8,000 to 192,000 tokens, caching reduced time to first token by 33 per cent at the short end and 91 per cent at the long end. The company reports a median 6.1-times speed-up. At 192,000 tokens, time to first token fell from 40.6 seconds to 3.7 seconds.
Those figures are company measurements and will not translate directly to every application. End-to-end latency also includes networking, output generation, tool calls and any processing performed outside SambaCloud. Developers should measure their own traffic distribution and cache-hit rate before forecasting savings.
Important limits at launch
Prompt caching currently applies only to MiniMax M2.7. Requests to other models will report zero cached tokens. The maximum cacheable prefix is 192,000 tokens, and entries are evicted automatically according to traffic patterns. SambaNova does not promise a fixed time to live for each prefix.
That variable lifetime means applications should treat caching as an optimisation rather than a permanent store. They must continue to send the complete prompt required for a correct response and should not assume that a previous request will remain available.
For suitable workloads, the launch provides a straightforward path to lower repeated-context costs without changing API semantics. The clearest next step is to inspect the new usage fields in production-like tests, confirm that stable content is positioned at the start of prompts and calculate savings from observed cache hits rather than headline discounts alone.