Repeated context becomes a billable design choice

Amazon Bedrock has extended prompt caching for open-weight models with an explicit mode that lets developers mark reusable prompt prefixes. The service already supports implicit caching for compatible models, where matching prefixes are selected automatically. Explicit breakpoints give applications more control over which system instructions, reference documents or tool definitions are written once and read across later requests. For agent developers, the feature targets a common inefficiency: sending the same large context on every model call even when only the latest user message or tool result has changed.

Prompt caching reuses the computed state of an identical prefix rather than processing those tokens again. Cache reads are priced below normal input for supported models, while some models charge a premium when a prefix is first written. Savings therefore depend on reuse. A large document cached for one request can cost more than processing it normally, whereas a stable agent prompt used hundreds of times may deliver material reductions in latency and input charges. Teams should model the expected read-to-write ratio before changing production prompts.

Implicit caching requires no request changes. Bedrock finds eligible matching prefixes on a best-effort basis, making it suitable for conversations and agent loops that already grow by appending new turns. The prompt should place stable material first and dynamic content last. Changing an early instruction invalidates reuse from that point forward. This ordering is operationally important: injecting timestamps, request IDs or user-specific text near the beginning can destroy cache efficiency even when most later content remains identical.

Implicit and explicit modes suit different prompts

Explicit mode adds prompt_cache_options and one or more prompt_cache_breakpoint markers through the Chat Completions API. Each breakpoint identifies a layer that may change at a different rate. A product manual can form one shared prefix, while tenant policy or tool definitions form another. Content after the last breakpoint remains uncached. Limits, minimum prefix lengths and time-to-live vary by model, so an application cannot assume that a caching strategy transfers unchanged across the Bedrock catalogue.

AWS demonstrates the feature with an OpenCode coding-agent session. The first request contains system instructions, tool definitions and the user task. After the model asks for a tool, the agent executes it locally and sends a second request containing the earlier context plus the result. Bedrock can read the first request from cache and process only the new tail. Coding agents are a strong fit because tool schemas and repository instructions may be repeated many times within one task.

Agents can reuse tool-heavy prefixes

Caching does not remove the security implications of the prompt. A poisoned instruction or excessive secret placed in a reusable prefix can influence every later request until the entry expires. Applications should construct prefixes from reviewed, versioned sources and keep credentials out of model context. Multi-tenant systems need clear separation so one customer never receives another customer’s cached state. Developers should verify the service’s isolation guarantees and use distinct identifiers or endpoints where their threat model requires stronger boundaries.

Observability is essential because a configured cache may not be producing hits. Bedrock responses include usage fields for cache reads and writes, while runtime metrics can show aggregate behaviour. Teams should monitor hit ratio, written tokens, read tokens, latency and total model cost by prompt version. A sudden fall in hits can reveal that an upstream component changed formatting. A high hit rate can still be a poor outcome if the cached prompt is unnecessarily long or produces lower-quality answers.

Cache metrics need outcome-level measurement

Prompt design and model quality remain connected. Moving volatile instructions after a breakpoint may improve reuse but alter how a model weighs them. Compressing reference material can cut cost while dropping details needed for accuracy. Every caching change should pass the same evaluation set as a model or prompt release, including adversarial and long-conversation cases. The business metric is cost per successful task, not cached tokens alone. Retries and human correction can overwhelm nominal savings from a faster call.

AWS notes that prompts and outputs on Bedrock are not shared with the model provider or used for model training, but customers still need to check regional support, licences and service terms for the chosen open-weight model. Explicit caching makes Bedrock’s OpenAI-compatible Chat Completions interface more practical for long-context agents. Its value will be greatest where prefixes are stable, reuse is frequent and developers can prove that the cache reduces end-to-end cost without weakening isolation or answer quality.

Versioning deserves special attention. A cached prefix should carry an application-visible prompt version so releases can be compared, rolled back and correlated with evaluation results. Deployments should warm a new version deliberately rather than mixing unexplained cache misses into user traffic. When a policy or tool definition changes urgently, operators also need confidence that the old prefix will no longer influence calls, even if its nominal lifetime has not expired.