Prompt Caching Cuts Agentic AI Costs by Up to 80%, but Only if You Design for It

An AI agent that re-reads the same 100,000-token codebase at every step is paying full price for work it already did. Prompt caching is how you stop paying twice.

For CFOs and technical directors, the issue is less about which model to pick than about how the prompt is built. The same model can cost three or four times more or less depending on whether the stable parts of the prompt stay stable.

Why agent loops burn tokens

An agent works in a cycle of reasoning, calling a tool, observing the result, and planning again. At every turn, the model has to take in its whole working context again. That includes system instructions, compliance guardrails, tool definitions, retrieved documents and the growing conversation history.

The model keeps no memory between calls, so step ten means recomputing steps one through nine. One production analysis found that about 62% of an average enterprise agent's inference bill goes to re-sending and re-processing identical static context.

The cost gap between chat and agents is large. EY estimates that orchestrating agentic AI can cost more than 30 times as much as running a standard chatbot. It puts the move from a $0.04 query-and-response session to a multi-agent task at upward of $1.20 per completed task. Gartner projects that inference cost per agent-based workflow will rise more than fivefold by 2028.

How prompt caching works, in plain terms

To read text, a transformer model computes "key" and "value" vectors for every token in relation to the tokens before it. For a 100,000-token codebase, that calculation is the main source of both delay and cost.

Caching keeps those computed vectors in the provider's memory between requests. When the next request arrives, the model loads the stored state and adds only the new tokens, instead of starting from zero.

The rules are strict, and they decide whether you save money:

• Exact prefix match. Two prompts share a cache only if they are identical from the very first token. A rotating user ID, a timestamp or a changed space near the top breaks the match.
• Time limits. GPU memory is scarce, so providers evict entries that are not used within a time window.
• Minimum size. Anthropic's Claude 5.5 models need at least 512 tokens to trigger the cache. OpenAI's GPT-5.6 needs 1,024.

When the match fails, the system quietly bills you at the full input rate. Nothing errors out. The invoice just grows.

What Claude 5.5 charges, and the arithmetic

Anthropic released the Claude 5.5 family between September 22 and October 7, 2026, aimed at enterprise agent work, software engineering and high-volume data processing. The pricing follows a simple bet. Writing to the cache costs extra, and reading from it costs very little.

Writing a cache costs 1.25x the standard input price for a 5-minute lifetime, or 2.0x for a 1-hour lifetime. Reads on Opus 5.5 and Sonnet 5.5 are priced at 0.05x the base input price, a 95% discount.

Opus 5.5 lists at $4.00 per million input tokens. A 100,000-token system prompt therefore costs $0.40 to process uncached. Writing it to a 5-minute cache costs $0.50, and each read inside the window costs $0.02. Sonnet 5.5 lists at $2.00 per million, with cache reads at $0.10 per million.

Here is what that means for a 20-step agent run. It is our own illustration using those list prices. It counts only the 100,000-token stable prefix, ignores new tokens and output, and assumes every step lands within the 5-minute window.

A working cache cuts that slice of the bill by about 89%. A broken one costs more than having no cache at all. After a single reuse, one write plus one read already costs less than two uncached calls.

Anthropic's higher tier follows the same logic. Fable 5.1 and Mythos 5.1 kept their standard price at $10.00 per million input tokens but cut the cache read price by 75%, to $0.25 per million.

How the other vendors price memory

Billing philosophies differ, and the right one depends on your traffic pattern. Steady loops suit short lifetimes. Bursty or low-frequency workloads may suit something else.

The table follows the report's October 2026 comparison. Google's explicit mode drops the write premium and charges for storage by the hour instead. On Gemini 3.8 Flash that is $0.50 per million tokens per hour, plus the discounted read rate. That suits a large knowledge base that must stay warm through unpredictable, low-frequency traffic.

DeepSeek's hit price is halved again off-peak, to $0.003 per million tokens, but you get little control over eviction. OpenAI's tooling includes cache-hit dashboards, which gives finance teams visibility without extra telemetry.

What measured results look like

In January 2026, PwC researchers published "Don't Break the Cache," an evaluation of prompt caching for long-running agent tasks. They ran more than 500 agent sessions across OpenAI, Anthropic and Google models, using a 10,000-token system prompt.

Proper caching cut API cost by 41% to 80% against a no-cache baseline, with statistical significance. Best-case savings were 79% to 81% on GPT-5.2 and 78% to 79% on Claude Sonnet 4.5. Time to first token improved by 13% to 31%.

The same study showed the failure case. Naive strategies that let dynamic tool results break the prefix made latency worse by 10% to 18%. In those runs the system kept paying the write premium for a prefix it never read back, which cost more than leaving caching off.

A July 2026 production analysis from Deriv AI shows the dollar effect. Its assistant processes roughly 30 billion input tokens a month. At a 20% cache hit rate the bill was about $30,750 a month. After restructuring prompts to reach an 85.8% hit rate, input-token costs fell 77% to $8,540. The difference, $22,210 a month, came from the same model.

Four design rules that protect the savings

1. Put stable content first and variable content last. System instructions, guardrails, tool definitions and large reference documents belong at the top of the prompt. User IDs, timestamps and the actual question go at the very end. Putting a timestamp inside the system prompt silently wipes out the hit rate.

2. Measure the hit rate. Anthropic's usage data reports cache creation and cache read tokens, and OpenAI reports cached tokens. A hit rate below 70% during an agent's intermediate steps signals prompt bloat or broken prefixes. Check your telemetry too: integration errors between OpenTelemetry and Anthropic have been documented to double-count cached tokens and inflate apparent cost.

3. Attribute cost and cap it. Tag every request so token use and cache efficiency map back to a business unit, developer and workflow. That lets a runaway loop hit a quota before it drains a department's budget. Prune tool lists to what each agent needs, and have tools return short summaries or IDs rather than raw data dumps.

4. Layer other savings on top. Semantic caching matches requests by meaning (the report cites a 0.8 similarity threshold) and returns an earlier answer without calling the model at all. For work that is not time-sensitive, such as nightly reports or bulk classification, batch APIs from OpenAI and Anthropic give a 50% discount for jobs completed within 24 hours. Caching discounts stack on top of batch discounts.

Build vs. buy when caching changes the math

Cheap API calls make custom agents look easy. One finance pilot, a variance-analysis agent for the financial close, might cost only $150 in tokens to reach a working prototype. Analysts call that figure a red herring, because the real cost hides a 20-to-30x multiplier in engineering, scaling, testing and upkeep.

The report's numbers for a custom build:

• A production-grade custom agent typically costs $20,000 to $80,000 and takes 8 to 12 weeks.
• At $150 an hour, a minimal 30-day build is $36,000 in labor alone.
• Upkeep is estimated at 20% of the build cost each year, or $7,200 on that example.
• Hosting, telemetry, vector memory management and prompt versioning come on top.

Buying has its own limits. Off-the-shelf software rarely handles deep integration with proprietary core systems or bespoke data-residency rules. Platform fees also grow with usage. One example moves from $19,000 in year one to $32,000 in year two.

Volume decides it. Below 5,000 interactions a month, no-code or low-code platforms give a better return. Analysts also use an 80% predictability rule: if 80% of a workflow follows bounded, predictable patterns, an agent pays off. Past 10,000 interactions a month, platform licenses of $1,500 to $2,000 a month begin to exceed the roughly $600 a month a well-built custom stack spends on optimized, cached API calls.

Most enterprises end up in the middle. They buy orchestration platforms or managed infrastructure, then build their own logic, prompt structure and tool definitions on top. The prompt structure is where the caching savings come from, so that is the layer worth owning.

If you are weighing a custom build against a platform, it can also help to look at the broader modernization picture. Hidden costs in data migration, integration work and parallel runs often add 30% to 45% to budgets, as covered in our breakdown of why modernization costs tend to climb past the sticker price.

Security and compliance questions to settle first

Cached context sits in provider memory, and that raises two issues. The first is timing. A cache hit returns its first token faster than a miss, and that difference can leak information about what is stored. Stanford researchers audited 17 API providers and found that seven used global cache sharing across unrelated users. A separate NDSS 2025 study, PROMPTPEEK, showed attackers in a shared serving environment reconstructing a victim's prompt token by token.

Providers responded. At least five changed their implementations and enforced isolation. Anthropic isolates caches per workspace on the Claude API and per organization on AWS Bedrock and Google Cloud. It also states that prompt caching is eligible for Zero Data Retention, with only temporary tensors held in volatile memory.

The second issue is regulatory. Under GDPR Article 6, each processing operation needs a lawful basis, and a cache hit arguably processes earlier data on behalf of a new request. As of late 2026, no major provider has given definitive guidance. Most also offer no endpoint to delete a specific cache entry before its lifetime ends. Regulated firms should treat the prompt cache as a logged data store under normal residency and retention governance.

If your agents touch regulated data, it is worth aligning caching plans with your wider architecture. Clear separation of layers and automated design checks, like those described in our enterprise application architecture approach, make it easier to prove where data lives and how it flows.

A starting checklist for the next 90 days

1. Pull the numbers. Find your cache hit rate by workflow. If you cannot see it, that is the first fix.
2. Audit prompt structure. Look for timestamps, IDs and per-user text above stable content.
3. Match the lifetime to the traffic. Steady loops fit 5-minute caches. Sparse traffic may fit a longer lifetime or a storage-fee model.
4. Move deferrable work to batch. Nightly and bulk jobs should not pay real-time prices.
5. Set quotas and tags. Every agent should have an owner and a spending limit.
6. Get legal sign-off. Settle isolation, retention and residency terms before sensitive data enters a cache.

Conclusion

Agent costs rise because the model re-reads the same context at every step. Caching stores that work and charges roughly 5% to 10% of the input price to reuse it, depending on the vendor. The savings depend on prompt discipline: the same model produced a $30,750 bill and an $8,540 bill in one documented case.

Treat the stable prompt as a governed asset, measure the hit rate, and own the prompt layer even when you buy the platform. If you want a second pair of eyes on an agent's cost structure or a build-vs-buy decision, Baytech Consulting is glad to talk it through. For teams that want help designing, building and running these systems, our AI-powered development services and AI integration practice are set up to handle both the engineering and the ongoing optimization.

FAQ

What is prompt caching in one sentence?

The provider stores the computed state of the start of your prompt, so later requests that begin with identical text only pay a small read fee instead of full processing.

How much does it really save?

PwC's study of more than 500 agent sessions found cost reductions of 41% to 80% against no caching. Deriv AI cut input-token costs by 77% by raising its hit rate.

Can caching ever cost more than not caching?

Yes. If the prefix changes every call, you keep paying the write premium and never get a read discount. On Opus 5.5 the 20-step illustration above comes to $10.00 against $8.00 uncached.

Is a 5-minute or 1-hour cache better on Claude?

The 1-hour write costs 2.0x instead of 1.25x. The cheaper 5-minute option fits tight loops, and the longer one fits agents that are used less often.

Is cached data safe?

Providers have tightened isolation since the Stanford findings, and Anthropic states that caching is eligible for Zero Data Retention. Data-residency questions remain open, and most providers offer no way to delete a single entry early.

Should we build our own agent or buy a platform?

Under 5,000 interactions a month, a low-code platform usually wins. Above 10,000 predictable interactions a month, a well-built cached custom stack can cost far less than licenses. Many firms combine both, often with a dedicated team handling the custom work while a platform covers lighter-weight use cases.

Supporting Links

• How prompt caching cuts AI agent costs
• Agentic AI cost estimates and Gartner inference forecast
• Prompt caching across AI providers, 2026 pricing comparison
• "Don't Break the Cache": prompt caching for long-horizon agentic tasks (PwC)
• Prompt caching and production AI agent costs (Deriv AI)
• How to build an AI agent: costs and volume thresholds
• The hidden data residency problem in prompt caching

About Baytech

At Baytech Consulting, we specialize in guiding businesses through this process, helping you build scalable, efficient, and high-performing software that evolves with your needs. Our MVP first approach helps our clients minimize upfront costs and maximize ROI. Ready to take the next step in your software development journey? Contact us today to learn how we can help you achieve your goals with a phased development approach.

About the Author

Bryan Reynolds is an accomplished technology executive with more than 25 years of experience leading innovation in the software industry. As the CEO and founder of Baytech Consulting, he has built a reputation for delivering custom software solutions that help businesses streamline operations, enhance customer experiences, and drive growth.

Bryan’s expertise spans custom software development, cloud infrastructure, artificial intelligence, and strategic business consulting, making him a trusted advisor and thought leader across a wide range of industries.