Can anyone explain why the prefix cache is tied to effort?
I frequently run Fable at xhigh effort to run statistical modeling way above my undergraduate understanding. Claude Fable produces Masters-degree level output, and then I spend lots of round trips asking it to explain different parts to me.
The first part absolutely uses the extra effort, but the interrogation exercise is something a much simpler model, or the same model with much less effort, could answer.
One trick is to simply as for a fast answer when talking to a high effort model, when working interactively. Sounds stupid but I do this all the time and it works. Just tell it you are working interactively now and need ultrafast answers with no thinking.
I would be really curious to know as well, why effort is linked to cache as its quite inconveniant. Is it possible the token used to indicate effort is only passed once at the start, not per thinking trace, or quite simply that different efforts have different model weights?
I’m guessing that there’s a system prompt at the top telling the model about its reasoning budget. So when you switch reasoning effort it busts the cache.
Bro: superintelligent machine line go up AI AGI software solved automate everything
Also bro: Run /clearbetween tasks. This prevents prior irrelevant context from being sent back to the model, which can reduce token usage.
Set your model and effort level before you start. Changing either one mid-conversation can bust your prompt cache, which can increase token cost.
@-mention files instead of naming them. The file gets attached to your message directly, which saves a Read call, or a search if Claude has to go find it.
Add quiet flags to noisy commands, or run them in a subagent. Command output is added to the conversation just like a file, and stays there for the rest of the session.
Run /context once in a fresh session. It shows what's loaded (CLAUDE.md, MCP tool definitions), so you can cut out anything unnecessary.
/compact before you take a break from your keyboard. The prompt cache expires after an hour, and summarizing a conversation is much cheaper while it's still cached.
After having used Codex for a promotional month, and now using Claude, Claude is not as efficient with finding relevant information. I can give it the one file it should be using and then it goes off and greps parent directories for more context. It’s also incredibly slow at producing results because of this side work. In this article, it seems like they are catching up to what GitHub copilot users had already been doing since the cost restructuring in June.
I'm finding that unexpected cache rewrites cost me huge.
I have 1h cache TTL set, and do nothing to cause rewrite (response in time, no model/effort/tool changes).
At 400K tokens in, I'll write a message, and /usage shows only a small increase in cache write. On the next message, cache writes shows 800K, and by the end, I often hit 2M cache writes with no explanation.
This seems to happen when: using /btw, asking it to review code, other random times. Anyone know what's going on?
I mean, it feels hard not to laugh at this type of blog post. My cynical interpretation is that this is a type of passing the buck to engineers in enterprise settings ("Stop spending tokens. Did you read the value maximization blog post? It is your fault.")
Oh yes, Claude will do all sorts of different things -- it depends on how you use it! You should totally learn all of these little finicky things ... because now completing your tasks cost money. It's not "free" anymore haha like when you used your old text editor, what are you a grandpa?
Oh, and those things will definitely change, as we (the priests of Claude) are vibe coding the system you use to do your little "tasks" ... right, you can't see how it works ... the code is not available. It's all good, just trust us -- we're totally looking out for you.
I mean it is utterly ridiculous to talk around this model of development. There are so many walls between you and doing the thing you want to do.
Agents are great, but the notion of "best tricks" for how to best use an opaque costful tool which will, by all odds, be completely different in a few months time is quite funny.
You know what won't change? A fucking text editor. Or your pi config, or a local model you run and trust.
I'm trying to understand your point of view, but it kind of just sounds like you're against learning how to use tools efficiently?
I mean, agentic coding software is hardly the first tool to exist where learning some idiosyncrasies of how to use it well can result in more efficiency and cost savings.
I think I'm happy about the first two. The third I suppose I care less about, just because I've kind of become used to it from decades working on the internet where many businesses/tools/apps are more like services and less like physical tools that never change.
> The third I suppose I care less about, just because I've kind of become used to it
I've never become used to it. My impression is that the constant churn has accelerated. Plausible drivers are (1) normalize novelty as desirable (like fast fashion), (2) product developer/designer incentive structures that reward revolutionary change over progressive refinement. The global switch to subscription models and continuous deployment didn't help.
> more like services and less like physical tools that never change.
I'm not sure that constant change is a characteristic feature of services, especially not professional services.
It used to be that you bought a piece of software and used that version until you decided it was worth upgrading, like a particular physical tool. The software still evolved, just like the design of physical tools can, in principle, evolve.
All that said, agentic AI tooling is evolving so rapidly I'm not sure an expectation of stability is realistic.
I agree that the third seems to be implied by industry, but I'd argue that it's not clear that it is necessary -- and it is subtle whether or not it is beneficial?
My contention is that we should be building towards less churn, not more. I'm aware that some churn is the cost of engaging in any sort of enterprise, but I'm deeply suspicious of an AI company inserting themselves between me, and the tasks I wish to do with my device -- with a completely opaque system that I can't really "learn".
dude, if you try to do harness development yourself you will realize that most things said in this blogpost is shared with any ${sufficiently_advanced_harness}. this is not really claude-specific, this is just how this class of tools, OSS or not, works
That's not my complaint. I know well the concerns of agent harnesses.
My complaint is that this is a low-dimensional projection of a system which I have no insight into, and therefore, I cannot evaluate the tips myself against their source.
Am I to believe the creators, knowing full well that the source will, as Boris Cherny put it in a recent interview, be deleted and rewritten from scratch at the release of the next big model?
Further: I'm responding to content in the blog post itself:
> Until pretty recently, the tools you wrote code with were a flat fee (or free). Your editor cost the same whether you fixed one test or fifty that afternoon, so an individual task didn't really have a price of its own.
I find this type of prose ridiculous. It conveys "this is the way things are now, get used to it".
I do all that, but an 'AS-BUILT' full review of my project still eats 3x my 5 hour budget on max 100€. Meanwhile, my 20€ GPT never hit a limit. Different, but just saying.
Part of the cynic in me just wants to ask "why not make a better harness by default?" The other cynic in me knows I'm about to see a hundred post on 'HOW TO 10X CLAUDE" from the ai bros and I'm already tired.
I guess if I had to ask something (as someone who doesn't use CC as their daily driver), how much control do you have on subagents and roughly how do define or know when a session is getting too long? I know the answer is "when the model is getting worse" but worse is doing a lot of lifting in that sentence.
Yeah, I sort of feel like they could just do this in Claude Code for us in some way. I mean they already run some mini classifier on whether a given prompt is waiting on input, so they could do the same to detect some of these cases, and just handle it.
If you have to explain that someone is "holding it wrong"(1), that is product error, not a user error.
This feels like the Anthropic version of "You're holding it wrong" (1)
1) https://www.wired.com/2010/06/iphone-4-holding-it-wrong/
Can anyone explain why the prefix cache is tied to effort?
I frequently run Fable at xhigh effort to run statistical modeling way above my undergraduate understanding. Claude Fable produces Masters-degree level output, and then I spend lots of round trips asking it to explain different parts to me.
The first part absolutely uses the extra effort, but the interrogation exercise is something a much simpler model, or the same model with much less effort, could answer.
One trick is to simply as for a fast answer when talking to a high effort model, when working interactively. Sounds stupid but I do this all the time and it works. Just tell it you are working interactively now and need ultrafast answers with no thinking.
I would be really curious to know as well, why effort is linked to cache as its quite inconveniant. Is it possible the token used to indicate effort is only passed once at the start, not per thinking trace, or quite simply that different efforts have different model weights?
I’m guessing that there’s a system prompt at the top telling the model about its reasoning budget. So when you switch reasoning effort it busts the cache.
do people use the Caveman, RTK plugins
Bro: superintelligent machine line go up AI AGI software solved automate everything
Also bro: Run /clearbetween tasks. This prevents prior irrelevant context from being sent back to the model, which can reduce token usage. Set your model and effort level before you start. Changing either one mid-conversation can bust your prompt cache, which can increase token cost. @-mention files instead of naming them. The file gets attached to your message directly, which saves a Read call, or a search if Claude has to go find it. Add quiet flags to noisy commands, or run them in a subagent. Command output is added to the conversation just like a file, and stays there for the rest of the session. Run /context once in a fresh session. It shows what's loaded (CLAUDE.md, MCP tool definitions), so you can cut out anything unnecessary. /compact before you take a break from your keyboard. The prompt cache expires after an hour, and summarizing a conversation is much cheaper while it's still cached.
Author not bro, sorry misgender
After having used Codex for a promotional month, and now using Claude, Claude is not as efficient with finding relevant information. I can give it the one file it should be using and then it goes off and greps parent directories for more context. It’s also incredibly slow at producing results because of this side work. In this article, it seems like they are catching up to what GitHub copilot users had already been doing since the cost restructuring in June.
I'm finding that unexpected cache rewrites cost me huge.
I have 1h cache TTL set, and do nothing to cause rewrite (response in time, no model/effort/tool changes).
At 400K tokens in, I'll write a message, and /usage shows only a small increase in cache write. On the next message, cache writes shows 800K, and by the end, I often hit 2M cache writes with no explanation.
This seems to happen when: using /btw, asking it to review code, other random times. Anyone know what's going on?
I mean, it feels hard not to laugh at this type of blog post. My cynical interpretation is that this is a type of passing the buck to engineers in enterprise settings ("Stop spending tokens. Did you read the value maximization blog post? It is your fault.")
Oh yes, Claude will do all sorts of different things -- it depends on how you use it! You should totally learn all of these little finicky things ... because now completing your tasks cost money. It's not "free" anymore haha like when you used your old text editor, what are you a grandpa?
Oh, and those things will definitely change, as we (the priests of Claude) are vibe coding the system you use to do your little "tasks" ... right, you can't see how it works ... the code is not available. It's all good, just trust us -- we're totally looking out for you.
I mean it is utterly ridiculous to talk around this model of development. There are so many walls between you and doing the thing you want to do.
Agents are great, but the notion of "best tricks" for how to best use an opaque costful tool which will, by all odds, be completely different in a few months time is quite funny.
You know what won't change? A fucking text editor. Or your pi config, or a local model you run and trust.
I'm trying to understand your point of view, but it kind of just sounds like you're against learning how to use tools efficiently?
I mean, agentic coding software is hardly the first tool to exist where learning some idiosyncrasies of how to use it well can result in more efficiency and cost savings.
It's very easy to understand:
- I'm happy to learn how to use tools efficiently
- I like to be able to inspect my tools
- I'm against tools changing underneath me
Are you against any of these points?
I think I'm happy about the first two. The third I suppose I care less about, just because I've kind of become used to it from decades working on the internet where many businesses/tools/apps are more like services and less like physical tools that never change.
> The third I suppose I care less about, just because I've kind of become used to it
I've never become used to it. My impression is that the constant churn has accelerated. Plausible drivers are (1) normalize novelty as desirable (like fast fashion), (2) product developer/designer incentive structures that reward revolutionary change over progressive refinement. The global switch to subscription models and continuous deployment didn't help.
> more like services and less like physical tools that never change.
I'm not sure that constant change is a characteristic feature of services, especially not professional services.
It used to be that you bought a piece of software and used that version until you decided it was worth upgrading, like a particular physical tool. The software still evolved, just like the design of physical tools can, in principle, evolve.
All that said, agentic AI tooling is evolving so rapidly I'm not sure an expectation of stability is realistic.
I agree that the third seems to be implied by industry, but I'd argue that it's not clear that it is necessary -- and it is subtle whether or not it is beneficial?
My contention is that we should be building towards less churn, not more. I'm aware that some churn is the cost of engaging in any sort of enterprise, but I'm deeply suspicious of an AI company inserting themselves between me, and the tasks I wish to do with my device -- with a completely opaque system that I can't really "learn".
dude, if you try to do harness development yourself you will realize that most things said in this blogpost is shared with any ${sufficiently_advanced_harness}. this is not really claude-specific, this is just how this class of tools, OSS or not, works
That's not my complaint. I know well the concerns of agent harnesses. My complaint is that this is a low-dimensional projection of a system which I have no insight into, and therefore, I cannot evaluate the tips myself against their source.
Am I to believe the creators, knowing full well that the source will, as Boris Cherny put it in a recent interview, be deleted and rewritten from scratch at the release of the next big model?
Further: I'm responding to content in the blog post itself:
> Until pretty recently, the tools you wrote code with were a flat fee (or free). Your editor cost the same whether you fixed one test or fifty that afternoon, so an individual task didn't really have a price of its own.
I find this type of prose ridiculous. It conveys "this is the way things are now, get used to it".
Does that make sense?
I do all that, but an 'AS-BUILT' full review of my project still eats 3x my 5 hour budget on max 100€. Meanwhile, my 20€ GPT never hit a limit. Different, but just saying.
Part of the cynic in me just wants to ask "why not make a better harness by default?" The other cynic in me knows I'm about to see a hundred post on 'HOW TO 10X CLAUDE" from the ai bros and I'm already tired.
I guess if I had to ask something (as someone who doesn't use CC as their daily driver), how much control do you have on subagents and roughly how do define or know when a session is getting too long? I know the answer is "when the model is getting worse" but worse is doing a lot of lifting in that sentence.
Yeah, I sort of feel like they could just do this in Claude Code for us in some way. I mean they already run some mini classifier on whether a given prompt is waiting on input, so they could do the same to detect some of these cases, and just handle it.
If you have to explain that someone is "holding it wrong"(1), that is product error, not a user error.
(1) https://www.wired.com/2010/06/iphone-4-holding-it-wrong/