491–500 of 620 posts
Earlier quoted context omitted.
Which plan are you on? I could see that happening with Pro (which I think defaults to Sonnet?), would be surprised with Max…
Pro. It even gave me $20 free credits, and exhausted free credits nearly instantly.
Earlier quoted context omitted.
> > Why are you handwaving things away though? I've got you on max effort. I even patched the system prompts to reduce this. In my experience, prompts like this one, which 1) ask for a reason behind an answer (when the model won't actually be able to provide one), 2) are somewhat standoff-ish, don't work well at all. You'll just have the model go the other way. What works much better is to tell the model to take a st…
> What works much better is to tell the model to take a step back and re-evaluate. I desperately hate that modern tooling relies on “did you perform the correct prayer to the Omnissiah” > to add some entropy to get it away from the local optimum Is that what it does? I don't think thats what it does, technically. I think thats just anthropomorphizing a system that behaves in a non deterministic way. A more menaingful…
"Why did you do that?" (Me, just wanting to understand)
"You're right I should have done the opposite" (starts implementing the opposite without seeking approval, etc.
But if you agree with it it won't do that, so it isn't simply a case of randomly rerunning prompts.
If anyone's had 4.7 update any documents so far - notice how concise it is at getting straight to the point. It rewrote some of my existing documentation (using Windsurf as the harness), not sure I liked the decrease in verbosity (removed columns and combined / compressed concepts) but it makes sense in respect to the model outputting less to save cost. To me this seems more that it's trained to be concise by default…
I'm a retired mathematician hoping to finish a second proof of a major theorem before I die. AI needs to understand my math and help me code. What I spend on AI isn't going to deplete my retirement savings. So far, Opus 4.7 seems a bit smarter than Opus 4.6 for my use case. That's my only concern. Is an $80 bottle of wine a better value than a $20 or $40 bottle of wine? Pretty much never. If there are those of us wil…
And what's missing in all these token count complaints is that 4.7 is actually cheaper overall anyways because it produces fewer output tokens.
Earlier quoted context omitted.
The actual rate isn’t relevant for the discussion
Well it might. If the actual rate is .9x then it matters a lot. Or even if it's like 1.1x, is the cost worth the return?
Meta pays $750k+ TC and makes far more profit/eng, do you think they care about $5k/eng/mo in inference? A 1.1x increase would be so significant that it would justify the cost easily, especially when you can just compress comps to make up for it
We dropped Claude. It's pretty clear this is a race to the bottom, and we don't want a hard dependency on another multi-billion dollar company just to write software We'll be keeping an eye on open models (of which we already make good use of). I think that's the way forward. Actually it would be great if everybody would put more focus on open models, perhaps we can come up with something like the "linux/postgres/git…
What open models are truly competing with both Claude Code and Opus 4.7 (xhigh) at this stage?
The open model mentality is also just so bizarre to me. You're going to use an inferior model to save, what, a couple hundred bucks a month? Is your time really worth that little?
No one working on a serious project at a serious company is downgrading their agent's intelligence for a marginal cost saving. Downgrading your model is like downgrading the toilet paper on your yacht.
Earlier quoted context omitted.
It’s a shitload of data, and it only works if all the tokens are 100% identical, i.e. all the attention values are exactly the same. Typically it’s cached for about 5 minutes, you can pay extra for longer caches.
If I have a conversation with claude then come back 30 minutes later to resume the conversation, the KV values for that prefill prefix are going to be exactly the same. That's the whole point of this caching in the first place. If you're willing to incur a latency penalty on a "cold resume" (which is fine for most use-cases), why couldn't they just move it to disk. The size of the KV cache should scale on the order o…
Correct, when you’re using the API you can choose between 60 minute or 5 minute cache writes for this reason, but I believe the subscription doesn’t offer this. 60 minute cache writes are about 25% more expensive than regular cache writes.
I don’t have insights into internals at Anthropic so I don’t know where the pain point is for increasing cache sizes.