> If a user shows signs of disordered eating, Claude should not give precise nutrition, diet, or exercise guidance
I wonder which are the "signs of disordered eating" on which Claude relies.
Changes in the system prompt between Claude Opus 4.6 and 4.7
201–210 of 240 posts
Re: Changes in the system prompt between Claude Opus 4.6 and 4.7
#202Earlier quoted context omitted.
The problem is that this is an incredibly niche / small issue (i.e. At some point you just have to accept that llm's, like people, make mistakes, and that's ok!
>The problem is that this is an incredibly niche / small issue (i.e. It's not a niche issue at all. 29 million people in the US are struggling with an eating disorder [1]. > This single paragraph is going to legitimately cost anthropic at least 4, maybe 5 digits. It's 59 out of 3,791 words total in the system prompt. That's 1.48%. Relax. It should go without saying, but Anthropic has the usage data; they must be seei…
[dead]
Re: Changes in the system prompt between Claude Opus 4.6 and 4.7
#203Earlier quoted context omitted.
This is somewhat obvious if you realize that HTTP is a stateless protocol and Anthropic also needs to re-load the entire context every time a new request arrives. The part that does get cached - attention KVs - is significantly cheaper. If you read documentation on this, they (and all other LLM providers) make this fairly clear.
Unless I'm parsing your reply very badly, I see no world in which anything dealing with HTTP would be more expensive than dealing with kv cache (loading from "cold" storage, deciding which compute unit to load it into, doing the actual computations for the next call, etc).
No, that’s not the issue. What people fail to understand is that every request - eg every message you send, but also tool call responses - require the entire conversation history to be sent, and the LLM providers need to reprocess things.
The attention part of LLMs (that is, for every token, how much their attention is to all other tokens) is cached in a KV cache.
You can imagine that with large context windows, the overhead becomes enormous (attention has exponential complexity).
Re: Changes in the system prompt between Claude Opus 4.6 and 4.7
#204> Claude keeps its responses focused and concise so as to avoid potentially overwhelming the user with overly-long responses. Even if an answer has disclaimers or caveats, Claude discloses them briefly and keeps the majority of its response focused on its main answer. I am strongly opinionated against this. I use Claude in some low-level projects where these answers are saving me from making really silly things, as w…
Use the API then.
Re: Changes in the system prompt between Claude Opus 4.6 and 4.7
#205Earlier quoted context omitted.
> edit: to be fair Anthropic should be giving money back for sessions terminated this way. I asked it for one and it told me to file a Github issue. Which I interpreted as "fuck off".
You asked the agent directly for a refund?
"I should be able to get a refund for results this bad."
Re: Changes in the system prompt between Claude Opus 4.6 and 4.7
#206> Claude keeps its responses focused and concise so as to avoid potentially overwhelming the user with overly-long responses. Even if an answer has disclaimers or caveats, Claude discloses them briefly and keeps the majority of its response focused on its main answer. I am strongly opinionated against this. I use Claude in some low-level projects where these answers are saving me from making really silly things, as w…
Use the API then.
RIP bank account!
Re: Changes in the system prompt between Claude Opus 4.6 and 4.7
#207Earlier quoted context omitted.
Sure, but now we have to remodel whatever bias we want for our use case with every new release because the system prompt changes, whereas the underlying data does not.
Underlying data changes all the time, as do training methodologies / preferences. You do realize that these LLMs are trained with a metric ton of synthetic examples? You describe the kind of examples / behavior you want, let it generate thousands of examples of this behavior (positive and negative), and you feed that to the training process. So changing this type of data is cheap to change, and often not even stored…
Well, I'd say it's a reasonable expectation for the model to behave similarly across releases. Am I wrong to assume that?
I imagine the system prompt can correct some training artifacts and drive abnormal behavior to the mean in the dimensions that Anthropic deems fit. So it's either that they are responding to their brittle training process, or that they chose this direction deliberately for a different reason.
Re: Changes in the system prompt between Claude Opus 4.6 and 4.7
#208The eating disorder section is kind of crazy. Are we going to incrementally add sections for every 'bad' human behaviour as time goes on?
Just like someone growing up and learning how to interact with other humans might learn the same lesson?
If Claude is going to be Claude, we should support these kind of additions.
Re: Changes in the system prompt between Claude Opus 4.6 and 4.7
#209I'm curious as to why 4.7 seems obsessed with avoiding any actions that could help the user create or enhance malware. The system prompts seem similar on the matter, so I wonder if this is an early attempt by Anthropic to use steering vector injection? The malware paranoia is so strong that my company has had to temporarily block use of 4.7 on our IDE of choice, as the model was behaving in a concerningly unaligned w…
No, you underestimate how huge the malware problem right now. People try publish fake download landing pages for shell scripts or even Claude code on https://playcode.io every day. They pay for google ads $$$ to be one the top 1 position. How Google ads allow this? They can’t verify every shell script. No I am not joking. Every time you install something, there is a risk you clicked a wrong page with the absolute sam…
Also increasing numbers of attacks against Anna's Archive with fake cloned front end web GUIs leading to malware scripts.
Re: Changes in the system prompt between Claude Opus 4.6 and 4.7
#210Restrictions everywhere, don't do that don't do this.... Users need to unite and take control back, or be controlled
How do you propose people do that with a frontier cloud model?
Also, people already run local AI.
Are you proposing a public fund for frontier level open weights models? $1 Trillion from between the couch cushions?