Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude 4 System Card

simonwillison.net

Post reply on HN
Print view View original

31–40 of 264 posts

Re: Claude 4 System Card

#31

Post by antirez »

aabhay wrote: post #5

Given the cited stats here and elsewhere as well as in everyday experience, does anyone else feel that this model isn’t significantly different, at least to justify the full version increment? The one statistic mentioned in this overview where they observed a 67% drop seems like it could easily be reduced simply by editing 3.7’s system prompt. What are folks’ theories on the version increment? Is the architecture sig…

It works better when using tools, but the LLM itself it is not powerful from the POV of reasoning. Actually Sonnet 4 seems weaker than Sonnet 3.7 in many instances.

Re: Claude 4 System Card

#32

Post by benreesman »

aabhay wrote: post #5

Given the cited stats here and elsewhere as well as in everyday experience, does anyone else feel that this model isn’t significantly different, at least to justify the full version increment? The one statistic mentioned in this overview where they observed a 67% drop seems like it could easily be reduced simply by editing 3.7’s system prompt. What are folks’ theories on the version increment? Is the architecture sig…

The API version I'm getting for Opus 4 via gptel is aligned in a way that will win me back to Claude if its intentional and durable. There seems to be maybe some generalized capability lift but its hard to tell, these things are aligment constrained to a level below earlier frontier models and the dynamic cost control and what not is a liability for people who work to deadlines. Its net negative.

The 3.7 bait and switch was the last straw for me and closed frontier vendors or so I said, but I caught a candid, useful, Opus 4 today on a lark, and if its on purpose its like a leadership shakeup level change. More likely they just don't have the "fuck the user" tune yet because they've only run it for themsrlves.

I'm not going to make plans contingent on it continuing to work well just yet, but I'm going to give it another audition.

Re: Claude 4 System Card

#33

Post by FieryTransition »

colonCapitalDee wrote: post #27
aabhay wrote: post #5

Given the cited stats here and elsewhere as well as in everyday experience, does anyone else feel that this model isn’t significantly different, at least to justify the full version increment? The one statistic mentioned in this overview where they observed a 67% drop seems like it could easily be reduced simply by editing 3.7’s system prompt. What are folks’ theories on the version increment? Is the architecture sig…

I'm noticing much more flattery ("Wow! That's so smart!") and I don't like it

Turns out tuning LLMs on human preferences leads to sycophantic behavior, they even wrote about it themselves, guess they wanted to push the model out too fast.

Re: Claude 4 System Card

#34

Post by huksley »

> ...told something in the system prompt like “take initiative,” it will frequently take very bold action. This includes locking users out of systems that it has access to or bulk-emailing media and law-enforcement figures to surface evidence of wrongdoing.

So if you ask it to aid in wrongdoing, it might behave that way, but who guarantees it will not hallucinate and do the same when you ask for something innocuous?

Cursor IDE runs all the commands AI asks for with the same privilege as you have.

Re: Claude 4 System Card

#35

Post by rorytbyrne »

lsy wrote: post #24

It’s honestly a little discouraging to me that the state of “research” here is to make up sci fi scenarios, get shocked that, e.g., feeding emails into a language model results in the emails coming back out, and then write about it with such a seemingly calculated abuse of anthropomorphic language that it completely confuses the basic issues at stake with these models. I understand that the media laps this stuff up s…

When we use LLMs as agents, this errant behaviour matters - regardless of whether it comes from sci-fi “emergent sentience” or just autocomplete of the training data. It puts a soft constraint on how we can use agentic autocomplete.

Re: Claude 4 System Card

#36

Post by saaaaaam »

colonCapitalDee wrote: post #27
aabhay wrote: post #5

Given the cited stats here and elsewhere as well as in everyday experience, does anyone else feel that this model isn’t significantly different, at least to justify the full version increment? The one statistic mentioned in this overview where they observed a 67% drop seems like it could easily be reduced simply by editing 3.7’s system prompt. What are folks’ theories on the version increment? Is the architecture sig…

I'm noticing much more flattery ("Wow! That's so smart!") and I don't like it

Yup, I mentioned this in another thread. I quickly find it unbearable and makes me not trust Claude. Really damaging.

Re: Claude 4 System Card

#37

Post by albert_e »

karimf wrote: post #26
albert_e wrote: post #23

OT > data provided by data-labeling services and paid contractors someone in my circle was interested in finding out how people participate in these exercises and if there are any "service providers" that do the heavy lifting of recruiting and managing this workforce for the many AI/LLM labs globally or even regionally they are interested in remote work opportunities that could leverage their (post-graduate level) ed…

https://mercor.com/

Seems to be a perfect starting point-- passed on -- thanks!

Re: Claude 4 System Card

#38

Post by angusturner »

lsy wrote: post #24

It’s honestly a little discouraging to me that the state of “research” here is to make up sci fi scenarios, get shocked that, e.g., feeding emails into a language model results in the emails coming back out, and then write about it with such a seemingly calculated abuse of anthropomorphic language that it completely confuses the basic issues at stake with these models. I understand that the media laps this stuff up s…

Agree the media is having a field day with this and a lot of people will draw bad conclusions about it being sentient etc.

But I think the thing that needs to be communicated effectively is that these these “agentic” systems could cause serious havoc if people give them too much control.

If an LLM decides to blackmail an engineer in service of some goal or preference that has arisen from its training data or instructions, and actually has the ability to follow through (bc people are stupid enough to cede control to these systems), that’s really bad news.

Saying “it’s just doing autocomplete!” totally misses the point.

Re: Claude 4 System Card

#40

Post by mattkevan »

albert_e wrote: post #23

OT > data provided by data-labeling services and paid contractors someone in my circle was interested in finding out how people participate in these exercises and if there are any "service providers" that do the heavy lifting of recruiting and managing this workforce for the many AI/LLM labs globally or even regionally they are interested in remote work opportunities that could leverage their (post-graduate level) ed…

My Reddit feed is absolutely spammed with data annotation job ads, looking specifically for maths tutors and coders.

Does not feel like roles with long-term prospects.

Post reply on HN