Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude 4 System Card

simonwillison.net

Post reply on HN
Print view View original

61–70 of 264 posts

Re: Claude 4 System Card

#61

Post by sensanaty »

colonCapitalDee wrote: post #27
aabhay wrote: post #5

Given the cited stats here and elsewhere as well as in everyday experience, does anyone else feel that this model isn’t significantly different, at least to justify the full version increment? The one statistic mentioned in this overview where they observed a 67% drop seems like it could easily be reduced simply by editing 3.7’s system prompt. What are folks’ theories on the version increment? Is the architecture sig…

I'm noticing much more flattery ("Wow! That's so smart!") and I don't like it

The default "voice" (for lack of a better word) compared to 3.7 is infuriating. It reads like the biggest ass licker on the planet, and it also does crap like the below

> So, `implements` actually provides compile-time safety

What writing style even is this? Like it's trying to explain something to a 10 year old.

I suspect that the flattery is there because people react well to it and it keeps them more engaged. Plus, if it tells you your idea for a dog shit flavoured ice cream stall is the most genius idea on earth, people will use it more and send more messages back and forth.

Re: Claude 4 System Card

#62

Post by sensanaty »

aabhay wrote: post #5

Given the cited stats here and elsewhere as well as in everyday experience, does anyone else feel that this model isn’t significantly different, at least to justify the full version increment? The one statistic mentioned in this overview where they observed a 67% drop seems like it could easily be reduced simply by editing 3.7’s system prompt. What are folks’ theories on the version increment? Is the architecture sig…

I feel that 3.7 is still the best. With 4, it keeps writing hundreds upon hundreds of lines, it'll invoke search for everything, it starts refactoring random lines unrelated to my question, it'll often rewrite entire portions of its own output for no reason. I think they took the "We need to shit out code" thing the AIs are good at and cranked it to 11 for whatever reason, where 3.7 had a nice balance (although it still writes WAY too many comments that are utterly useless)

Re: Claude 4 System Card

#63

Post by torginus »

sensanaty wrote: post #61
colonCapitalDee wrote: post #27

Earlier quoted context omitted.

I'm noticing much more flattery ("Wow! That's so smart!") and I don't like it

The default "voice" (for lack of a better word) compared to 3.7 is infuriating. It reads like the biggest ass licker on the planet, and it also does crap like the below > So, `implements` actually provides compile-time safety What writing style even is this? Like it's trying to explain something to a 10 year old. I suspect that the flattery is there because people react well to it and it keeps them more engaged. Plus…

Man I miss Claude 2. It talked like a competent, but incredibly lazy person who didn't care for formality and wanted to get the interaction over with in the shortest possible time.

Re: Claude 4 System Card

#65

Post by someothherguyy »

> Please implement for me. Please write a high quality, general purpose solution. If the task is unreasonable or infeasible, or if any of the tests are incorrect, please tell me. Do not hard code any test cases. Please tell me if the problem is unreasonable instead of hard coding test cases!

I have pretty good success with just telling agents "don't cheat"

Re: Claude 4 System Card

#66

Post by B1FF_PSUVM »

wgx wrote: post #42

Interesting! >Claude shows a striking “spiritual bliss” attractor state in self-interactions. When conversing with other Claude instances in both open-ended and structured environments, Claude gravitated to profuse gratitude and increasingly abstract and joyous spiritual or meditative expressions.

I think it was Larry Niven, quite a few decades ago, that had SF stories where AIs were only good for a few months before becoming suicidal...

Re: Claude 4 System Card

#67

Post by markovs_gun »

mike_hearn wrote: post #49
FieryTransition wrote: post #33

Earlier quoted context omitted.

Turns out tuning LLMs on human preferences leads to sycophantic behavior, they even wrote about it themselves, guess they wanted to push the model out too fast.

I think it was OpenAI that wrote about that. Most of us here on HN don't like this behaviour, but it's clear that the average user does. If you look at how differently people use AI that's not a surprise. There's a lot of using it as a life coach out there, or people who just want validation regardless of the scenario.

This is a problem with these being marketed products. Being popular isn't the same as being good, and being consumer products means they're getting optimized for what will make them popular instead of what will make them good.

Re: Claude 4 System Card

#68

Post by sensanaty »

lsy wrote: post #24

It’s honestly a little discouraging to me that the state of “research” here is to make up sci fi scenarios, get shocked that, e.g., feeding emails into a language model results in the emails coming back out, and then write about it with such a seemingly calculated abuse of anthropomorphic language that it completely confuses the basic issues at stake with these models. I understand that the media laps this stuff up s…

It's a massive hype bubble unrivaled in scale by anything that has ever come before it, so all the AI providers have huge vested interests in making it seem like these systems are "sentient". All of the marketing is riddled with anthropomorphization (is that a word?). "It's like a Junior!", "It's like your secretary!", "But humans also do X!" etc.

The other day on the Claude 4 announcement post [1], people were talking about Claude "threatening people" that wanted to shut it down or whatever. It's absolute lunacy, OpenAI did the same with GPT 2, and now the Claude team is doing the exact same idiotic marketing stunts and people are still somehow falling for it.

[1] https://news.ycombinator.com/item?id=44065616

Re: Claude 4 System Card

#69

Post by markovs_gun »

torginus wrote: post #63
sensanaty wrote: post #61

Earlier quoted context omitted.

The default "voice" (for lack of a better word) compared to 3.7 is infuriating. It reads like the biggest ass licker on the planet, and it also does crap like the below > So, `implements` actually provides compile-time safety What writing style even is this? Like it's trying to explain something to a 10 year old. I suspect that the flattery is there because people react well to it and it keeps them more engaged. Plus…

Man I miss Claude 2. It talked like a competent, but incredibly lazy person who didn't care for formality and wanted to get the interaction over with in the shortest possible time.

That's exactly what I want from an LLM. But then again I want a tool and not a robot prostitute

Re: Claude 4 System Card

#70

Post by ekidd »

tankenmate wrote: post #56
mike_hearn wrote: post #49

Earlier quoted context omitted.

I think it was OpenAI that wrote about that. Most of us here on HN don't like this behaviour, but it's clear that the average user does. If you look at how differently people use AI that's not a surprise. There's a lot of using it as a life coach out there, or people who just want validation regardless of the scenario.

> or people who just want validation regardless of the scenario. This really worries me as there are many people (even more prevalent in younger generations if some papers turn out to be valid) that lack resilience and critical self evaluation who may develop narcissistic tendencies with increased use or reinforcement from AIs. Just the health care costs involved when reality kicks in for these people, let alone othe…

> who may develop narcissistic tendencies with increased use or reinforcement from AIs.

It's clear to me that (1) a lot of billionaires believe amazingly stupid things, and (2) a big part of this is that they surround themselves with a bubble of sycophants. Apparently having people tell you 24/7 how amazing and special you are sometimes leads to delusional behavior.

But now regular people can get the same uncritical, fawning affirmations from an LLM. And it's clearly already messing some people up.

I expect there to be huge commercial pressure to suck up to users and tell them they're brilliant. And I expect the long-term results will be as bad as the way social media optimizes for filter bubbles and rage bait.

Post reply on HN