Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Sonnet 4.6

anthropic.com

Post reply on HN
Print view View original

431–440 of 1001 posts

Re: Claude Sonnet 4.6

#431

Post by cma »

brador wrote: post #423
WarmWash wrote: post #266

Earlier quoted context omitted.

I was just thinking earlier today how in an alternate universe, probably not too far removed from our own, Google has a monopoly on transformers and we are all stuck with a single GPT-3.5 level model, and Google has a GPT-4o model behind the scenes that it is terrified to release (but using heavily internally).

Now think about how often the patent system has stifled and stalled and delayed advancement for decades per innovation at a time. Where would we be if patents never existed?

To be fair, Google has a patent on the transformer architecture. Their page rank patent monopoly probably helped fund the R&D.

Re: Claude Sonnet 4.6

#432

Post by raddan »

MattGaiser wrote: post #409
zmmmmm wrote: post #400

I see a big focus on computer use - you can tell they think there is a lot of value there and in truth it may be as big as coding if they convincingly pull it off. However I am still mystified by the safety aspect. They say the model has greatly improved resistance. But their own safety evaluation says 8% of the time their automated adversarial system was able to one-shot a successful injection takeover even with saf…

Does it matter? "Security" and "performance" have been regular HN buzzwords for why some practice is a problem and the market has consistently shown that it doesn't value those that much.

Thank god most of the developers of security sensitive applications do not give a shit about what the market says.

Re: Claude Sonnet 4.6

#433

Post by ModernMech »

gordonhart wrote: post #90
gallerdude wrote: post #55

I always grew up hearing “competition is good for the consumer.” But I never really internalized how good fierce battles for market share are. The amount of competition in a space is directly proportional to how good the results are for consumers.

Remember when GPT-2 was “too dangerous to release” in 2019? That could have still been the state in 2026 if they didn’t YOLO it and ship ChatGPT to kick off this whole race.

Yeah, and Jurassic Park wouldn't have been a movie if they decided against breeding the dinosaurs.

Re: Claude Sonnet 4.6

#434

Post by vineyardmike »

WarmWash wrote: post #266
gordonhart wrote: post #90

Earlier quoted context omitted.

Remember when GPT-2 was “too dangerous to release” in 2019? That could have still been the state in 2026 if they didn’t YOLO it and ship ChatGPT to kick off this whole race.

I was just thinking earlier today how in an alternate universe, probably not too far removed from our own, Google has a monopoly on transformers and we are all stuck with a single GPT-3.5 level model, and Google has a GPT-4o model behind the scenes that it is terrified to release (but using heavily internally).

This was basically almost real.

Before ChatGPT was even released, Google had an internal-only chat tuned LLM. It went "viral" because some of the testers thought it was sentient and it caused a whole media circus. This is partially why Google was so ill equipped to even start competing - they had fresh wounds of a crazy media circus.

My pet theory though is that this news is what inspired OpenAI to chat-tune GPT-3, which was a pretty cool text generator model, but not a chat model. So it may have been a necessary step to get chat-llms out of Mountain View and into the real world.

https://www.scientificamerican.com/article/google-engineer-c...

https://www.theguardian.com/technology/2022/jul/23/google-fi...

Re: Claude Sonnet 4.6

#435

Post by zone411 »

They're improved compared to 4.5 on my Extended NYT Connections benchmark (https://github.com/lechmazur/nyt-connections/).

Sonnet 4.6 Thinking 16K scores 57.6 on the Extended NYT Connections Benchmark. Sonnet 4.5 Thinking 16K scored 49.3.

Sonnet 4.6 No Reasoning scores 55.2. Sonnet 4.5 No Reasoning scored 47.4.

Re: Claude Sonnet 4.6

#436

Post by jpalepu »

zozbot234 wrote: post #403
zmmmmm wrote: post #400

I see a big focus on computer use - you can tell they think there is a lot of value there and in truth it may be as big as coding if they convincingly pull it off. However I am still mystified by the safety aspect. They say the model has greatly improved resistance. But their own safety evaluation says 8% of the time their automated adversarial system was able to one-shot a successful injection takeover even with saf…

Isn't "computer use" just interaction with a shell-like environment, which is routine for current agents?

Interesting question! In this context, "computer use" means the model is manipulating a full graphical interface, using a virtual mouse and keyboard to interact with applications (like Chrome or LibreOffice), rather than simply operating in a shell environment.

Re: Claude Sonnet 4.6

#437

Post by Jach »

MrCheeze wrote: post #321
andrewchilds wrote: post #244

Many people have reported Opus 4.6 is a step back from Opus 4.5 - that 4.6 is consuming 5-10x as many tokens as 4.5 to accomplish the same task: https://github.com/anthropics/claude-code/issues/23706 I haven't seen a response from the Anthropic team about it. I can't help but look at Sonnet 4.6 in the same light, and want to stick with 4.5 across the board until this issue is acknowledged and resolved.

In my experience with the models (watching Claude play Pokemon), the models are similar in intelligence, but are very different in how they approach problems: Opus 4.5 hyperfocuses on completing its original plan, far more than any older or newer version of Claude. Opus 4.6 gets bored quickly and is constantly changing its approach if it doesn't get results fast. This makes it waste more time on"easy" tasks where the…

I haven't kept up with the Claude plays stuff, did it ever actually beat the game? I was under the impression that the harness was artificially hampering it considering how comparatively more easily various versions of ChatGPT and Gemini had beat the game and even moved on to beating Pokemon Crystal.

Re: Claude Sonnet 4.6

#438

Post by tempestn »

stavros wrote: post #88
tempestn wrote: post #50

Earlier quoted context omitted.

Because Opus 4.5 was released like a month ago and state of the art, and now the significantly faster and cheaper version is already comparable.

Opus 4.5 was November, but your point stands.

Fair. Feels like a month!

Re: Claude Sonnet 4.6

#439

Post by hibikir »

gallerdude wrote: post #55

I always grew up hearing “competition is good for the consumer.” But I never really internalized how good fierce battles for market share are. The amount of competition in a space is directly proportional to how good the results are for consumers.

Competition is great, but it's so much better when it is all about shaving costs. I am afraid that what we are seeing here is an arms race with no moat: Something that will behave a lot like a Vickrey auction. The competitors all lose money in the investment, and since a winner takes all, and it never makes sense to stop the marginal investment when you think you have a chance to win, ultimately more resources are spent than the value ever created.

This might not be what we are facing here, but seeing how little moat anyone on AI has, I just can't discount the risk. And then instead of the consumers of today getting a great deal, we zoom out and see that 5x was spent developing the tech than it needed to, and that's not all that great economically as a whole. It's not as if, say, the weights from a 3 year old model are just useful capital to be reused later, like, say, when in the dot com boom we ended up with way too much fiber that was needed, but that could be bought and turned on profitably later.

Re: Claude Sonnet 4.6

#440

Post by general_reveal »

zmmmmm wrote: post #400

I see a big focus on computer use - you can tell they think there is a lot of value there and in truth it may be as big as coding if they convincingly pull it off. However I am still mystified by the safety aspect. They say the model has greatly improved resistance. But their own safety evaluation says 8% of the time their automated adversarial system was able to one-shot a successful injection takeover even with saf…

If the world becomes dependent on computer-use than the AI buildout will be more than validated. That will require all that compute.
Post reply on HN