Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Sonnet 4.5

anthropic.com

Post reply on HN
Print view View original

341–350 of 819 posts

Re: Claude Sonnet 4.5

#341

Post by nchmy »

asadotzler wrote: post #315
Redster wrote: post #268

Earlier quoted context omitted.

Simonw is a cheerful and straightforward AI journalist who likes to show and not just tell. He has done a good job aggregating and documenting the progress of LLM tools and models. As I understand it, OpenAI and Anthropic have both wisely decided to make sure he has up to date info because they know he'll write about it. Thanks for all your work, Simon! You're my favorite journalist in this space and I really appreci…

AI blogger seems more appropriate than journalist.

are you aware of any "ai journalists"? Because simonw does great work, so perhaps blogger is what people should aspire towards?

Re: Claude Sonnet 4.5

#342

Post by usaar333 »

falcor84 wrote: post #174
wohoef wrote: post #96

Earlier quoted context omitted.

Just a few months ago people were still talking about exponential progress. The fact that we’re already going for just linear progress is not a good sign

Linear growth on a 0-100 benchmark is quite likely an exponential increase in capability.

Except it is sublinear. Sonnet 4 was 10.2% above sonnet 3.7 after 3 months.

Re: Claude Sonnet 4.5

#343

Post by motoxpro »

Implicated wrote: post #274
iagooar wrote: post #258

Anecdotal evidence. I have a fairly large web application with ~200k LoC. Gave the same prompt to Sonnet 4.5 (Claude Code) and GPT-5-Codex (Codex CLI). "implement a fuzzy search for conversations and reports either when selecting "Go to Conversation" or "Go to Report" and typing the title or when the user types in the title in the main input field, and none of the standard elements match, a search starts with a 2s de…

I'm not trying to be offensive here, feel the need to indicate that. But that prompt leads me to believe that you're going to get rather 'random' results due to leaving SO much room for interpretation. Also, in my experience, punctuation is important - particularly for pacing and grouping of logical 'parts' of a task and your prompt reads like a run on sentence. Making a lot of assumptions here - but I bet if I were…

I agree. The sibling responses also give some insight into why a lot of people don't get a lot of value out of coding agents.

Re: Claude Sonnet 4.5

#344

Post by mmgeorgi »

robotswantdata wrote: post #283
iagooar wrote: post #258

Anecdotal evidence. I have a fairly large web application with ~200k LoC. Gave the same prompt to Sonnet 4.5 (Claude Code) and GPT-5-Codex (Codex CLI). "implement a fuzzy search for conversations and reports either when selecting "Go to Conversation" or "Go to Report" and typing the title or when the user types in the title in the main input field, and none of the standard elements match, a search starts with a 2s de…

I had similar experience, not good enough yet to come back for the Claude max plan. Sticking with ChatGPT pro sub and gpt5 codex on high.

do you ever hit your pro quota?

Re: Claude Sonnet 4.5

#345

Post by levocardia »

zurfer wrote: post #57
zurfer wrote: post #31

Same price and a 4.5 bp jump from 72.7 to 77.2 SWEBench Pretty solid progress for roughly 4 months.

Also getting a perfect score on AIME (math) is pretty cool. Tongue in cheek: if we progress linearly from here software engineering as defined by SWE bench is solved in 23 months.

Pretty sure there is a subset of SWE bench problems that are either ill-posed or not possible with the intended setup; I think I remember seeing another company excluding a fraction of them for that reason. So maxing out SWEBench might only be ~95%.

I'm most interested to see the METR time horizon results - that is the real test for whether we are "on-trend"

Re: Claude Sonnet 4.5

#346

Post by GoatInGrey »

Implicated wrote: post #274
iagooar wrote: post #258

Anecdotal evidence. I have a fairly large web application with ~200k LoC. Gave the same prompt to Sonnet 4.5 (Claude Code) and GPT-5-Codex (Codex CLI). "implement a fuzzy search for conversations and reports either when selecting "Go to Conversation" or "Go to Report" and typing the title or when the user types in the title in the main input field, and none of the standard elements match, a search starts with a 2s de…

I'm not trying to be offensive here, feel the need to indicate that. But that prompt leads me to believe that you're going to get rather 'random' results due to leaving SO much room for interpretation. Also, in my experience, punctuation is important - particularly for pacing and grouping of logical 'parts' of a task and your prompt reads like a run on sentence. Making a lot of assumptions here - but I bet if I were…

This would explain the LLM implementing the feature in a way you didn't prefer. But this does not explain why Sonnet would deliver a broken implementation that does not work in even the most basic sense.

Also, there is a threshold in which the time it takes to develop a prompt, allow the agent to run, review its output, and go through iterative loops to correct errors or implementation problems, can exceed the time it takes me (a lazy human) to achieve the same end result.

Pair this with the bypassing of the generation effect, reduced prefrontal dopamine, and increased working memory load (in part due to minimal motor-cognitive integration), and AI-generated code in contexts with legal and financial repercussions can be a much worse deal than using your own fingers.

Re: Claude Sonnet 4.5

#347

Post by DrewADesign »

Implicated wrote: post #329
SirMaster wrote: post #316

Earlier quoted context omitted.

But isn't the end goal to be able to get useful results without so much prompting? I mean in the movies for example, advanced AI assistants do amazing things with very little prompting. Seems like that's what people want. To me, the fact that so many people basically say "you are prompting it wrong" is knock against the tech and the model. If people want to say that these systems are so smart at what they can do, the…

> But isn't the end goal to be able to get useful results without so much prompting? See below about context. > I mean in the movies for example, advanced AI assistants do amazing things with very little prompting. Seems like that's what people want. Movies != real life > To me, the fact that so many people basically say "you are prompting it wrong" is knock against the tech and the model. If people want to say that…

> But you're comparing the LLMs to humans

These things are being marketed as super intelligent magic answer machines. Judging them using the criteria the marketing teams have provided is completely reasonable.

> Movies != real life

Nobody claimed it was. This is about desires and expectations. The people charging money for these services and taking stacks of cash that would’ve otherwise been in in dev’s paychecks while doing so haven’t even tried to temper those expectations. They made their beds…

Re: Claude Sonnet 4.5

#348

Post by simianwords »

sipjca wrote: post #162
alach11 wrote: post #17

I'm really interested in the progress on computer use. These are the benchmarks to watch if you want to forecast economic disruption, IMO. Mastery of computer use takes us out of the paradigm of task-specific integrations with AI to a more generic interface that's way more scalable.

Maybe this is true? But it's not clear to me this methodology will ever be quite as good as native tool calling. Or maybe I don't know the benchmark well enough, I just assume it's vision based Perhaps Tesla FSD is a similar example where in practice self driving with vision should be possible (humans), but is fundamentally harder and more error prone than having better data. It seems to me very error prone and expen…

Looks like RPA vs API debate all over again

Re: Claude Sonnet 4.5

#349

Post by GoatInGrey »

stavros wrote: post #326
AppleBananaPie wrote: post #323

Earlier quoted context omitted.

To me it seems like an arbitrary number that I'm not even sure how someone could accurately measure it but maybe I've missed something :)

I took it as a ballpark of how the GP felt, rather than a rigorous measurement.

Which would be ironic as LLM usage has been observed to increase the sensation of productivity even when productivity is measurably reduced. Not to mention the "vibe" component of vibe coding

Re: Claude Sonnet 4.5

#350

Post by Forgeties79 »

mh- wrote: post #304
Forgeties79 wrote: post #221

Earlier quoted context omitted.

> I worry everyone is chasing benchmarks to the detriment of general performance. I’m not sure this is entirely what you’re driving at, but the example I always think of in my head is “I want an AI agent that will scan through my 20 to 30,000 photos, remove all the duplicates, then organize them all in some coherent fashion.” that’s the kind of service I need right now, and it feels like something AI should be able t…

The reality of that specific ask is it would not be difficult to build, but I believe it would be extremely difficult to build and offer at a price that users would pay for. So you're unlikely to find a commercial offering that does that using a (V)LM.

Yeah I imagine so. Hell I would pay like $100 for them to just do it once. If they really could do it with like 99% accuracy I would pay upwards of $300 tbh. Still, that’s probably not good enough lol
Post reply on HN