Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Sonnet 5

anthropic.com

Post reply on HN
Print view View original

751–760 of 822 posts

Re: Claude Sonnet 5

#751

Post by ACCount37 »

lawrjone wrote: post #739
sixtyj wrote: post #132

Earlier quoted context omitted.

I have tried to rewrite an article with GLM-5.2 and with Sonnet 4.6. Completely different results as LLM is non-deterministic. But GLM-5.2 made a lot of subtle mistakes that needed to be corrected by hand. On the opposite, Sonnet found and corrected all mistakes in the second round. Similar situation was with planning and coding. GLM-5.2 seems to be good “on paper” but the real usage results was different. And I am n…

We have found similar when plugging GLM 5.2 into actual benchmarks in our product. The open-source models are really dialled into the public benchmarks, until you try them in context you won't have a solid idea of how they perform (Sonnet is a higher quality model than 5.2, both in prose, reasoning, and alignment).

Most benchmarks notoriously undercount multi-turn instruction following too. And it's where open source models (and Gemini) lose big time.

Re: Claude Sonnet 5

#752

Post by drob518 »

tskj wrote: post #686
drob518 wrote: post #493

Earlier quoted context omitted.

Inference is the phase where they make money. But the question is whether they can be profitable overall as training continues to balloon.

I think the case for this is pretty strong actually. Last year my company was maybe willing to pay $100 a month to Anthropic (per developer). Today we're all on the $300 plan without any hesitation. If Fable ever becomes available as the default model, I imagine my company would be willing to pay in the $500-$1000 range per month per developer.

Okay, but that still has a limit, right? Do training costs have a limit? Everyone is in the frothy stage of this technology wave and they continue to buy more, but training the next model requires exponential increases in model sizes to get the same sorts of model performance increases, which suggests exponential cost increases, too (even ignoring temporary cost factors such as RAM price increases). You say your company will double or triple what they are paying today; how far are they willing to go? At some point they are going to have to cut developers to fund it (e.g., cut half the developers and give the survivors each an AI assistant with $180k in token budget, captured from the salary savings), but that also presupposes the productivity gains are there to support it.

Re: Claude Sonnet 5

#753

Post by [deleted] »

sisiissjz77 wrote: post #671
dimitrios1 wrote: post #566

Earlier quoted context omitted.

I am another skilled senior, have been coding since I was 7, although you have a few more years of experience on me, and am commenting here just for the goldilocks moment, as I have read and reflected on both of your comments and find my reality is somewhere in the middle. On personal projects, where I am in charge of all the hats (product development, UI, UX, backend, security, server admin, etc) -- absolutely crazy…

It’s simultaneously simple and deceptively difficult to coax a growing system into staying sane. It reminds me of forcing a fractal into growth yet somehow letting it remain similar at all scales, manually. Scaling in this sense is not operational (“servers”), but conceptual (“features”). I don’t want to be a downer but I find many devs are not great at this. Very clever folks, but they tend to not see these issues c…

[deleted]

Re: Claude Sonnet 5

#754

Post by somenameforme »

ngruhn wrote: post #692
itopaloglu83 wrote: post #245

Earlier quoted context omitted.

More and more I find myself trying to stop Opus from doing something stupid, and at every turn I need to tell it to stop overcomplicating things. I think the models are being optimized for wealth extraction from users and companies, instead of solving problems. I don't know why Opus would try to create an entire library when I told it specifically to do something simple that would take 2-3 lines of Python.

> I think the models are being optimized for wealth extraction from users and companies, instead of solving problems. I don't think so. Expect that in a market with high vendor lock-in but that's not the case here. The market is extremely competitive and switching cost are near zero. Anthropic can't afford to pull shit like this and sacrifice quality.

You'd also expect that in a market where the players think they're in a bubble.

Re: Claude Sonnet 5

#755

Post by LUmBULtERA »

fluidcruft wrote: post #711
LUmBULtERA wrote: post #703

Earlier quoted context omitted.

Did Anthropic have Opus 4.8 and Sonnet 5 switched in the Agentic Search chart at first?

No, and the original had everything more expensive. There's a comparison here: https://www.reddit.com/r/ClaudeAI/comments/1ukgqwr/looks_lik... The explanation Anthropic gave for the update doesn't address how the x-axis needed to range up to $50 previously and only $10 now. In any case the pass rates are also lower. Probably the difference between whatever it is people notice when they say models become "nerfed".

Huh, super interesting. Thanks!

Re: Claude Sonnet 5

#756

Post by pheewma »

hdjrudni wrote: post #698
mosbyllc wrote: post #693

Claude is a great model for me, but unfortunately, its quota is often insufficient. It seems that many people are now considering Codex as an alternative. If the quota is sufficient, I believe many people will continue to use the Claude Code model.

Codex is not better anymore. It appears they nerfed their quota a few weeks ago. I never used to hit my 5 hr limit, now I always do. Sometimes in like 2 prompts.

Codex was running a 2x usage promotion from around the time when Claude introduced rate limiting during peak hours, until May 31st. The various relevant subreddits were (more) insufferable: just 1000 posts per day to the tune of "Just switched the codex! So much more usage!" only to have that tone flip immediately after the promo ran out.

Re: Claude Sonnet 5

#757

Post by overgard »

everforward wrote: post #424
overgard wrote: post #193

Earlier quoted context omitted.

That's a really good point. I think if there wasn't the insane amount of money involved and these were treated as tools instead, they would probably be MORE productive. I think a person working hand in hand with an AI instead of delegating is the sweet spot of making things fast while also not losing understanding or control of the system. You are absolutely right that these companies can't justify their valuations i…

> I'm thinking the future of this tech will likely be better tooling with better IDE integrations rather than "Claude plz make me a SaaS kthx" I think this sort of thinking is a trap, because it presumes that all software has the same constraints. There's a spectrum of requirements between "chuck this over the wall at Claude, it only has to work once" and "this is a literal rocket ship, formally verify the whole thin…

Yeah I mean, there are definitely little scripts and stuff in my codebase that I'm happy to have an AI spit out and never look at because it's not important; but those are nice wins around the margins, not a justification of the "EVERYTHING HAS CHANGED!" insanity we've been seeing these past couple of years.

The thing is though, for that purpose, AI is just an incremental improvement. I mean, in the 80s and 90s people were quickly building crappy-but-useful things with basic or visual basic or spreadsheets and access databases, in the 2000s people built awful but functional websites in PHP, etc. I think AI is an improvement on those "scratch my itch quickly" things, but it's not a replacement for good practices.

Re: Claude Sonnet 5

#758

Post by tskj »

rimliu wrote: post #746
tskj wrote: post #684

Earlier quoted context omitted.

Well yeah obviously they have to stop reinvesting more than they make at some point to become profitable. To be clear, I think what Dario was saying was that if you consider each model training + deployment as a company , meaning all expenses and taxes, it was still profitable. Whether he's lying is another question, but seems unlikely.

Wow, you actually think any of those AI companies are profitable? Would you be interested in some bridges?

No, but if you'd actually read what I wrote, you'd see that what I said (that Dario said) was: if you consider each model as its own company, they've all been individually profitable. However, they keep re-investing that (and then some) into even bigger, more expensive models each time, causing the company to look unprofitable.

That being said, Anthropic did report being profitable this review quarter (Q2), so it's not as unreasonable as you claim.

Also Google is a pretty major AI company, and they're _insanely_ profitable.

Re: Claude Sonnet 5

#759

Post by Terretta »

ngruhn wrote: post #692
itopaloglu83 wrote: post #245

Earlier quoted context omitted.

More and more I find myself trying to stop Opus from doing something stupid, and at every turn I need to tell it to stop overcomplicating things. I think the models are being optimized for wealth extraction from users and companies, instead of solving problems. I don't know why Opus would try to create an entire library when I told it specifically to do something simple that would take 2-3 lines of Python.

> I think the models are being optimized for wealth extraction from users and companies, instead of solving problems. I don't think so. Expect that in a market with high vendor lock-in but that's not the case here. The market is extremely competitive and switching cost are near zero. Anthropic can't afford to pull shit like this and sacrifice quality.

Anthropic can't afford to pull shit like this and sacrifice quality.

And yet, the Java language exists.

For market share, ANTHROP\C needs to optimize for the vast mediocrity that are mid-bell-curve users and enterprises.

This adaptation tends to come with significant drag for the right ends of the bell curve firms or teams.

Unless ANTHROP\C have a separate objective function by cohort and ensure that doesn't regress, improving results for the emerging middle will nerf tools from point of view of those with high in-domain expertise.

Re: Claude Sonnet 5

#760

Post by adam_arthur »

grim_io wrote: post #669
adam_arthur wrote: post #319

Earlier quoted context omitted.

I've found disabling reasoning entirely but adding a "reason" to the JSON response from the LLM to work significantly faster and consume many fewer tokens for narrowly scoped prompts. At least for Claude family models. e.g. { "reason": " ", "selection": " " } I'm sure native reasoning produces more accurate results, but for my use case the quality was about the same, and the model would reason for thousands of tokens…

What you are doing, is producing an unnecessary summary of the result, not reasoning that models do to come up with the result. I don't get what value you get out of this.

This is the same concept as Chain of Thought.

Just that when "native" thinking is used it's hidden from the end result via special tags. If you force a model to reason about a result before producing the result, you get more accurate results.

Because "reason" comes before the selection, it has to think through why it is producing the result beforehand (e.g. produce a block of text that makes the correct answer statistically more likely to be sampled from the distribution. Giving it a property name does influence the direction of the thinking, but it's the same concept. You can call the property "yourThinking" too.

It sounds like you may be thinking too highly/mysteriously about how LLMs work.

At the end of the day they stream completely unstructured text outputs and all behaviors on top of that are just parsing XML-like tags to do tool calls, hide thinking etc (which they were trained to produce in certain circumstances).

There is no special "thinking process" it is a stream of text in the response that is simply wrapped in tags (or similar)

Post reply on HN