Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Sonnet 5

anthropic.com

Post reply on HN
Print view View original

681–690 of 822 posts

Re: Claude Sonnet 5

#681

Post by DanielHB »

illiac786 wrote: post #624
jimbo808 wrote: post #610

Earlier quoted context omitted.

I think I might have written a comment similar to yours maybe 6 months or a year ago. I'm not quite sure to respond to these sorts of replies. I have used LLMs/Claude Code quite extensively professionally and was a very early adopter, have built tooling around LLM/agentic development, and genuinely embraced it. They aren't useless, but the short term gains you think you're getting come at a very steep price that you…

I think the uncomfortable debate is not about skill atrophy as a general phenomenon (it’s happening anyway, doesn’t matter how much we debate it) but rather, _which_ skills are atrophying and if these skills are now superfluous/worthless or not. If you don’t use a skill, it’s like a gene a species doesn’t need anymore, it will atrophy. Is that bad and if yes, why? Skill atrophy is not intrinsically bad. I don’t know…

Since compilers became a thing Assembly language knowledge atrophied[1] across the workforce.

Since automatic memory management became a thing memory management and pointers knowledge atrophied[1] across the workforce (although not nearly to the same degree).

I think the pattern here is that compilers almost always output better machine code than humans, automatic memory management doesn't output better machine code than skilled humans can very (especially with modern languages that give you a lot build-time safety checks).

And even then, there is still demand for assembly knowledge in the workforce, it is just very niche.

I don't think LLMs will ever be good enough to "almost always" output better code than humans. But, like automatic memory management, it will likely make some types of programming more niche.

The key thing here is that compilers are deterministic, deterministic tools have way less variance in output quality. Automatic memory management is not as deterministic as a compiler because it happens at runtime. LLMs output build-time code, but the can be drastically different if I sneeze too hard.

[1]: as in % of the workforce, not absolute numbers. Hard to get exact figures on this, but I think we have more experienced people actively using Assembly today than we had before compilers became the default (late 80s). We probably have more active C/C++ programmers today than before Java became popular (early 2000s).

Re: Claude Sonnet 5

#682

Post by x3ro »

mingusrude wrote: post #631
jimbo808 wrote: post #621

Earlier quoted context omitted.

Technical debt and skill atrophy Technical debt due to accumulated excessively verbose, badly architected, often redundant, feature-bloated code which always looks good, even upon earnest review, but actually sucks and becomes extremely difficult to maintain in ways which are not obvious in code review. The issue is this: your tooling can help, and can make you feel better, and you might think you wrote all the promp…

> ...badly architected, often redundant, feature-bloated code which always looks good, even upon earnest review, but actually sucks and becomes extremely difficult to maintain in ways which are not obvious in code review. I can only judge from my own experience but with or without LLMs, these are the codebases that I have worked with during most of my career. To me, much of the question is whether LLMs produce worse…

In my experience, even if the code is not worse, there is so much more of it that people quickly stop understanding what is going on. Now you could argue „the LLM knows“, but that comes at an insane token cost, even if it works well.

This is in the interest of big AI companies: if they quasi-monopolize the skills entire sectors of the economy need in order to function, that will be great (for them).

Everyone keeps comparing this to compilers, but I don’t need a multiple-hundred dollar subscription to use LLVM. And people didn’t stop understanding how computers work either, just because they used C. And yeah, maybe local LLMs will become the norm, and I hope so. But market forces (hardware prices) certainly are working against that right now.

Re: Claude Sonnet 5

#683

Post by Traubenfuchs »

itopaloglu83 wrote: post #245
Jcampuzano2 wrote: post #216

I'm struggling to understand why I'd ever use this instead of just using a lower effort level for opus given on many of the benchmarks listed the cost per task rises above opus at anything higher than medium effort. Only thing I can think of is for when someone is out of opus credits. Of course there are API billing use cases but I'd probably still just use opus on low.

More and more I find myself trying to stop Opus from doing something stupid, and at every turn I need to tell it to stop overcomplicating things. I think the models are being optimized for wealth extraction from users and companies, instead of solving problems. I don't know why Opus would try to create an entire library when I told it specifically to do something simple that would take 2-3 lines of Python.

It's really bad when you let opus do investigations on broken java or infrastructure stuff. It starts decompiling .jar, sometimes multiple versions of the same dependency, reading every single kubernetes/terraform file and loading all the logs and info kubectl offers.

Re: Claude Sonnet 5

#684

Post by tskj »

4rf wrote: post #499
tskj wrote: post #276

Earlier quoted context omitted.

Dario has publicly claimed each model has been profitable, even accounting for its training costs; it's just that each new model is exponentially more expensive to train than the last, so the income lags and it looks like the company is losing money overall. Now, we can't know if this is true unfortunately, but it's not directly contradicted by anything that's known publicly at least. I thought it was an interesting…

why are you listening to these idiots who have every incentive to spin the story as much as possible FCFF = EBIT(1-t)-Reinvestment I dont care about your gross profit - this kind of cash profit determines the value of operating assets.

Well yeah obviously they have to stop reinvesting more than they make at some point to become profitable. To be clear, I think what Dario was saying was that if you consider each model training + deployment as a company, meaning all expenses and taxes, it was still profitable.

Whether he's lying is another question, but seems unlikely.

Re: Claude Sonnet 5

#685

Post by Traubenfuchs »

scrollop wrote: post #604
itopaloglu83 wrote: post #245

Earlier quoted context omitted.

More and more I find myself trying to stop Opus from doing something stupid, and at every turn I need to tell it to stop overcomplicating things. I think the models are being optimized for wealth extraction from users and companies, instead of solving problems. I don't know why Opus would try to create an entire library when I told it specifically to do something simple that would take 2-3 lines of Python.

"I think the models are being optimized for wealth extraction from users and companies, instead of solving problems." YES! They introduced the new tokenizer to increase token generation by upto 33%. On top of this, Anthropic are generating almost twice as much revenue per paid user than openai - whilst their subscriptions have lower usage limits than openai's: https://youtu.be/gK-7TKC7kvY?si=kx0qPE1rw-UCI-Jn&t=650

This slot machine has access to your bank account and can decide how much to play on its own!

Re: Claude Sonnet 5

#686

Post by tskj »

drob518 wrote: post #493
NorwegianDude wrote: post #367

Earlier quoted context omitted.

A common extreme misconception is that inference is expensive and that providers are loosing a lot of money. Inference is extremely lucrative and profitable.

Inference is the phase where they make money. But the question is whether they can be profitable overall as training continues to balloon.

I think the case for this is pretty strong actually. Last year my company was maybe willing to pay $100 a month to Anthropic (per developer). Today we're all on the $300 plan without any hesitation. If Fable ever becomes available as the default model, I imagine my company would be willing to pay in the $500-$1000 range per month per developer.

Re: Claude Sonnet 5

#687

Post by Traubenfuchs »

sixtyj wrote: post #132
conradkay wrote: post #19

Wow, seems worse even on price/performance than GLM 5.2, which is only 744b parameters. From the system card: "On CyberGym vulnerability discovery, Claude Sonnet 5 is less capable than Sonnet 4.6, and far less capable than Opus 4.8 and Mythos 5 As with the other evaluations in this section, these results were achieved with all safeguards turned off. When run with our default mitigations, Sonnet 5 scored a 0 on CyberG…

I have tried to rewrite an article with GLM-5.2 and with Sonnet 4.6. Completely different results as LLM is non-deterministic. But GLM-5.2 made a lot of subtle mistakes that needed to be corrected by hand. On the opposite, Sonnet found and corrected all mistakes in the second round. Similar situation was with planning and coding. GLM-5.2 seems to be good “on paper” but the real usage results was different. And I am n…

> Completely different results as LLM is non-deterministic.

You'd need to produce this like 20 times by each model and then do 2x20x20 cross comparisons by both models and ultimately distill the 2x20x20 comparison results into two reports of how they differ.

In this non deterministic computing future, everything else is voodoo, feelings and "vibes".

Re: Claude Sonnet 5

#688

Post by JumpCrisscross »

ChrisLTD wrote: post #366
JumpCrisscross wrote: post #176

Earlier quoted context omitted.

> no way to justify their valuations if they get downgraded to a pair programming tool I think there is. Pair today doesn’t mean they’re locked into that forever.

Their valuations don't make sense as just programming tools, period. Forget about if they are still human driven.

> Their valuations don't make sense as just programming tools, period

Yup. I think we agree. These valuations aren’t made or unmade by whether their tools are being used as vibe agents or pair programmers.

Re: Claude Sonnet 5

#689

Post by Traubenfuchs »

Sol- wrote: post #45

Wonder if the whole cyber paranoia leads to their models ultimately generating less secure code. After all, if it has the ability to generate safe code, it would imply that it knows something about cybersecurity, which could surely be used to hack all the banks in the world.

I think the cool kids call this "staying away from the vector space of highly skilled security engineers".
Post reply on HN