Live data from Hacker News

Claude 3.7 Sonnet and Claude Code

anthropic.com

831–840 of 1001 posts

Re: Claude 3.7 Sonnet and Claude Code

#831

You can get your HN profile analyzed by it and it's pretty funny :) https://hn-wrapped.kadoa.com/ I'm using this to test the humor of new models.

Okay, I feel like there might've been a breakthrough here. After watching Karpathy's video [0], he mentioned how hard it is for LLMs to have humor and be funny but it seems like Claude 3.7 really nailed it this time?

Like, most of these posts are legit funny.

[0] - https://www.youtube.com/watch?v=7xTGNNLPyMI

Re: Claude 3.7 Sonnet and Claude Code

#832
post #729

Ahha, recently my daugher come to me with 3rd grade math problem. "Without rearranging the digits 1 2 3 4 5, insert mathematical operation signs and, if necessary, parentheses between them so that the resulting expression equals 40 and 80. The key is that you can combine digits (like 12+3/45) but you cannot change their order from the original sequence 1,2,3,4,5" Grok3, Claude, Deepseek, Qwen all failed to solve this…

Looks correct to me on 3.7 extended (albeit with loooots of thinking) although I'm incredibly exhausted so I might not be mathing correctly: https://claude.ai/share/dfb37c1a-f6a8-45a1-b987-e6d28e205080

It found two solutions for 40 but one of them omitted the 5 the other added a 3.

12 * 3 + 4 = 40

1 * 2 * 3 * 4 * 5 / 3 = 40

Re: Claude 3.7 Sonnet and Claude Code

#833
post #622

Earlier quoted context omitted.

The problem with the API is that it, as it says in the documentation, could cost $100/hr. I would pay $50/mo or something to be able to have reasonable use of Claude Code in a limited (but not as limited) way as through the web UI, but all of these coding tools seem to work only with the API and are therefore either too expensive or too limited.

> The problem with the API is that it, as it says in the documentation, could cost $100/hr. I've used https://github.com/cline/cline to get a similar workflow to their Claude Code demo, and yes it's amazing how quickly the token counts add up. Claude seems to have capacity issues so I'm guessing they decided to charge a premium for what they can serve up. +1 on the too expensive or too limited sentiment. I subscribed…

Claude Code does caching well fwiw. Looking my costs after a few code sessions (totaling $6 or so) the vast majority is cache read, which is great to see. Without caching it'd be wildly more expensive.

Like $5+ was cache read ($0.05/token vs $3/token) so it would have cost $300+

Re: Claude 3.7 Sonnet and Claude Code

#834

You can get your HN profile analyzed by it and it's pretty funny :) https://hn-wrapped.kadoa.com/ I'm using this to test the humor of new models.

> Your salary is so low even your legacy code feels sorry for you. > You're the only person on HN who thinks $800/month is a salary and not a cloud computing bill. ouch

On the bright side. Not many here could 10x their salary in a couple of years.

Re: Claude 3.7 Sonnet and Claude Code

#835

Is it actually good at solving complex code or is it just garbage and people are lying about it as usual? In my experience EXTENSIVELY using claude 3.5 sonnet you basically have to do everything complex or you're just introducing massive amounts of slop code into your code base that while functional is nowhere near good. And for anything actually complex like requires a lot of context to make a decision and has to be…

I've played with it the whole day (so take it with a grain of salt). My gut feeling is that it can produce a bigger ... "thing". I am calling it a "thing", because it looks very much as what you want, but the bigger it is - the more the chances of it being subtly (or not) wrong.

I usually ask the models to extend a small parser/tree-walking interpreter with a compiler/VM.

Up until Claude 3.7 the models would propose something lazy and obviously incomplete. 3.7 generated something that looks almost right, mostly works, but is so overcomplicated and broken in such a way, that I rather delete it and write it from scratch. Trying to get the model to fix it resulted in running in circles, spitting out pieces of code that didn't fit the existing ones etc.

Not sure if I prefer the former or the latter tbh.

Re: Claude 3.7 Sonnet and Claude Code

#836

Earlier quoted context omitted.

Yep totally agree. It will also depend who captures the most eyeballs. ChatGPT is already my default first place to check something, where it was Google for the previous 20+ years.

Eyeballs aren’t enough though. Unlike Google ChatGPT is very expensive to run. It’s unlikely they can just slap ads on it like Google did.

Inference costs will keep dropping. The stuff the average consumer does will be trivially cheap. More stuff will move on device. The edge capabilities of these models are already far beyond what the average person can use or comprehend.

The point I wonder about is the sustainability of every query being 30+ requests. Site owners aren't ready to have 98% of their requests be non-monetizable bot traffic. However, sites that have something to sell are..

Re: Claude 3.7 Sonnet and Claude Code

#837
post #756

I am not sure how good these Exercism tasks are for measuring how good at a model with coding. My experience is that these models could write a simple function and get it right if it does not require any out of the box thinking (so essentially offloading the boilerplate part of coding). When it comes to think creatively and have a much better solution to a specific task that would require the think 2-3 steps ahead th…

I think many of the "AI can do coding" narratives don't see what coding means in real situations. It's finding out why "jbdoe1337" added this large if/else around the entire function body back in 2016 - it seems important business logic, but the commit just says "updated code". And how the h*ll this interaction between the conf.ini files, the conf/something.json and the ENV vars works. Why sometimes the ENV var overr…

Exactly. People sold on AI replacing software engineers are missing the point. It is almost the say that better laptops are replacing software engineers. LLMs are just tools that make you faster. Finding bugs, writing documentation, etc. are very nice to accelerate but creative thinking is also a big part of the job.

Re: Claude 3.7 Sonnet and Claude Code

#838

Earlier quoted context omitted.

With no moat, they aren't placed to capture much value; moats are what stops market competition from driving prices to the zero economic profit level, and that's even without further competition from free products that are being produced by people who aren’t even trying to support themselves in the market you are selling into, which can make even the zero economic profit price untenable.

Market competition doesn't work in an instant; even without a moat, there's plenty of money they can capture before it evaporates. Think pouring water from the faucet into a sink with open drain - if you have high enough flow rate, you can fill the sink faster than it drains. Then, when you turn the faucet off, as the sink is draining, you can still collect plenty of water from it with a cup or a bucket, before the s…

> Market competition doesn't work in an instant; even without a moat, there's plenty of money they can capture before it evaporates.

Sure, in a hypothetical market where before they try to extract profits most participants aren't losing money with below-profitable prices trying to keep mindshare. But you’d need a breakthrough around which a participant had some kind lf a moat to get, even temporarily, there in the LLM market.

Re: Claude 3.7 Sonnet and Claude Code

#839

Earlier quoted context omitted.

With no moat, they aren't placed to capture much value; moats are what stops market competition from driving prices to the zero economic profit level, and that's even without further competition from free products that are being produced by people who aren’t even trying to support themselves in the market you are selling into, which can make even the zero economic profit price untenable.

Market competition doesn't work in an instant; even without a moat, there's plenty of money they can capture before it evaporates. Think pouring water from the faucet into a sink with open drain - if you have high enough flow rate, you can fill the sink faster than it drains. Then, when you turn the faucet off, as the sink is draining, you can still collect plenty of water from it with a cup or a bucket, before the s…

The startups that are using API credits seem like the most likely to be able to achieve a good return on capital. There is a pretty clear cost structure and it's much more straightforward whether you are making money or not.

The infrastructure side of things, tens of billions and probably hundreds of billions going in, may not be fantastic for investors. The return on capital should approach cost of capital if someone does their job correctly. Add in government investment and subsidies (in China, the EU, the United States) and it become extremely difficult to make those calculations. In the long term, I don't think the AI infrastructure will be overbuilt (datacenters, fabs), but like the telecom bubble, it is easy to end up in a position where there is a lot of excess capacity and the way you made your bet means getting wiped out.

Of course if you aren't the investor and it isn't your capital, then there is a tremendous amount of money to be made because you have nothing to lose. I've been around a long time, and this is the closest thing I've felt to that inflection point where the web took off.

Re: Claude 3.7 Sonnet and Claude Code

#840

Claude 3.7 Sonnet scored 60.4% on the aider polyglot leaderboard [0], WITHOUT USING THINKING. Tied for 3rd place with o3-mini-high. Sonnet 3.7 has the highest non-thinking score, taking that title from Sonnet 3.5. Aider 0.75.0 is out with support for 3.7 Sonnet [1]. Thinking support and thinking benchmark results coming soon. [0] https://aider.chat/docs/leaderboards/ [1] https://aider.chat/HISTORY.html#aider-v0750

And yet, "DeepSeek R1 + claude-3-5-sonnet-20241022" scores 64% on the same benchmark 30% cheaper.

It's amazing what Deepseek is putting on the table while being full open source.

Post reply on HN