Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude 3.7 Sonnet and Claude Code

anthropic.com

Post reply on HN
Print view View original

961–970 of 1001 posts

Re: Claude 3.7 Sonnet and Claude Code

#961

Post by tanaykarnik »

jonplackett wrote: post #956
jonplackett wrote: post #752

Earlier quoted context omitted.

I like to make up my own tests, that way you know it is actually thinking. Tests that require thinking about the physical world are the most revealing. My new favourite is: You have 2 minutes to cool down a cup of coffee to the lowest temp you can. You have two options: 1. Add cold milk immediately, then let it sit for 2 mins. 2. Let it sit for 2 mins, then add cold milk. Which one cools the coffee to the lowest temp…

Can’t edit my comment - this was meant to say NON-thinking models can’t do it. The thinking ones can and that’s what is interesting (face palm)

> all *but* the thinking models get it wrong

don't worry it's "all but", so you are fine.

Re: Claude 3.7 Sonnet and Claude Code

#962

Post by precompute »

hubraumhugo wrote: post #131

You can get your HN profile analyzed by it and it's pretty funny :) https://hn-wrapped.kadoa.com/ I'm using this to test the humor of new models.

Their website is really similar to Exa's! Kind of a red flag, considering they're both practically in the same side of the data curation.

Re: Claude 3.7 Sonnet and Claude Code

#963

Post by camdenreslink »

CamperBob2 wrote: post #914
berkes wrote: post #756

Earlier quoted context omitted.

I think many of the "AI can do coding" narratives don't see what coding means in real situations. It's finding out why "jbdoe1337" added this large if/else around the entire function body back in 2016 - it seems important business logic, but the commit just says "updated code". And how the h*ll this interaction between the conf.ini files, the conf/something.json and the ENV vars works. Why sometimes the ENV var overr…

Systems built from scratch with AI won't have these limitations, because only the model will ever see the code. It will implement a spec that's written in English or another human language. When the business requirements change, the spec will change. When that happens, the system will either modify its previously-written code or regenerate it from the ground up. Which strategy it chooses won't be especially interesti…

I haven't seen evidence that this will come to pass. But it's possible. English-language specs are ambiguous. Do you really think businesses with money on the line will tolerate an LLM making automated changes to a codebase and pushing them without a human in the loop? Even human programmers create outages (and we have "AGI" which is the holy grail). If an autonomous LLM creates outages 10% more frequently than a team of humans it is basically unusable. We would need to see a lot more improvement from current state of the art.

Re: Claude 3.7 Sonnet and Claude Code

#964

Post by dghlsakjg »

Terretta wrote: post #568
Aeolun wrote: post #503

Earlier quoted context omitted.

I don’t think you need to be a business to use the API? At least I’m fairly certain I’m using it in a personal capacity. You are never going to hit $120/month even with full-time usage (no guarantees of course, but I get to like $40/month).

Careful -- a solo dev using it professionally, meaning, coding with it as a pair coder (XP style), can easily spend $1500/week.

$1500 is 100 million output tokens, or 500 million input tokens for Claude 3.7.

The entire LOTR trilogy is ~.55 million tokens (1,200 pages, published).

If you are sending and receiving the text equivalent of several hundred copies of the LOTR trilogy every week, I don't think you are actually using AI for anything useful, or you are providing far too much context.

Re: Claude 3.7 Sonnet and Claude Code

#965

Post by james_marks »

smithcoin wrote: post #936
james_marks wrote: post #895

Earlier quoted context omitted.

Perhaps a magnitude more effective than copy/paste, perhaps not. But do I get more than $300/month of value from it, per developer? Almost certainly. The bottleneck was already checking the work for correctness and building my own trust / familiarity with new code. So it's made that problem slightly more pronounced, as it generates more code faster, with more surface area to debug when many new changes arrive at once…

You described the price increasing from $10/month to $300 (which is an order of magnitude). What I meant was when you are saying it equals out (in terms of cost for you) you would expect a similar increase in performance. If that makes sense.

I understand, but it's the wrong question IMO. It was dramatically under-priced before, and less so now.

Re: Claude 3.7 Sonnet and Claude Code

#966

Post by biker142541 »

Sadly, Claude 3.7 is still failing pretty hard on Svelte 5 even when provided latest docs in context. It just fails more confidently and further into otherwise decent code than 3.5. Ex: built a much more complex initial app, but used runes incorrectly and continued to use . Even when prompted with update doc snippets, it couldn't dig itself out of its hole.

We really still need a better unified workflow for working on the cutting edge of tech with LLMs, imo. This problem is the same with other frameworks/technologies undergoing recent changes.

Re: Claude 3.7 Sonnet and Claude Code

#967

Post by astrodude »

jonplackett wrote: post #752
nightpool wrote: post #542

Earlier quoted context omitted.

> 225 coding exercises from Exercism Has there been any effort taken to reduce data leakage of this test set? Sounds like these exercises were available on the internet pre-2023, so they'll probably be included in the training data for any modern model, no?

I like to make up my own tests, that way you know it is actually thinking. Tests that require thinking about the physical world are the most revealing. My new favourite is: You have 2 minutes to cool down a cup of coffee to the lowest temp you can. You have two options: 1. Add cold milk immediately, then let it sit for 2 mins. 2. Let it sit for 2 mins, then add cold milk. Which one cools the coffee to the lowest temp…

how do you know that this test doesn't already exist somewhere on the internet and LLMs were trained on it?

Re: Claude 3.7 Sonnet and Claude Code

#968

Post by srigi »

babyshake wrote: post #190
bcherny wrote: post #91

Hi everyone! Boris from the Claude Code team here. @eschluntz, @catherinewu, @wolffiex, @bdr and I will be around for the next hour or so and we'll do our best to answer your questions about the product.

One thing I would love to have fixed - I type in a prompt, the model produces 90% or even 100% of the answer, and then shows an error that the system is at capacity and can't produce an answer. And then the response that has already been provided is removed! Please just make it where I can still have access to the response that has been provided, even if it is incomplete.

To me it doesn’t look like a bug. I believe it is a intended “feature” pushed from high management - a dark patern to make plebs pay for answer that has overflowed the quota.

Re: Claude 3.7 Sonnet and Claude Code

#969

Post by genewitch »

jonplackett wrote: post #954
chairdoor wrote: post #930

Earlier quoted context omitted.

Isn't the answer milk first, then let sit? You only have 2 minutes, so if you're adding the milk after 2 minutes have already elapsed, then you've already exceeded the time limit, meaning the final measurement would take place before the milk is even poured in.

Adding the milk second is colder. The bigger the temp difference the more cooling. So by putting the milk in right away you make the temp difference between the surroundings and the coffee smaller = less cooling over your 2 mins. I like puzzles with a satisfying answer

Parent is complaining about being technically hotter because the time the temperature is read for logging is at 120seconds.

I notice this on HN more than places like reddit and Mastodon.

I think it's a benefit when writing requirements and programming to specs. It's less of a benefit when having a freeform discussion.

Re: Claude 3.7 Sonnet and Claude Code

#970

Post by genewitch »

Seanambers wrote: post #870
weatherlite wrote: post #798

Earlier quoted context omitted.

Like what economic changes? You can make a case people are 10% more productive in very specific fields (programming, perhaps consultancy etc). That's not really an earthquake, the internet/web was probably way more significant.

LLMs are fundamentally a new paradigm, it just isn't distributed yet. It's not like the web suddenly was just there, it came slow at first, then everywhere at once, the money came even later.

Government and healthcare workers have been using AI for notes for over a year in Louisiana; an additional anecdote to sibling.
Post reply on HN