Live data from Hacker News

Claude 3.7 Sonnet and Claude Code

anthropic.com

331–340 of 1001 posts

Re: Claude 3.7 Sonnet and Claude Code

#331
Claude 3.7 Sonnet scored 60.4% on the aider polyglot leaderboard [0], WITHOUT USING THINKING.

Tied for 3rd place with o3-mini-high. Sonnet 3.7 has the highest non-thinking score, taking that title from Sonnet 3.5.

Aider 0.75.0 is out with support for 3.7 Sonnet [1].

Thinking support and thinking benchmark results coming soon.

[0] https://aider.chat/docs/leaderboards/

[1] https://aider.chat/HISTORY.html#aider-v0750

Re: Claude 3.7 Sonnet and Claude Code

#332

You can get your HN profile analyzed by it and it's pretty funny :) https://hn-wrapped.kadoa.com/ I'm using this to test the humor of new models.

> You've mentioned 'simple is robust' so many times that we're starting to think your dating profile just says 'uncomplicated and sturdy'.

> For someone who builds tools to automate everything, you sure spend a lot of time manually explaining why automation is the future on HN.

> Your obsession with sandboxed code execution suggests you've been traumatized by at least one production outage caused by an intern's unreviewed PR.

So good it hurts!

Re: Claude 3.7 Sonnet and Claude Code

#333

Earlier quoted context omitted.

Thanks! We all dogfood Claude every day to do our own work here, and solving our own pain points is more exciting to us than abstract benchmarks. Getting things done require a lot of booksmarts, but also a lot of "street smarts" - knowing when to answer quickly, when to double back, etc

Just want to say nice job and keep it up. Thrilled to start playing with 3.7. In general, benchmarks seem to very misleading in my experience, and I still prefer sonnet 3.5 for _nearly_ every use case- except massive text tasks, which I use gemini 2.0 pro with the 2M token context window.

I find the webdev arena tends to match my experience with models much more closely than other benchmarks: https://web.lmarena.ai/leaderboard. Excited to see how 3.7 performs!

Re: Claude 3.7 Sonnet and Claude Code

#334

Earlier quoted context omitted.

Can you give some insight into how you chose the reply limit length? It seems to cut off many useful programs that are 80%-90% done and if the limit were just a little higher it would be a source of extraordinary benefit.

If you can reproduce that, would you mind reporting it with /bug?

Just tried it with claude 3.7 sonnet, here is the share: https://claude.ai/share/68db540d-a7ba-4e1f-882e-f10adf64be91 and it doesn't finish outputing the program. (It's missing the rest of the application function and the main function).

Here are steps to reproduce.

Background/environment:

ChatGPT helped me build this complete web browser in Python:

https://taonexus.com/publicfiles/feb2025/71toy-browser-with-...

It looks like this, versus the eventual goal: https://imgur.com/a/j8ZHrt1

in 1055 lines. But eventually it couldn't improve on it anymore, ChatGPT couldn't modify it at my request so that inline elements would be on the same line.

If you want to run it just download it and rename it to .py, I like Anaconda as an environment, after reading the code you can install the required libraries with:

conda install -c conda-forge requests pillow urllib3

then run the browser from the Anaconda prompt by just writing "python " followed by the name of the file.

2.

I tried to continue to improve the program with Claude, so that in-line elements would be on the same line.

I performed these reproduceable steps:

1. copied the code and pasted it into a Claude chat window with ctrl-v. This keeps it in the chat as paste.

2. Gave it the prompt "This complete web browser works but doesn't lay out inline elements inline, it puts them all on a new line, can you fix it so inline elements are inline?"

It spit out code until it hit section 8 out of 9 which is 70% of the way through and gave the error message "Claude hit the max length for a message and has paused its response. You can write Continue to keep the chat going". Screenshot:

https://imgur.com/a/oSeiA4M

So I wrote "Continue" and it stops when it is 90% of the way done.

Again it got stuck at 90% of the way done, second screenshot in the above album.

So I wrote "Continue" again.

It just gave an answer but it never finished the program. There's no app entry in the program, it completely omited the rest of the main class itself and the callback to call it, which would be like:

        def run(self):
            self.root.mainloop()
    
    ###############################################################################
    # main
    ###############################################################################
    
    if __name__=="__main__":
        sys.setrecursionlimit(10**6)
        app=ToyBrowser()
        app.run()
so it only output a half-finished program. It explained that it was finished.

I tried telling it "you didn't finish the program, output the rest of it" but doing so just got it stuck rewriting it without finishing it. Again it said it ran into the limit, again I said Continue, and again it didn't finish it.

The program itself is only 1055 lines, it should be able to output that much.

Re: Claude 3.7 Sonnet and Claude Code

#335

Kagi LLM benchmark updated with general purpose and thinking mode for Sonnet 3.7. https://help.kagi.com/kagi/ai/llm-benchmark.html Appears to be second most capable general purpose LLM we tried (second to gemini 2.0 pro, in front of gpt-4o). Less impressive in thinking mode, about at the same level as o1-mini and o3-mini (with 8192 token thinking budget). Overall a very nice update, you get higher quality and higher…

Nice, but where is Grok?

Re: Claude 3.7 Sonnet and Claude Code

#336
It’s amazingly good, but it will be scaringly good when there will be a way to include the entire codebase in the context and let it create and run various parts of a large codebase autonomously. Right now I can only do patch work and give specific code snippets to make it work. Excited to try this new version out, I’m sure I won’t be disappointed,

Edit: I just tried claude code CLI and it's a good compromise, it works pretty well, it does the discovery by itself instead of loading the whole codebase into context

Re: Claude 3.7 Sonnet and Claude Code

#337
at Augment (https://augmentcode.com) we were one of the partner who tested 3.7 pre-launch. And it has been a pretty significant increase in quality and code understanding. Happy to answer some questions

FYI, We use Claude 3.7 has part of the new features we are shipping around Code Agent & more.

Re: Claude 3.7 Sonnet and Claude Code

#338

Earlier quoted context omitted.

DeepSeek’s models are vastly overhyped (FWIW I have access to them via Kagi, Windsurf, and Cursor - I regularly run the same tests on all three). I don’t think it matters that V3 is free when even R1 with its extra compute budget is inferior to Claude 3.5 by a large margin - at least in my experience in both bog standard React/Svelte frontend code and more complex C++/Qt components. After only half an hour of using C…

Hey there! I’m a fellow Qt developer and I really like your takes. Would you like to connect? My socials are on my profile.

We’ve already connected! Last year I think, because I was interested in your experience building a block editor (this was before your blog post on the topic). I’ve been meaning to reconnect for a few weeks now but family life keeps getting in the way - just like it keeps getting in the way of my implementing that block editor :)

I especially want to publish and send you the code for that inspector class and selector GUI that dumps the component hierarchy/state, QML source, and screenshot for use with Claude. Sadly I (and Claude) took some dumb shortcuts while implementing the inspector class that both couples it to proprietary code I can’t share and hardcodes some project specific bits, so it’s going to take me a bit of time to extricate the core logic.

I haven’t tried it with 3.7 but based on my tree-sitter QSyntaxHighlighter and Markdown QAbstactListModel tests so far, it is significantly better and I suspect the work Anthropic has done to train it for computer use will reap huge rewards for this use case. I’m still experimenting with the nitty gritty details but I think it will also be a game changer for testing in general, because combining computer use, gammaray-like dumps, and the Spix e2e testing API completes the full circle on app context.

Re: Claude 3.7 Sonnet and Claude Code

#339

You can get your HN profile analyzed by it and it's pretty funny :) https://hn-wrapped.kadoa.com/ I'm using this to test the humor of new models.

> You've recommended Marginalia search so many times, we're starting to think you're either the developer or just really enjoy websites that look like they were designed in 1998.

Actually quite funny.

[1] https://hn-wrapped.kadoa.com/jddj?share

Re: Claude 3.7 Sonnet and Claude Code

#340

It’s amazingly good, but it will be scaringly good when there will be a way to include the entire codebase in the context and let it create and run various parts of a large codebase autonomously. Right now I can only do patch work and give specific code snippets to make it work. Excited to try this new version out, I’m sure I won’t be disappointed, Edit: I just tried claude code CLI and it's a good compromise, it wor…

FWIW, there's a project to turn it into something similar, though I think it's lacking the "entire in context" part and runs into rate limits quick with Claude.

https://github.com/All-Hands-AI/OpenHands

The few times I've tested it out though it fails fairly quick and gets hung up (usually on setting up the project while testing with Kotlin / Go).

Post reply on HN