Tied for 3rd place with o3-mini-high. Sonnet 3.7 has the highest non-thinking score, taking that title from Sonnet 3.5.
Aider 0.75.0 is out with support for 3.7 Sonnet [1].
Thinking support and thinking benchmark results coming soon.
331–340 of 1001 posts
Tied for 3rd place with o3-mini-high. Sonnet 3.7 has the highest non-thinking score, taking that title from Sonnet 3.5.
Aider 0.75.0 is out with support for 3.7 Sonnet [1].
Thinking support and thinking benchmark results coming soon.
You can get your HN profile analyzed by it and it's pretty funny :) https://hn-wrapped.kadoa.com/ I'm using this to test the humor of new models.
> For someone who builds tools to automate everything, you sure spend a lot of time manually explaining why automation is the future on HN.
> Your obsession with sandboxed code execution suggests you've been traumatized by at least one production outage caused by an intern's unreviewed PR.
So good it hurts!
Earlier quoted context omitted.
Thanks! We all dogfood Claude every day to do our own work here, and solving our own pain points is more exciting to us than abstract benchmarks. Getting things done require a lot of booksmarts, but also a lot of "street smarts" - knowing when to answer quickly, when to double back, etc
Just want to say nice job and keep it up. Thrilled to start playing with 3.7. In general, benchmarks seem to very misleading in my experience, and I still prefer sonnet 3.5 for _nearly_ every use case- except massive text tasks, which I use gemini 2.0 pro with the 2M token context window.
Earlier quoted context omitted.
Can you give some insight into how you chose the reply limit length? It seems to cut off many useful programs that are 80%-90% done and if the limit were just a little higher it would be a source of extraordinary benefit.
If you can reproduce that, would you mind reporting it with /bug?
Here are steps to reproduce.
Background/environment:
ChatGPT helped me build this complete web browser in Python:
https://taonexus.com/publicfiles/feb2025/71toy-browser-with-...
It looks like this, versus the eventual goal: https://imgur.com/a/j8ZHrt1
in 1055 lines. But eventually it couldn't improve on it anymore, ChatGPT couldn't modify it at my request so that inline elements would be on the same line.
If you want to run it just download it and rename it to .py, I like Anaconda as an environment, after reading the code you can install the required libraries with:
conda install -c conda-forge requests pillow urllib3
then run the browser from the Anaconda prompt by just writing "python " followed by the name of the file.
2.
I tried to continue to improve the program with Claude, so that in-line elements would be on the same line.
I performed these reproduceable steps:
1. copied the code and pasted it into a Claude chat window with ctrl-v. This keeps it in the chat as paste.
2. Gave it the prompt "This complete web browser works but doesn't lay out inline elements inline, it puts them all on a new line, can you fix it so inline elements are inline?"
It spit out code until it hit section 8 out of 9 which is 70% of the way through and gave the error message "Claude hit the max length for a message and has paused its response. You can write Continue to keep the chat going". Screenshot:
So I wrote "Continue" and it stops when it is 90% of the way done.
Again it got stuck at 90% of the way done, second screenshot in the above album.
So I wrote "Continue" again.
It just gave an answer but it never finished the program. There's no app entry in the program, it completely omited the rest of the main class itself and the callback to call it, which would be like:
def run(self):
self.root.mainloop()
###############################################################################
# main
###############################################################################
if __name__=="__main__":
sys.setrecursionlimit(10**6)
app=ToyBrowser()
app.run()
so it only output a half-finished program. It explained that it was finished.I tried telling it "you didn't finish the program, output the rest of it" but doing so just got it stuck rewriting it without finishing it. Again it said it ran into the limit, again I said Continue, and again it didn't finish it.
The program itself is only 1055 lines, it should be able to output that much.
Kagi LLM benchmark updated with general purpose and thinking mode for Sonnet 3.7. https://help.kagi.com/kagi/ai/llm-benchmark.html Appears to be second most capable general purpose LLM we tried (second to gemini 2.0 pro, in front of gpt-4o). Less impressive in thinking mode, about at the same level as o1-mini and o3-mini (with 8192 token thinking budget). Overall a very nice update, you get higher quality and higher…
Edit: I just tried claude code CLI and it's a good compromise, it works pretty well, it does the discovery by itself instead of loading the whole codebase into context
FYI, We use Claude 3.7 has part of the new features we are shipping around Code Agent & more.
Earlier quoted context omitted.
DeepSeek’s models are vastly overhyped (FWIW I have access to them via Kagi, Windsurf, and Cursor - I regularly run the same tests on all three). I don’t think it matters that V3 is free when even R1 with its extra compute budget is inferior to Claude 3.5 by a large margin - at least in my experience in both bog standard React/Svelte frontend code and more complex C++/Qt components. After only half an hour of using C…
Hey there! I’m a fellow Qt developer and I really like your takes. Would you like to connect? My socials are on my profile.
I especially want to publish and send you the code for that inspector class and selector GUI that dumps the component hierarchy/state, QML source, and screenshot for use with Claude. Sadly I (and Claude) took some dumb shortcuts while implementing the inspector class that both couples it to proprietary code I can’t share and hardcodes some project specific bits, so it’s going to take me a bit of time to extricate the core logic.
I haven’t tried it with 3.7 but based on my tree-sitter QSyntaxHighlighter and Markdown QAbstactListModel tests so far, it is significantly better and I suspect the work Anthropic has done to train it for computer use will reap huge rewards for this use case. I’m still experimenting with the nitty gritty details but I think it will also be a game changer for testing in general, because combining computer use, gammaray-like dumps, and the Spix e2e testing API completes the full circle on app context.
You can get your HN profile analyzed by it and it's pretty funny :) https://hn-wrapped.kadoa.com/ I'm using this to test the humor of new models.
Actually quite funny.
It’s amazingly good, but it will be scaringly good when there will be a way to include the entire codebase in the context and let it create and run various parts of a large codebase autonomously. Right now I can only do patch work and give specific code snippets to make it work. Excited to try this new version out, I’m sure I won’t be disappointed, Edit: I just tried claude code CLI and it's a good compromise, it wor…
https://github.com/All-Hands-AI/OpenHands
The few times I've tested it out though it fails fairly quick and gets hung up (usually on setting up the project while testing with Kotlin / Go).