Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude 3.7 Sonnet and Claude Code

anthropic.com

Post reply on HN
Print view View original

251–260 of 1001 posts

Re: Claude 3.7 Sonnet and Claude Code

#251

Post by freediver »

Kagi LLM benchmark updated with general purpose and thinking mode for Sonnet 3.7.

https://help.kagi.com/kagi/ai/llm-benchmark.html

Appears to be second most capable general purpose LLM we tried (second to gemini 2.0 pro, in front of gpt-4o). Less impressive in thinking mode, about at the same level as o1-mini and o3-mini (with 8192 token thinking budget).

Overall a very nice update, you get higher quality and higher speed model at same price.

Hope to enable it in Kagi Assistant within 24h!

Re: Claude 3.7 Sonnet and Claude Code

#253

Post by falcor84 »

dingnuts wrote: post #144
amelius wrote: post #38

Earlier quoted context omitted.

It stopped being revolutionary and is now mostly evolutionary, though.

it's been evolutionary for a long time. I fine-tuned a GPT-2 based chat bot that could form complete sentences back in like 2017 It's been so long that I'm not even certain which YEAR I set that up.

Where do you draw the line? If going from forming sentences to achieving medal level success on IMO questions, doing extensive web research on its own and writing entire SaaS apps based on a prompt in under 10 years is just "evolutionary", then it's one heck of an evolution.

Re: Claude 3.7 Sonnet and Claude Code

#254

Post by grav »

Claude 3.7 Sonnet seems to have a context window of 64.000 via the API:

  max_tokens: 4242424242 > 64000, which is the maximum allowed number of output tokens for claude-3-7-sonnet-20250219
I got a max of 8192 with Claude 3.5 sonnet.

Re: Claude 3.7 Sonnet and Claude Code

#255

Post by tonyhart7 »

throwup238 wrote: post #195
bicx wrote: post #141

Earlier quoted context omitted.

Claude 3.5 has been fantastic in Windsurf. However, it does cost credits. DeepSeek V3 is now available in Windsurf at zero credit cost, which was a major shift for the company. Great to have variable options either way. I’d highly recommend anyone check out Windsurf’s Cascade feature for agentic-like code writing and exploration. It helped save me many hours in understanding new codebases and tracing data flows.

DeepSeek’s models are vastly overhyped (FWIW I have access to them via Kagi, Windsurf, and Cursor - I regularly run the same tests on all three). I don’t think it matters that V3 is free when even R1 with its extra compute budget is inferior to Claude 3.5 by a large margin - at least in my experience in both bog standard React/Svelte frontend code and more complex C++/Qt components. After only half an hour of using C…

I seen people switch from claude due to cost to another model notably deepseek tbh I think it still depends on model trained data on

Re: Claude 3.7 Sonnet and Claude Code

#256

Post by lebovic »

bakugo wrote: post #214
latetomato wrote: post #206

Earlier quoted context omitted.

Hello! Member of the API team here. We're unable to find issues with the /v1/models endpoint—can you share more details about your request? Feel free to email me at suzanne@anthropic.com. Thank you!

It always returns a Not Found error for me. Using the curl command copied directly from the docs: $ curl https://api.anthropic.com/v1/models --header "x-api-key: $ANTHROPIC_API_KEY" --header "anthropic-version: 2023-06-01" {"type":"error","error":{"type":"not_found_error","message":"Not found"}} Edit: Tried creating a different API key and it works with that one. Weird.

If you can reproduce the issue with the other API key, I'd also love to debug this! Feel free to share the curl -vv output (excluding the key) with the Anthropic email address in my profile

Re: Claude 3.7 Sonnet and Claude Code

#257

Post by eschluntz »

jumploops wrote: post #104

> "[..] in developing our reasoning models, we’ve optimized somewhat less for math and computer science competition problems, and instead shifted focus towards real-world tasks that better reflect how businesses actually use LLMs.” This is good news. OpenAI seems to be aiming towards "the smartest model," but in practice, LLMs are used primarily as learning aids, data transformers, and code writers. Balancing "intell…

Thanks! We all dogfood Claude every day to do our own work here, and solving our own pain points is more exciting to us than abstract benchmarks.

Getting things done require a lot of booksmarts, but also a lot of "street smarts" - knowing when to answer quickly, when to double back, etc

Re: Claude 3.7 Sonnet and Claude Code

#258

Post by koakuma-chan »

grav wrote: post #254

Claude 3.7 Sonnet seems to have a context window of 64.000 via the API: max_tokens: 4242424242 > 64000, which is the maximum allowed number of output tokens for claude-3-7-sonnet-20250219 I got a max of 8192 with Claude 3.5 sonnet.

Context window is how long your prompt can be. Output tokens is how long its response can be. What you sent says its response can be 64k tokens at maximum.

Re: Claude 3.7 Sonnet and Claude Code

#259

Post by eschluntz »

sealthedeal wrote: post #222
eschluntz wrote: post #160

Earlier quoted context omitted.

We are definitely aware of this (and working on it for the web UI), and that's why Claude Code goes directly through the API!

I haven't been able to find ClaudeCLI for pubic access yet. Would love to use.

>>> npm install -g @anthropic-ai/claude-code

>>> claude

Re: Claude 3.7 Sonnet and Claude Code

#260

Post by ben_w »

crowcroft wrote: post #220
jumploops wrote: post #104

> "[..] in developing our reasoning models, we’ve optimized somewhat less for math and computer science competition problems, and instead shifted focus towards real-world tasks that better reflect how businesses actually use LLMs.” This is good news. OpenAI seems to be aiming towards "the smartest model," but in practice, LLMs are used primarily as learning aids, data transformers, and code writers. Balancing "intell…

Sometimes I wonder if there is overfitting towards benchmarks (DeepSeek is the worst for this to me). Claude is pretty consistently the chat I go back to where the responses subjectively seem better to me, regardless of where the model actually lands in benchmarks.

> Sometimes I wonder if there is overfitting towards benchmarks

There absolutely is, even when it isn't intended.

The difference between what the model is fitting to and reality it is used on is essentially every problem in AI, from paperclipping to hallucination, from unlawful output to simple classification errors.

(Ok, not every problem, there's also sample efficiency, and…)

Post reply on HN