Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Sonnet 4 now supports 1M tokens of context

anthropic.com

Post reply on HN
Print view View original

581–590 of 706 posts

Re: Claude Sonnet 4 now supports 1M tokens of context

#581

Post by ehnto »

xwolfi wrote: post #541
ehnto wrote: post #532

Earlier quoted context omitted.

I disagree, I may not have the whole codebase in my head in one moment but I have had all of it in my head at some point, and it is still there, that is not true of an LLM. I use LLMs and am impressed by them, but they just do not approximate a human in this particular area. My ability to break a problem down does not start from listing the files out and reading a few. I have a high level understanding of the whole p…

You only worked on very small codebase then. When you work on giant ones, you Ctrl+F a lot, build a limited model of the problem space, and pray the unit tests will catch anything you might have missed...

We're measuring lengths of string, but I would not say I have worked on small projects. I am very familiar with discovery, and have worked on a lot of large legacy projects that have no tests just fine.

Re: Claude Sonnet 4 now supports 1M tokens of context

#582

Post by animuchan »

jbs789 wrote: post #572
PaulDavisThe1st wrote: post #561

Earlier quoted context omitted.

> I disagree, I may not have the whole codebase in my head in one moment but I have had all of it in my head at some point, and it is still there, that is not true of an LLM. All 3 points (you have had all of it your head at some point, it is still there, that is not true of an LLM) are mere conjectures, and not provable at this time, certainly not in the general case. You may be able to show this of some codebases f…

I'm not sure the idea that a developer maintains a high level understanding is all that controversial...

The trend for this idea's controversiality is shown on this very small chart: /

Re: Claude Sonnet 4 now supports 1M tokens of context

#583

Post by energy123 »

jack_pp wrote: post #552
aliljet wrote: post #16

This is definitely one of my CORE problem as I use these tools for "professional software engineering." I really desperately need LLMs to maintain extremely effective context and it's not actually that interesting to see a new model that's marginally better than the next one (for my day-to-day). However. Price is king. Allowing me to flood the context window with my code base is great, but given that the price has su…

maybe we need LLMs trained on ASTs or create a new symbolic way to represent software that's faster to grok by LLMs and have a translator so we can verify the code

You could probably build a decent agentic harness that achieves something similar.

Show the LLM a tree and/or call-graph representation of your codebase (e.g. `cargo diagram` and `cargo-depgraph`), which is token efficient.

And give the LLM a tool call to see the contents of the desired subtree. More precise than querying a RAG chunk or a whole file.

You could also have another optional tool call which routes the text content of the subtree through a smaller LLM that summarizes it into a maximum density snippet, which the LLM can use for a token efficient understanding of that subtree during early the planning phase.

But I'd agree that an LLM built natively around AST is a pretty cool idea.

Re: Claude Sonnet 4 now supports 1M tokens of context

#584

Post by ehnto »

airbreather wrote: post #542
ehnto wrote: post #532

Earlier quoted context omitted.

I disagree, I may not have the whole codebase in my head in one moment but I have had all of it in my head at some point, and it is still there, that is not true of an LLM. I use LLMs and am impressed by them, but they just do not approximate a human in this particular area. My ability to break a problem down does not start from listing the files out and reading a few. I have a high level understanding of the whole p…

you will have abstractions - black boxing, interface overviews etc, humans can only hold so much detail in current context memory, some say 7 items on average.

Of course, but even those blackoxes are not empty, they've got a vague picture inside them based on prior experience. I have been doing this for a while so most things are just various flavours of the same stuff, especially in enterprise software.

The important thing in this context is that I know it's all there, I don't have to grep the codebase to fill up my context, and my understanding of the holistic project does not change each time I am booted up.

Re: Claude Sonnet 4 now supports 1M tokens of context

#585

Post by otabdeveloper4 »

simianwords wrote: post #184
eldenring wrote: post #179

Earlier quoted context omitted.

Serving a model efficiently at 1M context is difficult and could be much more expensive/numerically tricky. I'm guessing they were working on serving it properly, since its the same "model" in scores and such.

Thanks - still not clear what they did really. Some inference time hacks?

Most likely still 32k tokens under the hood, but with some context slicing/averaging hacks to make inference not error out on infinite input.

(That's what I do locally with llama.cpp)

Re: Claude Sonnet 4 now supports 1M tokens of context

#586

Post by mixedCase »

rocqua wrote: post #567
wraptile wrote: post #525

Earlier quoted context omitted.

Right, the LLM doesn't need to know all of the code under utils.parse_id to know that this call will parse the ID. The best LLM results I get is when I manually define the the relative code graph of my problem similar how I'd imagine it my head which seems to provide optimal context. So bigger isn't really better.

I wonder why we can't have one LLM generate this understanding for another? Perhaps this is where teaming of LLMs gets its value. In managing high and low level context in different context windows.

This is a thing and doesn't require a separate model. You can set up custom prompts that will, based on another prompt describing the task to achieve, generate information about the codebase and a set of TODOs to accomplish the task, generating markdown files with a summarized version of the relevant knowledge and prompting you again to refine that summary if needed. You can then use these files to let the agent take over without going on a wild goose chase.

Re: Claude Sonnet 4 now supports 1M tokens of context

#587

Post by manmal »

ants_everywhere wrote: post #322
aliljet wrote: post #16

This is definitely one of my CORE problem as I use these tools for "professional software engineering." I really desperately need LLMs to maintain extremely effective context and it's not actually that interesting to see a new model that's marginally better than the next one (for my day-to-day). However. Price is king. Allowing me to flood the context window with my code base is great, but given that the price has su…

> I really desperately need LLMs to maintain extremely effective context The context is in the repo. An LLM will never have the context you need to solve all problems. Large enough repos don't fit on a single machine. There's a tradeoff just like in humans where getting a specific task done requires removing distractions. A context window that contains everything makes focus harder. For a long time context windows we…

> The context is in the repo

No it’s in the problem at hand. I need to load all related files, documentation, and style guides into the context. This works really well for smaller modules, but currently falls apart after a certain size.

Re: Claude Sonnet 4 now supports 1M tokens of context

#588

Post by brulard »

mathiaspoint wrote: post #483
brulard wrote: post #482

Earlier quoted context omitted.

> These issues are inherent to the technology That's simply false. Even if LLMs don't produce correct and valid code on first shot 100% times of the cases, if you use an agent, it's simply a matter of iterations. I have claude code connected to Playwright, context7 for docs and to Playwright, so it can iterate by itself if there are syntax errors, runtime errors or problems with the data on the backend side. Currentl…

It definitely isn't. LLMs often end up stuck in weird corners they just don't get and need someone familiar with the theory of what they're working on to unstick them. If the agent is the same model as the code generator it won't be able to on its own.

I was getting to stuck state with Gemini and to lesser extent with Sonnet 4, but my cases were resolved by Opus. I think it is mostly due to size of the task and if you split it in advance to smaller chunks, all these models has much higher probability to resolve.

Re: Claude Sonnet 4 now supports 1M tokens of context

#589

Post by brulard »

spaceywilly wrote: post #535
ramshanker wrote: post #514

Earlier quoted context omitted.

I mean the website of AI providers. chatgpt.com , gemini.google.com , claude.ai and so on.

I’ve had more success this way as well. I will use the model via web ui, paste in the relevant code, and ask it to implement something. It spits out the code, I copy it back into the ide, and build. I tried Claude Code but I find it goes off the rails too easily. I like the chat through the UI because it explains what it’s doing like a senior engineer would

Well, this is the way we could do it for 2 years already, but basically you are doing the transport layer for the process, which can not be efficient. If you really want to have tight control of what exactly the LLM sees, than that's still an option. But you only get so far with this approach.

Re: Claude Sonnet 4 now supports 1M tokens of context

#590

Post by brulard »

nojs wrote: post #536
brulard wrote: post #482

Earlier quoted context omitted.

> These issues are inherent to the technology That's simply false. Even if LLMs don't produce correct and valid code on first shot 100% times of the cases, if you use an agent, it's simply a matter of iterations. I have claude code connected to Playwright, context7 for docs and to Playwright, so it can iterate by itself if there are syntax errors, runtime errors or problems with the data on the backend side. Currentl…

Could you explain your exact playwright setup in more detail? I’ve found that claude really struggles to end-to-end test complex features that require browser use. It gets stuck for several minutes trying to find the right button to click for example.

No special setup, just something along "test with playwright" in the process list. It can get stuck, but for me it was not often enough for me to care. If it happens, I push it in the right direction.
Post reply on HN