Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Sonnet 4 now supports 1M tokens of context

anthropic.com

Post reply on HN
Print view View original

671–680 of 706 posts

Re: Claude Sonnet 4 now supports 1M tokens of context

#671

Post by jacobr1 »

km144 wrote: post #649
HarHarVeryFunny wrote: post #642

Earlier quoted context omitted.

Even 1 MB context is only roughly 20K LOC so pretty limiting, especially if you're also trying to fit API documents or any other lengthy material into the context. Anthropic also recently said that they think that longer/compressed context can serve as an alternative (not sure what was the exact wording/characterization they used) to continual/incremental learning, so context space is also going to be competing with…

As you alluded to at the end of your post—I'm not really convinced 20k LOC is very limiting. How many lines of code can you fit in your working mental model of a program? Certainly less than 20k concrete lines of text at any given time. In your working mental model, you have broad understandings of the broader domain. You have broad understandings of the architecture. You summarize broad sections of the program into…

This is basically how claude code works today. You have it /init a description of the project structure into CLAUDE.md that is used for each invocation. There is some implicit knowledge in the project about common frameworks and languages. Then when working on something between the explicit and implicit knowledge and the task at hand it will grep for relevant material in the project, load either full or parts of files, and THEN it will start working on the task. But it dynamically builds the context of the codebase based on searching for the relevant bit. Short-circuiting this by having a good project summary makes it more efficient - but you don't need to literally copy in all the code files.

Re: Claude Sonnet 4 now supports 1M tokens of context

#672

Post by jacobr1 »

Wowfunhappy wrote: post #314
rootnod3 wrote: post #20

Earlier quoted context omitted.

Flooding the context also means increasing the likelihood of the LLM confusing itself. Mainly because of the longer context. It derails along the way without a reset.

I keep reading this, but with Claude Code in particular, I consistently find it gets smarter the longer my conversations go on, peaking right at the point where it auto-compacts and everything goes to crap. This isn't always true--some conversations go poorly and it's better to reset and start over--but it usually is.

I've found there usually is some key context that is missing. Maybe it is project structure or a sampling of some key patterns from different parts of the codebase, or key data models. Getting those into CLAUDE.md reduces the need to keep building up (as large) context.

As an example for one project, I realized things were getting better after it started writing integration tests. I wasn't sure if that was the act of writing the test forced it to reason about the they black box way the system would be used, or if there was another factor. Turns out it was just example usage. Extracting out the usage patterns into both the README and CLAUDE.md was itself a simple request, then I got similar performance on new tasks.

Re: Claude Sonnet 4 now supports 1M tokens of context

#673

Post by HarHarVeryFunny »

jacobr1 wrote: post #671
km144 wrote: post #649

Earlier quoted context omitted.

As you alluded to at the end of your post—I'm not really convinced 20k LOC is very limiting. How many lines of code can you fit in your working mental model of a program? Certainly less than 20k concrete lines of text at any given time. In your working mental model, you have broad understandings of the broader domain. You have broad understandings of the architecture. You summarize broad sections of the program into…

This is basically how claude code works today. You have it /init a description of the project structure into CLAUDE.md that is used for each invocation. There is some implicit knowledge in the project about common frameworks and languages. Then when working on something between the explicit and implicit knowledge and the task at hand it will grep for relevant material in the project, load either full or parts of file…

Interesting - thanks!

Re: Claude Sonnet 4 now supports 1M tokens of context

#674

Post by epolanski »

mcintyre1994 wrote: post #601
epolanski wrote: post #496

Earlier quoted context omitted.

I really find your experience strikingly different than mine, I'll share you my flow: - step A: ask AI to write a featureA-requirements.md file at the root of the project, I give it a general description for the task, then have it ask me as many questions as possible to refine user stories and requirements. It generally comes up with a dozen or more of questions, of which multiples I would've not thought about and fo…

You’d probably like Kiro, it seems to be built specifically for this sort of spec-driven development.

How would it be better than what I'm doing with Claude?

Re: Claude Sonnet 4 now supports 1M tokens of context

#675

Post by scott_s »

jimbokun wrote: post #660
scott_s wrote: post #644

Earlier quoted context omitted.

> Of course, because I am not new to the problem, whereas an LLM is new to it every new prompt. That is true for the LLMs you have access to now. Now imagine if the LLM had been trained on your entire code base. And not just the code, but the entire commit history, commit messages and also all of your external design docs. And code and docs from all relevant projects. That LLM would not be new to the problem every pr…

Why haven’t the bug AI companies been pursuing that approach, vs just ramping up context window size?

Because training one family of models with very large context windows can be offered to the entire world as an online service. That is a very different business model from training or fine-tuning individual models specifically for individual customers. Someone will figure out how to do that at scale, eventually. It might require the cost of training to reduce significantly. But large companies with the resources to do this for themselves will do it, and many are doing it.

Re: Claude Sonnet 4 now supports 1M tokens of context

#676

Post by anonym29 »

varyherb wrote: post #80

I believe this can be configured in Claude Code via the following environment variable: ANTHROPIC_BETAS="context-1m-2025-08-07" claude

Tested this morning. Worked wonderfully, except ran into output issues. Attempted to patched the minified Claude file's CLAUDE_CODE_MAX_OUTPUT_TOKENS hard limit of 32000 on Sonnet to 64000, which worked, and I was able to generate outputs above 32000 tokens, but this coincided with a breakage of the 1m context window for me. Still testing and playing around with this, but this may be getting patched?

Re: Claude Sonnet 4 now supports 1M tokens of context

#678

Post by brulard »

jimbokun wrote: post #666
brulard wrote: post #473

Earlier quoted context omitted.

I think you misunderstand how context in current LLMs works. To get the best results you have to be very careful to provide what is needed for immediate task progression, and postpone context thats needed later in the process. If you give all the context at once, you will likely get quite degraded output quality. Thats like if you want to give a junior developer his first task, you likely won't teach him every corner…

It seems like LLM need to become experts at managing their OWN context. Selectively gripping and searching the code to pull into context only those parts relevant to the task at hand.

That's what I'm thinking about a lot. Something like the models "activate" just some subset of parameters when working (if I understand the new models correctly). So that model could activate parts of context which are relevant for the task at hand

Re: Claude Sonnet 4 now supports 1M tokens of context

#679

Post by brulard »

ants_everywhere wrote: post #629
brulard wrote: post #473

Earlier quoted context omitted.

I think you misunderstand how context in current LLMs works. To get the best results you have to be very careful to provide what is needed for immediate task progression, and postpone context thats needed later in the process. If you give all the context at once, you will likely get quite degraded output quality. Thats like if you want to give a junior developer his first task, you likely won't teach him every corner…

> I think you misunderstand how context in current LLMs works. Thanks but I don't and I'm not sure why you're jumping to this conclusion. EDIT: Oh I think you're talking about the last bit of the comment! If you read the one before I say that feeding it the entire repo isn't a great idea. But great idea or not, people want to do it, and it illustrates that as context window increases it creates demand for even larger…

I said that based on you saying you exhaust a million token context windows easily. I'm no expert on that, but I think the current state of LLMs works best if you are not approaching that 1M token limit, because large context (reportedly) deteriorates response quality quickly. I think state of the art usage is managing context in tens or low hundreds thousands tokens at most and taking advantage of splitting tasks across subtasks in time, or splitting context across multiple "expert" agents (see sub-agents in claude code).

Re: Claude Sonnet 4 now supports 1M tokens of context

#680

Post by adastra22 »

jimbokun wrote: post #665
vidarh wrote: post #600

Earlier quoted context omitted.

The comparison would be apt if the LLM was trained on your codebase .

Isn’t that the problem? I don’t see any progress on incrementally training LLMs on specific projects. I believe it’s called fine tuning, right? Why isn’t that the default approach anywhere instead of the hack of bigger “context windows”?

Because fine-tuning can be used to remove restrictions from a model, so they don't give us plebs access to that.
Post reply on HN