Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Sonnet 4 now supports 1M tokens of context

anthropic.com

Post reply on HN
Print view View original

41–50 of 706 posts

Re: Claude Sonnet 4 now supports 1M tokens of context

#41

Post by siwatanejo »

alexchamberlain wrote: post #35
aliljet wrote: post #16

This is definitely one of my CORE problem as I use these tools for "professional software engineering." I really desperately need LLMs to maintain extremely effective context and it's not actually that interesting to see a new model that's marginally better than the next one (for my day-to-day). However. Price is king. Allowing me to flood the context window with my code base is great, but given that the price has su…

I'm not sure how, and maybe some of the coding agents are doing this, but we need to teach the AI to use abstractions, rather than the whole code base for context. We as humans don't hold the whole codebase in our hear, and we shouldn't expect the AI to either.

I do think AIs are already using abstractions, otherwise you would be submitting all the source code of your dependencies into the context.

Re: Claude Sonnet 4 now supports 1M tokens of context

#42

Post by sdesol »

aliljet wrote: post #16

This is definitely one of my CORE problem as I use these tools for "professional software engineering." I really desperately need LLMs to maintain extremely effective context and it's not actually that interesting to see a new model that's marginally better than the next one (for my day-to-day). However. Price is king. Allowing me to flood the context window with my code base is great, but given that the price has su…

> I really desperately need LLMs to maintain extremely effective context

I actually built this. I'm still not ready to say "use the tool yet" but you can learn more about it at https://github.com/gitsense/chat.

The demo link is not up yet as I need to finalize an admin tool but you should be able to follow the npm instructions to play around with.

The basic idea is, you should be able to load your entire repo or repos and use the context builder to help you refine it. Or you can can create custom analyzers that you can do 'AI Assisted' searches with like execute `!ask find all frontend code that does [this]` and the because the analyzer knows how to extract the correct metadata to support that query, you'll be able to easily build the context using it.

Re: Claude Sonnet 4 now supports 1M tokens of context

#43

Post by F7F7F7 »

aliljet wrote: post #27
rootnod3 wrote: post #20

Earlier quoted context omitted.

Flooding the context also means increasing the likelihood of the LLM confusing itself. Mainly because of the longer context. It derails along the way without a reset.

How do you know that?

What do you think happens when things start falling outside of its context window? It loses access to parts of your conversation.

And that’s why it will gladly rebuild the same feature over and over again.

Re: Claude Sonnet 4 now supports 1M tokens of context

#45

Post by [deleted] »

alexchamberlain wrote: post #35
aliljet wrote: post #16

This is definitely one of my CORE problem as I use these tools for "professional software engineering." I really desperately need LLMs to maintain extremely effective context and it's not actually that interesting to see a new model that's marginally better than the next one (for my day-to-day). However. Price is king. Allowing me to flood the context window with my code base is great, but given that the price has su…

I'm not sure how, and maybe some of the coding agents are doing this, but we need to teach the AI to use abstractions, rather than the whole code base for context. We as humans don't hold the whole codebase in our hear, and we shouldn't expect the AI to either.

[deleted]

Re: Claude Sonnet 4 now supports 1M tokens of context

#47

Post by anonz4FWNqnX »

aliljet wrote: post #27
rootnod3 wrote: post #20

Earlier quoted context omitted.

Flooding the context also means increasing the likelihood of the LLM confusing itself. Mainly because of the longer context. It derails along the way without a reset.

How do you know that?

I've had similar experiences. I've gone back and forth between running models locally and using the commercial models. The local models can be incredibly useful (gemma, qwen), but they need more patience and work to get them to work.

One advantage to running locally[1] is that you can set the context length manually and see how well the llm uses it. I don't have an exact experience to relay, but it's not unusual for models to be allow longer contexts, but ignore that context.

Just making the context big doesn't mean the LLM is going to use it well.

[1] I've using lm studio on both a macbook air and a macbook pro. Even a macbook air with 16G can run pretty decent models.

Re: Claude Sonnet 4 now supports 1M tokens of context

#48

Post by kvirani »

sdesol wrote: post #42
aliljet wrote: post #16

This is definitely one of my CORE problem as I use these tools for "professional software engineering." I really desperately need LLMs to maintain extremely effective context and it's not actually that interesting to see a new model that's marginally better than the next one (for my day-to-day). However. Price is king. Allowing me to flood the context window with my code base is great, but given that the price has su…

> I really desperately need LLMs to maintain extremely effective context I actually built this. I'm still not ready to say "use the tool yet" but you can learn more about it at https://github.com/gitsense/chat . The demo link is not up yet as I need to finalize an admin tool but you should be able to follow the npm instructions to play around with. The basic idea is, you should be able to load your entire repo or rep…

Wait that's not how Cursor etc work? (I made assumptions)

Re: Claude Sonnet 4 now supports 1M tokens of context

#50

Post by F7F7F7 »

TheDong wrote: post #25
jasonthorsness wrote: post #4

Earlier quoted context omitted.

What do you recommend doing instead? I've been using Claude Code a lot but am still pretty novice at the best practices around this.

Have the AI produce a plan that spans multiple files (like "01 create frontend.md", "02 create backend.md", "03 test frontend and backend running together.md"), and then create a fresh context for each step if it looks like re-using the same context is leading it to confusion. Also, commit frequently, and if the AI constantly goes down the wrong path ("I can't create X so I'll stub it out with Y, we'll fix it later")…

Inventivatbly the files become a mess of their own. Changes and learnings from one part of the plan often dont result in adaptation to impacted plans down chain.

In the end you have a mish mash of half implemented plans and now you’ve lost context too. Which leads to blowing tokens on trying to figure out what’s been implemented, what’s half baked, and what was completely ignored.

Any links to anyone who’s built something at scale using this method? It always sounds good on paper.

I’d love to find a system that works.

Post reply on HN