Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Sonnet 4 now supports 1M tokens of context

anthropic.com

Post reply on HN
Print view View original

211–220 of 706 posts

Re: Claude Sonnet 4 now supports 1M tokens of context

#211

Post by mamp »

giancarlostoro wrote: post #136
aliljet wrote: post #27

Earlier quoted context omitted.

How do you know that?

Here's a paper from MIT that covers how this could be resolved in an interesting fashion: https://hanlab.mit.edu/blog/streamingllm The AI field is reusing existing CS concepts for AI that we never had hardware for, and now these people are learning how applied Software Engineering can make their theoretical models more efficient. It's kind of funny, I've seen this in tech over and over. People discover new thing, the…

Unfortunately, I think the context rot paper [1] found that the performance degradation when context increased still occurred in models using attention sinks.

1. https://research.trychroma.com/context-rot

Re: Claude Sonnet 4 now supports 1M tokens of context

#212

Post by photon_lines »

sdesol wrote: post #137
photon_lines wrote: post #110

Earlier quoted context omitted.

Sorry -- I keep seeing this being used but I'm not entirely sure how it differs from most of human thinking. Most human 'reasoning' is probabilistic as well and we rely on 'associative' networks to ingest information. In a similar manner - LLMs use association as well -- and not only that, but they are capable of figuring out patterns based on examples (just like humans are) -- read this paper for context: https://ar…

I'm not sure if I would say human reasoning is 'probabilistic' unless you are taking a very far step back and saying based on how the person lived, they have ingrained biases (weights) that dictates how they reason. I don't know if LLMs have a built in scepticism like humans do, that plays a significant role in reasoning. Regardless if you believe LLMs are probabilistic or not, I think what we are both saying is cont…

'I don't know if LLMs have a built in scepticism like humans do' - humans don't have an 'in built skepticism' -- we learn in through experience and through being taught how to 'reason' within school (and it takes a very long time to do this). You believe that this is in-grained but you may have forgotten having to slog through most of how the world works and being tested when you went to school and when your parents taught you these things. On the context component: yes, context is vitally important (just as it is with humans) -- you can't produce a great solution unless you understand the 'why' behind it and how the current solution works so I 100% agree with that.

Re: Claude Sonnet 4 now supports 1M tokens of context

#213

Post by tombot »

GodelNumbering wrote: post #201
verall wrote: post #151

Earlier quoted context omitted.

> This week, I used it to write ESP32 firmware and a Linux kernel driver. I'm not meaning to be negative at all, but was this for a toy/hobby or for a commercial project? I find that LLMs do very well on small greenfield toy/hobby projects but basically fall over when brought into commercial projects that often have bespoke requirements and standards (i.e. has to cross compile on qcc, comply with autosar, in-house bu…

This is my experience too. Also, their propensity to jump into code without necessarily understanding the requirement is annoying to say the least. As the project complexity grows, you find yourself writing longer and longer instructions just to guardrail. Another rather interesting thing is that they tend to gravitate towards sweep the errors under the rug kind of coding which is disastrous. e.g. "return X if we don…

> their propensity to jump into code without necessarily understanding the requirement is annoying to say the least.

Then don't let it, collaborate on the spec, ask Claude to make a plan. You'll get far better results

https://www.anthropic.com/engineering/claude-code-best-pract...

Re: Claude Sonnet 4 now supports 1M tokens of context

#216

Post by tombot »

shmoogy wrote: post #188
blitztime wrote: post #146

Earlier quoted context omitted.

How do you keep the context.md updated as the code changes?

I tell Claude to update it generally but you can probably use a hook

This, while it has context of the current problem, just ask Claude to re-read it's own documentation and think of things to add that will help it in the future

Re: Claude Sonnet 4 now supports 1M tokens of context

#217

Post by joduplessis »

As far as coding goes Claude seems to be the most competent right now, I like it. GPT5 is abysmal - I'm not sure if they're bugs, or what, but the new release takes a good few steps back. Gemini still a hit and miss - and Grok seems to be a poor man's Claude (where code is kind of okay, a bit buggy and somehow similar to Claude).

Re: Claude Sonnet 4 now supports 1M tokens of context

#218

Post by GodelNumbering »

oceanplexian wrote: post #161
verall wrote: post #151

Earlier quoted context omitted.

> This week, I used it to write ESP32 firmware and a Linux kernel driver. I'm not meaning to be negative at all, but was this for a toy/hobby or for a commercial project? I find that LLMs do very well on small greenfield toy/hobby projects but basically fall over when brought into commercial projects that often have bespoke requirements and standards (i.e. has to cross compile on qcc, comply with autosar, in-house bu…

I work in FAANG, have been for over a decade. These tools are creating a huge amount of value, starting with Copilot but now with tools like Claude Code and Cursor. The people doing so don’t have a lot of time to comment about it on HN since we’re busy building things.

I don't see how FAANG is relevant here. But the 'FAANG' I used to work at had an emergent problem of people throwing a lot of half baked 'AI-powered' code over the wall and let reviewers deal with it (due to incentives, not that they were malicious). In orgs like infra where everything needs to be reviewed carefully, this is purely a burden

Re: Claude Sonnet 4 now supports 1M tokens of context

#219

Post by sdesol »

nomel wrote: post #209
sdesol wrote: post #72

Earlier quoted context omitted.

LLMs (current implementation) are probabilistic so it really needs the actual code to predict the most likely next tokens. Now loading the whole code base can be a problem in itself, since other files may negatively affect the next token.

No, it doesn’t, nor do we. It’s why abstractions and documentations exist. If you know what a function achieves, and you trust it to do that, you don’t need to see/hold its exact implementation in your head.

But documentation doesn't include styling or preferred pattern, which is why I think a lot people complain that the LLM will just produce garbage. Also documentation is not guaranteed to be correct or up to date. To be able to produce the best code based on what you are hoping for, I do think having the actual code is necessary unless styling/design patterns are not important, then yes documentation will be suffice, provided they are accurate and up to date.

Re: Claude Sonnet 4 now supports 1M tokens of context

#220

Post by novaleaf »

PickledJesus wrote: post #181
novaleaf wrote: post #149

Earlier quoted context omitted.

where do you see it in CC?

I got a notification when I opened it, indicating that the default had changed, and I can see it on /model. Only on a max (20x) account, not there on a Pro one.

thanks, FYI I'm on a max 20x also and I don't see it!
Post reply on HN