Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Sonnet 4 now supports 1M tokens of context

anthropic.com

Post reply on HN
Print view View original

691–700 of 706 posts

Re: Claude Sonnet 4 now supports 1M tokens of context

#691

Post by adastra22 »

HarHarVeryFunny wrote: post #690
adastra22 wrote: post #688

Earlier quoted context omitted.

No, a YouTube video won’t convince me over repeated, verified lab experiments.

So what do you make of the video - do you think it's fake, or are you just making the distinction between eidetic memory and photographic memory? There are so many well documented cases of idiot savants with insane memory skills in various areas (books, music, dates/events, etc), that this type of snapshot visual memory (whatever you want to call it) doesn't seem surprising in that context - it'd really be a bit odd…

I do not watch YouTube, sorry.

Hearsay is not reliable. Yes there are stories of savants. When you put them in a lab and actually see how good their memory is, it turns out to be roughly the same as everyone else's. The stories aren't true.

(They may be better at remembering weird facts or something, but when you actually calculate the information entropy of what they are remembering, it ends up being within the ballpark of what a neurotypical person remembers across of general span of life. That's why these people are idiot savants (to use your term). They allocate all their memory points to weird trivia and none to everyday common knowledge.

Re: Claude Sonnet 4 now supports 1M tokens of context

#692

Post by menaerus »

jimbokun wrote: post #660
scott_s wrote: post #644

Earlier quoted context omitted.

> Of course, because I am not new to the problem, whereas an LLM is new to it every new prompt. That is true for the LLMs you have access to now. Now imagine if the LLM had been trained on your entire code base. And not just the code, but the entire commit history, commit messages and also all of your external design docs. And code and docs from all relevant projects. That LLM would not be new to the problem every pr…

Why haven’t the bug AI companies been pursuing that approach, vs just ramping up context window size?

Well, we don't really know if they aren't doing exactly that for their internal code repos, right?

Conceptually, there is no difference between fine-tuning the LLM for being a law expert of specific country and fine-tuning the LLM for being an expert for given codebase. Former is already happening and is public. Latter is not yet public but I believe it is happening.

Reason why big co are pursuing generic LLMs is because they serve as a foundation for basically any other derivative and domain-specific work.

Re: Claude Sonnet 4 now supports 1M tokens of context

#693

Post by viccis »

pvorb wrote: post #449
viccis wrote: post #359

Earlier quoted context omitted.

I agree. For me it's a modern version of that good ol "rails new" scaffolding with Ruby on Rails that got you started with a project structure. It makes sense because LLMs are particularly good at tasks that require little more knowledge than just a near perfect knowledge of the documentation of the tooling involved, and creating a well organized scaffold for a greenfield project falls squarely in that area. For lega…

I'm wondering what exact issue you are referring to with Databricks? I can't remember a time I had to change a line I wrote during the past 2.5 years I've been using it. Or are you talking about non-breaking changes?

They have changed a lot of their DLT (not even called that anymore lol, it's Lakeflow Pipelines now I think) syntax. I tried asking ChatGPT to convert a very simple Python one to Spark SQL, and it gave me a bunch of outdated SQL syntax.

Aside from that, if you use their Python connector package, it's a shit show to put it mildly. For example, 15.4 works with serverless but tells you (via deprecation warning) it doesn't and that you need to use 15.1 (which lacks a lot of variant stuff). So then you decide screw it I'm gonna just update to 16, except that serverless (which works on 15.4) doesn't work on 16.0 or 17.0.

Re: Claude Sonnet 4 now supports 1M tokens of context

#694

Post by menaerus »

m4r71n wrote: post #307
brokegrammer wrote: post #232

Many people are confused about the usefulness of 1M tokens because LLMs often start to get confused after about 100k. But this is big for Claude 4 because it uses automatic RAG when the context becomes large. With optimized retrieval thanks to RAG, we'll be able to make good use of those 1M tokens.

How does this work under the hood? Does it build an in-memory vector database of the input sources and runs queries on top of that data to supplement the context window?

RAG commonly implies some sort of vector database to be built and which will then be used for response augmentation. If it operates over the repo, I believe it will index your codebase using those vector embeddings.

Re: Claude Sonnet 4 now supports 1M tokens of context

#695

Post by HarHarVeryFunny »

adastra22 wrote: post #691
HarHarVeryFunny wrote: post #690

Earlier quoted context omitted.

So what do you make of the video - do you think it's fake, or are you just making the distinction between eidetic memory and photographic memory? There are so many well documented cases of idiot savants with insane memory skills in various areas (books, music, dates/events, etc), that this type of snapshot visual memory (whatever you want to call it) doesn't seem surprising in that context - it'd really be a bit odd…

I do not watch YouTube, sorry. Hearsay is not reliable. Yes there are stories of savants. When you put them in a lab and actually see how good their memory is, it turns out to be roughly the same as everyone else's. The stories aren't true. (They may be better at remembering weird facts or something, but when you actually calculate the information entropy of what they are remembering, it ends up being within the ball…

If you refuse to look at evidence, then your opinion isn't worth much, is it?

Re: Claude Sonnet 4 now supports 1M tokens of context

#696

Post by HarHarVeryFunny »

adastra22 wrote: post #691
HarHarVeryFunny wrote: post #690

Earlier quoted context omitted.

So what do you make of the video - do you think it's fake, or are you just making the distinction between eidetic memory and photographic memory? There are so many well documented cases of idiot savants with insane memory skills in various areas (books, music, dates/events, etc), that this type of snapshot visual memory (whatever you want to call it) doesn't seem surprising in that context - it'd really be a bit odd…

I do not watch YouTube, sorry. Hearsay is not reliable. Yes there are stories of savants. When you put them in a lab and actually see how good their memory is, it turns out to be roughly the same as everyone else's. The stories aren't true. (They may be better at remembering weird facts or something, but when you actually calculate the information entropy of what they are remembering, it ends up being within the ball…

> They allocate all their memory points to weird trivia and none to everyday common knowledge.

I think it's more complex than that - it's they way they are forming memories (i.e. what they remember) that is different to a normal person. In a normal person surprise/novelty (prediction failure) is the major learning signal that causes us to remember something - we're selective in what gets remembered (this is just mechanically how a normally operating brain works), whereas the savant appears to remember everything in certain modalities.

I don't think that "using up all their memory" is why savants are "idiots", but rather just a reflection of something more severe that is wrong.

Re: Claude Sonnet 4 now supports 1M tokens of context

#697

Post by snowfield »

scott_s wrote: post #644
ehnto wrote: post #627

Earlier quoted context omitted.

> If you're completely new to the problem then ... yes, it does. Of course, because I am not new to the problem, whereas an LLM is new to it every new prompt. I am not really trying to find a fair comparison because I believe humans have an unfair advantage in this instance, and am trying to make that point, rather than compare like for like abilities. I think we'll find even with all the context clues from MCPs and…

> Of course, because I am not new to the problem, whereas an LLM is new to it every new prompt. That is true for the LLMs you have access to now. Now imagine if the LLM had been trained on your entire code base. And not just the code, but the entire commit history, commit messages and also all of your external design docs. And code and docs from all relevant projects. That LLM would not be new to the problem every pr…

AI training doesn't work like that. you don't train it on context, you train it on recognition and patterns.

Re: Claude Sonnet 4 now supports 1M tokens of context

#698

Post by scott_s »

snowfield wrote: post #697
scott_s wrote: post #644

Earlier quoted context omitted.

> Of course, because I am not new to the problem, whereas an LLM is new to it every new prompt. That is true for the LLMs you have access to now. Now imagine if the LLM had been trained on your entire code base. And not just the code, but the entire commit history, commit messages and also all of your external design docs. And code and docs from all relevant projects. That LLM would not be new to the problem every pr…

AI training doesn't work like that. you don't train it on context, you train it on recognition and patterns.

You train on data. Context is also data. If you want a model to have certain data, you can bake it into the model during training, or provide it as context during inference. But if the "context" you want the model to have is big enough, you're going to want to train (or fine-tune) on it.

Consider that you're coding a Linux device driver. If you ask for help from an LLM that has never seen the Linux kernel code, has never seen a Linux device driver and has never seen all of the documentation from the Linux kernel, you're going to need to provide all of this as context. And that's both going to be onerous on you, and it might not be feasible. But if the LLM has already seen all of that during training, you don't need to provide it as context. Your context may be as simple as "I am coding a Linux device driver" and show it some of your code.

Re: Claude Sonnet 4 now supports 1M tokens of context

#699

Post by jama211 »

abc_lisper wrote: post #363
jama211 wrote: post #272

Earlier quoted context omitted.

I dunno, first time I tried an LLM I was getting so annoyed because I just wanted it to go through a css file and replace all colours with variables defined in root, and it kept missing stuff and spinning and I was getting so frustrated. Then a friend told me I should instead just ask it to write a script which accomplishes that goal, and it did it perfectly in one prompt, then ran it for me, and also wrote another s…

Which one did you use and when was this? I mean, no body gets anything working right the first time. You got to spend a few days atleast trying to understand the tool

It’s just a simple example of how knowing how to use a tool can make all the difference, and that can be improved upon with time. I’m not sure why you’re taking umbrage with that idea.

I know this style of arguing you’re going for. If I answer your questions, you’ll attack the specific model or use case I was in, or claim it was too simple/basic a use case, or some other nitpick about the specifics instead of in good faith attempting to take my point as stated. I won’t allow you to force control of the frame of the conversation by answering your questions, also because the answers wouldn’t do anything to change the spirit of my main point.

Re: Claude Sonnet 4 now supports 1M tokens of context

#700

Post by jama211 »

parpfish wrote: post #295
jama211 wrote: post #266

Earlier quoted context omitted.

Yeah sometimes I feel like a unicorn because I don’t really care about code at all, so long as it conforms to decent standards and does what it needs to do. I honestly believe engineers often overestimate the importance of elegance in code too, to the point of not realising the slow down of a project due to overly perfect code is genuinely not worth it.

i dont care if the code is elegant, i care that the code is consistent . do the same thing in the same way each time and it lets you chunk it up and skim it much easier. if there are little differences each time, you have to keep asking yourself "is it done differently here for a particular reason?"

Absolutely fair, and a great method.
Post reply on HN