Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Opus 4.6

anthropic.com

Post reply on HN
Print view View original

881–890 of 1001 posts

Re: Claude Opus 4.6

#881

Post by soulofmischief »

scwoodal wrote: post #857
soulofmischief wrote: post #854

Earlier quoted context omitted.

I am having trouble with 4.6 following the most basic of instructions. As an example, I asked it to commit everything in the worktree. I stressed everything and prompted it very explicitly, because even 4.5 sometimes likes to say, "I didn't do that other stuff, I'm only going to commit my stuff even though he said everything". It still only committed a few things. I had to ask again. And again. I had to ask four time…

Tell it what git commands to explicitly run and in what order for your desired outcome instead of “commit everything in the worktree” This prompt will work better across any/all models.

I have seen many cases of Claude ignoring extremely specific instructions to the point that any further specificity would take more information to express than just doing it myself.

Re: Claude Opus 4.6

#882

Post by paodealho »

rendx wrote: post #878
sigmoid10 wrote: post #822

Earlier quoted context omitted.

Most people still don't realize that general public world knowledge is not really a test for a model that was trained on general public world knowledge. I wouldn't be surprised if even proprietary content like the books themselves found their way into the training data, despite what publishers and authors may think of that. As a matter of fact, with all the special deals these companies make with publishers, it is ge…

> I wouldn't be surprised if even proprietary content like the books themselves found their way into the training data No need for surprises! It is publicly known that the corpus of 'shadow libraries' such as Library Genesis and Anna's Archive were specifically and manually requested by at least NVIDIA for their training data [1], used by Google in their training [2], downloaded by Meta employees [3] etc. [1] https:/…

also:

"Researchers Extract Nearly Entire Harry Potter Book From Commercial LLMs"

https://www.aitechsuite.com/ai-news/ai-shock-researchers-ext...

Re: Claude Opus 4.6

#883

Post by ehatr »

grey-area wrote: post #818
ck_one wrote: post #482

Just tested the new Opus 4.6 (1M context) on a fun needle-in-a-haystack challenge: finding every spell in all Harry Potter books. All 7 books come to ~1.75M tokens, so they don't quite fit yet. (At this rate of progress, mid-April should do it ) For now you can fit the first 4 books (~733K tokens). Results: Opus 4.6 found 49 out of 50 officially documented spells across those 4 books. The only miss was "Slugulus Eruc…

Surely the corpus Opus 4.6 ingested would include whatever reference you used to check the spells were there. I mean, there are probably dozens of pages on the internet like this: https://www.wizardemporium.com/blog/complete-list-of-harry-p... Why is this impressive? Do you think it's actually ingesting the books and only using those as a reference? Is that how LLMs work at all? It seems more likely it's predicting t…

The poster you reply to works in AI. The marketing strategy is to always have a cute Pelican or Harry Potter comment as the top comment for positive associations.

The poster knows all of that, this is plain marketing.

Re: Claude Opus 4.6

#884

Post by sigmoid10 »

rendx wrote: post #878
sigmoid10 wrote: post #822

Earlier quoted context omitted.

Most people still don't realize that general public world knowledge is not really a test for a model that was trained on general public world knowledge. I wouldn't be surprised if even proprietary content like the books themselves found their way into the training data, despite what publishers and authors may think of that. As a matter of fact, with all the special deals these companies make with publishers, it is ge…

> I wouldn't be surprised if even proprietary content like the books themselves found their way into the training data No need for surprises! It is publicly known that the corpus of 'shadow libraries' such as Library Genesis and Anna's Archive were specifically and manually requested by at least NVIDIA for their training data [1], used by Google in their training [2], downloaded by Meta employees [3] etc. [1] https:/…

The big AI houses are all in involved in varying degrees of litigation (all the way to class action lawsuits) with the big publishing houses. I think they at least have some level of filtering for their training data to keep them legally somewhat compliant. But considering how much copyrighted stuff is spread blisfully online, it is probably not enough to filter out the actual ebooks of certain publishers.

Re: Claude Opus 4.6

#885

Post by zaphirplane »

grey-area wrote: post #818
ck_one wrote: post #482

Just tested the new Opus 4.6 (1M context) on a fun needle-in-a-haystack challenge: finding every spell in all Harry Potter books. All 7 books come to ~1.75M tokens, so they don't quite fit yet. (At this rate of progress, mid-April should do it ) For now you can fit the first 4 books (~733K tokens). Results: Opus 4.6 found 49 out of 50 officially documented spells across those 4 books. The only miss was "Slugulus Eruc…

Surely the corpus Opus 4.6 ingested would include whatever reference you used to check the spells were there. I mean, there are probably dozens of pages on the internet like this: https://www.wizardemporium.com/blog/complete-list-of-harry-p... Why is this impressive? Do you think it's actually ingesting the books and only using those as a reference? Is that how LLMs work at all? It seems more likely it's predicting t…

Why doesn’t you ask it and find out ;)

Re: Claude Opus 4.6

#886

Post by Squarex »

JV00 wrote: post #870
Squarex wrote: post #431

Earlier quoted context omitted.

Claude sucks at non English languages. Gemini and ChatGPT are much better. Grok is the worst. I am a native Czech speaker and Claude makes up words and Grok sometimes respond in Russian. So while I love it for coding, it’s unusable for general purpose for me.

I tried coding in Italian with Claude and it sounds somewhat less professional than in English. Like it uses different language than what you would expect in the context. In the end I felt the result on the work per se was pretty much the same, just his comments sound strange. Thinking about it again, it's probably because Italian developers don't really speak pure Italian between themselves, we use a lot of English…

yeah, i overextrapolated it on my specific case on the czech language, but for me the difference is quite large and the czech internet has been quite active in the history, the computer linguistic department on the charles university is world tier... there is plenty of czech literature. it should not be that much of a problem to be profecient on it for major labs

Re: Claude Opus 4.6

#887

Post by zaphirplane »

legitster wrote: post #72

I'm still not sure I understand Anthropic's general strategy right now. They are doing these broad marketing programs trying to take on ChatGPT for "normies". And yet their bread and butter is still clearly coding. Meanwhile, Claude's general use cases are... fine. For generic research topics, I find that ChatGPT and Gemini run circles around it: in the depth of research, the type of tasks it can handle, and the qual…

Correct me if I’m wrong aren’t they the innovators of multiple things like skills sub agents mcp and whatever this memory thing is agents files

Seriously they are the apple iPhone or AWS of LLM a decade or so ago.

Re: Claude Opus 4.6

#888

Post by throw10920 »

ehatr wrote: post #883
grey-area wrote: post #818

Earlier quoted context omitted.

Surely the corpus Opus 4.6 ingested would include whatever reference you used to check the spells were there. I mean, there are probably dozens of pages on the internet like this: https://www.wizardemporium.com/blog/complete-list-of-harry-p... Why is this impressive? Do you think it's actually ingesting the books and only using those as a reference? Is that how LLMs work at all? It seems more likely it's predicting t…

The poster you reply to works in AI. The marketing strategy is to always have a cute Pelican or Harry Potter comment as the top comment for positive associations. The poster knows all of that, this is plain marketing.

This sounds compelling, but also something that an armchair marketer would have theorycrafted without any real-world experience or evidence that it actually works - and I searched online and can't find any references to something like it.

Do you have a citation for this?

Re: Claude Opus 4.6

#889

Post by ActionHank »

ck_one wrote: post #482

Just tested the new Opus 4.6 (1M context) on a fun needle-in-a-haystack challenge: finding every spell in all Harry Potter books. All 7 books come to ~1.75M tokens, so they don't quite fit yet. (At this rate of progress, mid-April should do it ) For now you can fit the first 4 books (~733K tokens). Results: Opus 4.6 found 49 out of 50 officially documented spells across those 4 books. The only miss was "Slugulus Eruc…

The books were likely in the training data, I don't know that it's that impressive.

Re: Claude Opus 4.6

#890

Post by grey-area »

LanceJones wrote: post #551
ck_one wrote: post #482

Just tested the new Opus 4.6 (1M context) on a fun needle-in-a-haystack challenge: finding every spell in all Harry Potter books. All 7 books come to ~1.75M tokens, so they don't quite fit yet. (At this rate of progress, mid-April should do it ) For now you can fit the first 4 books (~733K tokens). Results: Opus 4.6 found 49 out of 50 officially documented spells across those 4 books. The only miss was "Slugulus Eruc…

Assuming this experiment involved isolating the LLM from its training set?

Of course it didn't. Not sure you really can do that - LLMs are a collection of weights from the training set, take away the training set and they don't really exist. You'd have to train one from scratch excluding these books and all excerpts and articles about them somehow, which would be very expensive and I'm pretty sure the OP didn't do that.

So the test seems like a nonsensical test to me.

Post reply on HN