Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic says Alibaba illicitly extracted Claude AI model capabilities

reuters.com

Post reply on HN
Print view View original

561–570 of 1001 posts

Re: Anthropic says Alibaba illicitly extracted Claude AI model capabilities

#561

Post by monkey_monkey »

agos wrote: post #512
handoflixue wrote: post #419

Earlier quoted context omitted.

> But if you show them a jailbreak of their model that bypasses their safety, they'll tell you that any model can eventually be jailbroken so don't worry about safety. They claim two things: 1) The specific, available jailbreak for Fable 5 is not dangerous - this has been confirmed by multiple experts, and there is no credible evidence against this claim (in other words, Anthropic is probably correct) 2) It is imposs…

I'm pretty sure that Gödel incompleteness theorem and its consequences pretty much guarantee #2

Exactly. It's impossible to guarantee #2 doesn't happen (ie protect against all jailbreaks) for any system of sufficient complexity.

Re: Anthropic says Alibaba illicitly extracted Claude AI model capabilities

#563

Post by whizzter »

Whoa, Antrophic,etc are really running afraid that their IPO's are gonna crash when people realize that the open models are Good Enough(TM).

So I'd put it at 30% that this is a ruse, say that Qwen 3.5,etc is tainted by training by them and start issuing DMCA takedowns to protect the IPO valuation (Or they'll hold off on that, getting a DMCA takedown could backfire spectacularly if others do that to them).

Re: Anthropic says Alibaba illicitly extracted Claude AI model capabilities

#564

Post by nixon_why69 »

JumpCrisscross wrote: post #443
0xbadcafebee wrote: post #11

There's two basic kinds of distillation: 1) the massive [and dumb] method where you ask a question and use the answer as reinforcement (Black Box), and 2) more targeted distillation where you use one model to directly inform/train/guide another model (RLAIF). The latter is basically fine-tuning the model with direction from another model. Thousands of businesses do this every day to fine-tune. This is almost certainl…

> These complaints of distillation are inflating the problem They’re also missing the point. What would have happened to a member of the Manhattan Project who, through personal pursuit of profit, neglected their duty enough to let the bomb leak?

The companies are all for-profit companies, its not like they're selling out some national security goal for profit, profit is the point.

Anthropic already heavily restricts Chinese traffic but that only jams up researchers and regular Joes. Anyone motivated enough can hop a flight to Singapore with an nvme drive in their pocket.

Re: Anthropic says Alibaba illicitly extracted Claude AI model capabilities

#565

Post by gf000 »

Charon77 wrote: post #537
handoflixue wrote: post #419

Earlier quoted context omitted.

> But if you show them a jailbreak of their model that bypasses their safety, they'll tell you that any model can eventually be jailbroken so don't worry about safety. They claim two things: 1) The specific, available jailbreak for Fable 5 is not dangerous - this has been confirmed by multiple experts, and there is no credible evidence against this claim (in other words, Anthropic is probably correct) 2) It is imposs…

> If #2 was false, surely some other LLM lab would have done it by now. This is a logical flaw. LLM that is immune to jailbreak _could_ exist, but not yet, or maybe nobody talks about it. Yes there's a market, but all of these AI boom is too recent to make any claims.

Like how would you even define what a jailbreak is?

Re: Anthropic says Alibaba illicitly extracted Claude AI model capabilities

#566

Post by maccard »

ascorbic wrote: post #553
nicce wrote: post #540

Earlier quoted context omitted.

Yet they did not need to destroy the models which were trained with them?

Using them was allowed as fair use – it was the downloading of the pirated copies that was infringement. That's why Anthropic switched to scanning paper books.

> That's why Anthropic switched to scanning paper books.

After they threw away all the tainted data from the pirated books, right?

Re: Anthropic says Alibaba illicitly extracted Claude AI model capabilities

#567

Post by nicce »

ascorbic wrote: post #553
nicce wrote: post #540

Earlier quoted context omitted.

Yet they did not need to destroy the models which were trained with them?

Using them was allowed as fair use – it was the downloading of the pirated copies that was infringement. That's why Anthropic switched to scanning paper books.

In a different world it is not fair use. The benefits of the crime should be always taken off. If you isolate the training and pirating, you may say that it was fair, but that completely misses the point. The sole purpose of pirating (aka crime) was to train the models.

Re: Anthropic says Alibaba illicitly extracted Claude AI model capabilities

#568

Post by theshackleford »

wilg wrote: post #375
user_7832 wrote: post #369

Earlier quoted context omitted.

Oh, Anthropic, the company that hoover'd up everyone else's data, and is now unhappy when others are doing to it what it did to others? The same Anthropic?

Yes, this joke/point has been made 10,000 times in this thread in almost every comment, and on every other previous thread. Thank you!

And yet it seems to continue to need to be repeated. If the shoe fits and all.

Re: Anthropic says Alibaba illicitly extracted Claude AI model capabilities

#569

Post by trymas »

handoflixue wrote: post #527
trymas wrote: post #437

Earlier quoted context omitted.

> Humans have spent millenia harvesting and distilling each other's IP You maybe somewhat correct, but also copyright lawyers wouldn’t have work if it would be up for grabs to take others IP willy nilly just because “shoulders of giants and all that”.

I mean, there's an obvious difference between "distributing copies" (which is what the law was designed to prevent) and "training an LLM". We already managed "banning LLM output that contains copyrighted text" - it's much easier to just pirate a copy of the text. So I think the copyright lawyers will continue to have work as long as human written texts are worth buying.

> I mean, there's an obvious difference between "distributing copies" (which is what the law was designed to prevent) and "training an LLM".

What's the difference between me/you downloading an mp3 through torrents for personal use (not distributing) while risking criminal punishment in most of the western world and BigCorp downloading petabytes worth of copyrighted works "to train an LLM" and resell it?

Can me/you do the same, when police comes to mine/your door?

"Dear police, don't lock me up - I was just going to train an LLM!"

Re: Anthropic says Alibaba illicitly extracted Claude AI model capabilities

#570

Post by kgeist »

SubiculumCode wrote: post #519

Everyone here praising these Chinese companies for their smarts (sure they are smart) has been ignoring this very big fact, they're improvements have mostly been by being parasitic on the leading edge SOTA models, not from some inherent innovation advantage. They are as innovative as their western counterparts, but they lack the compute, so their keeping up within months of those SOTA models depends on other means, l…

>like distillation attacks. I don't blame them; its the obvious only strategy when you cant compete in compute

>distillation attacks are the only vector to keep up

It's demonstrably wrong, they invest in architectural improvements as well, for example, DeepSeek's compressed attention. When you lack compute, you need fast training/fast inference, and distillation alone doesn't solve it. From what I understand, that kind of distillation "attack" (28 mln exchanges) only slightly improves instruction tuning/reasoning traces. If the base model is crap, distilling Claude on a few million exchanges alone won't magically make your model as good as Chinese models currently are (or magically make inference faster on the limited hardware they have). And training the base model needs a proper training run. Serving users at scale needs optimized architectures as well, especially with test-time compute and ever growing context lengths. That's where architectural innovations are happening in Chinese labs when it comes to compute.

Post reply on HN