In an other news, a terrorist organization practicing torture at daily level just released a public denunciation of the evil forces they are fighting against, guided by their holy mission of making progress in social morality for all of us.
Anthropic says Alibaba illicitly extracted Claude AI model capabilities
611–620 of 1001 posts
Re: Anthropic says Alibaba illicitly extracted Claude AI model capabilities
#612Earlier quoted context omitted.
It's too late to prevent distillation of some capabilities, like writing code or finding vulnerabilities [1]. But an AI lab can continue to produce immense economic value without releasing the model publicly for potential distillation. For example, it could use a model solely in-house to develop therapeutics. Hopefully there's a future where others can access frontier models, but it's not neccessary if preventing pro…
My long-term prediction for the sector is that frontier models will be so expensive that they will only be available for grant-funded projects at research institutions, like supercomputer clusters were 25 years ago.
Why? Well it depends, most evidence is suggesting that Anthropic and OpenAI are making a lot of money on inference so the question is whether its more profitable for them to sell 100X tokens for Y, or 1X tokens for 100Y. In most industries with high fixed costs and low variable costs and unlimited scalability (like LLM providers) the first option ends up being much more profitable
Re: Anthropic says Alibaba illicitly extracted Claude AI model capabilities
#613I'll play the world's smallest violin for Dario
Re: Anthropic says Alibaba illicitly extracted Claude AI model capabilities
#614Reminds me a bit of the anecdote of Steve Jobs complaining about people ripping off the Mac GUI, in the mid to late 1980s, when he gave no public acknowledgement to the work done by Xerox on the Alto and Star operating system. "you're trying to rip off what I've already ripped off!" Crawl the whole Internet to build a gargantuan sized LLM and then complain you're being copied...
I think you meant a quote attributed to Bill Gates: "Well, Steve, I think there's more than one way of looking at it. I think it's more like we both had this rich neighbor named Xerox and I broke into his house to steal the TV set and found out that you had already stolen it."
I thought Xerox demoed something they haven’t implemented yet, and Apple turned a mockup into a real GUI.
Re: Anthropic says Alibaba illicitly extracted Claude AI model capabilities
#615Everyone here praising these Chinese companies for their smarts (sure they are smart) has been ignoring this very big fact, they're improvements have mostly been by being parasitic on the leading edge SOTA models, not from some inherent innovation advantage. They are as innovative as their western counterparts, but they lack the compute, so their keeping up within months of those SOTA models depends on other means, l…
Haven't we all been parasitic towards china for the last half century though? They were our source of cheap labor and they got out of being the ones being used by playing everybody and stealing knowledge.
This all feels like everybody is playing everybody dirty.
Re: Anthropic says Alibaba illicitly extracted Claude AI model capabilities
#616Earlier quoted context omitted.
Using them was allowed as fair use – it was the downloading of the pirated copies that was infringement. That's why Anthropic switched to scanning paper books.
> That's why Anthropic switched to scanning paper books. Could they not just subscribe to the academic publishers like universities do? Or buy eBooks? I don't understand how the "scanning" part is relevant here other than used physical books being cheaper perhaps?
Bulk second-hand books are a lot cheaper than ebooks. Also not all books are available as ebooks, and ebooks have terms of service that presumably prevent them being used for training.
Re: Anthropic says Alibaba illicitly extracted Claude AI model capabilities
#617What exactly is illicit about what they did? Legally, model output cannot be protected by IP laws whether domestic or international. The most they can hope for is civil relief which is a stretch given the literally illicit methods they used to train their models. Ahtoropic got treated the same way it has been treating everyone else. This is the bed they made and now they, too, have to sleep in it.
Anthropic is master of Newspeak (see previously bugs -> vulnerabilities wrt Mythos). Distillation violates their terms of service, which is a civil offense, not a criminal one. It is not illicit, illegal nor breaks any laws.
Re: Anthropic says Alibaba illicitly extracted Claude AI model capabilities
#618Earlier quoted context omitted.
Using them was allowed as fair use – it was the downloading of the pirated copies that was infringement. That's why Anthropic switched to scanning paper books.
If using the books is fair use, then distilling the model, which is just a derived product of those books is also fair use. These companies are trying to have their cake and eat it too.
Probably, yes. It's likely just a breach in their terms of service. You'll note that they're not suing them – they're trying to get the government to do their work for them.
Re: Anthropic says Alibaba illicitly extracted Claude AI model capabilities
#619Earlier quoted context omitted.
> These complaints of distillation are inflating the problem to make it sound worse than it is Unfortunately, the Reuters piece itself is complicit in this dramatization. The lede paragraph parrots Anthropic's talking point that distillation is an "attack", without using quotes that would alert the reader that this framing is a corporate talking point. Distillation is NOT an attack.
Reuters is probably the most rigorous news agency in the world. > it said was the largest known attack > Anthropic said in the letter it was supportive of the U.S. government's efforts to combat the attacks both times the word "attack" appears it's clearly stated that the word was used by the company, it's a direct company quote. actually putting it into quotes would be editorializing > Unfortunately, the Reuters pie…
Well, let’s say you put the picture of some political figure, and put in highly contrasted red, bold large catchy font, "TERRORIST THAT KILLED MILLION PEOPLE", then below that in barely visible contrast, in tiny discrete letters, "is what this person probably will claim to be against".
This whole sentence technically will be correct, 100% guarantee, whatever this person actually even said or think.
From a propaganda point of view, framing the elements of language is even more important than what the statements actually states to be true or possibly true.
Re: Anthropic says Alibaba illicitly extracted Claude AI model capabilities
#620Earlier quoted context omitted.
My long-term prediction for the sector is that frontier models will be so expensive that they will only be available for grant-funded projects at research institutions, like supercomputer clusters were 25 years ago.
Why? Well it depends, most evidence is suggesting that Anthropic and OpenAI are making a lot of money on inference so the question is whether its more profitable for them to sell 100X tokens for Y, or 1X tokens for 100Y. In most industries with high fixed costs and low variable costs and unlimited scalability (like LLM providers) the first option ends up being much more profitable
Literally nobody is making money on inference