781–790 of 1001 posts
“Distillation attack” are we joking here. If anything these models should be compelled to be public since they have been trained off public data. What an absurd overreach to call this an attack. It’s clear they are scapegoating national security and China at this point to build an anti-competitive moat. I generally really like Anthropic’s work and models but stuff like this scares me for the future. We are positionin…
“Distillation attack” are we joking here. If anything these models should be compelled to be public since they have been trained off public data. What an absurd overreach to call this an attack. It’s clear they are scapegoating national security and China at this point to build an anti-competitive moat. I generally really like Anthropic’s work and models but stuff like this scares me for the future. We are positionin…
Two wrongs don't make a right
Everyone in AI industry wants to fight dirty, but gets angry when their competitor fights dirty as well. And I’ve mentioned it before, how I generally like Ant and its products.
When bots open the same board 1 million times per day it is web scraping to train the AI model and OK. When someone asks 150 thousand questions it is now distilling.
On an unrleated note, 150k qieries feels like nothing?
Scrapers seem to account for 50% total internet trafic.
Do they use different methodology since it is suddenly bad when scraping happens to them?
For all the complaints about Anthropic many of you still give them your money! Stop using it. I don't care if they claim they are the best model. I stopped paying OpenAI and Anthropic 2 years ago once they started going for regulatory capture! They started whining once Llama3 was released and was good! Before the chinese models got strong.
There's two basic kinds of distillation: 1) the massive [and dumb] method where you ask a question and use the answer as reinforcement (Black Box), and 2) more targeted distillation where you use one model to directly inform/train/guide another model (RLAIF). The latter is basically fine-tuning the model with direction from another model. Thousands of businesses do this every day to fine-tune. This is almost certainl…
> These complaints of distillation are inflating the problem to make it sound worse than it is Unfortunately, the Reuters piece itself is complicit in this dramatization. The lede paragraph parrots Anthropic's talking point that distillation is an "attack", without using quotes that would alert the reader that this framing is a corporate talking point. Distillation is NOT an attack.
From the article -
> 28.8 million exchanges with Claude through almost 25,000 fraudulent accounts
wouldn't that be considered an attack? Not sure what I'm missing here.
Earlier quoted context omitted.
If saying “plz don’t distill me” is your moat, you don’t have a moat.
No. What will happen is it will turn dark. No public release. National Security uses only, or in carefully vetted industry settings.
Earlier quoted context omitted.
> These complaints of distillation are inflating the problem to make it sound worse than it is Unfortunately, the Reuters piece itself is complicit in this dramatization. The lede paragraph parrots Anthropic's talking point that distillation is an "attack", without using quotes that would alert the reader that this framing is a corporate talking point. Distillation is NOT an attack.
> Distillation is NOT an attack. From the article - > 28.8 million exchanges with Claude through almost 25,000 fraudulent accounts wouldn't that be considered an attack? Not sure what I'm missing here.
Earlier quoted context omitted.
> These complaints of distillation are inflating the problem to make it sound worse than it is Unfortunately, the Reuters piece itself is complicit in this dramatization. The lede paragraph parrots Anthropic's talking point that distillation is an "attack", without using quotes that would alert the reader that this framing is a corporate talking point. Distillation is NOT an attack.
> Distillation is NOT an attack. From the article - > 28.8 million exchanges with Claude through almost 25,000 fraudulent accounts wouldn't that be considered an attack? Not sure what I'm missing here.
Pot calling kettle black.
Earlier quoted context omitted.
Byproduct? It’s essentially the only part of an LLM that is useful, because it’s the WHOLE product! It’s the same reason why DRM for audio and video is a non sequitur - if you want a person to see or hear audio or video, eventually at the end of the chain, it’s going to be converted to audio for the ear and light for the eyes - that’s why you attach your tap. Without a model generating tokens, what’s the point. So if…
That's why the harness is moving server-side: because generating tokens is not the actual point of the model, not for the users. Especially with tool calling giving us agents that can act, most of the tokens generated are not, themselves, critical to the end users. Specifically, a lot of tokens goes into orchestrating actual tool calls, and then most "thinking tokens" are only relevant to users only in so far as they…
Earlier quoted context omitted.
> These complaints of distillation are inflating the problem to make it sound worse than it is Unfortunately, the Reuters piece itself is complicit in this dramatization. The lede paragraph parrots Anthropic's talking point that distillation is an "attack", without using quotes that would alert the reader that this framing is a corporate talking point. Distillation is NOT an attack.
> Distillation is NOT an attack. From the article - > 28.8 million exchanges with Claude through almost 25,000 fraudulent accounts wouldn't that be considered an attack? Not sure what I'm missing here.