741–750 of 1001 posts
Earlier quoted context omitted.
If saying “plz don’t distill me” is your moat, you don’t have a moat.
No. What will happen is it will turn dark. No public release. National Security uses only, or in carefully vetted industry settings.
Earlier quoted context omitted.
>This is great for competition! Chinese vendors offering a cheaper solution = what economics told me the free market was all about. Yeah, like all those Chinese bootleggers selling DVDs for a few dollars rather than $20. Free market! https://news.ycombinator.com/item?id=48664814
It's quite curious how multi billion dollar enterprises can't compete with a Chinese bootlegger with a big jacket, tbh. Imagine having such a warchest and being so bad at business, lol.
Earlier quoted context omitted.
It's not about how big your dataset is - it's about how you use it. I jest, but I'm also completely serious. 1T tokens from Claude can teach a model something 1T tokens scraped from the open web can't. Things like "how an LLM can problem solve effectively", or "how an LLM should use tools", or "how to construct reasoning chains", or "when to double check", or "what innate capabilities an LLM can or can't rely on". Th…
Can you back up this with hard data and evidence? Most research converges to the idea that RL on synthetic data makes models worse, not better. If what you claim was anywhere near that relevant, than we would've long achieved singularity by simply feeding increasingly better output to the training of the next model in a loop. Yet this doesn't work. 25 million turns on Claude output is a small amount, yet an expensive…
You are missing a mountain of nuance by generalizing the existence of a hole there.
Earlier quoted context omitted.
Glad you pointed this out. I believe the sequence was that Jobs himself got a shorter demo during his first visit with no prior arrangements. He then negotiated bringing back a group of his key people to get a more in depth demo and that included the stock deal. When Apple was accused of 'ripping off' PARC, Steve didn't seem keen to bring up this rather salient point. I suspect it may have been a combination of wanti…
> the million dollar stock deal could seem a bit like trading beads to Native Americans for Manhattan Island But in both cases the value only existed because of the people offering the deal. XeroX doing nothing with a UI or native Americans doing nothing with some land would mean the UI and the land would continue to be worth nothing. It was the others coming with ideas and effort that made them valuable.
You just reveal your own ignorance by equivocating value with monetary value.
There's two basic kinds of distillation: 1) the massive [and dumb] method where you ask a question and use the answer as reinforcement (Black Box), and 2) more targeted distillation where you use one model to directly inform/train/guide another model (RLAIF). The latter is basically fine-tuning the model with direction from another model. Thousands of businesses do this every day to fine-tune. This is almost certainl…
Just for the sake of clarity:
0. Full distillation uses logits of the teacher model - that's much more information than the text itself. This is a kind of distillation used inside labs, but one can't distill Claude this way as logits are not available via API.
1. Supervised fine-tuning on synthetic data might be called blackbox distillation. I guess that's what you meant in your case (1).
2. Reinforcement learning (like RLAIF) uses least amount of information from the teacher, i.e. only few bits per task.
Earlier quoted context omitted.
Anthropic raped everyone without asking and stole their labor to build their career-commoditizing tech. Distillation is Robin Hooding it back so that one trillion dollar company doesn't reap all the benefits of their automation of the workforce. Distillation is Prometheus bringing fire from the gods to give to ordinary humans. Something we all own anyway, but that was kept from us. Distillation is freedom. Everyone s…
Eaaaaasy now, the Chinese labs aren't freedom fighters on behalf the common man. They're not non-profits, they're not neutral transnational organizations only dedicated to open source efforts. They're Chinese companies offering open source models now as loss leaders to keep themselves in the game because they know virtually nobody, especially in the corporate world, would contract with them and give them access to th…
There's two basic kinds of distillation: 1) the massive [and dumb] method where you ask a question and use the answer as reinforcement (Black Box), and 2) more targeted distillation where you use one model to directly inform/train/guide another model (RLAIF). The latter is basically fine-tuning the model with direction from another model. Thousands of businesses do this every day to fine-tune. This is almost certainl…
https://huggingface.co/Jackrong/Qwen3.5-27B-Claude-4.6-Opus-...
Earlier quoted context omitted.
> These complaints of distillation are inflating the problem to make it sound worse than it is Unfortunately, the Reuters piece itself is complicit in this dramatization. The lede paragraph parrots Anthropic's talking point that distillation is an "attack", without using quotes that would alert the reader that this framing is a corporate talking point. Distillation is NOT an attack.
The standard of neutrality that people here pretend to require from news organizations is not even remotely realistic. It was a timely story from Reuters. They do fast news feeds, like APnews. Could it have been better or more accurate? Sure, they could have gone into why distillation may or may not be seen as "an attack". But then it would have been a more involved story, defeating the purpose of a news feed. The Re…
Until very recently, all of modern civilization was built by people who got their news at most once a day. Reputable bureaus like Reuters took that day to get it right.
I’m not the national security advisor, so I don’t need a push notification that there was an earthquake in Nepal, or a bullshit rush-job briefing on Chinese AI distillation tactics.
There's two basic kinds of distillation: 1) the massive [and dumb] method where you ask a question and use the answer as reinforcement (Black Box), and 2) more targeted distillation where you use one model to directly inform/train/guide another model (RLAIF). The latter is basically fine-tuning the model with direction from another model. Thousands of businesses do this every day to fine-tune. This is almost certainl…
Why can't OSS software rival closed source software? It should be an open market, at least "somewhat", what's happening for real? EU providers will also get banned, if they reach or exceed US model capabilties?
Closed source providers can close your account at a whim like and destroy your business and then use the data you supplied them to create a competitor (Meta, Google, OpenAI, Anthrophic).