Regardless of whether they achieved parity via distillation, or whether they got here via independently constructing a model from scratch, it was always going to end this way for the frontier American labs. Distillation “attacks” are not attacks. The frontier labs “distilled” all existing human written knowledge into their models, there was always going to be a second class lab that would distill that model into a ch…
Almost all markets depend on some form of regulation whether its as simple as "leave everyone alone but no stealing" or "every participant has to source every object through mountains of red tape." Thus far the US has not really chosen to go the Chinese rare-earth method yet. The problem with distillation attacks is the end result is everyone who is not doing them is going to deal with some kind of regulation whether…
The Kimi K3 Moment
161–170 of 644 posts
Re: The Kimi K3 Moment
#162Kimi K3 is really good, but it’s obviously worse than Fable, usually worse than Opus, in my experience.
Re: The Kimi K3 Moment
#163I never truly understood what the intended business model around LLMs was. Get them widespread through cheap pricing and then jacking it up? Being the only ones that had a viable product so to get the ability to extract as much value as you want from AI? I don't understand how a product that: - is interfaced with and is deeply linked to natural language, so everything you produce (sessions, history, etc) is in Markdo…
The valuation is based on one lab getting a decisive first advantage, and turning that into a durable self-improving advantage that can never be caught up to. If any can pull it off (a gigantic if), they will effectively own most AI value, and the people who own their shares will live happily ever after. Divide your investment between the labs that could plausibly do this, and your EV may not be dreadful.
Re: The Kimi K3 Moment
#164Earlier quoted context omitted.
Is this some form of rage bait? 2005 we hadn't the GPUs, we have today. There are other factors, but I think this is the big one. The mathematics of building an LLM are really old, we just hadn't the hardware to do the needed calculations.
Right. Therefor it's not simply a derivative of information. The hardware is required to build the model. Software as well. The model uses information, it is not "distilled" from it. "Distillation" literally means to separate and take some components out of something. You can distill how a model works from a model. You cant distill a model from information because the information does not contain the model. People ar…
Re: The Kimi K3 Moment
#165Earlier quoted context omitted.
How is distillation an "attack" but gigascraping the Internet to the point of crashing servers and everyone needs Cloudflare and Anubis now not an "attack"? I'm not aiming for a what about kickflip here: I'm saying we need to either agree on some rules or stop crying foul. Maybe the coherent legal theory is that neural networks and intellectual property don't interact. That would be weird but it would be consistent,…
I think you've basically got the legal theory. Training a neural network isn't prohibited by copyright law so if you can legally get your hands on something (e.g. by sending a GET request to someone with rights to serve the contents of their web page, or by buying a book) without signing a contract to not train on it, you can train on it. But the American AI companies only let you query their models if you first sign…
Maybe today. I doubt it tomorrow. Legal and not legal, largely, has to answer to the population sooner or later. Ultimately, humanity decides legality. And I don't think the frontier labs will get a pass from humanity in the midterm, let alone the long term. I think you'll see the rules change towards something more "intent" driven. And there's absolutely no difference in intent between Frontier labs and everyone chasing them.
Frontier labs just want the door closed behind them, as do their investors, because they know the money will never be recouped if others can do the same magic tricks.
Re: The Kimi K3 Moment
#166Earlier quoted context omitted.
> western governments Are you talking about the US, specifically? Why would other countries, that don't share the same anxiety about China as the US, would be troubled with the this?
Well, many EU countries like Italy and Germany officially freaked out about DeepSeek, ordering it to be banned from app stores etc.
It has nothing to do with running open models, especially in hardware within Europe.
Re: The Kimi K3 Moment
#167Re: The Kimi K3 Moment
#168Earlier quoted context omitted.
> western governments Are you talking about the US, specifically? Why would other countries, that don't share the same anxiety about China as the US, would be troubled with the this?
The US might pressure them?
Re: The Kimi K3 Moment
#169Regardless of whether they achieved parity via distillation, or whether they got here via independently constructing a model from scratch, it was always going to end this way for the frontier American labs. Distillation “attacks” are not attacks. The frontier labs “distilled” all existing human written knowledge into their models, there was always going to be a second class lab that would distill that model into a ch…
>Distillation “attacks” are not attacks. The frontier labs “distilled” all existing human written knowledge into their models So why didnt we have these LLMs in 2005?