341–350 of 742 posts
Earlier quoted context omitted.
I fear they are laying the groundwork to ban US citizens from using Chinese models, so that they can make sure that Brockman gets his money's worth.
why would you fear that? At least it's something to fight unfair business practices. As for American AI companies violating copyright there's a venue for that, the courts. File a case if you feel your copyright has been violated and get your day in court.
Earlier quoted context omitted.
Generating a set of weights is not learning.
That is a strong statement. I guess you are telling Machine Learning to go fuck itself.
Does this matter? Distillation is not illegal by every definition of the word. There are millions of samples available on huggingface and models explicitely trained on output produced by fable. There has been no action taken against them. Another example is that it appears that the upper limit of what you can do is ultimately dependent on people working on the model, otherwise grok would be a LOT more competitive pre…
I wouldn't be surprised at all if US labs are also distilling Chinese models, except we'd never know since they can simply self-host them
Earlier quoted context omitted.
Perhaps even more importantly, the current frontier LLM models are self-admittedly the product of enormous quantities of copyright infringement and even less savory inputs, so calling them out for distilling the fruit of that tainted tree reads as highly hypocritical at best.
While I agree on a moral level, I think there is a distinction to be made. Training a SOTA model takes a huge amount of resources and expertise so the people doing the training are adding a lot of value along the way. I think this is much less true for distillation (which is kind of the whole point). ed: to clarify, I totally agree that a huge chunk of the value in LLMs is coming from the source material. My point wa…
If that is the whole point you need to clarify why this is the case on an objective level.
I would say building a comparable model using any means necessary (just like what Anthropic and OAI did) at a lower cost is actually more valuable to soceity and Monshoot is arguably generating more value with less.
Earlier quoted context omitted.
All of the models stole the entirety of written knowledge on the internet to train. They are being sued for the few cases where we have some proof of what they did because of some whistleblowers, all the rest will just go unpunished. They breached Github TOS, robot.txt's, copyright, patents every form of IP protection under the sun from a billion sources. It's just ridiculous for the thieves to cry about someone else…
Do you care about the law or not? I think theft is bad everywhere, not just when Anthropic does it.
Earlier quoted context omitted.
Perhaps even more importantly, the current frontier LLM models are self-admittedly the product of enormous quantities of copyright infringement and even less savory inputs, so calling them out for distilling the fruit of that tainted tree reads as highly hypocritical at best.
While I agree on a moral level, I think there is a distinction to be made. Training a SOTA model takes a huge amount of resources and expertise so the people doing the training are adding a lot of value along the way. I think this is much less true for distillation (which is kind of the whole point). ed: to clarify, I totally agree that a huge chunk of the value in LLMs is coming from the source material. My point wa…
Earlier quoted context omitted.
People that think the Chinese are only able to copy western tech are in for a wakeup call. Actually, that has already happened in many domains, it's just that most western people (USA especially) won't admit it.
my assertion isn't that china isn't able to surpass western AI. I think it may well happen. I've been to china many times and am well aware of how ahead they are in many technological/societal areas. at the same time, I don't buy the idea that distillation is unimportant in assessing what Chinese labs are capable of. If it wasn't, why did Kimi's release timing coincide so well with Fable's launch? and if Anthropic ha…
For some reason folks seem to think that China can take action and then other countries can't also take action or respond to that action and it comes up again and again. China has hypersonic missiles! Pack it up boys time to go home. Nothing we can do. Dang shucks. China distilled American AI models, welp time to just close it all down and let's just write off those trillions of dollars and all the literal geniuses financing and building these things. Oh well China can just copy American models while we spend all the money! Ok we just stop developing models and we'll just copy their models. China will flood the market with their cheap products! Nope can't do anything like, oh, idk, not buy any of those products or just raise the prices on them in local markets. It's never-ending. I don't understand the lack of capacity to reason about other actors that takes commonly takes place. And that's just China, never mind other general issues.
Here's a site that asks the same questions to 22 models and compares how similar their responses are. https://typebulb.com/u/lab/you-re-relatively-right/full According to these results GLM 5.2 is very similar to Google Gemini and Kimi K3 is very similar to Fable 5. The American frontier labs are not similar to each other.
I also think the Fable accusation is wrong and it was most likely Opus 4.8 which itself is likely a distillation of Fable.