391–400 of 742 posts
Earlier quoted context omitted.
> Distillation is impossible to stop Lots of things are impossible or very difficult to stop completely but measures can be taken to reduce their prevalence.
Sure, but the problem is that it hurts legit people too.
rules for thee but not for me
Plenty of HN readers feel this way and it's a good point, but it has also become an entirely cliché response which pops up like mushrooms anytime "distillation" appears. That means it's against the site guidelines, which ask: " Eschew flamebait. Avoid generic tangents. Omit internet tropes. " - https://news.ycombinator.com/newsguidelines.html I don't mean to pick on you personally! It's just that reflexive responses…
When I see a lot of the responses, even cliche ones, it tells me people are passionate about the issue. Right now I think that's an interesting piece of information.
One of the replies: > @MehdiKarech > I don't remember letting Anthropic or Open Ai scrapping my GitHub, my research gate and all my online writings L O L https://xcancel.com/MehdiKarech/status/2080000779859939678#m
How much credible , provable evidence? None really.
proof is generally not even needed
Earlier quoted context omitted.
Sure, but the problem is that it hurts legit people too.
> the problem is that it hurts legit people too. What hurts other people too?
Earlier quoted context omitted.
Perhaps even more importantly, the current frontier LLM models are self-admittedly the product of enormous quantities of copyright infringement and even less savory inputs, so calling them out for distilling the fruit of that tainted tree reads as highly hypocritical at best.
While I agree on a moral level, I think there is a distinction to be made. Training a SOTA model takes a huge amount of resources and expertise so the people doing the training are adding a lot of value along the way. I think this is much less true for distillation (which is kind of the whole point). ed: to clarify, I totally agree that a huge chunk of the value in LLMs is coming from the source material. My point wa…
If the distilled model is cheaper, then it's just LLM's getting LLM'ed.
Earlier quoted context omitted.
I like this comment because its argument only makes sense if you assume that the entire world's output of books and art did not require a huge amount of resources and expertise to make, nor did it add any value. It's the most CS-major take ever!
If turning other peoples copyrighted work into a model is transformative enough to be protected then so is distilling that model into a different, better, model.
Earlier quoted context omitted.
Perhaps even more importantly, the current frontier LLM models are self-admittedly the product of enormous quantities of copyright infringement and even less savory inputs, so calling them out for distilling the fruit of that tainted tree reads as highly hypocritical at best.
No - distillation is not data inputs. Raw materials vs. Value add. They are different things, like ore and metal. Distillation is a new thing we need to understand, it's probably closer to IP than not.