351–360 of 430 posts
I'm strictly speaking never going to think of model distillation as "stealing." It goes against the spirit of scientific research, and besides every tech company has lost my permission to define what I think of as theft forever
I found the discussion around inference scaling with the 'Wait' hack so surreal. The fact such an ingeniously simple method can impact performance makes me wonder how many low-hanging fruit we're still missing. So weird to think that improvements on a branch of computer science is boiling down to conjuring the right incantation words, how you even change your mindset to start thinking this way?
Agreed. Here are three things that I find surreal about the s1 paper. (1) The abstract changed how I thought about this domain (advanced reasoning models). The only other paper that did that for me was the "Memory Resource Management in VMware ESX Server". And that paper got published 23 years ago. (2) The model, data, and code are open source at https://github.com/simplescaling/s1 . With this, you can start training…
I found the discussion around inference scaling with the 'Wait' hack so surreal. The fact such an ingeniously simple method can impact performance makes me wonder how many low-hanging fruit we're still missing. So weird to think that improvements on a branch of computer science is boiling down to conjuring the right incantation words, how you even change your mindset to start thinking this way?
Earlier quoted context omitted.
This is something I have been suppressing since I don't want to become chicken little. Anyone who isn't terrified by the last 3 months probably doesn't really understand what is happening. I went from accepting I wouldn't see a true AI in my lifetime, to thinking it is possible before I die, to thinking it is possible in in the next decade, to thinking it is probably in the next 3 years to wondering if we might see i…
This frightens mostly people whose identity is built around "intelligence", but without grounding in the real world. I've yet to see really good articulations of what, precisely we should be scared of. Bedroom superweapons? Algorithmic propaganda? These things have humans in the loop building them. And the problem of "human alignment" is one unsolved since Cain and Abel. AI alone is words on a screen. The sibling thr…
It has certainly had this impact on my identity; I am unclear how well-grounded I really am*.
> I've yet to see really good articulations of what, precisely we should be scared of.
What would such an articulation look like, given you've not seen it?
> Bedroom superweapons? Algorithmic propaganda? These things have humans in the loop building them.
Even with current limited systems — which are not purely desk workers, they're already being connected to and controlling robots, even by amateurs — AI lowers the minimum human skill level needed to do those things.
The fear is: how far are we from an AI that doesn't need a human in the loop? Because ChatGPT was almost immediately followed by ChaosGPT, and I have every reason to expect people to continue to make clones of ChaosGPT continuously until one is capable of actually causing harm. (As with 3d-printed guns, high chance the first ones will explode in the face of the user rather than the target).
I hope we're years away, just as self driving cars turned out to be over-promised and under-delivered for the last decade — even without a question of "safety", it's going to be hard to transition the world economy to one where humans need not apply.
> And the problem of "human alignment" is one unsolved since Cain and Abel.
Yes, it is unsolved since time immemorial.
This has required us to not only write laws, but also design our societies and institutions such that humans breaking laws doesn't make everything collapse.
While I dislike the meme "AI == crypto", one overlap is that both have nerds speed-running discovering how legislation works any why it's needed — for crypto, specifically financial legislation after it explodes in their face; for AI, to imbue the machine with a reason to approximate society's moral code, because they see the problem coming.
--
* Dunning Kruger applies; and now I have first-hand experience of what this feels like from the inside, as my self-perception of how competent I am at German has remained constant over 7 years of living in Germany and improving my grasp of the language the entire time.
Earlier quoted context omitted.
How are you defining "reasoning"? Because I see these sorts of gnostic assertion about LLMs all the time about how they "definitely aren't doing " by gesturing at the technical things it's doing, with no attempts to actually justify the negative assertion. It often comes across as privileged reason trying to justify that of course the machine isn't doing some ineffable thing only meat-brains do.
From my other ridiculous comment, as I do entertain simulation theory in my understanding of God: Reasoning as we know it could just be a mechanism to fill in gaps in obviously sparse data (we absolutely do not have all the data to render reality accurately, you are seeing an illusion). Go reason about it all you want. The LLM doesn’t know anything. We determine what output is right, even if the LLM swears the output…
(Also, if I might give a recommendation, you might be the type of person to enjoy Unsong by Scott Alexander https://unsongbook.com/)
Earlier quoted context omitted.
> The intelligence that will be available to the average technically literate individual will be frightening. That's not the scary part. The scary part is the intelligence at scale that could be available to the average employer . Lots of us like to LARP that we're capitalists, but very few of us are. There's zero ideological or cultural framework in place to prioritize the well being of the general population over t…
> AI, especially accelerating AI, is bad news for anyone who needs to work for a living. It's not going to lead to a Star Trek fantasy. It means an eventual phase change for the economy that consigns us (and most consumer product companies) to wither and fade away. How would that work? If there are no consumers then why even bother producing? If the cost of labor and capital trends towards zero then the natural conse…
> If the producers refuse to lower their prices then they either don’t participate in the market (which also means their production is pointless) or ensure some other way that the consumers can buy their products.
Imagine you're a billionaire with a data centre and golden horde of androids.
You're the consumer, the robots make stuff for you; they don't make stuff for anyone else, just you, in the same way and for the same reason that your power tools and kitchen appliances don't commute to work — you could, if you wanted, lend them to people, just like those other appliances, but you'd have to actually choose to, it wouldn't be a natural consequence of the free market.
Their production is, indeed, pointless. This doesn't help anyone else eat. The moment anyone can afford to move from "have not" to "have", they drop out of the demand market for everyone else's economic output.
I don't know how big the impact of dropping out would be: the right says "trickle down economics" is good and this would be the exact opposite of that; while the left criticism's of trickle-down economics is that in practice the super-rich already have so much stuff that making them richer doesn't enrich anyone else who might service them, so if the right is correct then this is bad but if the left is correct then this makes very little difference.
Unfortunately, "nobody knows" is a great way to get a market panic all by itself.
Earlier quoted context omitted.
At most it would be illicit copying. Though it's poetic justice that OpenAI is complaining about someone else playing fast and loose with copyright rules.
The First Amendment is not just about free speech, but also the right to read, the only question is if AI has that right.
Earlier quoted context omitted.
> in that a distilled model of an LLM is like a JPEG of a photo That's an interesting analogy, because I've always thought of the hidden states (and weights and biases) of an LLMs as a compressed version of the training data.
And what is compression but finding the minimum amount of information required to reproduce a phenomena? I.e. discovering natural laws.
Earlier quoted context omitted.
At most it would be illicit copying. Though it's poetic justice that OpenAI is complaining about someone else playing fast and loose with copyright rules.
The First Amendment is not just about free speech, but also the right to read, the only question is if AI has that right.
I'm strictly speaking never going to think of model distillation as "stealing." It goes against the spirit of scientific research, and besides every tech company has lost my permission to define what I think of as theft forever
Maybe but something has gotta pay the bills to justify the cutting edge. I guess it's a similar problem to researching medicine.