11–20 of 95 posts
Earlier quoted context omitted.
The whole show in ML these days is in the infra around massively distributed, redundant, tunable, train the same fucking transformer we’ve all been using since 2017.
If you can find a more efficient or effective approach than this “redundant” one you’ll be rich. There’s a lot of smart people working in this space.
Earlier quoted context omitted.
I tend to agree that the math gets a lot simpler if the sun doesn’t rotate around a mystical axiom that humans are “special”.
High-energy physicists are about as serious as it gets, but they’re still dragging their feet around many worlds because it has uncomfortable implications for consciousness and free will. Homo Sapiens has a serious agenda around being something other than the smartest chimpanzee.
I’m well aware that this is an unpopular opinion. When your game plan is take a multi-head attention transformer and overfit it on as much of the Internet as we have A100s to fit it on, you’re going to get a distribution, and if you sample from it, it’s going to sound like the Internet. People read too much into this. The technology is obviously powerful borderline dangerous, but it’s still just maximizing P(this-sen…
I’m well aware that this is an unpopular opinion. When your game plan is take a multi-head attention transformer and overfit it on as much of the Internet as we have A100s to fit it on, you’re going to get a distribution, and if you sample from it, it’s going to sound like the Internet. People read too much into this. The technology is obviously powerful borderline dangerous, but it’s still just maximizing P(this-sen…
It’s so unpopular that literally every post on HN/proggit related to language models has the same comment ;)
Earlier quoted context omitted.
High-energy physicists are about as serious as it gets, but they’re still dragging their feet around many worlds because it has uncomfortable implications for consciousness and free will. Homo Sapiens has a serious agenda around being something other than the smartest chimpanzee.
No, they're "dragging their feet" because it's impossible to test, it's unfalsifiable.
If we just admit the possibility that the math was right all along we win, the math works, and we lose “I am a unique, distinct entity.”
I’m on team math.
I’m well aware that this is an unpopular opinion. When your game plan is take a multi-head attention transformer and overfit it on as much of the Internet as we have A100s to fit it on, you’re going to get a distribution, and if you sample from it, it’s going to sound like the Internet. People read too much into this. The technology is obviously powerful borderline dangerous, but it’s still just maximizing P(this-sen…
Worth noting that in the limit the distribution P(human did/said A | context B) is a complete definition of human behavior. If you could model this perfectly, that would be a perfect model of a human ie identical.
But extremely serious scientists, very smart people, are still drawing epicycles on blackboards studying “consciousness”.
I’m well aware that this is an unpopular opinion. When your game plan is take a multi-head attention transformer and overfit it on as much of the Internet as we have A100s to fit it on, you’re going to get a distribution, and if you sample from it, it’s going to sound like the Internet. People read too much into this. The technology is obviously powerful borderline dangerous, but it’s still just maximizing P(this-sen…
But are we doing any different?
Just a side note, I was showing some friends gpt3 and asking some trivial questions, trying to impress them, when one of them asked me: "Is it any different from like asking google for the same information?" I mean, In a pure naive and practical point of view she wasn't that wrong.
I’m well aware that this is an unpopular opinion. When your game plan is take a multi-head attention transformer and overfit it on as much of the Internet as we have A100s to fit it on, you’re going to get a distribution, and if you sample from it, it’s going to sound like the Internet. People read too much into this. The technology is obviously powerful borderline dangerous, but it’s still just maximizing P(this-sen…
> I’m well aware that this is an unpopular opinion It’s so unpopular that literally every post on HN/proggit related to language models has the same comment ;)
I used to do this for a living.
I’m well aware that this is an unpopular opinion. When your game plan is take a multi-head attention transformer and overfit it on as much of the Internet as we have A100s to fit it on, you’re going to get a distribution, and if you sample from it, it’s going to sound like the Internet. People read too much into this. The technology is obviously powerful borderline dangerous, but it’s still just maximizing P(this-sen…
> but it’s still just maximizing P(this-sentence-was-on-the-internet). But are we doing any different?