191–200 of 215 posts
Earlier quoted context omitted.
can you explain the math of how you got to 25 gpus? everyone seems to know these conversions and idk if i missed the memo or something
220 Bp * 8 = 1760 Bp 1760 Bp takes ~ 1760 Gb of (V)RAM when 8-bit quantized. Plus you will need some memory for state. So you will need at least 1760/24=74 consumer-grade GPUs (3090/4090) or 1760/80=22 professional-grade GPUs (A100/H100).
> GPT-4: 8 x 220B experts trained with different data/task distributions and 16-iter inference. There was a post on HackerNews the other day about a 13B open source model. Any 220B open source models? Why or why not? I wonder what the 8 categories were. I wonder what goes into identifying tokens and then trying to guess which category/model you should look up. What if tokens go between two models, how do the models r…
same. i wish i had asked george instead of nodding along like an idiot. he probably wouldnt know but at least he’d speculate in interesting ways.
Earlier quoted context omitted.
> The gap with GPT-4 is not that big... Oh it is though. I've tried several OS models and nothing comes even close to GPT-4. Turns out ClosedAI has a moat after all.
AI development is taking place at breathtaking velocity and GPT-4 isn't the end of the road. OpenAI's moat relies on them improving at a faster pace than competitors. I wouldn't take it for granted that OpenAI's lead will remain the same in a year as it is today. Very possibly they could expand their lead. Also possibly, their lead could vanish.
Earlier quoted context omitted.
Pretty sure this is how the human brain works. If we add models that can transform information into updates to existing models, then add a model that can process streams of information and determine which models to update with the information (and can also apply that process to itself), we start to approach a primitive brain, no?
> Pretty sure this is how the human brain works How we thought it works in the 1960's yes.
Earlier quoted context omitted.
This is probably not the path to GI. First we would need a precise scientific formalism to accurately describe intelligence, which currently does not exist. Second, it may or may not end up being tied to consciousness, and there's a thing called the hard problem of consciousness, that possibly might not be solvable. It might end up being the kind of thing where if you want to accurately model consciousness, you would…
To accurately model consciousness it seems like we'd only need a computer the size of our brains, we'd just need to be as efficient as it.
It's very easy to say "brain is made of fundamental particles and forces, all we have to do is create a similar configuration or a model of them," but it's precisely in the task of understanding the higher order patterns of those fundamental particles and forces where we seem to run into some serious challenges that as of yet remain unaddressed.
The AI/ML way of approaching this is more of a top-down approach, where we just sort of ignore the fact that we don't understand how our own brains/minds work and just try to build something kind of like it in the folksy sense. I'm not discouraging that approach, but I'm very curious to see where it will lead us.
Earlier quoted context omitted.
I remember watching a live stream [1] of him going through Twitter engineering articles and trying to reverse engineer the Twitter frontend and backend, and he clearly had absolutely no clue of how anything remotely related to the web works. He was just clicking around compulsively, jumping between stuff randomly without even reading it, while not understanding what he was looking at. I have no idea how he's successf…
> I have no idea how he's successful in tech, judging from what I saw there. I think judging only from what you saw there is the issue. If you look somewhere like Wikipedia [0], you'll see he was the first person to jailbreak the iPhone, the first person to achieve hypervisor access to the PS3's CPU, he took first place in several notable CTFs (at least one time as a one-person team), he worked on Google's Project Ze…
What's 16 iter inference?
Earlier quoted context omitted.
220 Bp * 8 = 1760 Bp 1760 Bp takes ~ 1760 Gb of (V)RAM when 8-bit quantized. Plus you will need some memory for state. So you will need at least 1760/24=74 consumer-grade GPUs (3090/4090) or 1760/80=22 professional-grade GPUs (A100/H100).
thank you for this! 1Bp ~ 1Gb VRAM 8bit is a very nice equivalence to hold in my head.
Earlier quoted context omitted.
They mean 25 x RTX 4090 GPUs. 4090 is a model number
And to give a bit more context, it's one of the top consumer grade cards available (and has 24GB of RAM). It costs on the order of $1.6k instead of $15-25k of H100.