Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT4 is 8 x 220B params = 1.7T params

twitter.com

Post reply on HN
Print view View original

171–180 of 215 posts

Re: GPT4 is 8 x 220B params = 1.7T params

#171

Post by mrtranscendence »

TeMPOraL wrote: post #94
startupsfail wrote: post #57

Earlier quoted context omitted.

It’s funny that this post is trending on HN right next to the post about a paper showing how to build a model 1000x smaller than 1.7T that can code better than LLMs 10x larger.

I don't find it funny, I find it scary and mind-blowing: the impact of these headlines is additive - this one confirms the effectiveness of combining models, and the other one suggests you could cut the model size a couple orders of magnitude if you train on clean enough data. Together, this points at a way to achieve both GPT-4 that fits on your phone, and a much more powerful model that's not larger than GPT-4 is n…

This doesn't seem likely in the near term. An iPhone 13 Pro has 6gb of memory, which might be enough for a single 1.3b parameter model if you purged everything else out of RAM. Combining 8 (or even 4) of them on a single phone won't happen anytime soon at the rate phones improve.

Plus, none of the smaller models are really appreciably close to GPT-4 on most metrics. It's not clear to me you could get there at all with 1.3b models. Maybe somebody gets there someday with 65b models, but then you're far out of the reach of phones.

Re: GPT4 is 8 x 220B params = 1.7T params

#172

Post by albrewer »

devinprater wrote: post #153
andreyk wrote: post #34

weird title, note that the tweet said "so yes, GPT4 is *technically* 10x the size of GPT3, and all the small circle big circle memes from January were actually... in the ballpark?" It's really 8 models that are 220B, which is not the same as one model that is 1.7T params. There have been 1T+ models via mixtures of experts for a while now. Note also the follow up tweet: "since MoE is So Hot Right Now, GLaM might be th…

Oh, is that how Bing's image descriptions work? If you upload an image and ask for a description, it'll give a really good one. But if you ask it where it got it from, it says it got it from another algorithm. So is it using another expert AI to describe, and then feed it into GPT4 for conversion into language?

Pretty sure this is how the human brain works. If we add models that can transform information into updates to existing models, then add a model that can process streams of information and determine which models to update with the information (and can also apply that process to itself), we start to approach a primitive brain, no?

Re: GPT4 is 8 x 220B params = 1.7T params

#173

Post by generalizations »

Reminds me of this: https://en.wikipedia.org/wiki/Society_of_Mind

> In his book of the same name, Minsky constructs a model of human intelligence step by step, built up from the interactions of simple parts called agents, which are themselves mindless. He describes the postulated interactions as constituting a "society of mind", hence the title.

Re: GPT4 is 8 x 220B params = 1.7T params

#174

Post by mrtranscendence »

fnands wrote: post #146
ChatGTP wrote: post #142

Earlier quoted context omitted.

How are. 254090 GPUs supposed to run inside a robot ?

They mean 25 x RTX 4090 GPUs. 4090 is a model number

I think the question stands. You can't fit 25 4090s in a robot (unless we're talking about something massive with an equally massive battery), and even if you could an LLM wouldn't be appropriate for driving a robot.

Given the pace of improvements I don't see how you compress 25 4090s into a single GPU in 7 years. A 4090 isn't 25 times the power of a GTX 980, it's closer to maybe three times.

Re: GPT4 is 8 x 220B params = 1.7T params

#176

Post by chriskanan »

janalsncm wrote: post #61

At a minimum he glossed over the multimodal capabilities of GPT-4. If they use the same set of tokens, it’s unclear how this doesn’t pollute text training data. If they use separate tokens, the model size should be bigger.

This is the same strategy used by Palm-E and Mini-GPT4. They have a projection layer that transforms the vision embeddings into the token embedding space.

Re: GPT4 is 8 x 220B params = 1.7T params

#177

Post by swyx »

zarzavat wrote: post #118
DaiPlusPlus wrote: post #116

Earlier quoted context omitted.

Well, they're not wrong... Given a highly-parallelizable numerics job, and all other things being equal, a 20 core 2.0Ghz machine will run the job in the same time as a 40 core 1.0Ghz machine. ...though usually we'd just give it in FLOPS - though FLOPS doesn't describe non-float operation performance. Probably just a sales/marketing person who learned a new technical word and wants to sound knowledgable to the custom…

At the risk of torturing this analogy beyond recognition, it’s more like having an 8 core CPU but with a weak 150W PSU such that you can only run one core at a time.

in my understanding with an 8 way MoE youre still running all 8 for every request. only if you’re making a router model then itd be the “weak PSU” analogy. and even then a sufficiently perfect router would essentially have to know enough about its routes to almost simulate them which idk if i can believe works

Re: GPT4 is 8 x 220B params = 1.7T params

#178

Post by swyx »

nirushiv wrote: post #136
ddalex wrote: post #114

Earlier quoted context omitted.

if you have a basket with 4 apples and a basket with 3 pears, are you not having 7 fruits ? just because the first selection layer is very thin doesn't mean that the network cannot be considered composable

You can’t claim it’s a 7-fruit Basket, unless you find a way to fit them in the same basket

but you can claim its a 7 fruit grouping of baskets, we’re just in a territory where the fruit count matters less and less now

which happens to be consistent with what openai is telling us

Re: GPT4 is 8 x 220B params = 1.7T params

#179

Post by swyx »

LarsDu88 wrote: post #82

That's on the order of 25 4090 GPUs to run inference. Not a crazy number by any means. We will see consumer robots running that by the end of the decade, mark my words.

can you explain the math of how you got to 25 gpus? everyone seems to know these conversions and idk if i missed the memo or something
Post reply on HN