GPT4 is 8 x 220B params = 1.7T params
twitter.com
GPT4 is 8 x 220B params = 1.7T params
1–10 of 215 posts
Re: GPT4 is 8 x 220B params = 1.7T params
#2Is this still orders of magnitude smaller than a human brain?
How many? Based on current human neurons/synapses knowledge?
Re: GPT4 is 8 x 220B params = 1.7T params
#3Is this still orders of magnitude smaller than a human brain? How many? Based on current human neurons/synapses knowledge?
2 orders of magnitude smaller, assuming 100T synaptic connections in the human brain.
Re: GPT4 is 8 x 220B params = 1.7T params
#4Is this still orders of magnitude smaller than a human brain? How many? Based on current human neurons/synapses knowledge?
We really have no idea how to directly compare the two.
Also, vast portions of the human brain are dedicated to the visual cortex, smelling, breathing, muscle control... things which have value to us but which don't contribute to knowledge work when evaluating how many parameters it would take to replace human knowledge work.
Re: GPT4 is 8 x 220B params = 1.7T params
#5I wouldn’t trust anything geohot says. He doesn’t have access to any inside information.
Re: GPT4 is 8 x 220B params = 1.7T params
#6Is this still orders of magnitude smaller than a human brain? How many? Based on current human neurons/synapses knowledge?
We really have no idea how to directly compare the two. Also, vast portions of the human brain are dedicated to the visual cortex, smelling, breathing, muscle control... things which have value to us but which don't contribute to knowledge work when evaluating how many parameters it would take to replace human knowledge work.
It's really interesting that human organism requires so much computational power to support live.
Re: GPT4 is 8 x 220B params = 1.7T params
#7I wouldn’t trust anything geohot says. He doesn’t have access to any inside information.
He doesn't strike me as the type of person to lie (except when trolling). His reputation is solid enough that I'm sure he's had discussions with people in the space.
Re: GPT4 is 8 x 220B params = 1.7T params
#8Earlier quoted context omitted.
We really have no idea how to directly compare the two. Also, vast portions of the human brain are dedicated to the visual cortex, smelling, breathing, muscle control... things which have value to us but which don't contribute to knowledge work when evaluating how many parameters it would take to replace human knowledge work.
It's really interesting that human organism requires so much computational power to support live.
I think it's even more interesting that the required amount of energy to do that high computational work isn't that high. Evolution has been working on it for a long time, and some things are really inefficient but overall it does an OK job at making squishy machines.
Re: GPT4 is 8 x 220B params = 1.7T params
#9I wouldn’t trust anything geohot says. He doesn’t have access to any inside information.
https://twitter.com/soumithchintala/status/16712671501017210...
It looks like at least one other person has also heard the same information.
Re: GPT4 is 8 x 220B params = 1.7T params
#10GPT-4 is 1.7T params in the same way that an AMD Ryzen 9 7950X is 72 GHz.