Awesome. I like where Tesla R&D money is going.
An inside look at the custom CPUs in Tesla's Dojo Supercomputer
11–20 of 132 posts
Re: An inside look at the custom CPUs in Tesla's Dojo Supercomputer
#12Is this power from the top method something other people do?
Power is from the bottom. Cooling is from the top and bottom. What you see on top are voltage regulators soldered directly to the wafer.
Re: An inside look at the custom CPUs in Tesla's Dojo Supercomputer
#13Now assume that Moore's law stays healthy and in a few decades we're carrying around dojo-equivalent smart phones. Or implants. What do our apps do with all that compute? Detailed convincing AR, accurate real-time translation, identify almost any object in sight, turing-test passing answering machine. We'll be carrying around neural-net training hardware that is different than the stuff between our ears, often worse…
Performance per watt is not improving that much, nor is feature size. You'll probably never be carrying around a supercomputer with this many FLOPS.
Even with reversible computing?
Re: An inside look at the custom CPUs in Tesla's Dojo Supercomputer
#14Earlier quoted context omitted.
Moore’s law is quite dead. Transistor density isn’t doubling. We’re still getting speed increases due to architectural improvements, but transistor density is capping out because thermals are a problem. That’s why you see these massive wafer designs and they’re not shrinking.
> Transistor density isn’t doubling. It pretty much continues to double: https://en.m.wikipedia.org/wiki/Transistor_count#/media/File...
Raw transistor count isn't the same as density.
Re: An inside look at the custom CPUs in Tesla's Dojo Supercomputer
#15Now assume that Moore's law stays healthy and in a few decades we're carrying around dojo-equivalent smart phones. Or implants. What do our apps do with all that compute? Detailed convincing AR, accurate real-time translation, identify almost any object in sight, turing-test passing answering machine. We'll be carrying around neural-net training hardware that is different than the stuff between our ears, often worse…
Moore’s law is quite dead. Transistor density isn’t doubling. We’re still getting speed increases due to architectural improvements, but transistor density is capping out because thermals are a problem. That’s why you see these massive wafer designs and they’re not shrinking.
Both TSMC and Intel have transistor density increases on their process roadmaps, although they definitely aren't doubling.
Re: An inside look at the custom CPUs in Tesla's Dojo Supercomputer
#16Insane, truly insane. Meanwhile laptop CPUs, with the notable exception of M1, are impossible to cool.
Not sure that's relevant here - the reason they can cool this is because they're custom-designing housing, server racks, etc for these chips based on the amount of power they draw. You could cool pretty much anything if you were able to give it this much love and care.
Re: An inside look at the custom CPUs in Tesla's Dojo Supercomputer
#17Now assume that Moore's law stays healthy and in a few decades we're carrying around dojo-equivalent smart phones. Or implants. What do our apps do with all that compute? Detailed convincing AR, accurate real-time translation, identify almost any object in sight, turing-test passing answering machine. We'll be carrying around neural-net training hardware that is different than the stuff between our ears, often worse…
I wish I had your optimism. All I expect are better ways to deliver targeted advertising.
[deleted]
Re: An inside look at the custom CPUs in Tesla's Dojo Supercomputer
#18Now assume that Moore's law stays healthy and in a few decades we're carrying around dojo-equivalent smart phones. Or implants. What do our apps do with all that compute? Detailed convincing AR, accurate real-time translation, identify almost any object in sight, turing-test passing answering machine. We'll be carrying around neural-net training hardware that is different than the stuff between our ears, often worse…
> What do our apps do with all that compute?
javascript
Re: An inside look at the custom CPUs in Tesla's Dojo Supercomputer
#19Now assume that Moore's law stays healthy and in a few decades we're carrying around dojo-equivalent smart phones. Or implants. What do our apps do with all that compute? Detailed convincing AR, accurate real-time translation, identify almost any object in sight, turing-test passing answering machine. We'll be carrying around neural-net training hardware that is different than the stuff between our ears, often worse…
Moore’s law is quite dead. Transistor density isn’t doubling. We’re still getting speed increases due to architectural improvements, but transistor density is capping out because thermals are a problem. That’s why you see these massive wafer designs and they’re not shrinking.
Moore’s law was about count, not density.
Re: An inside look at the custom CPUs in Tesla's Dojo Supercomputer
#20Earlier quoted context omitted.
Performance per watt is not improving that much, nor is feature size. You'll probably never be carrying around a supercomputer with this many FLOPS.
Even with reversible computing?
What about cloud services. A 5G connection to a supercomputer on demand.