Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An inside look at the custom CPUs in Tesla's Dojo Supercomputer

semianalysis.com

Post reply on HN
Print view View original

91–100 of 132 posts

Re: An inside look at the custom CPUs in Tesla's Dojo Supercomputer

#91

Post by _ph_ »

hirundo wrote: post #2

Now assume that Moore's law stays healthy and in a few decades we're carrying around dojo-equivalent smart phones. Or implants. What do our apps do with all that compute? Detailed convincing AR, accurate real-time translation, identify almost any object in sight, turing-test passing answering machine. We'll be carrying around neural-net training hardware that is different than the stuff between our ears, often worse…

If have a recent iPhone, you are carrying some special purpose neural net hardware around in your pocket already. I think this is a trend which is going to continue. First obvious application is, that speech recognition moves onto the phones themselves, no network required any more.

Re: An inside look at the custom CPUs in Tesla's Dojo Supercomputer

#92

Post by cedilla »

chrisseaton wrote: post #80
volta83 wrote: post #75

The fact that they didn't do this: > Their current training cluster would be the 5th largest supercomputer if Tesla stopped all real workloads, ran Linpack, and submitted it to the Top500 list. which is trivial to do, and pretty much a must when bringing up the cluster to make sure its working properly, so much that most clusters do this on every maintainance, along with another bunch of benchmarks; and that they say…

I don't get it - they're using it for actual work, rather than burning power to run useless benchmarks for bragging rights - and you think that makes it hype?! Surely it's the opposite - running benchmarks rather than doing something useful is hype.

Running benchmarks is not only useful but necessary. It's a point where planning meets the real world. Private clusters are also routinely benchmarked, often as part of validation.

In the end if someone just tells you their supercomputer "would" be in the TOP500, they just tell you that it is definitely not in the TOP500. There might be good reasons for that, but still, it's like claiming that you would win Olympic gold if only you'd bother to participate.

Re: An inside look at the custom CPUs in Tesla's Dojo Supercomputer

#93

Post by cedilla »

chrisseaton wrote: post #85
volta83 wrote: post #83

Earlier quoted context omitted.

You have never built a super computer, have you? You have to connect thousands of cables, have hundreds of nodes, with thousands of components, everything interconnected, and if you connect one wrong, the computer outputs incorrect results. You have to routinely update the software, and if a software upgrade introduces a 20% perf regression (which happens), then your 10 MWh cluster starts burning 2MWh for nothing. Or…

> run these benchmarks as an integration tests on every single cluster maintenance Why run someone else's benchmark and not your own application to test performance? And what's the point of submitting to Top500? Why do you care how your system ranks? What's the business or technical purpose in that?

For the same reason that we don't test engines during a race. Only what's under test should change, with the rest being fixed.

It would be much more difficult to adapt some of your own applications as a test. If the result are bad, how would you even know if it's the application's fault or a problem with the cluster? LINPACK on the other hand is well understood and there has been tons of work to make sure that it uses all the power your cluster can deliver.

Re: An inside look at the custom CPUs in Tesla's Dojo Supercomputer

#94

Post by elurg »

volta83 wrote: post #75

The fact that they didn't do this: > Their current training cluster would be the 5th largest supercomputer if Tesla stopped all real workloads, ran Linpack, and submitted it to the Top500 list. which is trivial to do, and pretty much a must when bringing up the cluster to make sure its working properly, so much that most clusters do this on every maintainance, along with another bunch of benchmarks; and that they say…

In the presentation they were quite open that they got to the point of running real loads but only on a single tile on a bench.

Not clear what you're disputing here?

Re: An inside look at the custom CPUs in Tesla's Dojo Supercomputer

#95

Post by volta83 »

elurg wrote: post #94
volta83 wrote: post #75

The fact that they didn't do this: > Their current training cluster would be the 5th largest supercomputer if Tesla stopped all real workloads, ran Linpack, and submitted it to the Top500 list. which is trivial to do, and pretty much a must when bringing up the cluster to make sure its working properly, so much that most clusters do this on every maintainance, along with another bunch of benchmarks; and that they say…

In the presentation they were quite open that they got to the point of running real loads but only on a single tile on a bench. Not clear what you're disputing here?

> Not clear what you're disputing here?

The article, that assumes "FLOPS on paper == FLOPS in practice".

Re: An inside look at the custom CPUs in Tesla's Dojo Supercomputer

#96

Post by ilovefood »

volta83 wrote: post #75

The fact that they didn't do this: > Their current training cluster would be the 5th largest supercomputer if Tesla stopped all real workloads, ran Linpack, and submitted it to the Top500 list. which is trivial to do, and pretty much a must when bringing up the cluster to make sure its working properly, so much that most clusters do this on every maintainance, along with another bunch of benchmarks; and that they say…

The more 3D renders in a presentation the more skepticism I develop. I noticed there is a direct leading indicator of a stock's price in relation to the quality of the 3D renders in the company's presentations/PR events. This is of course only anecdotal evidence but I'm pretty sure the hypothesis can hold it's ground against 50% of "ML Papers" today.

Re: An inside look at the custom CPUs in Tesla's Dojo Supercomputer

#97

Post by dmitriid »

ilovefood wrote: post #96
volta83 wrote: post #75

The fact that they didn't do this: > Their current training cluster would be the 5th largest supercomputer if Tesla stopped all real workloads, ran Linpack, and submitted it to the Top500 list. which is trivial to do, and pretty much a must when bringing up the cluster to make sure its working properly, so much that most clusters do this on every maintainance, along with another bunch of benchmarks; and that they say…

The more 3D renders in a presentation the more skepticism I develop. I noticed there is a direct leading indicator of a stock's price in relation to the quality of the 3D renders in the company's presentations/PR events. This is of course only anecdotal evidence but I'm pretty sure the hypothesis can hold it's ground against 50% of "ML Papers" today.

This reminds of Magic Leap. 3 billion dollars invested in a 3D render.

Re: An inside look at the custom CPUs in Tesla's Dojo Supercomputer

#98

Post by shaklee3 »

TaylorAlexander wrote: post #77
creato wrote: post #64

Earlier quoted context omitted.

Every hardware vendor ever says that. It's probably true for the people that built the software stack. Whether it is true for third parties is TBD.

There’s no third party here. As far as I can tell Tesla designed the chips and wrote the software stack to work together. They don’t want to rely on third parties for their critical infrastructure and AI chip design is vital to their success. At least this is what I can grok as an outsider.

Right, I understand the claims, but as you can see with AMD there's a ton of work to actually be competitive, and AMD has a large team working on it. I'm just skeptical at the claim the the software is of any use to anyone but a small group.

Re: An inside look at the custom CPUs in Tesla's Dojo Supercomputer

#99

Post by Sanguinaire »

chrisseaton wrote: post #85
volta83 wrote: post #83

Earlier quoted context omitted.

You have never built a super computer, have you? You have to connect thousands of cables, have hundreds of nodes, with thousands of components, everything interconnected, and if you connect one wrong, the computer outputs incorrect results. You have to routinely update the software, and if a software upgrade introduces a 20% perf regression (which happens), then your 10 MWh cluster starts burning 2MWh for nothing. Or…

> run these benchmarks as an integration tests on every single cluster maintenance Why run someone else's benchmark and not your own application to test performance? And what's the point of submitting to Top500? Why do you care how your system ranks? What's the business or technical purpose in that?

HPL is a stress test with a notionally useful output measurement - just about the most effective way of pushing CPU load to the limit, and it tells you what fraction of the theoretical maximum FLOPS you can actually achieve given the other system constraints like memory and network throughput.

Re: An inside look at the custom CPUs in Tesla's Dojo Supercomputer

#100

Post by addicted »

ilovefood wrote: post #96
volta83 wrote: post #75

The fact that they didn't do this: > Their current training cluster would be the 5th largest supercomputer if Tesla stopped all real workloads, ran Linpack, and submitted it to the Top500 list. which is trivial to do, and pretty much a must when bringing up the cluster to make sure its working properly, so much that most clusters do this on every maintainance, along with another bunch of benchmarks; and that they say…

The more 3D renders in a presentation the more skepticism I develop. I noticed there is a direct leading indicator of a stock's price in relation to the quality of the 3D renders in the company's presentations/PR events. This is of course only anecdotal evidence but I'm pretty sure the hypothesis can hold it's ground against 50% of "ML Papers" today.

Sure, your heuristic may be suspicious of 3D renders, but has it considered men in robot costumes?
Post reply on HN