Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An inside look at the custom CPUs in Tesla's Dojo Supercomputer

semianalysis.com

Post reply on HN
Print view View original

111–120 of 132 posts

Re: An inside look at the custom CPUs in Tesla's Dojo Supercomputer

#111

Post by N1H1L »

volta83 wrote: post #83
chrisseaton wrote: post #80

Earlier quoted context omitted.

I don't get it - they're using it for actual work, rather than burning power to run useless benchmarks for bragging rights - and you think that makes it hype?! Surely it's the opposite - running benchmarks rather than doing something useful is hype.

You have never built a super computer, have you? You have to connect thousands of cables, have hundreds of nodes, with thousands of components, everything interconnected, and if you connect one wrong, the computer outputs incorrect results. You have to routinely update the software, and if a software upgrade introduces a 20% perf regression (which happens), then your 10 MWh cluster starts burning 2MWh for nothing. Or…

Hi fellow supercomputer person!

My experience is on Summit & am now working with Frontier setup.

Re: An inside look at the custom CPUs in Tesla's Dojo Supercomputer

#112

Post by WoodenChair »

smolder wrote: post #61
WoodenChair wrote: post #22

Earlier quoted context omitted.

Sure, but if you look at the raw transistor count of a series of similarly sized dies (take Apple's A series for example), it holds up. It only doesn't hold up if you look at Intel the last 5 years. If you look at series' made on TSMC or Samsung, it holds up.

If you look at CPU performance relative to density as it has progressed over the last few decades, there's a clear decline in speed improvement. Denser only means faster to a point, regardless of Intel's process update failures.

Not sure what you’re replying to. The statement I’m replying to didn’t say anything about performance. It said “ Transistor density isn’t doubling.”

Re: An inside look at the custom CPUs in Tesla's Dojo Supercomputer

#113

Post by volta83 »

chrisseaton wrote: post #85
volta83 wrote: post #83

Earlier quoted context omitted.

You have never built a super computer, have you? You have to connect thousands of cables, have hundreds of nodes, with thousands of components, everything interconnected, and if you connect one wrong, the computer outputs incorrect results. You have to routinely update the software, and if a software upgrade introduces a 20% perf regression (which happens), then your 10 MWh cluster starts burning 2MWh for nothing. Or…

> run these benchmarks as an integration tests on every single cluster maintenance Why run someone else's benchmark and not your own application to test performance? And what's the point of submitting to Top500? Why do you care how your system ranks? What's the business or technical purpose in that?

When a company claims that their new vaccine cures cancer, but is against applying for FDA approval because "no reasons given", you seem to be the kind of person that camps in front of their offices overnight to get the first 10 shots, all in the neck.

Re: An inside look at the custom CPUs in Tesla's Dojo Supercomputer

#114

Post by chrisseaton »

N1H1L wrote: post #110
chrisseaton wrote: post #85

Earlier quoted context omitted.

> run these benchmarks as an integration tests on every single cluster maintenance Why run someone else's benchmark and not your own application to test performance? And what's the point of submitting to Top500? Why do you care how your system ranks? What's the business or technical purpose in that?

Is this some sort of a post-truth joke? If you are claiming something is the best, then without any comparable metrics across systems we are in the Twilight zone.

But they've got no obligation to prove anything to random people on the internet. They aren't asking for peer review. They probably don't want to waste time and energy running a benchmark to compete in someone else's top N list. What's in it for them?

Re: An inside look at the custom CPUs in Tesla's Dojo Supercomputer

#115

Post by chrisseaton »

volta83 wrote: post #113
chrisseaton wrote: post #85

Earlier quoted context omitted.

> run these benchmarks as an integration tests on every single cluster maintenance Why run someone else's benchmark and not your own application to test performance? And what's the point of submitting to Top500? Why do you care how your system ranks? What's the business or technical purpose in that?

When a company claims that their new vaccine cures cancer, but is against applying for FDA approval because "no reasons given", you seem to be the kind of person that camps in front of their offices overnight to get the first 10 shots, all in the neck.

But FDA approval is mandatory, and participation in top N lists is just marketing. And they aren't even asking you to use their machine!

Re: An inside look at the custom CPUs in Tesla's Dojo Supercomputer

#116

Post by volta83 »

chrisseaton wrote: post #114
N1H1L wrote: post #110

Earlier quoted context omitted.

Is this some sort of a post-truth joke? If you are claiming something is the best, then without any comparable metrics across systems we are in the Twilight zone.

But they've got no obligation to prove anything to random people on the internet. They aren't asking for peer review. They probably don't want to waste time and energy running a benchmark to compete in someone else's top N list. What's in it for them?

> But they've got no obligation to prove anything to random people on the internet.

They are a publicly traded company.

I'm a tesla investor and deserve to know.

If they are right and their hardware is 10x better than the competitor, then I'm happy. If they are wrong and they could have get 10x better outcomes by spending 100x less money, I'll be very pissed off.

> They probably don't want to waste time and energy running a benchmark to compete in someone else's top N list.

It costs no time. It's the first thing anybody does when building any cluster. It's part of accepting the materials from your suppliers to check that they didn't sell you shit.

Re: An inside look at the custom CPUs in Tesla's Dojo Supercomputer

#117

Post by chrisseaton »

volta83 wrote: post #116
chrisseaton wrote: post #114

Earlier quoted context omitted.

But they've got no obligation to prove anything to random people on the internet. They aren't asking for peer review. They probably don't want to waste time and energy running a benchmark to compete in someone else's top N list. What's in it for them?

> But they've got no obligation to prove anything to random people on the internet. They are a publicly traded company. I'm a tesla investor and deserve to know. If they are right and their hardware is 10x better than the competitor, then I'm happy. If they are wrong and they could have get 10x better outcomes by spending 100x less money, I'll be very pissed off. > They probably don't want to waste time and energy ru…

> I'm a tesla investor and deserve to know.

I'm not an expert in securities law, but I don't believe being an investor entitles you to demand arbitrary proprietary information you want from a company. Otherwise people would buy one Apple share and find out what specs the new iPhone will have.

Re: An inside look at the custom CPUs in Tesla's Dojo Supercomputer

#118

Post by volta83 »

chrisseaton wrote: post #115
volta83 wrote: post #113

Earlier quoted context omitted.

When a company claims that their new vaccine cures cancer, but is against applying for FDA approval because "no reasons given", you seem to be the kind of person that camps in front of their offices overnight to get the first 10 shots, all in the neck.

But FDA approval is mandatory, and participation in top N lists is just marketing. And they aren't even asking you to use their machine!

Dude i don't know what you don't get it.

Tesla is saying their hardware is 10x better than the competition.

I don't believe it. They don't publish any numbers, and everybody else does.

You believe them. Good for you.

I don't. I think not doing that is extremely suspicious, because if their cluster can be turned on, they have the results. So the reality of this is 100% certain that their results just suck, and they don't publish them to save face.

You believe their shit is so good, they didn't even test it while installing it. Good for you.

You think there is a masterplan to keep the results secret. Good for you.

I think their results are just horrible, and that's why they don't show them.

With any serious company, this wouldn't even be a discussion, because they would just show the verified facts about what their cluster can do or shut up about it if its really so secret.

You don't see the NSA or the DOD bragging around about being 100x better than "the competition".

To me they look forced to sell that they are doing something about these clusters to... drive hype up and get the stocks back up, and since their numbers are actually horrible, this is what we get.

Re: An inside look at the custom CPUs in Tesla's Dojo Supercomputer

#119

Post by formerly_proven »

elurg wrote: post #94
volta83 wrote: post #75

The fact that they didn't do this: > Their current training cluster would be the 5th largest supercomputer if Tesla stopped all real workloads, ran Linpack, and submitted it to the Top500 list. which is trivial to do, and pretty much a must when bringing up the cluster to make sure its working properly, so much that most clusters do this on every maintainance, along with another bunch of benchmarks; and that they say…

In the presentation they were quite open that they got to the point of running real loads but only on a single tile on a bench. Not clear what you're disputing here?

I only work on a lowly private cluster but running standard benchmarks is utterly routine here (it is in fact automated). As others with HPC experience pointed out, running benchmarks is pretty much mandatory when bringing a new system up, not just to ensure it actually performs as promised, but also to weed out bad and marginal components. You do one or two weeks of intense benchmarking and testing and you're assured numerous nodes will fail and need parts replaced. When you apply patches, you benchmark again. Why? Because the suppliers fix for "code XYZ crashes nodes" is probably "let's uh just reduce power limits by a few %". Or because of unintentional issues limiting performance. When a node crashes, you benchmark it again. Why? Because linpack and friends are good at making marginal hardware fail.

So it is literally unbelievable that Tesla not just stands up a cluster, but created their own hardware to do so, and didn't run any quotable benchmark and only has the theoretic FLOPS numbers for marketing.

Re: An inside look at the custom CPUs in Tesla's Dojo Supercomputer

#120

Post by ipsum2 »

formerly_proven wrote: post #79
ipsum2 wrote: post #37

Taking everything at face value: Dojo is overall a very impressive project! - Communication speed is one of the biggest bottlenecks to large models, so their bandwidth of 4TBps is very smart. - They claim a 1.3x perf/watt improvement, which is not really that great for ASICs compared to GPUs. Perf/watt is probably the most important number in datacenters. - They only use SRAM, no DRAM. This is a huge mistake, which l…

The thermal solution and packaging looks a lot like mainframe systems from the 80s (IBM, Siemens, Japanese). Siemens H100: https://pbs.twimg.com/media/Ee56Q2bWkAA8vBE?format=jpg&name=... https://pbs.twimg.com/media/Ee56Q2ZX0AEqAWo?format=jpg&name=... IBM: https://www.youtube.com/watch?v=xQ3oJlt4GrI

That was great, thanks for linking it.
Post reply on HN