Earlier quoted context omitted.
> What makes you think that Telsa, a company with far less AI workers and knowledge, an far less money than the above companies can out design and out build them? Presumably because Elon himself will be involved in the design, and Elon, as we all know, is one of the world's great thinkers. ;)
Elon is one of the worlds greatest talent poachers, and that is much better than being a great thinker.
Tesla turns on 10k-node Nvidia H100 Cluster
101–110 of 135 posts
Re: Tesla turns on 10k-node Nvidia H100 Cluster
#102Earlier quoted context omitted.
He delivered on them though right? Also reusable rockets didn’t exist?
John Carmack on a shoestring budget nearly got this working at Armadillo. If he had more money and time he would've had it working a half decade before SpaceX.
Re: Tesla turns on 10k-node Nvidia H100 Cluster
#103Earlier quoted context omitted.
Got it, so no then right. I love Carmack, but having an idea for something is infinitely easier than actually successfully executing on it, especially as incredibly successfully as SpaceX.
Rocketry involves building a lot of prototypes and blowing up a lot of things when mistakes happen. Carmack did execute, they had a working rocket, and with time they could've solved the software issues, but they didn't have the funding to blow up dozens of prototypes. Carmack drove that project, he wrote code, he built rocket engines himself, he ran missions. Elon, notably, doesn't do any of that. He just had more m…
Re: Tesla turns on 10k-node Nvidia H100 Cluster
#104I'm confused. The article from September 1 linked to here is strangely future-tense ("But the firm’s latest investment in 10,000 of the company’s H100 GPUs dwarfs the power of this supercomputer....This AI cluster, worth more than $300 million, will offer a peak performance..."). It links to a Tom's Hardware article ( https://www.tomshardware.com/news/teslas-dollar300-million-a... ) from August 28 that says "Tesla is…
> high editorial standards of such tech-press luminaries as "TechRadar" and "Hacker News". If you would’ve just scrolled just a little bit on that Twitter post that you linked. You would’ve seen these: https://x.com/sawyermerritt/status/1696012091964915744 https://x.com/tim_zaman/status/1695488119729238147 Also, just FYI. Sawyer posts most of the Tesla and SpaceX breaking news on Twitter before major outlets even wri…
Re: Tesla turns on 10k-node Nvidia H100 Cluster
#105Earlier quoted context omitted.
10k H100 chips is considered a very large cluster. The third fastest supercomputer in the world is Microsoft’s eagle with 14k H100s https://www.top500.org/lists/top500/2023/11/
Ah, gotcha, so the fact that its 10,000 chips for one dedicated cluster that makes it large, as opposed to Azure which has an order of magnitude more GPUS but rents many of those out.
Re: Tesla turns on 10k-node Nvidia H100 Cluster
#106Earlier quoted context omitted.
Interesting choice of words... I take you work for OpenAI? :) How large is their/'your' cluster? Probably the biggest in the world by now..
Parent is almost certainly talking about Frontier, the supercomputer with the US Department of Energy[0]. [0] - https://top500.org/system/180047/
Re: Tesla turns on 10k-node Nvidia H100 Cluster
#107Earlier quoted context omitted.
> If you would’ve just scrolled just a little bit on that Twitter post that you linked. You would’ve seen these: I don't see those when I scroll. I see "Buckle up everyone, the acceleration of progress is about to get nutty!" and this is the end of the post? Maybe I'm misusing this thing? > https://x.com/tim_zaman/status/1695488119729238147 So another guy who claims to be a Tesla employee says (again, strangely futur…
> Maybe I'm misusing this thing? That seems to be the case here. ;) > So another guy who claims to be a Tesla employee says (again, strangely future tense) that this is true? I mean, I am willing to believe--'cause he paid $20 for a blue check--that he probably is a Tesla employee. Another case of misuse? Here’s a tip for you. When you see a company logo/icon on someone's Twitter/X profile. That means they are verifi…
Like, whats the actual news story here?
Totally agree there’s a lack of critical thinking at play.
Also, I think the X.com links only work if you have a login or something, fyi.
Re: Tesla turns on 10k-node Nvidia H100 Cluster
#108Earlier quoted context omitted.
That Telsa owners can use their cars to make money while they are working as robo taxis -let's just say he vastly underestimates effort it takes to make progress - FSD is not there yet.
Vaporware assumes it will never happen. Is that the case you think or is it that he was vastly over optimistic? Very likely the latter.
Re: Tesla turns on 10k-node Nvidia H100 Cluster
#109I understand that the H100 is NVidia's leading edge chip, but can someone let me know if 10K is considered to be a big cluster? I've never worked inside one of the leading edge AI companies like OpenAI, Google, Microsoft or Meta. Is this comparable to what they would work with? My first guess is that it seems much smaller. And if you are running many parallel training jobs then you are getting about 1,000 chips at mo…
Last I heard, the estimate was that NVIDIA would build 550k units in 2023, so 2% of all production — especially as at least six others (your four plus Apple and at least one intelligence agency) will be of similar size by themselves — is certainly non-negligible.
Re: Tesla turns on 10k-node Nvidia H100 Cluster
#110Earlier quoted context omitted.
Out of interest, what did you use all that compute for?
ETH PoW. When ETH switched to PoS, we shut it all down. It sure was fun while it lasted, not many people on the planet have run that much compute. I did a lot of unique optimizations to autotune each individual GPU for performance by tweaking the software knobs on them. They are all snowflakes. Same model, different batches (heck, even same batch!), can produce wildly different performance results. Over the years, I…