Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tesla Dojo Custom AI Supercomputer at HC34

servethehome.com

Post reply on HN
Print view View original

31–40 of 102 posts

Re: Tesla Dojo Custom AI Supercomputer at HC34

#31

Post by londons_explore »

kjksf wrote: post #15
londons_explore wrote: post #12

If this stuff is any good, Tesla should make it available for everyone to use colab-style. A hosted Jupyter notebook in a sandboxed VM able to send jobs to this new silicon is something that might be possible to set up by a small team in a few months, and could turn into a billion dollar business. As a bonus, Tesla can use revenue from that to grow this supercomputer, while using any spare/unsold capacity for themsel…

It is already Tesla's plan to build AWS-style paid access to Dojo. I think they said that during the first AI Day. Here's a 19 minute supercut of AI Day: https://www.youtube.com/watch?v=keWEE9FwS9o

I suspect they've just deprioritized it for other work.

But I think that was the wrong strategic move - they should have opened it up, together with some 'Tesla AI' demo models, in a colab environment. They can hire new employees to do that - it is separate work from that involved in making the self driving car, and will not block or interfere.

The only reason I think they might not is that they don't want to step on Googles toes - there are very close links between Musk companies and Google, and a direct competitor to Googles TPU product might hurt that relationship more than it generates in revenue.

Re: Tesla Dojo Custom AI Supercomputer at HC34

#32

Post by rcMgD2BwE72F »

cma wrote: post #25
killyourcar wrote: post #11

Earlier quoted context omitted.

Musk is absolutely the reason I would never consider buying another Tesla, having just sold mine. Dude just straight up lies. He may not realize he's lying, but he lies constantly. Edit, because I'm getting downvoted. Here's some examples: self driving, battery swaps, robotic snake chargers, cybertruck windows, Bitcoin won't be converted to fiat, starlink speeds will improve, he will sell his home, first mars mission…

Dear Moon was supposed to be 2022. But he was supposed to land a manned dragon capsule on Mars in 2020 and before that 2018, plus passenger rocket flights from New York to Shanghai for between the price of a coach and business class ticket by 2028 (first made the claim in 2018, later doubled down on the 2028 date around 2020). https://www.theverge.com/2017/2/17/14652026/spacex-red-drago... Starlink satellites were su…

>California gave them almost a billion for "delivering" them.

Source?

Re: Tesla Dojo Custom AI Supercomputer at HC34

#33

Post by mg »

nielsinho wrote: post #30
mg wrote: post #24

It's interesting to see to what extent matrix multiplication and backpropagation are dominating the AI space these days. I wouldn't be surprised if other approaches like genetic programming will make a comeback one day. If there are any papers out there arguing for/against neural networks to stay in the king's seat forever, I would love to see them.

You can take a look here: https://youtu.be/27PYlj-qNb0

That is a genetic algorithm at work.

With "genetic programming" I was referring to applying genetic algorithms to code. So that the result of the evolution is a program that performs well doing a task like driving a car.

Nice research on this has been done by John R. Koza. But nothing new came out in this area for quite a while.

Re: Tesla Dojo Custom AI Supercomputer at HC34

#34

Post by schiffern »

londons_explore wrote: post #31
kjksf wrote: post #15

Earlier quoted context omitted.

It is already Tesla's plan to build AWS-style paid access to Dojo. I think they said that during the first AI Day. Here's a 19 minute supercut of AI Day: https://www.youtube.com/watch?v=keWEE9FwS9o

I suspect they've just deprioritized it for other work. But I think that was the wrong strategic move - they should have opened it up, together with some 'Tesla AI' demo models, in a colab environment. They can hire new employees to do that - it is separate work from that involved in making the self driving car, and will not block or interfere. The only reason I think they might not is that they don't want to step on…

> They can hire new employees

The bottleneck isn't employees, it's chips.

Re: Tesla Dojo Custom AI Supercomputer at HC34

#35

Post by JustFinishedBSG »

mg wrote: post #24

It's interesting to see to what extent matrix multiplication and backpropagation are dominating the AI space these days. I wouldn't be surprised if other approaches like genetic programming will make a comeback one day. If there are any papers out there arguing for/against neural networks to stay in the king's seat forever, I would love to see them.

> I wouldn't be surprised if other approaches like genetic programming will make a comeback one day.

Why would they?

If you assume that your objective is kinda smooth genetic algorithms and related are guaranteed to suck. And most "real" world things of interest can be assumed to be pretty smooth

Re: Tesla Dojo Custom AI Supercomputer at HC34

#36

Post by londons_explore »

schiffern wrote: post #34
londons_explore wrote: post #31

Earlier quoted context omitted.

I suspect they've just deprioritized it for other work. But I think that was the wrong strategic move - they should have opened it up, together with some 'Tesla AI' demo models, in a colab environment. They can hire new employees to do that - it is separate work from that involved in making the self driving car, and will not block or interfere. The only reason I think they might not is that they don't want to step on…

> They can hire new employees The bottleneck isn't employees, it's chips.

If that were the case, they'd have already launched a 'demo' version with very low usage limits. Users can start building/porting their models, and then run them in a few months when the next batch of chips arrives.

Re: Tesla Dojo Custom AI Supercomputer at HC34

#37

Post by freemint »

londons_explore wrote: post #12

If this stuff is any good, Tesla should make it available for everyone to use colab-style. A hosted Jupyter notebook in a sandboxed VM able to send jobs to this new silicon is something that might be possible to set up by a small team in a few months, and could turn into a billion dollar business. As a bonus, Tesla can use revenue from that to grow this supercomputer, while using any spare/unsold capacity for themsel…

Tesla should focus on making full self driving cars finally work.

If you listened to their talk their silicon comes without support for access control and multi tenancy. Letting random people run on this will be an unlimited disaster.

Re: Tesla Dojo Custom AI Supercomputer at HC34

#38

Post by clouddrover »

freemint wrote: post #37
londons_explore wrote: post #12

If this stuff is any good, Tesla should make it available for everyone to use colab-style. A hosted Jupyter notebook in a sandboxed VM able to send jobs to this new silicon is something that might be possible to set up by a small team in a few months, and could turn into a billion dollar business. As a bonus, Tesla can use revenue from that to grow this supercomputer, while using any spare/unsold capacity for themsel…

Tesla should focus on making full self driving cars finally work. If you listened to their talk their silicon comes without support for access control and multi tenancy. Letting random people run on this will be an unlimited disaster.

Tesla should aim for full self parking first. They're behind other manufacturers on that:

https://www.youtube.com/watch?v=nsb2XBAIWyA

Re: Tesla Dojo Custom AI Supercomputer at HC34

#39

Post by dadof4 »

So they have one of the world’s most powerful super computers and they still can’t figure out how to make the car handle an incoming merge lane or how to properly drive down a two lane road without a lane lines…. I grew so frustrated with Tesla constantly trying to solve hard problems without mastering the easy stuff I sold the car.

Re: Tesla Dojo Custom AI Supercomputer at HC34

#40

Post by pclmulqdq »

loser777 wrote: post #22

The most striking thing about the architecture is that it appears so heterogeneous and complex. Considering the vast amount of software/machine learning engineering behind model/data/pipeline parallelism schemes like Megatron-LM and ZeRO (which target hardware topologies that seem almost simple by comparison) I'm curious what abstractions are in place to make this beast of an architecture friendly to programmers. Can…

I'm not sure I would call the architecture very complex. It's about as simple as you can make a scale-out supercomputer. I assume they essentially do static positioning of the cluster for training jobs, and have a translation layer from the TensorFlow middle-end to their thing. Google did a similar thing with their TPUs, so it makes sense that they would have architected TF to accept exotic supercomputers as backends.
Post reply on HN