Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Tesla Dojo Chip Is Impressive, but There Are Some Major Technical Issues

semianalysis.com

Post reply on HN
Print view View original

51–60 of 112 posts

Re: The Tesla Dojo Chip Is Impressive, but There Are Some Major Technical Issues

#51

Post by dimitrios1 »

jvanderbot wrote: post #44
dragontamer wrote: post #39

Earlier quoted context omitted.

> could we have had an autonomous rail system by now? We already have autonomous rail systems. Its called positive train control and was fully implemented like a year or two ago (mandated in 2009, but you know how government works, lol) https://en.wikipedia.org/wiki/Positive_train_control The train conductor has become more-and-more automated to remove the chance of human error. It works with a system of very reliabl…

I wonder if, for the cost spent on CPU-and-engineer-driven research and development of autonomous cars, if we could have had nationwide autonomous rail rolled out .

Nope, requires a sane society that gathers around and does whats best for the collective whole. What we have, however, is a kleoptocracy, and being as such, the opportunity to extract more wealth, faster, and easier, existed with selling the promise of autonomous vehicles. If the same incentives existed for autonomous rail, it would have happened already.

Re: The Tesla Dojo Chip Is Impressive, but There Are Some Major Technical Issues

#52

Post by samstave »

whatshisface wrote: post #40
jvanderbot wrote: post #37

As Hamming suggested in "Art of doing science and engineering", when you want to make something autonomous, you usually have to build a completely different device that solves the same problem, rather than automating the same device. I wonder. For all the money thrown into self-driving cars research , could we have had an autonomous rail system by now? The technology for mostly-autonomous rail is well understood. Mos…

The labor saving advantages of carrying 100 people on the same vehicle are so enormous that there is little motivation if any to quit paying conductors and engineers.

The profit motivation of having 100 people being required to buy gas is the anti-motivation in capitalist oligarchies

Re: The Tesla Dojo Chip Is Impressive, but There Are Some Major Technical Issues

#53

Post by HPsquared »

jvanderbot wrote: post #37

As Hamming suggested in "Art of doing science and engineering", when you want to make something autonomous, you usually have to build a completely different device that solves the same problem, rather than automating the same device. I wonder. For all the money thrown into self-driving cars research , could we have had an autonomous rail system by now? The technology for mostly-autonomous rail is well understood. Mos…

We could go the other way and have humanoid robot drivers that can get in and drive any car. Now that'd be difficult!

Re: The Tesla Dojo Chip Is Impressive, but There Are Some Major Technical Issues

#54

Post by modeless »

I'm glad people are exploring the design space. To some extent the training techniques and neural net architectures need to be tailored to the hardware. Nvidia isn't on top just because they're good at chip design, but because people have chosen to focus research effort on techniques that work well on Nvidia hardware. New hardware may allow new techniques to shine.

New hardware architectures can't really be used to their full potential without years of research into techniques that are suited for them. The more people who have access to the hardware, the faster we can discover those techniques. If Tesla is serious about their hardware project, they need to offer it to the public as some kind of cloud training system. They don't have enough people internally to develop everything themselves in a short enough time to remain competitive with the rest of the industry.

Re: The Tesla Dojo Chip Is Impressive, but There Are Some Major Technical Issues

#55

Post by 2bitencryption »

from the article:

> but the short of it is that their unique system on wafer packaging and chip design choices potentially allow an order magnitude advantage over competing AI hardware in training of massive multi-trillion parameter networks.

I kind of wonder if Tesla is building the Juicero of self-driving. [0]

Beautifully designed. An absolute marvel of engineering. The result of brilliant people with tons of money using every ounce of their knowledge to create something wonderful.

Except... you could just squeeze the bag. You could just use LIDAR. You could just use your hands to squish the fruit and get something just as good. You could just (etc etc).

No doubt future Teslas will be supercomputers on wheels. But what if all those trillions of parameters spent trying to compose 3D worlds out of 2D images is pointless if you can just get a scanner that operates in 3D space to begin with??

[0] https://www.theguardian.com/technology/2017/sep/01/juicero-s...

Re: The Tesla Dojo Chip Is Impressive, but There Are Some Major Technical Issues

#56

Post by jeffbee »

dragontamer wrote: post #33
zaptrem wrote: post #22

Earlier quoted context omitted.

I agree with your points on the robotaxi front, but there are many other problems that will totally benefit from a bigger training computer.

But Tesla isn't a cloud-provider company, nor is it a hardware company. None of the technical specs, assembly language, API, SDKs or whatnot have been released for Dojo. The model of "someone will find this training computer useful" is... fine. Google TPUs, NVidia DGX, Intel Xe-HPC, AMD MI100, Cerebras wafer scale AI. These are computers that nominally are aiming for the market of selling computers / APIs / SDKs that…

You forgot the Hauwei Ascend 910. 4096 of them in a rack is an easy exaflop.

Re: The Tesla Dojo Chip Is Impressive, but There Are Some Major Technical Issues

#57

Post by thesausageking »

The Q&A section on their compiler and software that the author links to is very interesting:

https://www.youtube.com/watch?v=j0z4FweCy4M&t=8047s

It sounds like they're going to have write a ton of custom software in order to use this hardware at scale. And, based on the team being speechless when asked a follow up question, it doesn't sound like they know (yet) how they're going to solve this.

Nvidia gets a lot of credit for their hardware advances, but what really what their chips work so well for deep learning was the huge software stack they created around CUDA.

Underestimating the software investment required has plagued a lot of AI chip startups. It doesn't sound like Tesla is immune to this.

Re: The Tesla Dojo Chip Is Impressive, but There Are Some Major Technical Issues

#58

Post by mynameisvlad »

jeffbee wrote: post #46
zdragnar wrote: post #41

Earlier quoted context omitted.

We could have possibly automated some existing rail, but I am not altogether certain that doing so would have lead to any significant improvements in cost or efficiency. Actually laying the infrastructure for mass transit via rail is an entirely different league of cost from what has been dumped into self driving cars. We have a hard enough time agreeing on how to do light rail transit in places that want it, and the…

A self-driving streetcar, that only has 1 degree of freedom, might be a practical problem to solve and maybe if it existed we would not have had to go through the 90-day outage of VTA light rail service that the Bay Area just experienced, an experience which in all likelihood killed VTA light rail forever.

The VTA control building/center was badly damaged in the shooting that caused the outage you're talking about. Surely a fully automated light rail line would require everything in said building to likely be running at full capacity to work.

Also, it was a result of a mass shooting. Have some tact.

Re: The Tesla Dojo Chip Is Impressive, but There Are Some Major Technical Issues

#59

Post by sgrove »

dragontamer wrote: post #49
jvanderbot wrote: post #44

Earlier quoted context omitted.

I wonder if, for the cost spent on CPU-and-engineer-driven research and development of autonomous cars, if we could have had nationwide autonomous rail rolled out .

https://railroads.dot.gov/train-control/ptc/positive-train-c... > On December 29, 2020, FRA announced that PTC technology is in operation on all 57,536 required freight and passenger railroad route miles, prior to the December 31, 2020 statutory deadline set forth by Congress. We got that. We literally got that.

I suspect the OP may be more interested in nationwide, convenient, affordable, automated passenger rail.

Which in the US is possibly (?) several orders of magnitude more in terms of capital investment compared to both electrifying the entire ICE fleet and automating it.

Re: The Tesla Dojo Chip Is Impressive, but There Are Some Major Technical Issues

#60

Post by arnaudsm »

2bitencryption wrote: post #55

from the article: > but the short of it is that their unique system on wafer packaging and chip design choices potentially allow an order magnitude advantage over competing AI hardware in training of massive multi-trillion parameter networks. I kind of wonder if Tesla is building the Juicero of self-driving. [0] Beautifully designed. An absolute marvel of engineering. The result of brilliant people with tons of money…

The Juicero comparison doesn't hold up. LIDAR is 10x more expensive than RGB, but neither reach lvl5 at the moment. I'm glad multiple companies try multiple paths, it's the best way to avoid a research dead-end.
Post reply on HN