I wonder what the current owners of Tilera and their series of Tile processors think about Teslas naming here
Proper noun?
Tesla Dojo Custom AI Supercomputer at HC34
71–80 of 102 posts
Re: Tesla Dojo Custom AI Supercomputer at HC34
#72Earlier quoted context omitted.
Musk promising FSD by the end of the year - year after year - is one major thing that tarnished the Tesla image in my mind. I now assume that he has to believe that in order to avoid lawsuits. Maybe the same is with the computing infrastructure. They need to build it to show that they honestly believed it could work - even though they failed to deliver on their promises many times. That said, they do make progress an…
Musk is absolutely the reason I would never consider buying another Tesla, having just sold mine. Dude just straight up lies. He may not realize he's lying, but he lies constantly. Edit, because I'm getting downvoted. Here's some examples: self driving, battery swaps, robotic snake chargers, cybertruck windows, Bitcoin won't be converted to fiat, starlink speeds will improve, he will sell his home, first mars mission…
The way I would put it is that he's been attempting to erect a reality distortion field to distract from that fact it will take much longer than he predicts to achieve some of his more distant goals. And that RDF basically means promising things that can't be delivered, for some time.
I almost bought a tesla a year ago based on the idea FSD was going to work at some point in the future but after Elon announced some last minute hardware changes to my model's radar (removing it), and I read about all the steps you should perform at pickup, I went and bought a toyota instead.
Re: Tesla Dojo Custom AI Supercomputer at HC34
#73Earlier quoted context omitted.
All of that is irrelevant to Musk calling the man a “pedophile” with zero evidence.
Pedophile was not the exact insult used why do you have it in quotes as if it were spoken verbatim?
You left out all the important details. No only did he say "pedo guy" specifically to implicate the guy as a pedophile, he hired a sketchy guy to investigate him.
Also: """In emails to Buzzfeed News in September 2018, Musk also called him a "child rapist" and accused him of moving to Chiang Rai for a "child bride who was about 12 years old." He added that he "f---ing hope[d]" Unsworth would sue him. """
The person we're talking about (Unsworth) was the known expert on that cave, who was handling an emergency situation. Elon inserted himself in this situation unnecessarily and when he was rebuffed, resorted to some very serious things to say and took unwise actions (https://www.businessinsider.com/elon-musk-convicted-felon-in...)
Re: Tesla Dojo Custom AI Supercomputer at HC34
#74Earlier quoted context omitted.
They'll be at a process disadvantage. D1 is allegedly TSMC 7nm, as per last year's information ( https://www.tomshardware.com/news/tesla-d1-ai-chip ). An ASIC can strip out features they don't need and save some space. But a good chunk of modern GPUs are memory-controllers, registers, and SIMD-cores. And modern GPUs (both AMD's MI250x and NVidia's A100) have 16-bit matrix multiplication units (aka: Tensor cores). Onc…
Nvidia’s stuff is good but it’s pretty high margin. They don’t give access to it for cheap,& in the last five years, the cost per unit performance has been nearly flat. They’re also more generalized than Tesla needs. Performance advantages from process shrinks have also stagnated. A good time for a custom approach.
NVidia is claiming 1000 16-bit Tensor-FLOPS on Hopper: https://developer.nvidia.com/blog/nvidia-hopper-architecture...
While Tesla is claiming less than 400 Tensor-FLOPS on D1.
So yeah, the claims of NVidia's GH100 / Hopper GPU are an order of magnitude faster than the D1. Which is no surprise, because when your transistors are less than 1/2 the size of the competition, you can easily have 2x the performance in a embarrassingly parallel problem.
--------
Note that the A100, released in 2020, offers 312 TFlops of 16-bit Tensor matrix-multiplication operations per second. Meaning D1's chip is barely competitive against the 2-year-old NVidia A100, let alone the next-generation Hopper.
And note that NVidia's server-GPUs (like A100 or GH100) already come in prepackaged supercomputer formats with extremely-high speed data-links between them. See the DGX line of NVidia supercomputers. https://www.nvidia.com/en-us/data-center/dgx-station-a100/
--------
You can't beat physics. Smaller transistors use less power, while more transistors offer more parallelism. A process-node advantage is huge.
Re: Tesla Dojo Custom AI Supercomputer at HC34
#75Earlier quoted context omitted.
Curious... did you sell your Tesla because he lies? or for other issues? What replacement car/company did you decide to go with? Volkswagen?
That's the tough thing. Tesla still has the early mover advantage in the EV space and despite all the dumb stuff, most other EVs can't yet compete in the basic driving experience for the price (assuming no FSD). I just want a Toyota Corolla or Camry-style car, for a decent price, that is electric. The i3 was kinda close to what I wanted in spirit, but they had to make it look all "tech" and "future-y".
Not a real answer, but I got a RAV 4 Prime- it's hybrid- but I never engage the engine. I've driven 6000 miles on two tanks of gas.
Re: Tesla Dojo Custom AI Supercomputer at HC34
#76Earlier quoted context omitted.
I can understand some of these 'lies' but most of them are not really lies. I can understand being angry about self driving, most of the rest of it is pretty absurd. Most of these are either things that are simple changes in strategy, mostly good choice regards to internal investment and roadmap. Others are research projects that were never promised to be products. I really don't understand how anybody can be angry a…
> Did you personally sign a contract with Tesla for a battery swap station or something? I've found that a surprisingly large number of people are really committed to the idea that battery swapping is the only way that EVs could work. These are personally offended that Tesla abandoned it. But doing battery swapping well is much more capital intensive than their supercharger strategy was. They could never have afforde…
And people act like some Tesla scammed California by not rolling out the technology widely. But this was never a requirement of the grand.
The whole point of such grants is to figure out if its a commercially viable solution or not, and it wasn't. Or at least not the best one.
Re: Tesla Dojo Custom AI Supercomputer at HC34
#77It's interesting to see to what extent matrix multiplication and backpropagation are dominating the AI space these days. I wouldn't be surprised if other approaches like genetic programming will make a comeback one day. If there are any papers out there arguing for/against neural networks to stay in the king's seat forever, I would love to see them.
I think you have to start to narrow your question before you get results. That is, for the use-case of fast moving sensor inputs (like in a car) or reviewing a million videos to find identifiable patterns, cNN performs, but other specific queries in domain knowledge or patterns of robotic movement, decision trees and random forest immediately differentiate themselves. So papers on specific topics that are not the specialty of cNN might demonstrate effectiveness, and leave it up to the reader/critic to compare widely against other uses overall.
Re: Tesla Dojo Custom AI Supercomputer at HC34
#78Earlier quoted context omitted.
It is already Tesla's plan to build AWS-style paid access to Dojo. I think they said that during the first AI Day. Here's a 19 minute supercut of AI Day: https://www.youtube.com/watch?v=keWEE9FwS9o
I suspect they've just deprioritized it for other work. But I think that was the wrong strategic move - they should have opened it up, together with some 'Tesla AI' demo models, in a colab environment. They can hire new employees to do that - it is separate work from that involved in making the self driving car, and will not block or interfere. The only reason I think they might not is that they don't want to step on…
>and a direct competitor to Googles TPU product might hurt that relationship more than it generates in revenue.
There is approximately zero chance that this is a consideration.
My guess would be that, like a lot of things under the Elon Musk umbrella, what they claim they are able to do in theory diverges far from reality. We've seen similar slideshows.
Re: Tesla Dojo Custom AI Supercomputer at HC34
#79Earlier quoted context omitted.
Maybe it not an easy problem?
I once drove a Tesla on a road that was asphalt covered over what appeared to earlier be a concrete road. The concrete was visible through cracks in the top asphalt surface in straight lines. To a camera, they sort of looked like lane lines, but they were right down the middle of the lanes! I was manually driving, but the car was constantly freaking out because it thought I was running off the road. :lol Yes, it can…
>I was manually driving, but the car was constantly freaking out because it thought I was running off the road. :lol Yes, it can be a hard problem.
If cracks in asphalt fooling the car's AI is a "hard problem", than full self-driving is doomed.
Re: Tesla Dojo Custom AI Supercomputer at HC34
#80The most striking thing about the architecture is that it appears so heterogeneous and complex. Considering the vast amount of software/machine learning engineering behind model/data/pipeline parallelism schemes like Megatron-LM and ZeRO (which target hardware topologies that seem almost simple by comparison) I'm curious what abstractions are in place to make this beast of an architecture friendly to programmers. Can…
I'm not sure I would call the architecture very complex. It's about as simple as you can make a scale-out supercomputer. I assume they essentially do static positioning of the cluster for training jobs, and have a translation layer from the TensorFlow middle-end to their thing. Google did a similar thing with their TPUs, so it makes sense that they would have architected TF to accept exotic supercomputers as backends…
Tensorflow (and pytorch) convert your computation graph (constructed in python) to XLA, which is then specialized to a specific hardware architecture. XLA is a good intermediate language and in fact, you can convert some memory movement in the graph to network calls, allowing you to run on parallel systems (like a cluster of GPUs or TPUs with their own non-host-based networking).
It still requires many experts, both to write the XLA to hardware translation, and ML engineers who know how to write TF python that executes quickly.
(note: Google has transitioned many projects to Jax, which also writes to XLA, as TF ended up being a bit of a pig with wings)