151–160 of 249 posts
The competition in this space is great but I can't help but wonder what would happen if instead all these companies pooled their resources and went after the goal collectively. There is so much duplication going on and the paths do not seem to me - as an outsider - to be all that divergent, which is usually a pre-condition for having a lot of independent efforts one of which will succeed. It's as if everybody wants t…
That would be a bad idea because like in evolutionary processes you need this diversity of ideas to locate better local optima even if it will take longer.
Awesome presentation. Crazy that they're developing their own training hardware too. It's going to be a very crowded space very soon. Can they really stay ahead of everyone else in the industry? Can it really be cheaper to staff up whole teams to design chips for cutting edge nodes, fabricate them, build supporting hardware and datacenters and compilers, than to just rent some TPUs on Google Cloud? I can see the case…
Earlier quoted context omitted.
> the highest profile people work for the big companies and don’t share their discoveries I have to strongly disagree with this for the specific case of AI/ML. The big company labs are publishing open access papers non-stop, often with code and sometimes even datasets. They're more open than some areas of academia, in fact.
Really? I heard the opposite from someone in the field, who told me that they do publish but it is never the really relevant stuff. I can’t really judge that myself to be honest
The competition in this space is great but I can't help but wonder what would happen if instead all these companies pooled their resources and went after the goal collectively. There is so much duplication going on and the paths do not seem to me - as an outsider - to be all that divergent, which is usually a pre-condition for having a lot of independent efforts one of which will succeed. It's as if everybody wants t…
What I've seen personally is what can be loosely termed "emergent consensus". Historical competitors (and often it gets whittled down to two giants, such as Boeing and Airbus) will work in secret on research. But after years of experimentation arrive at very similar outcomes. An optimal answer that could only be arrived at through constant trial and error, product evolution and iteration.
Regarding Karpathy's PyTorch presentation I don't thing anything that wasn't already public was revealed. The FSD board with custom NPUs is a Work of Art. I like that there are dual redundant streams. And the scale of the dataset is already well know: 4096 HD-images per step!
If I had to speculate, the "Dojo Cluster" may be envisioned as an effort to share data and compute with industry partners as a cloud SaaS product and ancillary revenue stream. But that is pure speculation ;)
Inside Tesla’s Neural Processor In The FSD Chip
https://fuse.wikichip.org/news/2707/inside-teslas-neural-pro...
Earlier quoted context omitted.
Google has Edge TPUs for use outside datacenters, and they don't support training. Neither do the chips Tesla made for their cars. It's a pretty different problem.
I wouldn't be so sure. Edge TPUs could be the exact same architecture than Google Cloud TPUs, but as you need less computation power for inferring than training, they have simply less transistors on the die and could be underclocked. In other words, Cloud TPUs could be the same architecture than Edge TPUs but scaled to an higher frequency and more packed. I guess we need sources to confirm.
Also, training requires a lot more RAM per unit of compute, since it needs to store all past layer activations, whereas for inference, that is unnecessary.
As far as I know, no player who has developed dedicated ML hardware (as opposed to using GPU's) uses the same hardware for both inference and training.
Awesome presentation. Crazy that they're developing their own training hardware too. It's going to be a very crowded space very soon. Can they really stay ahead of everyone else in the industry? Can it really be cheaper to staff up whole teams to design chips for cutting edge nodes, fabricate them, build supporting hardware and datacenters and compilers, than to just rent some TPUs on Google Cloud? I can see the case…
>Also, I'm really curious whether the custom hardware in the cars is benefiting them at all yet. Every feature they've released so far works fine on the previous generation hardware with 1/10 the compute power. The latest OTA finally brings a hardware v3 only feature, traffic cone visualization, and traffic cone automatic lane change.
The trick for level 5 is learning the mapping between the lidar point cloud and the video stream. It’s the best of both worlds.
Earlier quoted context omitted.
> mind boggling to think people trust this crap It's also mind boggling to think we currently trust organic tissue to do this crap, some of which is bathed in psychoactive chemicals. And yet we do, and as a result, horrendous catastrophes occur every minute of every day. > It's all fancy neural networks until someone dies No, that can't be the standard, not when people are dying right now in the current regime. Unles…
No, it needs to be a lower rate since it will kill at random. Today’s rate includes drunk drivers, people on their phone and other “unsafe” drivers. If you are an attentive driver your chance of death would actually go up if the overall death rate was the same.