201–210 of 249 posts
I'm still amazed that Teslas team isn't using a map... I know maps get outdated and are sometimes wrong, but having inaccurate knowledge of what's around the corner is far far more helpful than not having any clue whats around the corner. The smart solution would be to consider a map a probabilistic thing, which neural networks are really good at handling.
I'm still amazed that Teslas team isn't using a map... I know maps get outdated and are sometimes wrong, but having inaccurate knowledge of what's around the corner is far far more helpful than not having any clue whats around the corner. The smart solution would be to consider a map a probabilistic thing, which neural networks are really good at handling.
I'm still amazed Tesla has decided not to use lidar and instead just stick with cheap cameras. Better sensors are there, they're available, they're cheap and they can probably "see" better than plain old cameras... it doesn't make too much sense not to use them IMHO. But then again, I am not coding NNs for Tesla...
Awesome presentation. Crazy that they're developing their own training hardware too. It's going to be a very crowded space very soon. Can they really stay ahead of everyone else in the industry? Can it really be cheaper to staff up whole teams to design chips for cutting edge nodes, fabricate them, build supporting hardware and datacenters and compilers, than to just rent some TPUs on Google Cloud? I can see the case…
Nothing crazy about it. TPU-like stuff is ~10x the energy efficiency of GPUs and several times the speed. When you're spending megawatt-hours and days to train a single model, it adds up in both real and opportunity costs. Also, Google TPU TOS prohibits the use of TPUs for stuff that competes with Google (and I'm assuming with other companies under Alphabet umbrella), at Google's sole determination. Not that it would…
GPU advantage: more refined ecosystem and you can buy them for $TPU advantage: Colab has a free tier that lets you play with them at no charge and if GOOG has sweat more software engineering blood and tears into your model's functions, it will run better on them.
All IMO of course. And deep down it can get more complicated than that, but I salute GOOG for being the first company to ship competitive AI HW, doubly so at scale.
The competition in this space is great but I can't help but wonder what would happen if instead all these companies pooled their resources and went after the goal collectively. There is so much duplication going on and the paths do not seem to me - as an outsider - to be all that divergent, which is usually a pre-condition for having a lot of independent efforts one of which will succeed. It's as if everybody wants t…
> There is so much duplication going on In the self-driving world, the duplication is necessary - different companies are taking different directions, and nobody really knows which will work out. In the ML hardware world, the duplication is mostly unnecessary. People are developing their own inference hardware ASIC's because they're relatively simple (compared to designing a CPU from scratch, designing a TPU is prett…
So which one design you suggest we all use for all our ML needs?
The competition in this space is great but I can't help but wonder what would happen if instead all these companies pooled their resources and went after the goal collectively. There is so much duplication going on and the paths do not seem to me - as an outsider - to be all that divergent, which is usually a pre-condition for having a lot of independent efforts one of which will succeed. It's as if everybody wants t…
Earlier quoted context omitted.
> The competition in this space is great but I can't help but wonder what would happen if instead all these companies pooled their resources and went after the goal collectively. It would probably slow down. 9 women can't have a baby in 1 month. Besides that, the disagreements about approach, politics, or eventual competitive interests would probably bring things to a halt for a long time. I don't think the solutions…
Exactly. Look at Human Brain Project: $1B and 10 years later what exactly have they achieved? Just like you said - disagreements about approach, politics, or eventual competitive interests did bring things to a halt for a long time
Earlier quoted context omitted.
Exactly. Look at Human Brain Project: $1B and 10 years later what exactly have they achieved? Just like you said - disagreements about approach, politics, or eventual competitive interests did bring things to a halt for a long time
The human brain project was DOA from day #1. Unrealistic goals, no clear reason why more money would lead to better results and no concrete deliverables that anybody needed.
Awesome presentation. Crazy that they're developing their own training hardware too. It's going to be a very crowded space very soon. Can they really stay ahead of everyone else in the industry? Can it really be cheaper to staff up whole teams to design chips for cutting edge nodes, fabricate them, build supporting hardware and datacenters and compilers, than to just rent some TPUs on Google Cloud? I can see the case…
I would expect their training hardware to be something specifically aimed at optimizing memory bandwidth to support distributing training of their “shared” hydra feature. It’s interesting that the shared hydra feature extractor is able to converge as they keep adding more and more output predictions under a training regime of interleaving asynchronous updates to the model from different predictor networks ...
Seems to me the formula they are pursuing with custom hardware might be to support a strategy of 1. keep adding more predictions based on same feature 2. Increase the span of time represented by batches used to train the recurrent networks
Both pursuits seem very data efficient in terms of the amount of training data they could conceivably collect per unit time of observation ...
Custom hardware with a problem specific memory architecture aimed at efficiently supporting training with very large rnn time slices could be developed that’s more about “make it possible to train this proposed model at all” rather than “make it faster/cheaper to train existing common model architectures”. When custom hardware is required to make it possible to train the model they want, the validity of the hardware development cost bet might end up being more about the effectiveness of the model they think they want than it is about maintaining general purpose performance parity vs any off the shelf hardware options ...
Earlier quoted context omitted.
I'm still amazed Tesla has decided not to use lidar and instead just stick with cheap cameras. Better sensors are there, they're available, they're cheap and they can probably "see" better than plain old cameras... it doesn't make too much sense not to use them IMHO. But then again, I am not coding NNs for Tesla...
Kaparthy has a good presentation outlining why they aren’t focused on lidar.. it’s pretty compelling logic.
Earlier quoted context omitted.
citation, please
https://www.tesla.com/VehicleSafetyReport
Autopilot has the luxury of being able to / requiring disengagement in less-than-favorable driving conditions. Humans don't.
A fair comparison would be comparing against the miles/conditions/roads driven.