Wow
Andrej Karpathy talks about how Tesla's NNs are structured and trained [video]
61–70 of 249 posts
Re: Andrej Karpathy talks about how Tesla's NNs are structured and trained [video]
#62One thing I didn't quite understand is how training sub-graphs in parallel works. If you are editing a sub-graph of a monolith type model, aren't you affecting other graphs that have dependencies on the one you're editing? If these are independent graphs, then what's a "sub-graph" even mean?
Re: Andrej Karpathy talks about how Tesla's NNs are structured and trained [video]
#63Earlier quoted context omitted.
No, the appropriate standard for a self-driving car is the driver they replace. If they're safer than whoever would have been driving (be it on a saturday night or not) then they're a net win. Right now the average idiots are buying more Teslas than mail carriers and cab drivers, though that might change I guess. Numbers are numbers.
If you like numbers, then you'll like this one: Humans achieve 7 9's of reliability when driving a car, as measured for fatalities by time, or 8 9's if you measure by miles. I've yet to see a computer just stay up that long, never mind actually doing anything the whole time. (If you use the collision rate instead, the numbers are about 500 times worse - but that's still quite a lot of 9's.) People have this fixed tho…
You are using the 9s term wrong. It is supposed to be used with unitless numbers. Not crashes per hour or per mile.
Silly example of why: humans are zero nines of reliability if you talk about crashes per parsec.
Re: Andrej Karpathy talks about how Tesla's NNs are structured and trained [video]
#64Just listening to this talk scares me. The amount of errors - even in a seemingly normal, sunny day - is mind boggling to think people trust this crap. How can we rely on the output of eight cameras? This is not a kid's science project. It's all fancy neural networks until someone dies. Pretty callous and Silicon valley-mindset for such an important and critical function of the car. Will never buy a Tesla after havin…
Same. Watching the AI visualization of summons in-action was horrifying, and made clear why many have reported summons mode as resembling a drunk person navigating a parking lot.
Yes it's not perfected yet, which is why it requires human supervision for now. Having it operate in the wild as it is now (again, under human supervision) will help it become less horrifying, which I think you would agree is what we want.
Re: Andrej Karpathy talks about how Tesla's NNs are structured and trained [video]
#65His team is hiring; https://www.tesla.com/careers/job/software-engineerdeeplearn... https://www.tesla.com/careers/job/machine-learninginfrastruc... https://www.tesla.com/careers/job/machine-learningscientista...
Thanks Tim! Would love to apply, but still a student. Hoping to join in the future given how nicely orchestrated your team has been training nets.
Apply for internship. Doing well on an internship is a great way to get a foot in the door after graduation.
Re: Andrej Karpathy talks about how Tesla's NNs are structured and trained [video]
#66Meanwhile Waymo is way ahead.
Do you belittle everyone that gets second place in the Olympics because the winner is "way ahead"? Your comment just reeks of anger and hostility. It seems like you'd rather Tesla didn't try at all, and instead we all just give up and go back to the status quo.
Elon belittles lidar saying it is doomed and will never work yet Waymo and Cruise will probably be operating self driving taxi fleets in California next year. Tesla deserves getting dumped on for those comments because they are no where near self driving.
Re: Andrej Karpathy talks about how Tesla's NNs are structured and trained [video]
#67Earlier quoted context omitted.
citation, please
https://www.tesla.com/VehicleSafetyReport
The autopilot is a drive assist , I can explain that there are less accidents when drive assists is one because
- the driver is still present and will most of the time intervene and save his life(there are some youtube videos where the Tesla AI was trying to kill the driver and those incidents don't appear as accidents int the stats)
- "autopilot" is engaged mostly on highways , but the statistics are not accounting for this
- they are also comparing the safety of a new car versus the median of all cars(old and new, cheap or expensive) on all demographics(how many teens own a Tesla?)
I would like that Tesla make all the data open or have an independent group analyze it, including the disengagements, I wonder how isolated are the cases where the car is driving you into trucks or stone wall but the drivers intervened and saved themselves and also saved Tesla from a bad statistic.
Re: Andrej Karpathy talks about how Tesla's NNs are structured and trained [video]
#68Earlier quoted context omitted.
If you like numbers, then you'll like this one: Humans achieve 7 9's of reliability when driving a car, as measured for fatalities by time, or 8 9's if you measure by miles. I've yet to see a computer just stay up that long, never mind actually doing anything the whole time. (If you use the collision rate instead, the numbers are about 500 times worse - but that's still quite a lot of 9's.) People have this fixed tho…
You are using the 9s term wrong. It is supposed to be used with unitless numbers. Not crashes per hour or per mile. Silly example of why: humans are zero nines of reliability if you talk about crashes per parsec.
> Not crashes per hour or per mile.
But that's not what I measured. I measured safe minutes of driving as a ratio to unsafe minutes. Which is a unitless number.
For 7 9's I assumed a crash had a 5 minute lead-in of unsafe driving before the actual crash, and that average driving speed was 30 mph.
If you assume the bad driving is 30 seconds (for example the accident in Tuscon the Uber car saw the pedestrian around 30 seconds before crashing), then you can add another 9, making both figures 8 9's of reliability.
Re: Andrej Karpathy talks about how Tesla's NNs are structured and trained [video]
#69One thing I didn't quite understand is how training sub-graphs in parallel works. If you are editing a sub-graph of a monolith type model, aren't you affecting other graphs that have dependencies on the one you're editing? If these are independent graphs, then what's a "sub-graph" even mean?
In PyTorch you have full control on the graph and weight, everything feels like Python.
So feeding some of the learning between “sub-graph” is easy. Not sure if this is possible on Tensorflow/Keras?
He describes the sub-graph training in the context that they they have all the predictors in one big model, and with control of the network can feedforward and train sub-graph (read sub-parts) of the model.
Re: Andrej Karpathy talks about how Tesla's NNs are structured and trained [video]
#70Awesome presentation. Crazy that they're developing their own training hardware too. It's going to be a very crowded space very soon. Can they really stay ahead of everyone else in the industry? Can it really be cheaper to staff up whole teams to design chips for cutting edge nodes, fabricate them, build supporting hardware and datacenters and compilers, than to just rent some TPUs on Google Cloud?
I can see the case for doing their own edge hardware for the cars (barely), but I really don't think doing training hardware will pay off for them. If they're serious about it, they should spin it out as a separate business to spread the development cost over a larger customer base.
Also, I'm really curious whether the custom hardware in the cars is benefiting them at all yet. Every feature they've released so far works fine on the previous generation hardware with 1/10 the compute power. At some point won't they need to start training radically larger networks to take advantage of all that untapped compute power?