Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Andrej Karpathy talks about how Tesla's NNs are structured and trained [video]

youtube.com

Post reply on HN
Print view View original

101–110 of 249 posts

Re: Andrej Karpathy talks about how Tesla's NNs are structured and trained [video]

#101

Post by Udik »

chronic739i wrote: post #90
jeffshek wrote: post #86

Earlier quoted context omitted.

At Tesla's scale and priorities, they'd probably be less keen on using external cloud providers. Using TPUs at their scale would certainly require Google's AI consultants to supervise which isn't ideal for Tesla. Not agreeing or disagreeing with their decisions, but if you have the resources, you can certainly design a custom chip that performs a specific type of task very well that beats other competitors. Nvidia's…

> At Tesla's scale and priorities, they'd probably be less keen on using external cloud providers. By hardware-hours, Tesla is hardly one of the top companies training deep networks. Planet Labs (satellite imaging), Netflix, Pornhub, to name a few. What's the info they'd leak to the Google consultants? How much data or TPUs they're using? This is practically public information.

What does Netflix do with NNs?

Re: Andrej Karpathy talks about how Tesla's NNs are structured and trained [video]

#102

Post by semi-extrinsic »

panarky wrote: post #31
mindfulplay wrote: post #22

Just listening to this talk scares me. The amount of errors - even in a seemingly normal, sunny day - is mind boggling to think people trust this crap. How can we rely on the output of eight cameras? This is not a kid's science project. It's all fancy neural networks until someone dies. Pretty callous and Silicon valley-mindset for such an important and critical function of the car. Will never buy a Tesla after havin…

> mind boggling to think people trust this crap It's also mind boggling to think we currently trust organic tissue to do this crap, some of which is bathed in psychoactive chemicals. And yet we do, and as a result, horrendous catastrophes occur every minute of every day. > It's all fancy neural networks until someone dies No, that can't be the standard, not when people are dying right now in the current regime. Unles…

To really be a measurable improvement over humans, and especially an improvement that is statistically distinguishable from just letting the safety tech of 2019 percolate into the average car, self-driving needs to achieve fatality rates around 0.1 per billion miles.

Current averages for the US are around 10 per billion miles and decreasing; best countries in Europe are already below 5 per billion miles.

There's some really interesting progress both on vehicles monitoring driver attention, and on monitoring for alcohol in the air, that would yield substantial improvements even beyond 2019 tech. I have no doubt we'll see fatality rates below 1 per billion miles in Western Europe within a decade.

The corollary of the statistics quoted above is that you need to observe your self-driving vehicle system over tens of billions of miles before you even know if you're safer or not.

Re: Andrej Karpathy talks about how Tesla's NNs are structured and trained [video]

#103

Post by rawoke083600 »

mindfulplay wrote: post #28
freediver wrote: post #26

Earlier quoted context omitted.

You are right this approach is scary, and it is astonishingly innacurate (im a tesla owner). However the reason are not eight cameras. You should be able to drive fine with just one camera (thought experiment: could you drive a car 1000 miles from you, just by seeing what the driver of that car would see, no extra cameras, sensors or lidars?).

I work closely on ondevice OCR technology - something that has improved so much and still cannot figure out the difference between I and 1. And that took many decades not even years and in a very constrained problem space. Releasing a "self driving" metal torpedo even if it's a limited Summons should at least be regulated if not outright banned. I feel like even a single accident let alone death should put these idio…

How do we currently regulate and put behind bars the human drivers that drives the metal torpedoes like idiots ? Specifically the ones that are not caught.

Re: Andrej Karpathy talks about how Tesla's NNs are structured and trained [video]

#104

Post by choppaface »

jeffshek wrote: post #86
modeless wrote: post #70

Awesome presentation. Crazy that they're developing their own training hardware too. It's going to be a very crowded space very soon. Can they really stay ahead of everyone else in the industry? Can it really be cheaper to staff up whole teams to design chips for cutting edge nodes, fabricate them, build supporting hardware and datacenters and compilers, than to just rent some TPUs on Google Cloud? I can see the case…

At Tesla's scale and priorities, they'd probably be less keen on using external cloud providers. Using TPUs at their scale would certainly require Google's AI consultants to supervise which isn't ideal for Tesla. Not agreeing or disagreeing with their decisions, but if you have the resources, you can certainly design a custom chip that performs a specific type of task very well that beats other competitors. Nvidia's…

> At Tesla's scale and priorities, they'd probably be less keen on using external cloud providers.

Not sure if it’s still the case today, but previously Tesla’s training was done on-prem and with their own in-house Tensorflow clone.

And yes, if you get TPUs from GCloud, you are likely to be working with their engineers to get things working. Those engineers tend not to have much business conflict of interest, though. They want to help you because your problems are likely more interesting than what they’d otherwise be assigned.

Re: Andrej Karpathy talks about how Tesla's NNs are structured and trained [video]

#105

Post by jacquesm »

The competition in this space is great but I can't help but wonder what would happen if instead all these companies pooled their resources and went after the goal collectively. There is so much duplication going on and the paths do not seem to me - as an outsider - to be all that divergent, which is usually a pre-condition for having a lot of independent efforts one of which will succeed.

It's as if everybody wants to be the one to exclusively own the tech. Imagine every car manufacturer having a completely different take on what a car should be like from a safety perspective. We have standards bodies for a reason and given the fact that there are plenty of lives at stake here maybe for once the monetary angle should get a back-seat (pun intended) to safety and a joint effort is called for. That would also stop people dying because operators of unsafe software are trying to make up for their late entry by 'moving fast and breaking things' where in this case the things are pedestrians, cyclists and other traffic participants who have no share in the monetary gain.

Re: Andrej Karpathy talks about how Tesla's NNs are structured and trained [video]

#106

Post by choppaface »

throwaway010718 wrote: post #15
timzaman wrote: post #11

His team is hiring; https://www.tesla.com/careers/job/software-engineerdeeplearn... https://www.tesla.com/careers/job/machine-learninginfrastruc... https://www.tesla.com/careers/job/machine-learningscientista...

Any guess what the compensation is like for these positions ?

Less than Uber, but more than Waymo, who is only offering ~$20k-ish stock packages like a late start-up who expects to 10x. Depends on how you value Tesla stock, though. See levels.fyi

Re: Andrej Karpathy talks about how Tesla's NNs are structured and trained [video]

#107

Post by [deleted] »

choppaface wrote: post #106
throwaway010718 wrote: post #15

Earlier quoted context omitted.

Any guess what the compensation is like for these positions ?

Less than Uber, but more than Waymo, who is only offering ~$20k-ish stock packages like a late start-up who expects to 10x. Depends on how you value Tesla stock, though. See levels.fyi

[deleted]

Re: Andrej Karpathy talks about how Tesla's NNs are structured and trained [video]

#108

Post by paraschopra »

jacquesm wrote: post #105

The competition in this space is great but I can't help but wonder what would happen if instead all these companies pooled their resources and went after the goal collectively. There is so much duplication going on and the paths do not seem to me - as an outsider - to be all that divergent, which is usually a pre-condition for having a lot of independent efforts one of which will succeed. It's as if everybody wants t…

Standards shouldn't emerge too soon. I think for self driving tech, at the current stage, competition is good because there are lots of unsolved questions. Competition will ensure the best tech is ultimately available to consumers.

Of course, it's not a binary choice. Things like data should probably be pooled but the use of data in tech should compete.

Re: Andrej Karpathy talks about how Tesla's NNs are structured and trained [video]

#109

Post by paraschopra »

eanzenberg wrote: post #62

One thing I didn't quite understand is how training sub-graphs in parallel works. If you are editing a sub-graph of a monolith type model, aren't you affecting other graphs that have dependencies on the one you're editing? If these are independent graphs, then what's a "sub-graph" even mean?

I think their architecture might be their secret sauce. But I'm curious about this too.

Re: Andrej Karpathy talks about how Tesla's NNs are structured and trained [video]

#110

Post by jacquesm »

xiphias2 wrote: post #75
adrr wrote: post #66

Earlier quoted context omitted.

Elon belittles lidar saying it is doomed and will never work yet Waymo and Cruise will probably be operating self driving taxi fleets in California next year. Tesla deserves getting dumped on for those comments because they are no where near self driving.

Andrej Karpathy just started working on Tesla's software 2 years ago, before what Chris Lattner did was a mess (he wanted to just have 1 task that learns magically everything), Andrej had to start everything from scratch. Waymo had a 20 year advantage, but Google lost many key people there in the meantime as Larry Page didn't want to launch partial self driving. I think both approaches are great and I wouldn't want t…

> Larry Page didn't want to launch partial self driving

That's the responsible thing to do.

Post reply on HN