Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tesla deploys massive new Autopilot neural net in v9

electrek.co

Post reply on HN
Print view View original

61–70 of 84 posts

Re: Tesla deploys massive new Autopilot neural net in v9

#61

Post by shoyer »

Someone wrote: post #30

”When you increase the number of parameters (weights) in an NN by a factor of 5 you don’t just get 5 times the capacity and need 5 times as much training data. In terms of expressive capacity increase it’s more akin to a number with 5 times as many digits. So if V8’s expressive capacity was 10, V9’s capacity is more like 100,000.” I find it very, very hard to believe that. I know ‘expressive power’ is a fairly vague…

Strongly agreed. I'd love to see a paper a backing up this claim but I'm pretty sure it's just wrong.

Likewise, I would be quite surprised if Tesla is really pushing state of the art for the size of their vision models with what they've deployed in cars. Researchers have built some pretty big models...

Re: Tesla deploys massive new Autopilot neural net in v9

#62

Post by mannykannot »

X6S1x6Okd1st wrote: post #59
Someone wrote: post #30

”When you increase the number of parameters (weights) in an NN by a factor of 5 you don’t just get 5 times the capacity and need 5 times as much training data. In terms of expressive capacity increase it’s more akin to a number with 5 times as many digits. So if V8’s expressive capacity was 10, V9’s capacity is more like 100,000.” I find it very, very hard to believe that. I know ‘expressive power’ is a fairly vague…

Here is a highly specific definition of "expressive power" https://en.m.wikipedia.org/wiki/VC_dimension To me the statement doesn't seem that unreasonable

Thanks for bringing this to my attenton. In that article, however, it says for neural nets:

V is the set of nodes. Each node is a simple computation cell. E is the set of edges, Each edge has a weight. .... If the activation function is the sigmoid function and the weights are general, then the VC dimension is ... at most O(|E|^2.|V|^2) [apologies for the crappy formatting]

While Someone's quote from the article seems to be suggesting something exponential in the number of edges.

Re: Tesla deploys massive new Autopilot neural net in v9

#63

Post by X6S1x6Okd1st »

mannykannot wrote: post #62
X6S1x6Okd1st wrote: post #59

Earlier quoted context omitted.

Here is a highly specific definition of "expressive power" https://en.m.wikipedia.org/wiki/VC_dimension To me the statement doesn't seem that unreasonable

Thanks for bringing this to my attenton. In that article, however, it says for neural nets: V is the set of nodes. Each node is a simple computation cell. E is the set of edges, Each edge has a weight. .... If the activation function is the sigmoid function and the weights are general, then the VC dimension is ... at most O(|E|^2.|V|^2) [apologies for the crappy formatting] While Someone's quote from the article seem…

Good point

Re: Tesla deploys massive new Autopilot neural net in v9

#64

Post by Someone »

mannykannot wrote: post #62
X6S1x6Okd1st wrote: post #59

Earlier quoted context omitted.

Here is a highly specific definition of "expressive power" https://en.m.wikipedia.org/wiki/VC_dimension To me the statement doesn't seem that unreasonable

Thanks for bringing this to my attenton. In that article, however, it says for neural nets: V is the set of nodes. Each node is a simple computation cell. E is the set of edges, Each edge has a weight. .... If the activation function is the sigmoid function and the weights are general, then the VC dimension is ... at most O(|E|^2.|V|^2) [apologies for the crappy formatting] While Someone's quote from the article seem…

I haven’t even tried to hunt down the book referenced on Wikipedia, but I think it’s worse than that. The Wikipedia page says ”The VC dimension of a neural network is bounded as follows”. “Is bounded” is an expression in mathematics that is more about what we know about a problem, than about the problem itself (as a classical example, see https://en.wikipedia.org/wiki/Graham's_number#Context. Graham’s number ‘bounds’ a number whose value we know to be at least 13)

Given the huge range between that upper bound and the lower bound of Ω(|E|²), chances are that upper bound is far from tight (https://en.wikipedia.org/wiki/Upper_and_lower_bounds#Tight_b...).

Also, one line below the O(|E|².|V|²) you quoted:

”If the weights come from a finite family (e.g. the weights are real numbers that can be represented by at most 32 bits in a computer), then, for both activation functions, the VC dimension is at most O(|E|)”

Of course, they may use a different activation function, in which case that mathematical statement doesn’t apply, but I would think it’s more unlikely that applies than the claim made on the article we’re discussing.

For example, it would hugely surprise me if using an activation function that isn’t increasing or that has many large discontinuities behaves a lot better than the sigmoid surely used.

Re: Tesla deploys massive new Autopilot neural net in v9

#65

Post by Latteland »

vkoskiv wrote: post #60
threeseed wrote: post #34

Still completely reckless in my opinion to deploy any autonomous system without LIDAR or some other long range, night time capable sensor. Even if it's not reckless their competitors will offer low light and night time autonomous driving which will be a major advantage.

Teslas have a long-range radar on the front.

Yes, that's very useful in day to day driving, still not as good as lidar, radar doesn't have enough resolution.

Re: Tesla deploys massive new Autopilot neural net in v9

#66

Post by mtgx »

toomuchtodo wrote: post #58
threeseed wrote: post #57

Earlier quoted context omitted.

Some of us think that Tesla's technology is an experiment. An incredibly dangerous one.

Federal regulators do not agree with your assessment. https://thehill.com/policy/transportation/automobiles/315133... > Federal regulators "did not identify any defects" in Tesla's autopilot feature after a lengthy investigation of the technology, officials announced Thursday. > A six-month investigation failed to uncover any flaws with the autopilot's emergency breaking technology and other advanced features linked…

Sounds dubious. They seem to be focused on Autopilot working as intended. What if that is far from good enough?

Re: Tesla deploys massive new Autopilot neural net in v9

#67

Post by mtgx »

jaimex2 wrote: post #42
xvector wrote: post #37

Earlier quoted context omitted.

I do not get why your comment is so controversial - you are absolutely right. Conventional cameras alone are not trustworthy for self-driving, and this is part of the reason every respectable company venturing into self-driving is incorporation technologies like LIDAR. I find it sketchy that Tesla markets "future full-self-driving" when they are unlikely to have the hardware to make that a safe experience.

Ever wondered why in millions of years nature didnt evolve a LIDAR equivalent and most species use passive vision? I think Tesla is bang on the money going only with cameras. They are already better than the human eye.

Even at night? And in snow?

Re: Tesla deploys massive new Autopilot neural net in v9

#68

Post by mtgx »

AnthonyMouse wrote: post #54
threeseed wrote: post #45

Earlier quoted context omitted.

Eagles can see 4-8x further than humans can and many varieties of birds can see in additional spectrums to ours. And of course SONAR exists in whales, dolphins etc which is a corollary.

> Eagles can see 4-8x further than humans can and many varieties of birds can see in additional spectrums to ours. So can digital cameras. > And of course SONAR exists in whales, dolphins etc which is a corollary. The reason for (active) SONAR is the lack of ambient sound. An alternative for a car would be headlights. Or passive infrared cameras.

So does Tesla have cameras that see 4-8x better than a human then? Or do they use potato cameras that are useless at night like Uber did?

Re: Tesla deploys massive new Autopilot neural net in v9

#69

Post by nickik »

xvector wrote: post #37
threeseed wrote: post #34

Still completely reckless in my opinion to deploy any autonomous system without LIDAR or some other long range, night time capable sensor. Even if it's not reckless their competitors will offer low light and night time autonomous driving which will be a major advantage.

I do not get why your comment is so controversial - you are absolutely right. Conventional cameras alone are not trustworthy for self-driving, and this is part of the reason every respectable company venturing into self-driving is incorporation technologies like LIDAR. I find it sketchy that Tesla markets "future full-self-driving" when they are unlikely to have the hardware to make that a safe experience.

So basically you simple predefined what technology makes people 'respectable' and and what not. Seems to me that you are just assuming knowledge that actually nobody has.

Re: Tesla deploys massive new Autopilot neural net in v9

#70

Post by franciscrick1 »

mikestew wrote: post #56
jaimex2 wrote: post #42

Earlier quoted context omitted.

Ever wondered why in millions of years nature didnt evolve a LIDAR equivalent and most species use passive vision? I think Tesla is bang on the money going only with cameras. They are already better than the human eye.

Ever wondered why in millions of years nature didnt evolve a LIDAR equivalent and most species use passive vision? Because it compensated for poor sensor hardware with a processing unit and software that we have thus far been unable to even come close to replicating? Granted, solve that, and we’re just a few years away!

You say this ironically, and yet every indicator points toward this happening in the next 5-10 years!
Post reply on HN