Fat tails
Predicting where AI is going in 2020
51–60 of 88 posts
Re: Predicting where AI is going in 2020
#52Earlier quoted context omitted.
Control algorithms are a part of the problem. What about input data? There's nothing that comes close to NNs in answering a question, say, "Is there pedestrian ahead and what he/she will probably do?" A control system doesn't need to be end-to-end neural, by the way.
> What about input data? What about it? You get input data from data sources which, e.g., in a car it would be a sensor. The manufacturer of the sensor provides you with the guaranteed sensor accuracy for some inputs, which gives you the upper bound on the input error from that source. That is, in a reliable control pipeline, the upper bounds on the errors of data sources are known a priori. Sure, sensors can malfunc…
> That is, in a reliable control pipeline, the upper bounds on the errors of data sources are known a priori.
Now we have natural neural networks in the control loop for some reason despite their unknown error bounds. To give another example: is there a sensor for vehicle placement relative to road edges with known upper error bound, which is less than width of the road? No, we have GPS, radar, lidar, camera data that we need to interpret somehow.
A car that reliably avoids collisions (Can it, though? It needs to predict road situation to do it reliably), but can occasionally veer off the road, doesn't strike me as particularly safe.
> to be overridden by reliable control pipelines when required.
Those reliable pipelines needs to be mostly reactive. And there's a limit on what they can do. You can't avoid a collision when a car emerges from around a corner with 0.1 seconds to react. You need complex processing of those "simple" data sources to detect zones that can't be observed right now and to assess a probability of the said situation.
All in all, we already have unreliable human part in the control loop of a vehicle. A control system that is provably robust in all real world situations will be the ultimate achievement and not a prerequirement for wide use of self-driving cars.
Re: Predicting where AI is going in 2020
#53Earlier quoted context omitted.
> What about input data? What about it? You get input data from data sources which, e.g., in a car it would be a sensor. The manufacturer of the sensor provides you with the guaranteed sensor accuracy for some inputs, which gives you the upper bound on the input error from that source. That is, in a reliable control pipeline, the upper bounds on the errors of data sources are known a priori. Sure, sensors can malfunc…
> That is, in a reliable control pipeline, the upper bounds on the errors of data sources are known a priori. Now we have natural neural networks in the control loop for some reason despite their unknown error bounds. To give another example: is there a sensor for vehicle placement relative to road edges with known upper error bound, which is less than width of the road? No, we have GPS, radar, lidar, camera data tha…
> Now we have natural neural networks in the control loop for some reason despite their unknown error bounds.
Not in any automatic control loops. All control loops that do this have a human as the final piece of the pipeline, and that human is legally responsible for the outcome of the control loop. That defeats the point of automatic control.
> To give another example: is there a sensor for vehicle placement relative to road edges with known upper error bound, which is less than width of the road?
No, which is why these control pieplines have a human in control. For the experimental pipelines that do not have a human in the end of the control loop, they do have other control pipelines to avoid collisions, and the only that their control algorithms guarantee is a lack of collisions, not the ability for the car to stay on a lane. That is, the car might leave the lane under some conditions, but if it does, it will detect other objects and avoid crashing into them (although those objects might crash into it).
> All in all, we already have unreliable human part in the control loop of a vehicle. A control system that is provably robust in all real world situations will be the ultimate achievement and not a prerequirement for wide use of self-driving cars.
Right now, control loops without proven error bounds are not allowed by certification bodies on any control-loop in charge of preserving human lives in the aerospace, automotive, medical, and industrial robotics industries.
Allowing control-loops without known error bounds to be in charge of human lives would lower the current standards of these industries. Could be done (the government would need to create a new kind of regulation for this), but at this point it is unclear how that would look like.
Re: Predicting where AI is going in 2020
#54I love PyTorch, but I’m not confident the claim that it is the most popular is close to true. The cited link, which brings up a lot of new research is in PyTorch simply doesn’t account for the amount of TensorFlow in production. Sure, a lot of academics may be embracing PyTorch, but almost all production models have been in TensorFlow. Tesla is a huge notable example that’s using PyTorch at scale. I do suspect that t…
Depends on how you measure it, of course. However, stackoverflow survey, google trends, and github octoverse all show PyTorch is on a steep upward trajectory that recently reached effective parity with TensorFlow and has not yet started slowing down.
Re: Predicting where AI is going in 2020
#55Earlier quoted context omitted.
> That is, in a reliable control pipeline, the upper bounds on the errors of data sources are known a priori. Now we have natural neural networks in the control loop for some reason despite their unknown error bounds. To give another example: is there a sensor for vehicle placement relative to road edges with known upper error bound, which is less than width of the road? No, we have GPS, radar, lidar, camera data tha…
> Now we have natural neural networks in the control loop for some reason despite their unknown error bounds. Not in any automatic control loops. All control loops that do this have a human as the final piece of the pipeline, and that human is legally responsible for the outcome of the control loop. That defeats the point of automatic control. > To give another example: is there a sensor for vehicle placement relativ…
Well actually, they are allowed in the automotive world, re: Tesla.
They won't necessarily lower the standard for some applications if the alternative is just regular people at the wheel.
Re: Predicting where AI is going in 2020
#56Earlier quoted context omitted.
The encoding I was talking about may well be something more abstract than 'imitate humans'. Still, babies don't generally try to imitate the sound of rattles or household sounds nearly as much as speech, so I still conclude that it is a safe assumption that there is something about sounds made by humans that is inherently interesting to them for some reason (instead of being a learned behavior). Related to the second…
> try to imitate the sound of rattles or household sounds nearly as much as speech Well surely that's a case of the range of the vocal chords? Parrots are another intelligent creature that has better range and they imitate all sorts of sounds. > Related to the second, the rate at which we learn, and the very specific order we learn things in, points very strongly in the direction that there is some built-in model tha…
The range of the vocal chords is a reason why children can't successfully imitate these sounds, it doesn't directly explain why they wouldn't try.
Re: Predicting where AI is going in 2020
#57Earlier quoted context omitted.
I did try training a Pytorch BERT-derived model on TPUs (on Colab) and it didn't work out-of-the box. (Where didn't work=was using the CPU) I didn't dig into it to find out why.
(I'm one of the Cloud TPU product leads) We've seen multiple BERT-related PyTorch models training successfully on Cloud TPUs, including training at scale on large, distributed Cloud TPU Pod slices. Would you consider filing a GitHub issue at https://github.com/pytorch/xla or emailing pytorch-tpu@googlegroups.com to provide a bit more context about the specific issue you encountered? Here's the current PyTorch/TPU tro…
> BERT-related PyTorch models training successfully on Cloud TPUs
How do you see it? Do you look at your client's code?
Re: Predicting where AI is going in 2020
#58Earlier quoted context omitted.
> try to imitate the sound of rattles or household sounds nearly as much as speech Well surely that's a case of the range of the vocal chords? Parrots are another intelligent creature that has better range and they imitate all sorts of sounds. > Related to the second, the rate at which we learn, and the very specific order we learn things in, points very strongly in the direction that there is some built-in model tha…
The range of the vocal chords is a reason why children can't successfully imitate these sounds, it doesn't directly explain why they wouldn't try.
maybe they do try and we just shrug it off as gurgling. Kids do make funny noises when they're vocalising.
Re: Predicting where AI is going in 2020
#59Fat tails
This is the content I come to Hacker News for.
Re: Predicting where AI is going in 2020
#60Honesty, repeatability, numerical analysis. Canonicalization. Honesty: how many times was the exact same data processed? Was the result cherry picked and the best one published? For the sake of integrity how is it possible to scientifically improve on this result? (example, your AI outputs some life altering decision?) Repeatability: In science, if a result can be independently verified, it gives validity to the "con…
I would argue most machine learning papers that use public datasets have code available and are often also reproduced independently (sometimes just because of somebody's need to port between PyTorch/TensorFlow). Reproducibility is still a big problem in reinforcement learning, however. People are definitely thinking carefully about issues of noise and quantization error. Low-precision or quantized neural networks are…
>I would argue most machine learning papers that use public datasets have code available and are often also reproduced independently
lol have you ever tried? i have several github issues on published models because i couldn't recreate that have responses like "i don't remember the parameters i used and we've moved on".