Live data from Hacker News

A popular self-driving car dataset is missing labels for hundreds of pedestrians

blog.roboflow.ai

131–140 of 202 posts

Re: A popular self-driving car dataset is missing labels for hundreds of pedestrians

#131
post #12

Earlier quoted context omitted.

> And while a single frame might be missing a label, I bet that at-speed most everything important gets labeled correctly enough to be better than a distracted human driver, or the average human driver for that matter. Would you bet a family member? That a distracted driver is a hazard does not mean other drivers are safe, or even saf er .

>Would you bet a family member? "think of the children!" Lives at stake don't change anything here. The question is whether self-driving cars, even with the errors, are safer for people than regular drivers on average. If so, then absolutely yes everyone should bet their lives and their families'. Thousands of people are dying every day in cars. This is not something we need to wait for it to be perfect. It only need…

"The question is whether self-driving cars, even with the errors, are safer for people than regular drivers on average. If so..."

You could say "the question is whether aircraft, even with the errors, are safe for people than regular drivers on average. If so..." - then how should we change policy towards Boeing in light of the 737 MAX fiasco? Should we then avoid any adverse action towards them and focus on encouraging more people to fly?

If we declare that something is safer, particularly before it even exists, isn't there a danger of a feedback loop that prevents it from being safer?

Regular drivers kill people, but they are also generally vulnerable to the crashes that they cause. Boeing engineers, or the programmers of self-driving AIs, don't have their interests aligned with you, the occupant of the vehicle, nearly as much.

Re: A popular self-driving car dataset is missing labels for hundreds of pedestrians

#132
post #119

Earlier quoted context omitted.

IMO to be "scared" of self driving cars they need to be more dangerous than any other random car on the street today, and the bar for that is pretty low.

Nope, with other cars you know the factors that increase your risks (fog, drunk driving, distractions). You can make decisions, like not getting into a car with a drunk friend. With machine learning you never know when it might mistake the back of a semi for an overpass or soemthing.

Also, not to fear monger too much, but if this ends up happening there will inevitably be a period where people still assume that self driving cars are foolproof, and as a result, a lot of pedestrians/other drivers will get blamed for incidents that weren't in fact their fault.

Re: A popular self-driving car dataset is missing labels for hundreds of pedestrians

#133
post #12

Earlier quoted context omitted.

> And while a single frame might be missing a label, I bet that at-speed most everything important gets labeled correctly enough to be better than a distracted human driver, or the average human driver for that matter. Would you bet a family member? That a distracted driver is a hazard does not mean other drivers are safe, or even saf er .

>Would you bet a family member? "think of the children!" Lives at stake don't change anything here. The question is whether self-driving cars, even with the errors, are safer for people than regular drivers on average. If so, then absolutely yes everyone should bet their lives and their families'. Thousands of people are dying every day in cars. This is not something we need to wait for it to be perfect. It only need…

I would never buy a car that's just better than an average driver: many crashes come from driving while drunk, in a fog, on ice, etc.

I would only buy a car that's as good as a fully attentive, sober, skilled and well rested driver.

Besides, there will be unforceen issues that increase your likelihood of death: hacking, sensor failures, etc.

Re: A popular self-driving car dataset is missing labels for hundreds of pedestrians

#134

Earlier quoted context omitted.

You don't reach 0%, that's a straw man. The goal is better than human, and the 35,000+ vehicle-related fatalities that happen in the U.S. each year.

There's a disconnect here. People who talk about the danger of humans driving cars always seem to talk about the raw numbers, because humans drive cars a lot and the raw numbers are rather large. But when we talk about automated driving, it's in percentages, because it's not being done on the same scale. So to compare apples to apples, you'd have to convert the number of fatalities to an accuracy percentage. Have you…

> you'd have to convert the number of fatalities to an accuracy percentage

Telsa's early results for their very limited "self-driving" technology has shown a huge reduction in accidents for any given period of time the vehicles are on the road.

Re: A popular self-driving car dataset is missing labels for hundreds of pedestrians

#135
post #102

Earlier quoted context omitted.

> No one is putting an actual self-driving car on the market using this specific data set. Disingenuous to pretend this is any indication of the data using by serious companies in the space Are you sure about that? What about "non-serious" companies? Various fly-by-night self-driving startups? I mean, this sounds like the machine learning equivalent of "no serious business is pulling random bits of code from StackOve…

Are you sure about that? What about "non-serious" companies? Various fly-by-night self-driving startups? Fly-by-night self-driving startups will at some point run into issues with the NHTSA. George Hotz self-driving car project was shut down as soon as he announced that he'd start selling some prototype. States require permits just to be allowed to test self-driving prototypes on the open road. I don't know what the…

They haven't shut down, they are selling their prototype and I think they have recently put out a new model.

Re: A popular self-driving car dataset is missing labels for hundreds of pedestrians

#136

Earlier quoted context omitted.

I'm not sure what point you're trying to make, but it has nothing to do with some mislabeled examples. Every system using supervised training assets has labeling issues, and the world hasn't ended yet. Take a look at Google Translate. This is only "scary" if you're ignorant of the problem. The OP is selling something, and it's in their best interest to spread FUD to sell it.

That's probably the least charitable interpretation possible. No idea who the OP is, in any case. It's scary that the only way we know how to build something that detects pedestrians in an image with any kind of reliability is to use training data. You: All training data has labeling issues. Me: Training data is the only way we know how to build some aspects of systems. Other people here: Some of these systems are sa…

> No idea who the OP is, in any case.

The person that originally posted this thread that works for the company that's selling this FUD. At the bottom of the blog post:

"Roboflow accelerates your computer vision workflow through automated annotation quality assurance"

This is a non-issue exasperated by a for-profit corporation that's creating an "issue" they conveniently have a service to help you fix.

Re: A popular self-driving car dataset is missing labels for hundreds of pedestrians

#137

Earlier quoted context omitted.

The Uber vehicle that ended up killing the pedestrian in Arizona was running under conditions similar to the ones that you outline here. The only notable difference was that a person was monitoring the vehicle.

The only notable difference was that a person was present and supposed to be monitoring the vehicle. Fixed that for you

And it will happen again. There were cases of people forgetting Nuclear Weapons on a runway for a whole day by accident. People, eventually, will fuck up. Mislabelled datasets will be used.

Re: A popular self-driving car dataset is missing labels for hundreds of pedestrians

#138
post #102

Earlier quoted context omitted.

Are you sure about that? What about "non-serious" companies? Various fly-by-night self-driving startups? Fly-by-night self-driving startups will at some point run into issues with the NHTSA. George Hotz self-driving car project was shut down as soon as he announced that he'd start selling some prototype. States require permits just to be allowed to test self-driving prototypes on the open road. I don't know what the…

No one stopped Uber, no one has stopped Tesla (Tesla hasn't been able deal with regression testing on a lane finding system...)

[deleted]

Re: A popular self-driving car dataset is missing labels for hundreds of pedestrians

#139
post #130

Do they really think people are using these datasets for commercial applications?

Do you really think there is any possible fuckup that won't happen sooner or later?

Will staff of a nuclear silo forget to lock the door, and then fall asleep? Because that happened in US

Do you really think someone will put a plane in production with a single safety critical sensor with no backup or fall back?

Do you really think someone will pour dissolved uranium down the drain, starting a nuclear reaction and dying horribly?

Do you really think someone will crash a spacecraft into mars, by mixing up imperial and metric units?

Re: A popular self-driving car dataset is missing labels for hundreds of pedestrians

#140

Earlier quoted context omitted.

There's a disconnect here. People who talk about the danger of humans driving cars always seem to talk about the raw numbers, because humans drive cars a lot and the raw numbers are rather large. But when we talk about automated driving, it's in percentages, because it's not being done on the same scale. So to compare apples to apples, you'd have to convert the number of fatalities to an accuracy percentage. Have you…

> you'd have to convert the number of fatalities to an accuracy percentage Telsa's early results for their very limited "self-driving" technology has shown a huge reduction in accidents for any given period of time the vehicles are on the road.

That seems like it incorporates a lot of assumptions. I think it's best to slow down and realize that comparisons don't mean much if you're comparing the wrong things. The first step is to determine the first thing that you are comparing and exactly what it is. Then you can move on to the other half and determine whether it is appropriate.

Humans are much safer than people on average, when driving in conditions suitable for Autopilot.

Post reply on HN