Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A popular self-driving car dataset is missing labels for hundreds of pedestrians

blog.roboflow.ai

Post reply on HN
Print view View original

71–80 of 202 posts

Re: A popular self-driving car dataset is missing labels for hundreds of pedestrians

#71

Post by jxramos »

yeldarb wrote: post #2

This is really scary. I discovered this because we're working on converting and re-hosting popular datasets in many popular formats for easy use across models... I first noticed that there were a bunch of completely unlabeled images. Upon digging in, I was appalled that fully 1/3 of the images contained errors or omissions! Some are small (eg a part of a car on the edge of the frame or a ways in the distance not bein…

Does this dataset have an auxiliary field for each data point to note if it's been human reviewed? We used to train models with only those datapoints that had n=3 concordance in manual reviewers.

Re: A popular self-driving car dataset is missing labels for hundreds of pedestrians

#72

Post by lolc »

This just shows how clueless AI is still. Imagine your driving instructor insisted there was no person with a stroller on that corner. You'd dump them on the spot!

Only when the machines start to complain when they are fed shitty data, we can talk about them being fit to drive.

Re: A popular self-driving car dataset is missing labels for hundreds of pedestrians

#73

Post by craftinator »

rifung wrote: post #38
pathseeker wrote: post #30

Earlier quoted context omitted.

>Would you bet a family member? "think of the children!" Lives at stake don't change anything here. The question is whether self-driving cars, even with the errors, are safer for people than regular drivers on average. If so, then absolutely yes everyone should bet their lives and their families'. Thousands of people are dying every day in cars. This is not something we need to wait for it to be perfect. It only need…

> Lives at stake don't change anything here. The question is whether self-driving cars, even with the errors, are safer for people than regular drivers on average. If so, then absolutely yes everyone should bet their lives and their families'. Maybe logically that makes sense but from an ethical perspective I argue it's much more complicated than that (e.g. the trolley problem) In the current system if a human is at…

I'm voting for the "less dead people" option. Mostly because I'm a selfish person, been in automobile accidents caused by lapsing human attention, and I want it to be less likely that I'll die in a car crash.

Re: A popular self-driving car dataset is missing labels for hundreds of pedestrians

#74

Post by drcross »

tomnipotent wrote: post #15
yeldarb wrote: post #2

This is really scary. I discovered this because we're working on converting and re-hosting popular datasets in many popular formats for easy use across models... I first noticed that there were a bunch of completely unlabeled images. Upon digging in, I was appalled that fully 1/3 of the images contained errors or omissions! Some are small (eg a part of a car on the edge of the frame or a ways in the distance not bein…

> This is really scary. No, it's not even remotely "really scary". No one is putting an actual self-driving car on the market using this specific data set. Disingenuous to pretend this is any indication of the data using by serious companies in the space or is represented of the impact a few mislabelled samples have on the ability of these systems & algorithms to generalize.

Thank you. I can't believe the grandstand here.

Re: A popular self-driving car dataset is missing labels for hundreds of pedestrians

#75

Post by thanatropism »

haditab wrote: post #57

I acknowledge the issues in the dataset and that it has a lot of stars on github because it's from Udacity; but calling it 'a popular self-driving car dataset' is misleading as it implies this dataset is popularly used for self-driving cars when it is in fact only a small dataset Udacity uses to teach the basics of training neural networks for self-driving cars. I've been involved in the autonomous vehicle industry f…

Are these larger datasets routinely subject to the same kind of inspection this titanic.csv of self-driving car datasets?

Re: A popular self-driving car dataset is missing labels for hundreds of pedestrians

#76

Post by cameronbrown »

craftinator wrote: post #73
rifung wrote: post #38

Earlier quoted context omitted.

> Lives at stake don't change anything here. The question is whether self-driving cars, even with the errors, are safer for people than regular drivers on average. If so, then absolutely yes everyone should bet their lives and their families'. Maybe logically that makes sense but from an ethical perspective I argue it's much more complicated than that (e.g. the trolley problem) In the current system if a human is at…

I'm voting for the "less dead people" option. Mostly because I'm a selfish person, been in automobile accidents caused by lapsing human attention, and I want it to be less likely that I'll die in a car crash.

But it's not just about quantity. It's also _different_ people who will die. That radically alters things from an ethical perspective.

Re: A popular self-driving car dataset is missing labels for hundreds of pedestrians

#77

Post by tomnipotent »

jwandborg wrote: post #66
tomnipotent wrote: post #64

Earlier quoted context omitted.

> Is there any statistical/mathematical tool to completely eradicate or greatly diminish the effects of bad labeling Yes, it's called statistics and probability theory.

That's correct. I know what goes in and what comes out, not what happens in the middle. How does ~33% insanity in become Edit: Parent was edited, was previously (paraphrased) > I'm guessing you have no technical understanding of how this works

How does making up something ridiculous like "33% insanity" give you anything that's resembles a subject that we can discuss? Hyperbole in, hyperbole out.

Re: A popular self-driving car dataset is missing labels for hundreds of pedestrians

#78

Post by tomnipotent »

jwandborg wrote: post #69
tomnipotent wrote: post #64

Earlier quoted context omitted.

> Is there any statistical/mathematical tool to completely eradicate or greatly diminish the effects of bad labeling Yes, it's called statistics and probability theory.

> Yes it's called statistics and probability theory. My understanding of statistics is: - I can halve the % insanity by adding another 100% of good labels. - If I want to reduce the insanity of labels to 1/33th of ~33% I need to add another 3200% of good labels. - If I want to reduce the insanity to 0% I need to balance the bad labels with an infinite amount of good labels. Is there anything I'm missing entirely exce…

You don't reach 0%, that's a straw man. The goal is better than human, and the 35,000+ vehicle-related fatalities that happen in the U.S. each year.

Re: A popular self-driving car dataset is missing labels for hundreds of pedestrians

#79

Post by xkemp »

netsharc wrote: post #39
sgustard wrote: post #20

Earlier quoted context omitted.

People should learn not to go outside if they're not labelled.

What a funny future it'd be, if we have to wear something distinctive (giant QR codes?) so we don't get killed outside. As a side effect it would make tracking us much easier...

You seem to be joking, but isn't that pretty much the SQ? Not universally, but there is a lot of clothing incorporating high-viz features. Most prominent in children's clothes and work clothes, but I've seen reflective material that is almost invisible during daytime being used in business coats, apparently with an eye towards bankers on bikes and the like.

Re: A popular self-driving car dataset is missing labels for hundreds of pedestrians

#80

Post by a_t48 »

Hold on a second here - are unlabeled pixels used in training a NN to do detection? Will a typical NN get trained to label those pixels as “not a human”? I agree that they should be labeled, but it’s the difference between needing to throw more data at the problem (because you aren’t getting as much learning per image as you could) and actively training the car to do something bad.
Post reply on HN