Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A popular self-driving car dataset is missing labels for hundreds of pedestrians

blog.roboflow.ai

Post reply on HN
Print view View original

51–60 of 202 posts

Re: A popular self-driving car dataset is missing labels for hundreds of pedestrians

#51

Post by myself248 »

PeterisP wrote: post #29
danbrooks wrote: post #4

I work in the AV space. There's a lot of ambiguity in labeling. How should a crowd of people be annotated? A line of parked cars? A photograph of a car? Examine the training set!

A big thing is not that the example is missing, but that it counts as a negative example. I.e. if during training a ML system notices the ambiguous combination (i.e. a woman pushing a baby stroller or a crowd) and marks it as a pedestrian, then it gets penalized in a manner that teaches it to ignore these ambigious combinations and treat it as nothing; while in practice it should probably treat such ambiguous combina…

Likewise, I have yet to see one of these things that can recognize potholes and swerve to avoid them lest a wheel be ripped off.

Sooner or later it'd be nice to be able to drive one of these things in a place that isn't southern California.

Re: A popular self-driving car dataset is missing labels for hundreds of pedestrians

#52

Post by [deleted] »

yeldarb wrote: post #2

This is really scary. I discovered this because we're working on converting and re-hosting popular datasets in many popular formats for easy use across models... I first noticed that there were a bunch of completely unlabeled images. Upon digging in, I was appalled that fully 1/3 of the images contained errors or omissions! Some are small (eg a part of a car on the edge of the frame or a ways in the distance not bein…

[deleted]

Re: A popular self-driving car dataset is missing labels for hundreds of pedestrians

#53

Post by lahwran »

myself248 wrote: post #48
Animats wrote: post #18

These are manually tagged stills, right? Not video? That's a data set for training CAPCHA breakers, not self-driving. You need to use video, where you get to see the same objects at different ranges. Recognition gets better as you get closer. Then track the recognized objects backwards to when they first appear, and try to recognize them at smaller sizes.

I'm tangential to the field, but not directly in it myself, and I tend to agree. The goal of these systems should be to "perceive" their environment, not just to "recognize" it. Perception means understanding that just because a truck (that we recognized 3 frames ago) went behind a tree, doesn't mean it ceased being a truck. Furthermore, this knowledge should be used to refine the model, to say "hey I can still see t…

you're correct that this sort of persistent world modeling is needed for self driving cars, but from what I've heard from friends who work in the industry, both cruise and waymo have it. they're very far from using a plain CNN on their video cameras, they've got depth mapping and such and carefully constructed software making use of the perception data to model how the world will change and react to that. idk if it works well, but they definitely know they need it and are trying.

that said, I've driven a tesla on autopilot, and holy crap is was so incredibly bad. I'm optimistic about self driving cars in general, but not about tesla's. it will frequently lose track of the road lines at night and fail to make turns, suddenly beeping at you that you're in control now, with no warning! I only ever used it like cruise control, but I can't understand how anyone driving a tesla would dare use the tricks that allow bypassing the restrictions that prevent taking your hands off the wheel.

Re: A popular self-driving car dataset is missing labels for hundreds of pedestrians

#54

Post by yters »

pathseeker wrote: post #30
rhizome wrote: post #12

Earlier quoted context omitted.

> And while a single frame might be missing a label, I bet that at-speed most everything important gets labeled correctly enough to be better than a distracted human driver, or the average human driver for that matter. Would you bet a family member? That a distracted driver is a hazard does not mean other drivers are safe, or even saf er .

>Would you bet a family member? "think of the children!" Lives at stake don't change anything here. The question is whether self-driving cars, even with the errors, are safer for people than regular drivers on average. If so, then absolutely yes everyone should bet their lives and their families'. Thousands of people are dying every day in cars. This is not something we need to wait for it to be perfect. It only need…

I think people intuitively ascribe a moral dimension to whether people are involved in accidents, hence why they are more worried about dying on an airplane than in a car, even though the former is much less likely than the latter.

If I die in a plane, I have no control over the matter. If I die in a car, then at least I may have had control, and it could be chalked up to my inattentiveness, bad driving behavior, etc. The latter then implies the individual has direct control over whether they live or die, and can thus ensure life by being a responsible driver. I bet the same reasoning will be applied to self driving cars, even if they are orders of magnitude safer, just as the case with airplanes.

Re: A popular self-driving car dataset is missing labels for hundreds of pedestrians

#55

Post by viraptor »

Waterluvian wrote: post #44
netsharc wrote: post #39

Earlier quoted context omitted.

What a funny future it'd be, if we have to wear something distinctive (giant QR codes?) so we don't get killed outside. As a side effect it would make tracking us much easier...

IR reflective tape on outdoor clothing. Not the most outlandish idea. I hate it but I can see a future of it.

I really hope that gets verified before people start doing this (and I'm sure some will). Being classified as a lens flare / visual artefact is not a great way to die either.

Re: A popular self-driving car dataset is missing labels for hundreds of pedestrians

#56

Post by geraldcombs »

sgustard wrote: post #20
stared wrote: post #11

It is a self-correcting problem: these pedestrains won’t be present in the next dataset.

People should learn not to go outside if they're not labelled.

Ford has you covered. Literally.

https://media.ford.com/content/fordmedia/feu/en/news/2018/06...

Re: A popular self-driving car dataset is missing labels for hundreds of pedestrians

#57

Post by haditab »

I acknowledge the issues in the dataset and that it has a lot of stars on github because it's from Udacity; but calling it 'a popular self-driving car dataset' is misleading as it implies this dataset is popularly used for self-driving cars when it is in fact only a small dataset Udacity uses to teach the basics of training neural networks for self-driving cars.

I've been involved in the autonomous vehicle industry for a while and have been focused on perception for most of it. Most research papers will test their models on popular datasets for self-driving cars and show the results as a sort of benchmark. I've never seen this dataset mentioned anywhere. Heck the size of the dataset is an order of magnitude smaller than most of the popular ones as well.

This is just a github repo. That's it.

Re: A popular self-driving car dataset is missing labels for hundreds of pedestrians

#58

Post by dylan604 »

cmiles74 wrote: post #33
Fricken wrote: post #13

Well, if an autonomous vehicle outfit were running unmonitored Level 4 vehicles on public roads using only an open source data set I'd be worried. Even if it was labelled thoroughly and correctly, there isn't nearly enough data in any open source dataset to train an autonomous vehicle perception system that can operate safely without human supervision. This is not a safety critical issue.

The Uber vehicle that ended up killing the pedestrian in Arizona was running under conditions similar to the ones that you outline here. The only notable difference was that a person was monitoring the vehicle.

The only notable difference was that a person was present and supposed to be monitoring the vehicle.

Fixed that for you

Re: A popular self-driving car dataset is missing labels for hundreds of pedestrians

#59

Post by jacquesm »

geraldcombs wrote: post #56
sgustard wrote: post #20

Earlier quoted context omitted.

People should learn not to go outside if they're not labelled.

Ford has you covered. Literally. https://media.ford.com/content/fordmedia/feu/en/news/2018/06...

That's not a reply to the GP and actually a very good idea.

Re: A popular self-driving car dataset is missing labels for hundreds of pedestrians

#60

Post by jacquesm »

ffk wrote: post #41
JadeNB wrote: post #35

Earlier quoted context omitted.

> People should learn not to go outside if they're not labelled. I worry what will happen when this idea breeds with the "why worry about privacy if you've got nothing to hide?" fallacy.

Wouldn't it be nice if preserving your privacy simply meant not wearing your QR code?

It's just 'license plates for people'. And driving without a license plate is already illegal in many places. It would be a small step to require citizens to wear their nameber clearly visible at all times when outside. The penalty for failure would be a bit harsh. But on the plus side, all the corpses would be unidentified, so technically no-bodies...
Post reply on HN