Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

YOLOv5: State-of-the-art object detection at 140 FPS

blog.roboflow.ai

Post reply on HN
Print view View original

121–130 of 136 posts

Re: YOLOv5: State-of-the-art object detection at 140 FPS

#121

Post by bArray »

sillysaurusx wrote: post #87
rocauc wrote: post #85

Earlier quoted context omitted.

Hey all - OP here. We're not affiliated with Ultralytics or the other researchers. We're a startup that enables developers to use computer vision without being machine learning experts, and we support a wide array of open source model architectures for teams to try on their data: https://models.roboflow.ai Beyond that, we're just fans. We're amazed by how quickly the field is moving and we did some benchmarks that we…

YOLOv5 seems to have one important advantage over v4, which your post helped highlight: Fourth, YOLOv5 is small. Specifically, a weights file for YOLOv5 is 27 megabytes. Our weights file for YOLOv4 (with Darknet architecture) is 244 megabytes. YOLOv5 is nearly 90 percent smaller than YOLOv4. This means YOLOv5 can be deployed to embedded devices much more easily. Naming controversy aside, it's nice to have some model…

This is why I have so much doubt. To claim it's better in any meaningful way you need to show it on the same framework, varied datasets, varied input sizes and you should be able to use it in your detection problem and also see some benefits from the previous version.

> SIZE: YOLOv5 is about 88% smaller than YOLOv4 (27 MB vs 244 MB)

Is that a benefit of Darknet vs TF, YOLOv4 vs YOLOv5, or did you win the NN lottery [1]?

> SPEED: YOLOv5 is about 180% faster than YOLOv4 (140 FPS vs 50 FPS)

Again, where does this improvement come from?

> ACCURACY: YOLOv5 is roughly as accurate as YOLOv4 on the same task (0.895 mAP vs 0.892 mAP)

The difference in 0.1% accuracy can be huge, for example the difference between 99.9% and 100% could require an insanely larger neural network. Even much less that 99% accuracy, it seems clear to me that there can still be some limitations on accuracy from neural network size.

For example, if you really don't care so much for accuracy, you can really squeeze the network down [2].

[1] https://ai.facebook.com/blog/understanding-the-generalizatio...

[2] https://arxiv.org/abs/1910.03159

Re: YOLOv5: State-of-the-art object detection at 140 FPS

#122

Post by bArray »

rocauc wrote: post #85
bArray wrote: post #63

I'm just going to call this out as bullshit. This isn't YOLOv5. I doubt they even did a proper comparison between their model and YOLOv4. Someone asked it to not be called YOLOv5 and their response was just awful [1]. They also blew off a request to publish a blog/paper detailing the network [2]. I filed a ticket to get to the bottom of this with the creators of YOLOv4: https://github.com/AlexeyAB/darknet/issues/5920…

Hey all - OP here. We're not affiliated with Ultralytics or the other researchers. We're a startup that enables developers to use computer vision without being machine learning experts, and we support a wide array of open source model architectures for teams to try on their data: https://models.roboflow.ai Beyond that, we're just fans. We're amazed by how quickly the field is moving and we did some benchmarks that we…

> We're not affiliated with Ultralytics or the other

> researchers.

Unfortunately I am now unable to edit to reflect this better.

Re: YOLOv5: State-of-the-art object detection at 140 FPS

#123

Post by bArray »

joshvm wrote: post #92
bArray wrote: post #63

I'm just going to call this out as bullshit. This isn't YOLOv5. I doubt they even did a proper comparison between their model and YOLOv4. Someone asked it to not be called YOLOv5 and their response was just awful [1]. They also blew off a request to publish a blog/paper detailing the network [2]. I filed a ticket to get to the bottom of this with the creators of YOLOv4: https://github.com/AlexeyAB/darknet/issues/5920…

I somewhat agree on the naming issue. I don't think yolov5 is semantically very informative. But by the way, if you read the issues from a while back you'll see that AlexeyAB's fork basically scooped them, hence the version bump. Ultralytics probably would have called this Yolov4 otherwise. This repo has been in the works for a while. For history, Ultralytics originally forked the core code from some other Pytorch im…

> But by the way, if you read the issues from a while back

> you'll see that AlexeyAB's fork basically scooped them,

> hence the version bump.

Yeah that sucks, but it does mean they should have done some proper comparison with YOLOv4.

> This took a while, probably because there is actually very

> little documentation for Yolov3 and there was confusion

> over what the loss function actually ought to be. The

> darknet repo is totally uncommented C with lots of single

> letter variable names. AlexeyAB is a Saint.

Maybe I'm alone, but I found it quite readable. You can quite reasonably understand the source in a day.

> The v4 release was also quite contentious.

Kind of, I am personally still evaluating this network fully.

> I disagree on your second point though. Demanding a paper

> when the author says "we will later" is hardly a blow off.

Checkout the translation of "you can you up,no can no bb" (see other comments).

> And before we knock Glenn for this, as far as I know, he's

> running a business, not a research group.

I understand, but this seems very unethical to take the name of an open source framework and network that publishes it's improvements in some form, bump the version number and then claim it's faster without actually doing an apples to apples test. It would have seem appropriate to contact the person who carried the torch after pjreddie stepped down from the project.

Re: YOLOv5: State-of-the-art object detection at 140 FPS

#124

Post by lm28469 »

wiz21c wrote: post #43
lm28469 wrote: post #30

Earlier quoted context omitted.

The question is always the same: is every technical/scientific progress desirable ? But it seems that this question isn't asked anymore, "move fast and break things" am I right ? I'm much more worried about people using your arguments to try and shut down the discussion than people trying to open the debate, because once the mass surveillance/face recognition mass adoption pandora's box is open there won't be any way…

>>> When I see predator drones and FBI stingray planes above every major us cities during protests Can you provide evidence for this ? Not that I doubt it, but if I want to tell other people that story, I must have evidence to be believed :-) edit: ah of course, 30 seconds of duckduckgo just provide the info I need : https://thehill.com/homenews/house/501445-democrats-press-dh...

Yes and plenty of other sources:

https://www.aclu.org/blog/privacy-technology/surveillance-te...

https://www.wired.com/2016/01/california-police-used-stingra...

https://chicago.cbslocal.com/2014/12/06/activists-say-chicag...

Once the tech is out there it's simply a question of "when" will it be used for borderline illegal activities, especially in the US where you have these different entities (fbi, cia, nsa, dea, &c.) basically acting in their own bubble and doing whatever they want until it's leaked and/or gets outrageous enough to get the public attention.

I mean, there were unidentified armed forces marching in US streets last week, if people don't se this as the biggest red flag in recent US history I don't know what they need.

https://www.vice.com/en_us/article/akzvy8/unidentified-law-e...

https://www.nbcnews.com/politics/white-house/i-was-horrified...

Re: YOLOv5: State-of-the-art object detection at 140 FPS

#125

Post by homarp »

tropshop wrote: post #83
homarp wrote: post #53

Earlier quoted context omitted.

also, having models that can run locally on "cheap enough" devices is also quite interesting that you more or less understand. Compared to using API in the cloud or purchasing Hikvision cameras.

Just getting into this. Do you recommend any particular "dumb" camera devices with a quality stream?

what's your price range? indoor or outdoor? where do you want to do the inference?

Re: YOLOv5: State-of-the-art object detection at 140 FPS

#126

Post by jcims »

joshvm wrote: post #106
jcims wrote: post #104

Earlier quoted context omitted.

High resolution light field cameras would really help here as well. That seems a ways off though. Are you folks able to do any multi-spectral stuff? That seems interesting.

I work mostly with RGB/Thermal, if that counts. My PhD was in stereo/lidar fusion, so I've always been into mixing sensors :) I've also done some work on satellite imaging which is 13-band (Sentinel 2). Lots of people in ecology use the Parrot Sequoia which is four-band multispectral. There really isn't much published work in ML beyond RGB, which I find interesting - yes there's RGB-D and LIDAR but it's mostly for dr…

Whoa that's awesome! Love hearing contemporary technology used to detect/diagnose/monitor the environment and our ecological impact. Boots on ground will always be important but the horizontal scaling you can get out of imaging I would imagine really helps prioritize where you turn your attention. Thanks for the info and best of luck!

Re: YOLOv5: State-of-the-art object detection at 140 FPS

#127

Post by anthiras »

rocauc wrote: post #85
bArray wrote: post #63

I'm just going to call this out as bullshit. This isn't YOLOv5. I doubt they even did a proper comparison between their model and YOLOv4. Someone asked it to not be called YOLOv5 and their response was just awful [1]. They also blew off a request to publish a blog/paper detailing the network [2]. I filed a ticket to get to the bottom of this with the creators of YOLOv4: https://github.com/AlexeyAB/darknet/issues/5920…

Hey all - OP here. We're not affiliated with Ultralytics or the other researchers. We're a startup that enables developers to use computer vision without being machine learning experts, and we support a wide array of open source model architectures for teams to try on their data: https://models.roboflow.ai Beyond that, we're just fans. We're amazed by how quickly the field is moving and we did some benchmarks that we…

It's about time for Roboflow to pull this article. It seems highly unlikely that a 90 % smaller model would provide a similar accuracy, and the result seems to come from a small custom dataset only. Please make a real COCO comparison instead.

The YoloV5 repo itself shows performance comparable to YoloV3: https://github.com/ultralytics/yolov5#pretrained-checkpoints

Another comparison suggests YoloV5 is slightly WORSE than YoloV4: https://github.com/WongKinYiu/CrossStagePartialNetworks/issu...

Re: YOLOv5: State-of-the-art object detection at 140 FPS

#128

Post by DougBTX »

bArray wrote: post #123
joshvm wrote: post #92

Earlier quoted context omitted.

I somewhat agree on the naming issue. I don't think yolov5 is semantically very informative. But by the way, if you read the issues from a while back you'll see that AlexeyAB's fork basically scooped them, hence the version bump. Ultralytics probably would have called this Yolov4 otherwise. This repo has been in the works for a while. For history, Ultralytics originally forked the core code from some other Pytorch im…

> But by the way, if you read the issues from a while back > you'll see that AlexeyAB's fork basically scooped them, > hence the version bump. Yeah that sucks, but it does mean they should have done some proper comparison with YOLOv4. > This took a while, probably because there is actually very > little documentation for Yolov3 and there was confusion > over what the loss function actually ought to be. The > darknet…

> Checkout the translation of "you can you up,no can no bb" (see other comments).

Who actually is "WDNMD0-0"? Looks like the account was created to make just that one comment.

Re: YOLOv5: State-of-the-art object detection at 140 FPS

#129

Post by bArray »

anthiras wrote: post #127
rocauc wrote: post #85

Earlier quoted context omitted.

Hey all - OP here. We're not affiliated with Ultralytics or the other researchers. We're a startup that enables developers to use computer vision without being machine learning experts, and we support a wide array of open source model architectures for teams to try on their data: https://models.roboflow.ai Beyond that, we're just fans. We're amazed by how quickly the field is moving and we did some benchmarks that we…

It's about time for Roboflow to pull this article. It seems highly unlikely that a 90 % smaller model would provide a similar accuracy, and the result seems to come from a small custom dataset only. Please make a real COCO comparison instead. The YoloV5 repo itself shows performance comparable to YoloV3: https://github.com/ultralytics/yolov5#pretrained-checkpoints Another comparison suggests YoloV5 is slightly WORSE…

> It's about time for Roboflow to pull this article.

The article still adds value by suggesting how one would run the network and in general the site seems to be about collating different networks.

Perhaps a disclaimer could be good, reading something like: "the speed improvements mentioned in this article are currently being tested". As a publisher, when you print somebody else's words, unless quoted, they are said with your authority. The claims are very big and it doesn't feel like enough testing has been done yet to even verify that they hold true.

Re: YOLOv5: State-of-the-art object detection at 140 FPS

#130

Post by joshvm »

bArray wrote: post #123
joshvm wrote: post #92

Earlier quoted context omitted.

I somewhat agree on the naming issue. I don't think yolov5 is semantically very informative. But by the way, if you read the issues from a while back you'll see that AlexeyAB's fork basically scooped them, hence the version bump. Ultralytics probably would have called this Yolov4 otherwise. This repo has been in the works for a while. For history, Ultralytics originally forked the core code from some other Pytorch im…

> But by the way, if you read the issues from a while back > you'll see that AlexeyAB's fork basically scooped them, > hence the version bump. Yeah that sucks, but it does mean they should have done some proper comparison with YOLOv4. > This took a while, probably because there is actually very > little documentation for Yolov3 and there was confusion > over what the loss function actually ought to be. The > darknet…

On the whole I agree about darknet being readable, it seemed well written and I've found it useful to grok how training libraries are written. I think they've moved to other backends now for the main computation though.

But.. it was still very much undocumented (and there were details missing from the paper). I think this almost certainly led to some slowdown in porting to other frameworks. And the fact its written in C has probably limited how much people are willing to contribute to the project.

> Checkout the translation of "you can you up,no can no bb" (see other comments).

That's from an 11 day old github account with no history, not Ultralytics as far as I know.

> Kind of, I am personally still evaluating this network fully.

Contention referring to the community response rather than the performance of the model itself.

Post reply on HN