Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Object Detection at 1840 FPS with TorchScript, TensorRT and DeepStream

paulbridger.com

Post reply on HN
Print view View original

31–40 of 58 posts

Re: Object Detection at 1840 FPS with TorchScript, TensorRT and DeepStream

#31

Post by gugagore »

bufferoverflow wrote: post #6
cm2187 wrote: post #4

Out of curiosity, what are the possible use cases for object detection at >100 fps? I assume it would have to be objects that move very fast, i.e. nothing ordinary that I can think of. [edit] actually stupid question. I assume it's more about throughput than fps, i.e. be able to process lots of streams on the same machine, for instance for doing mass analysis of CCTV streams.

Self-driving. Ideally you want something around 1000fps and low latency, so it has time to react. I'm sure military and sports applications are obvious too.

I don't think you'll find a 1000 fps camera on a "standard" AV platform. And if you did, I imagine it would be too noisy to be useful without a ton of illumination.

Re: Object Detection at 1840 FPS with TorchScript, TensorRT and DeepStream

#32

Post by mozak1111 »

I see this and I immediately think of "trash sorting" at ultra high speed. If one can combine this with a bunch of accurate (laser precision) air guns, to shoot and move individual pieces of trash you can sort through a truck load of trash in a matter of seconds, perhaps in the air while they are being dumped! compare this approach with how we are currently doing it [0] - Somebody should get Elon Musk on this project right away!

[0] - https://www.youtube.com/watch?v=QbKA9uNgzYQ

Re: Object Detection at 1840 FPS with TorchScript, TensorRT and DeepStream

#33

Post by pantelisk »

cm2187 wrote: post #4

Out of curiosity, what are the possible use cases for object detection at >100 fps? I assume it would have to be objects that move very fast, i.e. nothing ordinary that I can think of. [edit] actually stupid question. I assume it's more about throughput than fps, i.e. be able to process lots of streams on the same machine, for instance for doing mass analysis of CCTV streams.

Cloud service that needs to serve multiple requests or process many video streams in parallel. (faster performance = less hardware required, bigger scale and potentially improve end user experience - save from their data being on the cloud of course).

On device (eg mobile phone) processing with battery usage that respects the user. Older hardware/models inclusion as well.

Of course the above aren't cases were the stream itself is 100+fps, but more of broader general benefits. For a 100+ fps stream.. well there are many things that go fast, imagine you wanted a robot that tracks or catches a fly before it takes off. Flies have a reaction time of 5ms (200fps), that's why it's hard for us to catch! Expand and apply the same concept to other things (that are fast, or happen very quickly) now...

Re: Object Detection at 1840 FPS with TorchScript, TensorRT and DeepStream

#34

Post by aaronblohowiak »

cm2187 wrote: post #4

Out of curiosity, what are the possible use cases for object detection at >100 fps? I assume it would have to be objects that move very fast, i.e. nothing ordinary that I can think of. [edit] actually stupid question. I assume it's more about throughput than fps, i.e. be able to process lots of streams on the same machine, for instance for doing mass analysis of CCTV streams.

Food processing, recycling separation? I can imagine lots of small parts moving fast

Re: Object Detection at 1840 FPS with TorchScript, TensorRT and DeepStream

#35

Post by unnouinceput »

cm2187 wrote: post #4

Out of curiosity, what are the possible use cases for object detection at >100 fps? I assume it would have to be objects that move very fast, i.e. nothing ordinary that I can think of. [edit] actually stupid question. I assume it's more about throughput than fps, i.e. be able to process lots of streams on the same machine, for instance for doing mass analysis of CCTV streams.

tomato / potato sorting while they are on conveyor. bigger fps, more objects to dump on said conveyor.

Re: Object Detection at 1840 FPS with TorchScript, TensorRT and DeepStream

#36

Post by tyfon »

mozak1111 wrote: post #32

I see this and I immediately think of "trash sorting" at ultra high speed. If one can combine this with a bunch of accurate (laser precision) air guns, to shoot and move individual pieces of trash you can sort through a truck load of trash in a matter of seconds, perhaps in the air while they are being dumped! compare this approach with how we are currently doing it [0] - Somebody should get Elon Musk on this project…

Trash sorting is probably better than self driving cars. I only see speed talk on this page and nothing about accuracy.

Musk needs like 99.9999% accuracy at near zero latency over several hours of operation. I think Tesla currently is at maybe 99.995% from driving my car. The last 0.005% results in phantom braking etc. It's actually a very hard nut to crack and I don't expect them to achieve the full self driving in all conditions for another 10-15 years maybe. The edge cases are just too many.

I like the trash idea though (or a Q/A robot at a factory etc).

Re: Object Detection at 1840 FPS with TorchScript, TensorRT and DeepStream

#37

Post by moron4hire »

briggers wrote: post #28
moron4hire wrote: post #26

Any word on latency? I didn't see anything in the article. I guess, since this is a synthetic test just pumping a single image file through repeatedly instead of an actual video stream, then it wouldn't realistically be measurable. But if latency is particularly low, this would be a boon for AR systems.

BTW, this is pumping the same video file through the network - not just a single file. I don't measure latency, but this is not a deep pipeline so it's easy to calculate.

Ok, I guess I misread that part.

Re: Object Detection at 1840 FPS with TorchScript, TensorRT and DeepStream

#38

Post by ebalit »

mozak1111 wrote: post #32

I see this and I immediately think of "trash sorting" at ultra high speed. If one can combine this with a bunch of accurate (laser precision) air guns, to shoot and move individual pieces of trash you can sort through a truck load of trash in a matter of seconds, perhaps in the air while they are being dumped! compare this approach with how we are currently doing it [0] - Somebody should get Elon Musk on this project…

Not related to Elon but there is a company called Pellenc ST in the south of France that work exactly on this kind of problem. You can see a video of one of their machines here [0].

I work at an AI consultancy [1] that help them use deep neural nets in these high throughput and low latency conditions. It's an interesting challenge and the performance than can be squeezed from modern hardware are indeed impressive.

0: https://youtu.be/XLciSGE82DY?t=280

1: https://neovision.fr

Re: Object Detection at 1840 FPS with TorchScript, TensorRT and DeepStream

#39

Post by liuliu »

moron4hire wrote: post #26

Any word on latency? I didn't see anything in the article. I guess, since this is a synthetic test just pumping a single image file through repeatedly instead of an actual video stream, then it wouldn't realistically be measurable. But if latency is particularly low, this would be a boon for AR systems.

Latency fundamentally limited by the model processing single frame, all-in-all, probably somewhere around 10 to 15ms depends on your input size (assuming VGA type of input). This is a great article talks about system-engineering for the vision pipeline, but to solve the latency issue, you need either a beefier processor (or more specialized processor) or a better tuned algorithm.

Re: Object Detection at 1840 FPS with TorchScript, TensorRT and DeepStream

#40

Post by zanethomas »

cm2187 wrote: post #4

Out of curiosity, what are the possible use cases for object detection at >100 fps? I assume it would have to be objects that move very fast, i.e. nothing ordinary that I can think of. [edit] actually stupid question. I assume it's more about throughput than fps, i.e. be able to process lots of streams on the same machine, for instance for doing mass analysis of CCTV streams.

Shooting down drown swarms.
Post reply on HN