Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Object Detection at 1840 FPS with TorchScript, TensorRT and DeepStream

paulbridger.com

Post reply on HN
Print view View original

51–58 of 58 posts

Re: Object Detection at 1840 FPS with TorchScript, TensorRT and DeepStream

#51

Post by nl »

janimo wrote: post #14

How portable are these techniques to other architectures? Could >100 FPS be realistically achieved today using only CPUs or mobile phones?

> Could >100 FPS be realistically achieved today using only CPUs or mobile phones?

Not yet.

Google's MediaPipe object detector (which is one of the most optimised mobile solutions around) can do "26fps on an Adreno 650 mobile GPU"[1].

The Adreno 650 is the GPU in the Snapdragon 865, ie the current high end SOC used by most non-Apple phones. This gives roughly the same performance as an iPhone 11.

[1] https://google.github.io/mediapipe/solutions/objectron.html

[2] https://www.tweaktown.com/news/69097/qualcomm-adreno-650-gpu...

[3] https://www.tomsguide.com/news/snapdragon-865-benchmarks

Re: Object Detection at 1840 FPS with TorchScript, TensorRT and DeepStream

#52

Post by analognoise »

handol wrote: post #49
Cerium wrote: post #42

Earlier quoted context omitted.

Sorting by optical recognition and air guns to separate a falling curtain of product into two output streams is already a product. The development of these machines are the reason that 10 or 15 years ago you stopped seeing bad beans in bulk bean bags. I am involved in the tea industry where they are used to sort tea by grade - stems, bad leaves, broken leaves, full leaves. Here is a diagram: https://www.satake-usa.co…

Reminds me of the library modernization drive in 'Rainbows End'. The book digitizer is basically a wood chipper with lights and high speed cameras in the debris chute.

There was a company in Japan that would digitize your books and destroy the original (to prevent infringement claims), iirc.

Re: Object Detection at 1840 FPS with TorchScript, TensorRT and DeepStream

#53

Post by briggers »

Grimm1 wrote: post #44

Good work getting TensorRT running we had a real pain in the butt recently when working with it and just opted to go with ONNXRuntime, their graph optimizer and their TensorRT backend -- may not be as fast as straight TensorRT from comparisons I've seen but it got us to a competitive inference and latency so we're happy with it.

Nice one! I've long been interested in the ONNX serving path.

Re: Object Detection at 1840 FPS with TorchScript, TensorRT and DeepStream

#54

Post by SequoiaHope »

Cerium wrote: post #42
mozak1111 wrote: post #32

I see this and I immediately think of "trash sorting" at ultra high speed. If one can combine this with a bunch of accurate (laser precision) air guns, to shoot and move individual pieces of trash you can sort through a truck load of trash in a matter of seconds, perhaps in the air while they are being dumped! compare this approach with how we are currently doing it [0] - Somebody should get Elon Musk on this project…

Sorting by optical recognition and air guns to separate a falling curtain of product into two output streams is already a product. The development of these machines are the reason that 10 or 15 years ago you stopped seeing bad beans in bulk bean bags. I am involved in the tea industry where they are used to sort tea by grade - stems, bad leaves, broken leaves, full leaves. Here is a diagram: https://www.satake-usa.co…

Yes but when I went to the recycling (sorting) facility in San Mateo they remarked that their plastic sorting systems work by infrared reflection and so cannot see black plastic. They said that because of this they are unable to process black plastic entirely. I got the sense that there’s room for improvement.

Bonus slow motion footage of their processing machines: https://imgur.com/gallery/IK5zKkO

Re: Object Detection at 1840 FPS with TorchScript, TensorRT and DeepStream

#55

Post by TaylorAlexander »

cm2187 wrote: post #4

Out of curiosity, what are the possible use cases for object detection at >100 fps? I assume it would have to be objects that move very fast, i.e. nothing ordinary that I can think of. [edit] actually stupid question. I assume it's more about throughput than fps, i.e. be able to process lots of streams on the same machine, for instance for doing mass analysis of CCTV streams.

Plenty of robotics tasks could benefit from high FPS tracking of single streams. Generally process tasks where faster=better. But yes, tracking many streams at once is useful too!

Re: Object Detection at 1840 FPS with TorchScript, TensorRT and DeepStream

#56

Post by TaylorAlexander »

raverbashing wrote: post #7
bufferoverflow wrote: post #6

Earlier quoted context omitted.

Self-driving. Ideally you want something around 1000fps and low latency, so it has time to react. I'm sure military and sports applications are obvious too.

Doubt Humans reaction times are much slower than that. In fact for some things it can take a whole second https://www.visualexpert.com/Resources/reactiontime.html Maybe racing sports have shorter reaction times, but I'd be frankly surprised if it was something 10fps for your average drive should be more than enough

For reference I am making a farming robot that goes at 1 meter per second and I run the main control loop at 10hz. I would absolutely run a car that goes freeway speeds at at least 100hz.

Re: Object Detection at 1840 FPS with TorchScript, TensorRT and DeepStream

#57

Post by janimo »

nl wrote: post #51
janimo wrote: post #14

How portable are these techniques to other architectures? Could >100 FPS be realistically achieved today using only CPUs or mobile phones?

> Could >100 FPS be realistically achieved today using only CPUs or mobile phones? Not yet. Google's MediaPipe object detector (which is one of the most optimised mobile solutions around) can do "26fps on an Adreno 650 mobile GPU"[1]. The Adreno 650 is the GPU in the Snapdragon 865, ie the current high end SOC used by most non-Apple phones. This gives roughly the same performance as an iPhone 11. [1] https://google.g…

Thanks for the links. I think there also isn't an API for accessing high FPS cameras on Android devices that support slow motion video capture.

Re: Object Detection at 1840 FPS with TorchScript, TensorRT and DeepStream

#58

Post by bufferoverflow »

raverbashing wrote: post #7
bufferoverflow wrote: post #6

Earlier quoted context omitted.

Self-driving. Ideally you want something around 1000fps and low latency, so it has time to react. I'm sure military and sports applications are obvious too.

Doubt Humans reaction times are much slower than that. In fact for some things it can take a whole second https://www.visualexpert.com/Resources/reactiontime.html Maybe racing sports have shorter reaction times, but I'd be frankly surprised if it was something 10fps for your average drive should be more than enough

I'm not sure what your argument is. Any self-driving system should strive to be much better than human drivers. It makes complete sense to have reaction time much better than human.
Post reply on HN