Live data from Hacker News

Object Detection at 1840 FPS with TorchScript, TensorRT and DeepStream

paulbridger.com

11–20 of 58 posts

Re: Object Detection at 1840 FPS with TorchScript, TensorRT and DeepStream

#11
post #8

A weird question, but since there's another article on HN right now about programming language energy efficiency https://news.ycombinator.com/item?id=24816733 any idea whether going from 9fps to 1840fps consumes the same power, 200x the power, or somewhere in between?

Great question, now I wish I'd recorded power consumption for all these experiments. Judging from cumulative hours of watching the output of nvidia-smi I've definitely seen a linearish relationship between utilization and power draw (with a non-zero floor of 30-40W).

Re: Object Detection at 1840 FPS with TorchScript, TensorRT and DeepStream

#12
post #4

Out of curiosity, what are the possible use cases for object detection at >100 fps? I assume it would have to be objects that move very fast, i.e. nothing ordinary that I can think of. [edit] actually stupid question. I assume it's more about throughput than fps, i.e. be able to process lots of streams on the same machine, for instance for doing mass analysis of CCTV streams.

Smart missiles and weapons i guess

[deleted]

Re: Object Detection at 1840 FPS with TorchScript, TensorRT and DeepStream

#13

Earlier quoted context omitted.

Self-driving. Ideally you want something around 1000fps and low latency, so it has time to react. I'm sure military and sports applications are obvious too.

Doubt Humans reaction times are much slower than that. In fact for some things it can take a whole second https://www.visualexpert.com/Resources/reactiontime.html Maybe racing sports have shorter reaction times, but I'd be frankly surprised if it was something 10fps for your average drive should be more than enough

100ms is generally considered to be the lower bound for a valid start, i.e. 99 would be considered a jump start in F1 iirc

Re: Object Detection at 1840 FPS with TorchScript, TensorRT and DeepStream

#15
post #4

Out of curiosity, what are the possible use cases for object detection at >100 fps? I assume it would have to be objects that move very fast, i.e. nothing ordinary that I can think of. [edit] actually stupid question. I assume it's more about throughput than fps, i.e. be able to process lots of streams on the same machine, for instance for doing mass analysis of CCTV streams.

While I'm not into object detection such as this, I can easily imagine this being part of a system where you want the rest of the system to have time to act on the information.

As such the point isn't that you can detect objects >N fps, but rather that the object detection shouldn't take more than X% of the time per cycle so that the overall cycle time can run at a given rate.

Re: Object Detection at 1840 FPS with TorchScript, TensorRT and DeepStream

#16

Earlier quoted context omitted.

Self-driving. Ideally you want something around 1000fps and low latency, so it has time to react. I'm sure military and sports applications are obvious too.

Doubt Humans reaction times are much slower than that. In fact for some things it can take a whole second https://www.visualexpert.com/Resources/reactiontime.html Maybe racing sports have shorter reaction times, but I'd be frankly surprised if it was something 10fps for your average drive should be more than enough

But machines aren't (yet) as capable as humans at driving-situation-recognition and driving-decision-making. One way they can compensate for those shortcomings is to be superior in other ways: 100% vigilance and super-fast reaction times/decision-making.

Re: Object Detection at 1840 FPS with TorchScript, TensorRT and DeepStream

#18
post #4

Out of curiosity, what are the possible use cases for object detection at >100 fps? I assume it would have to be objects that move very fast, i.e. nothing ordinary that I can think of. [edit] actually stupid question. I assume it's more about throughput than fps, i.e. be able to process lots of streams on the same machine, for instance for doing mass analysis of CCTV streams.

For me it's only exciting because it lowers the barrier of how much I can do with a much smaller system than double 2080ti's.

Re: Object Detection at 1840 FPS with TorchScript, TensorRT and DeepStream

#19
post #2

> There is evidence (measured using gil_load) that we were throttled by a fundamental Python limitation with multiple threads fighting over the Global Interpreter Lock (GIL). Can anyone comment on how often this is a problem and if this problem is truly fundamental to Python? Could it be solved in a Python 3.x release?

Yes, this is a fundamental part of Python. By default, a single Python process is single-threaded in the traditional sense. So, using "threads" (i.e. the Threading module) in Python is actually more like using fibers in some other language. They're not OS threads. So, if you're not waiting on I/O, then yes, the threads will fight over the GIL and performance will suffer. This is inherent to Python and will not be changed.

But there's a few more things that can be said about this. Python "threads" are really just a mental construct for designing programs. The selling point is that you can share variables and data between "threads" without having to worry about locks or data corruption or anything like that. It just works. But, even with that advantage, you're relying on Python to switch between "threads" on its own, and that could easily slow things down. If you're willing to drop the mental construct and go for better performance but still use a single process and be able to share variables, the asyncio module will let you control when the main Python process will move between points of code flow.

However, if you really want to use traditional multiple processes/threads just use the Multiprocessing module. It actually launches multiple Python processes and links them together. It's called in a similar fashion to Threading, so there isn't much code change for that part. But because it's no longer a single process - and no longer bound by the GIL - you can't share data between the processes as easily. With Multiprocessing, you'll need to create slightly more complex data structures (like a multiprocessing manager namespace) to share that data. It's not that hard, but it requires a bit of planning ahead of time.

Re: Object Detection at 1840 FPS with TorchScript, TensorRT and DeepStream

#20
post #8

A weird question, but since there's another article on HN right now about programming language energy efficiency https://news.ycombinator.com/item?id=24816733 any idea whether going from 9fps to 1840fps consumes the same power, 200x the power, or somewhere in between?

I see Rust is almost equal to C if not better in the graph, however, I think equally-skilled programmers in either language would show the Rust programmer spending more 'energy' programming and iterating than the C programmer, but then make the argument that the C program will use more 'energy' downstream if bugs slip in. In any case, an eye-opening metric on what I, and I am sure many, take for granted. Cool.
Post reply on HN