Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tensorflow on edge, or – Building a “smart” security camera with a Raspberry Pi

chollinger.com

Post reply on HN
Print view View original

41–50 of 83 posts

Re: Tensorflow on edge, or – Building a “smart” security camera with a Raspberry Pi

#41

Post by agibsonccc »

snowzach wrote: post #4
brutus1213 wrote: post #2

Nice writeup but the Raspberry Pi isn't running tensorflow. It is mentioned in the article that the author is sending images to an edge machine. The big question I had was about hardware video encoding/decoding ... doesn't really cover that. I've found sending single image frames over zeromq to be fairly limiting if you care about high frame rate/low latency processing. Key issue I have run into is while many chips s…

Shameless plug, check out DOODS: https://github.com/snowzach/doods It's a simple REST/gRPC API for doing object detection with Tensorflow or Tensorflow Lite. It will run on a Raspberry Pi. It actually did support the EdgeTPU hardware accelerator to make the Pi pretty quick for certain models. They broke something so I need to fix EdgeTPU support but it's still usable on the Pi withe the mobilenet models or inception…

Few questions:

1. Did you build this for your own use cases? Interesting side project?

2. How do you feel about the need for base64 being a requirement on the endpoints? Isn't GRPC the wrong medium for this? Also, what do you see as the main limitations right now? The models?

Re: Tensorflow on edge, or – Building a “smart” security camera with a Raspberry Pi

#42

Post by agibsonccc »

NikolaeVarius wrote: post #24
thebruce87m wrote: post #13

Earlier quoted context omitted.

This is where GStreamer normally steps in. A lot of hardware manufacturers provide a gstreamer plugin for their module. I’ve had experience with NVIDIA and atmel SoCs and that seemed to be the default path. Good luck with the gstreamer pipeline learning curve however!

Can confirm not fun

Could you elaborate on some of the problems you had overall?

Re: Tensorflow on edge, or – Building a “smart” security camera with a Raspberry Pi

#44

Post by agibsonccc »

alkonaut wrote: post #11

Would object detection like this work out of the box for deer like he demonstrates for humans? I need this for deer.

Hi, could you describe your use case a bit? Just an alarm trigger for deer in the backyard?

Re: Tensorflow on edge, or – Building a “smart” security camera with a Raspberry Pi

#45

Post by agibsonccc »

fareesh wrote: post #15

I am trying to achieve something similar but at a higher scale. I have about 48 different cameras where I want to count people and get their approximate location in the frame. I want to run an object detection model on all of those video streams simultaneously. My AWS instance maxes out after 7 simultaneous streams so I figured I don't really need real-time monitoring. One frame every couple of seconds, even every mi…

Hi Fareesh, I'd love to hear more about your use case. Email's in my profile.

Re: Tensorflow on edge, or – Building a “smart” security camera with a Raspberry Pi

#46

Post by shiftpgdn »

fareesh wrote: post #15

I am trying to achieve something similar but at a higher scale. I have about 48 different cameras where I want to count people and get their approximate location in the frame. I want to run an object detection model on all of those video streams simultaneously. My AWS instance maxes out after 7 simultaneous streams so I figured I don't really need real-time monitoring. One frame every couple of seconds, even every mi…

48 RTSP streams is a lot of bandwidth to consume at once. Why not use an edge PC or Jetson system to do it in small blocks? A new Jetson Xavier NX can do 8-12 streams depending on FPS and model.

Re: Tensorflow on edge, or – Building a “smart” security camera with a Raspberry Pi

#47

Post by quietbritishjim »

staycoolboy wrote: post #18

I've done this with a Jetson Xavier, 4 CCTV cameras and a PoE hub. You really want to use DeepStream and C/C++ for inference, not Python and TensorFlow. I'm streaming ~20 fps (17 to 30) 720P directly from my home IP4 address, and when a person is in-frame long enough and caught by the tracker, a stream goes to an AWS endpoint for storage. I've experimented with both SSDMobileNet and Yolo3, which are both pretty error…

> You really want to use ... C/C++ for inference, not Python ...

> You need ... C/C++, not Python.

I think this is a red herring. Usually for deep learning you just use Python to plug together the libraries that actually do the processing, and those are written in terms of C/C++. You can see that in the article where the numpy array returned from OpenCV's video capture API is passed directly to tensorflow. Python never touches the individual pixels of the image directly, and once that's inside tensorflow it's irrelevant that a Python object briefly represented it.

> with a Jetson Xavier

Well that's obviously the real difference. It's not even just the same general type of computer but a bit faster - the Jetson has a decent NVidia GPU on board whereas the Raspberry Pi is doing the processing on its extremely limp CPU. Indeed that's the whole point of the Jetson; it's basically an NVidia graphics card with extra components strapped to it to turn it into a full computer.

> You really want to use DeepStream ... not TensorFlow

I'm not familiar with DeepStream, so I'm not so sure about this, but again this is unlikely to make a great deal of difference. It's certainly not the main factor at play here: that's definitely the Jetson's GPU, which of course TensorFlow can certainly use (via CUDA and CUDNN, as does DeepStream). It's true that using TensorRT can provide a speed boost on a Jetson, but even that's possible with TensorFlow, although admittedly you have to remember to call it specifically but it's just three or four lines of (Python!) code. There are already so many ways it's unavoidable to tie yourself into NVidia's ecosystem, it seems like a bad idea to tie yourself in further in a totally avoidable way like this.

[Edit: I just realised that the image is being streamed to a remote computer that's doing the inference. The general point remains though. The totally different architecture (including having to transfer data over the network) and hardware are the actual reason for the performance difference, while C/C++ vs Python and DeepStream vs TensorFlow are tiny details.]

Re: Tensorflow on edge, or – Building a “smart” security camera with a Raspberry Pi

#48

Post by agibsonccc »

quietbritishjim wrote: post #47
staycoolboy wrote: post #18

I've done this with a Jetson Xavier, 4 CCTV cameras and a PoE hub. You really want to use DeepStream and C/C++ for inference, not Python and TensorFlow. I'm streaming ~20 fps (17 to 30) 720P directly from my home IP4 address, and when a person is in-frame long enough and caught by the tracker, a stream goes to an AWS endpoint for storage. I've experimented with both SSDMobileNet and Yolo3, which are both pretty error…

> You really want to use ... C/C++ for inference, not Python ... > You need ... C/C++, not Python. I think this is a red herring. Usually for deep learning you just use Python to plug together the libraries that actually do the processing, and those are written in terms of C/C++. You can see that in the article where the numpy array returned from OpenCV's video capture API is passed directly to tensorflow. Python nev…

If something were to be more "neutral" what would you hope to see exactly? Something performant is typically going to be framework/hardware specific.

Re: Tensorflow on edge, or – Building a “smart” security camera with a Raspberry Pi

#49

Post by nl »

scottlamb wrote: post #33
staycoolboy wrote: post #18

I've done this with a Jetson Xavier, 4 CCTV cameras and a PoE hub. You really want to use DeepStream and C/C++ for inference, not Python and TensorFlow. I'm streaming ~20 fps (17 to 30) 720P directly from my home IP4 address, and when a person is in-frame long enough and caught by the tracker, a stream goes to an AWS endpoint for storage. I've experimented with both SSDMobileNet and Yolo3, which are both pretty error…

> You need way more processing power than an RPi to do this at 30fps, and C/C++, not Python. (There are literally dozens of projects for the RPi and TFlow online but they all get like 0.1 fps or less by using Flask and browser reload of a PNG... great for POC but not for real video) I think 8 streams at 15 fps (aka 120 fps total) is possible with a ($35) Raspberry Pi 4 + ($75) Coral USB Accelerator. I say "I think" b…

The Orange Pi AI Stick Lite looks really interesting.

Here's the link: https://www.aliexpress.com/item/32958159325.html and it says the PLAI training tools are (now?) free on request.

Post reply on HN