Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tensorflow on edge, or – Building a “smart” security camera with a Raspberry Pi

chollinger.com

Post reply on HN
Print view View original

61–70 of 83 posts

Re: Tensorflow on edge, or – Building a “smart” security camera with a Raspberry Pi

#61

Post by gspr »

dheera wrote: post #51
hikarudo wrote: post #38

Earlier quoted context omitted.

I disagree. The term "edge computing" actually adds precision to a description of a distributed system. Nowadays, with a lot of machine learning inference happening on the cloud, when seeing the term "edge inference" you immediately know you don't have to send heavy bandwidth-clogging video streams to the cloud. Inference on the edge is a clear trend in computer vision applications, now that we each year there are be…

> Nowadays, with a lot of machine learning inference happening on the cloud Right, and if it's not on the cloud, it runs locally, as everything did before "cloud" became popular. We don't need to call it "edge" just to raise VC money or put out some PR. We can just say it runs locally, on-device, etc. If (big if) and when Adobe realizes that their Creative Cloud was a bad idea, are they going to call the next product…

> Right, and if it's not on the cloud, it runs locally, as everything did before "cloud" became popular. We don't need to call it "edge" just to raise VC money or put out some PR. We can just say it runs locally, on-device, etc.

To me, "edge" means more than just "not cloud". It's appropriately used when making the point that computations happen where the data is gathered and the output is required (which seems actually not to be the case in TFA, but still). It's when computations are not offloaded elsewhere at all, not just "not to the cloud".

Re: Tensorflow on edge, or – Building a “smart” security camera with a Raspberry Pi

#62

Post by scottlamb »

staycoolboy wrote: post #58
scottlamb wrote: post #33

Earlier quoted context omitted.

> You need way more processing power than an RPi to do this at 30fps, and C/C++, not Python. (There are literally dozens of projects for the RPi and TFlow online but they all get like 0.1 fps or less by using Flask and browser reload of a PNG... great for POC but not for real video) I think 8 streams at 15 fps (aka 120 fps total) is possible with a ($35) Raspberry Pi 4 + ($75) Coral USB Accelerator. I say "I think" b…

I started with an Rpi by itself. Then I tried a Coral USB stick. I also tried the Intel Neural Compute Stick 2. The Coral USB accelerator doesn't accelerate all of the layers, only some of them. The CPU has to do the rest of the work. Plus, you only get this speed if you preload an image into memory and blast it through the accelerator in a loop. This ignores getting the image INTO the accelerator, which requires res…

> The Coral USB accelerator doesn't accelerate all of the layers, only some of them.

My understanding is that with the pretrained models, everything happens on the TPU. If you use some lightweight transfer learning techniques to tweak the model [1], the last layer happens on the CPU. That's supposed to be insignificant, but I haven't actually tried it.

I'm very curious what you're using for a model. You're clearly further along than I am. Did you use your own cameras' data? Did you do transfer learning? (If so, what did you start from? you mentioned SSDMobileNet and Yolo3. Do you have a favorite?) Did you build a model from scratch?

Anyway, my point is that a similar project seems doable on a Raspberry Pi 4 with some extra hardware. I don't mean to say that you're Doing It Wrong for using a Xavier. I've thought about buying one of those myself...

[1] https://coral.ai/docs/edgetpu/models-intro/#transfer-learnin...

Re: Tensorflow on edge, or – Building a “smart” security camera with a Raspberry Pi

#63

Post by josteink »

> We’ll use a Raspberry Pi 4 with the camera module to detect video. ... Now, here’s an issue for you: My old RasPi runs a 32bit version of Raspbian.

So why not just use the 64-bit Ubuntu RPi image instead then?

https://ubuntu.com/download/raspberry-pi

Re: Tensorflow on edge, or – Building a “smart” security camera with a Raspberry Pi

#64

Post by quietbritishjim »

m463 wrote: post #56
quietbritishjim wrote: post #47

Earlier quoted context omitted.

> You really want to use ... C/C++ for inference, not Python ... > You need ... C/C++, not Python. I think this is a red herring. Usually for deep learning you just use Python to plug together the libraries that actually do the processing, and those are written in terms of C/C++. You can see that in the article where the numpy array returned from OpenCV's video capture API is passed directly to tensorflow. Python nev…

I think the difference with the jetson xavier is the tensor cores. The xavier is different from the pi (and even the jetson nano), like 100x different.

The Raspberry Pi doesn't have any "tensor cores" at all. According to Wikipedia, it actually does have a "Broadcom VideoCore IV" GPU, but I don't think this processor is ever used for deep learning. So if you did inference on the Pi then it would have to be on the CPU; inference is slower even on a meaty desktop CPU than on a GPU, never mind the low-powered CPU on the Pi.

That is all academic, as the whole point of the article is actually that the processing isn't done on the Pi but on the remote server. In that case the difference (if there even is one, I don't see a frame rate mentioned in the article) is indeed down to the difference in power of the respective GPUs, as you're alluding to, or to do with the fact that the article is having to stream the image frames over the network (it doesn't even seem to compress them) whereas the parent comment's idea just processes them locally.

Re: Tensorflow on edge, or – Building a “smart” security camera with a Raspberry Pi

#65

Post by gambiting »

brutus1213 wrote: post #2

Nice writeup but the Raspberry Pi isn't running tensorflow. It is mentioned in the article that the author is sending images to an edge machine. The big question I had was about hardware video encoding/decoding ... doesn't really cover that. I've found sending single image frames over zeromq to be fairly limiting if you care about high frame rate/low latency processing. Key issue I have run into is while many chips s…

Yeah I was hoping it was a raspberry pi maybe using one of these neural nets USB sticks, instead it was just using RPi as a dumb terminal for sending video. You could probably do the same with an old android phone set to stream video over lan.

Re: Tensorflow on edge, or – Building a “smart” security camera with a Raspberry Pi

#66

Post by quietbritishjim »

agibsonccc wrote: post #48
quietbritishjim wrote: post #47

Earlier quoted context omitted.

> You really want to use ... C/C++ for inference, not Python ... > You need ... C/C++, not Python. I think this is a red herring. Usually for deep learning you just use Python to plug together the libraries that actually do the processing, and those are written in terms of C/C++. You can see that in the article where the numpy array returned from OpenCV's video capture API is passed directly to tensorflow. Python nev…

If something were to be more "neutral" what would you hope to see exactly? Something performant is typically going to be framework/hardware specific.

Sorry, I'm not sure what you mean by "neutral". Are you talking about my suggestion to avoid DeepStream? If so:

The frameworks that work on multiple types of hardware, like TensorFlow and (probably most popular now) PyTorch, have separate backends for their different targets. Each of these backends have huge amounts of platform-specific code, and in the case of the Nvidia backend, that code is written in terms of CUDA just as DeepStream is. That's how they achieve good performance even though the top-level API is hardware generic. The overwhelming majority of deep learning code, both the actual learning and the inference, is written in terms of these frameworks rather than NVidia's proprietary framework. Admittedly I haven't played with NVidia's library, but I highly doubt there's a serious performance difference - it's even possible that the open-source libraries are faster due to the greater community (/Google) effort to optimise them.

It does look like DeepStream does a lot more of the processing pipeline than just the inference. In that case it's going to be a lot more tricky to get the whole pipeline on the GPU using those TensorFlow or PyTorch. At the end of the day, if only DeepStream does what you need, I'm not saying you necessarily shouldn't use it - just that you should ideally attempt to avoid it if reasonably possible.

Re: Tensorflow on edge, or – Building a “smart” security camera with a Raspberry Pi

#67

Post by scottlamb »

scottlamb wrote: post #57
scottlamb wrote: post #33

Earlier quoted context omitted.

> You need way more processing power than an RPi to do this at 30fps, and C/C++, not Python. (There are literally dozens of projects for the RPi and TFlow online but they all get like 0.1 fps or less by using Flask and browser reload of a PNG... great for POC but not for real video) I think 8 streams at 15 fps (aka 120 fps total) is possible with a ($35) Raspberry Pi 4 + ($75) Coral USB Accelerator. I say "I think" b…

> * iirc, I got the Coral USB Accelerator to do about 180 fps with this model. [edit: but don't trust my memory—it could have been as low as 100 fps.] Just dusted off my test program. 115.5 fps on my Intel NUC. I think that's the limit of this model on the Coral USB Accelerator, or very close to it. My Raspberry Pi 4 is still compiling...I might update with that number in a bit. Likely the H.264 decoding will be the…

72.2 fps on the Raspberry Pi 4 right now, with CPU varying between 150%–220%. I expect with some work I could max out the Coral USB Accelerator as the Intel NUC is likely doing already.

Re: Tensorflow on edge, or – Building a “smart” security camera with a Raspberry Pi

#69

Post by alkonaut »

agibsonccc wrote: post #44
alkonaut wrote: post #11

Would object detection like this work out of the box for deer like he demonstrates for humans? I need this for deer.

Hi, could you describe your use case a bit? Just an alarm trigger for deer in the backyard?

Yes, I'd point a camera at my precious vegetables and if a deer walks into the video feed, something that scares it off is triggered so it runs off before eating the whole garden.

Re: Tensorflow on edge, or – Building a “smart” security camera with a Raspberry Pi

#70

Post by quietbritishjim »

brutus1213 wrote: post #2

Nice writeup but the Raspberry Pi isn't running tensorflow. It is mentioned in the article that the author is sending images to an edge machine. The big question I had was about hardware video encoding/decoding ... doesn't really cover that. I've found sending single image frames over zeromq to be fairly limiting if you care about high frame rate/low latency processing. Key issue I have run into is while many chips s…

> Nice writeup but the Raspberry Pi isn't running tensorflow. It is mentioned in the article that the author is sending images to an edge machine.

Yeah I was a bit surprised by this, and although the article is very clear about it I think it's generated a bit of confusion in the comments here. My understanding of edge computing is that it means the processing of data is done at the point the data is captured, so to me that would mean right there on the raspberry pi. But the author considers their whole LAN to be the "edge", so basically anything that doesn't involve sending the data over the internet:

> ... doing the heavy lifting on a machine physically close to the edge node – in this case, running the Tensorflow Object detection. By doing so, we avoid roundtrips over the internet, as well as having to pay for Cloud compute on e.g., AWS or GCP.

I think their strategy of capturing the data on a very low-power device and then processing on a server on your network is a very reasonable one, I just wouldn't have used that term.

Post reply on HN