Would object detection like this work out of the box for deer like he demonstrates for humans? I need this for deer.
Tensorflow on edge, or – Building a “smart” security camera with a Raspberry Pi
11–20 of 83 posts
Re: Tensorflow on edge, or – Building a “smart” security camera with a Raspberry Pi
#12I've just started down this path using a plain RTSP-serving camera and a low end box using the Coral EdgeTPU to process the frames. It looks like there are a variety of solutions available. https://github.com/blakeblackshear/frigate https://docs.ambianic.ai/users/configure/ etc
Re: Tensorflow on edge, or – Building a “smart” security camera with a Raspberry Pi
#13Nice writeup but the Raspberry Pi isn't running tensorflow. It is mentioned in the article that the author is sending images to an edge machine. The big question I had was about hardware video encoding/decoding ... doesn't really cover that. I've found sending single image frames over zeromq to be fairly limiting if you care about high frame rate/low latency processing. Key issue I have run into is while many chips s…
This is where GStreamer normally steps in. A lot of hardware manufacturers provide a gstreamer plugin for their module. I’ve had experience with NVIDIA and atmel SoCs and that seemed to be the default path.
Good luck with the gstreamer pipeline learning curve however!
Re: Tensorflow on edge, or – Building a “smart” security camera with a Raspberry Pi
#14Is it really edge computing if the pi isn’t running tensorflow? I know the definition is kind of woolly. I wonder what the performance would be on a $100 jetson nano.
it's not going to the cloud, so I'd say that counts.
Re: Tensorflow on edge, or – Building a “smart” security camera with a Raspberry Pi
#15I am trying to achieve something similar but at a higher scale.
I have about 48 different cameras where I want to count people and get their approximate location in the frame.
I want to run an object detection model on all of those video streams simultaneously.
My AWS instance maxes out after 7 simultaneous streams so I figured I don't really need real-time monitoring. One frame every couple of seconds, even every minute could potentially suffice, since I am dealing with larger time-frames. Since I don't want to run too many instances at the same time, what are some viable strategies to achieve this?
My plan is to have 5-6 instances of the ML model loaded up and waiting to accept a frame. When one of them is ready, it will instruct one of the RTSP streams to send it a frame, which it will process and store / send the result to an application server. I feel like I may not even be able to consume so many RTSP streams at once (I've never tried so I don't know), so I may have to have some other method of priming the handshake etc. before the model asks for a frame to process.
Is there a better / non-hacky way of achieving this (i.e. managing the workload on a single GPU instance) ?
I don't have any control of the camera hardware at all.
Re: Tensorflow on edge, or – Building a “smart” security camera with a Raspberry Pi
#16 the Raspberry Pi isn't running tensorflow
Re: Tensorflow on edge, or – Building a “smart” security camera with a Raspberry Pi
#17Is it really edge computing if the pi isn’t running tensorflow? I know the definition is kind of woolly. I wonder what the performance would be on a $100 jetson nano.
The Jetson Nano is fantastic, and I use Tensorflow on it with two home security cameras. It is a wonderful device, and deserves far more attention than it gets.
I'm going to upgrade in the next week to a Jetson Xavier NX. Not because I need to, but because I like playing around and it's a silly powerful device.
I also run a NextDNS CLI client on it, various automation stuff, etc.
Re: Tensorflow on edge, or – Building a “smart” security camera with a Raspberry Pi
#18I've done this with a Jetson Xavier, 4 CCTV cameras and a PoE hub. You really want to use DeepStream and C/C++ for inference, not Python and TensorFlow.
I'm streaming ~20 fps (17 to 30) 720P directly from my home IP4 address, and when a person is in-frame long enough and caught by the tracker, a stream goes to an AWS endpoint for storage.
I've experimented with both SSDMobileNet and Yolo3, which are both pretty error prone but they do a much better job filtering out moving tree limbs and passing clouds, unlike Arlo.
You need way more processing power than an RPi to do this at 30fps, and C/C++, not Python. (There are literally dozens of projects for the RPi and TFlow online but they all get like 0.1 fps or less by using Flask and browser reload of a PNG... great for POC but not for real video)
I wrote very little of the code, honestly: only the capture pipe required a new C element. I started with NVidia DeepStream which is phenomenally well-written, and their built-in accelerated RTSP element, and added a custom GStreamer element that outputs a downsampled MPEG capture to the cloud when the upstream detector tracks an object. NVidia also wrote the tracker, you just need to provide an object detector like SSDMobileNet or YOLO. NVidia gets it.
The main 4 camera-pipe mux splits into the AI engine and into a tee to the RTSP server on one side and my capture element on the other side.
It was amazingly simple, and If I turn the CCD cameras down to 720P with h265 and a low bitrate, I don't need to turn on the noisy Xavier fan. The onboard Arm core does the detected downsampling (one camera only, a limitation right now) and pushes the video with a rest endpoint on a node server in the AWS cloud.
I'm very pleased with it, I haven't tested scaling but if I turned off the GPU governors I could easily go to 8 cameras. I went with PoE because WiFi can't handle the demand.
Re: Tensorflow on edge, or – Building a “smart” security camera with a Raspberry Pi
#19esp32-cam microcontroller costs around $6 and you have face detection build in. It have bluetooth and wifi and most of it's drivers code if not everything is on github. Only problem is you need to program it using arduino or other microcontroller hardware.
Re: Tensorflow on edge, or – Building a “smart” security camera with a Raspberry Pi
#20Earlier quoted context omitted.
Curious what you found limiting about zeromq? Just not enough throughput for high FPS? So far I’ve found it to be the sanest multicast solution since clients pull.
Issue isn't ZeroMQ. The simple/inefficient way to do it is to capture frames one at a time, and send them via ZeroMQ. Video is pretty bandwidth intensive .. the only reason things like YouTube work as smooth as they do is that they use codecs such H264/265 (which are proprietary unfortuantely) and stream compress frames over the network. Now doing the codec in software burns a lot of CPU as this is very math intensiv…
Aren't IP cameras capable of H264 already encoded the video before output it to the network? H264 video stream has very good compression ratio and shouldn't consume too much bandwidth.