Live data from Hacker News

Video Surveillance with YOLO+llava

github.com

51–60 of 70 posts

Re: Video Surveillance with YOLO+llava

#51
post #4

Can you specify ideal hardware (camera, computer) to deploy the solution? Thanks

I can recommend the Axis brand. Very user friendly while power user friendly as well, true local offerings. I personally bought mine used, it's an older model, and even then, it holds up really well.

+1 for Axis

Re: Video Surveillance with YOLO+llava

#52
post #4

Can you specify ideal hardware (camera, computer) to deploy the solution? Thanks

Here are hardware recommendations from another similar (and well established) project: [1] [2]. Even though they don't recommend Reolink cameras, I have both Amcrest and Reolink cameras working well with Frigate for more than a year now. [1] https://docs.frigate.video/frigate/hardware [2] https://github.com/blakeblackshear/frigate

Many Amcrest IP Cameras are manufactured by Dahua and use localized versions of Dahua firmware. The same applies to the Lorex brand in the United States.

Some things that matter when it comes to configuring your IP Cameras (Beyond security, etc): - Support for RTSP - Configurable Encoding Settings (e.g. h264 coded, bitrate, i-frame intervals, framerate) - Support for Substreams (i.e. a full-resolution main stream for recording, and at least one lower-resolution substream for preview/detection/etc) ...

Make sure the hardware you select is capable of the above.

Configurability will matter because Identification is not the same as Detection (Reference: "DORI" - Detection, Observation, Recognition, and Identification from IEC EN62676-4). If you want to be able to successfully identify objects or entities using your cameras, it will require more care than basic Observation or Detection.

Re: Video Surveillance with YOLO+llava

#53
post #27
post #23

Earlier quoted context omitted.

I have something similar. It's not tracking though. Drawing around 10W on a pi, around 7W on a Jetson.

not sure if i'm misunderstanding - you've got a similar GPU to a 1060 hooked up to a pi?

Not a pi. A Jetson. Still an arm SBC though.

Re: Video Surveillance with YOLO+llava

#54
post #46
post #20

This runs with a Geforce GTX 1060. By a quick search it's 120 W. Maybe it's only the peak power consumption but it's still a lot. Do commercial products, if there are any, consume that much power?

You can use a Coral USB Accelerator, doesn't use more than 10W.

Yeah. But it’s likely it’s an 8-bit quantised, likely very small model with a small number of parameters. Which translates into poor recall and lots of false positives.

How many parameters is the model you are using with hailo? And what’s the quantisation and what model is it actually ?

Re: Video Surveillance with YOLO+llava

#55
post #20

This runs with a Geforce GTX 1060. By a quick search it's 120 W. Maybe it's only the peak power consumption but it's still a lot. Do commercial products, if there are any, consume that much power?

There's a wide range of inference accelerators in commercial use.

For "edge" or embedded applications, an accelerator such as the Google Coral Edge TPU is a useful reference point where it is capable of up to 4 Trillion Operations per Second (4 TOPS), with up to 2 Watts of power consumption (2 TOPS/W), however the accelerator is limited to INT8 operations. It also has around 8 MB of memory for model storage.

Meanwhile a general purpose or gaming GPU can support a wider range of instructions, single-precision, double-precision floating point, integer, etc).

Geforce GTX 1060 for example: 4.375 TFLOPS (FP32) @ 120W (https://www.techpowerup.com/gpu-specs/geforce-gtx-1060-6-gb....)

There are commercial-oriented products that are optimized for particular operations and precision.

Here's a blog post discussing Google's 1st-generation ASIC TPU used in its datacenters: https://cloud.google.com/blog/products/ai-machine-learning/a...

(92 TOPS @ 700 Mhz - 40W)

https://arxiv.org/abs/1704.04760

Re: Video Surveillance with YOLO+llava

#56

Earlier quoted context omitted.

Here are hardware recommendations from another similar (and well established) project: [1] [2]. Even though they don't recommend Reolink cameras, I have both Amcrest and Reolink cameras working well with Frigate for more than a year now. [1] https://docs.frigate.video/frigate/hardware [2] https://github.com/blakeblackshear/frigate

Many Amcrest IP Cameras are manufactured by Dahua and use localized versions of Dahua firmware. The same applies to the Lorex brand in the United States. Some things that matter when it comes to configuring your IP Cameras (Beyond security, etc): - Support for RTSP - Configurable Encoding Settings (e.g. h264 coded, bitrate, i-frame intervals, framerate) - Support for Substreams (i.e. a full-resolution main stream for…

Isn’t it illegal now to import HIKvision and Dahua to the states now ?

Re: Video Surveillance with YOLO+llava

#57
post #56

Earlier quoted context omitted.

Many Amcrest IP Cameras are manufactured by Dahua and use localized versions of Dahua firmware. The same applies to the Lorex brand in the United States. Some things that matter when it comes to configuring your IP Cameras (Beyond security, etc): - Support for RTSP - Configurable Encoding Settings (e.g. h264 coded, bitrate, i-frame intervals, framerate) - Support for Substreams (i.e. a full-resolution main stream for…

Isn’t it illegal now to import HIKvision and Dahua to the states now ?

AFAIK, the FCC ban pertains to particular applications (or marketing of products for such applications). It did not apply to consumer applications.

"On November 25, 2022, the Federal Communications Commission (FCC) released new rules restricting equipment that poses national security risks from being imported to or sold in the United States. Under the new rules, the FCC will not issue new authorizations for telecommunications equipment produced by Huawei Technologies Company (Huawei) and ZTE Corporation (ZTE), the two largest telecommunications equipment manufacturers in the People’s Republic of China (PRC).

The FCC also will not authorize equipment produced by three PRC-based surveillance camera manufacturers—Hytera Communications (Hytera), Hangzhou Hikvision Digital Technology (Hikvision), and Dahua Technology (Dahua)—until the FCC approves these entities’ plans to ensure that their equipment is not marketed or sold for public safety purposes, government facilities, critical infrastructure, or other national security purposes. The FCC did not, however, revoke any of its prior authorizations for these companies’ equipment, although it sought comments on whether it should do so in the future."

https://crsreports.congress.gov/product/pdf/LSB/LSB10895/1

Re: Video Surveillance with YOLO+llava

#58

If you're interested in DIY security+AI, check out Frigate NVR( https://frigate.video/ ), Scrypted( https://www.scrypted.app/ ) and Viseron( https://viseron.netlify.app/ ).

I've been running frigate for a while now and I find it's object detection has a higher than preferred false-positive rate.

For instance, it kept thinking the tree in my back yard is a person. I find it hilarious that it often assigns a higher likelihood the tree is a person than me! I've needed to put a mask over the tree as a last resort.

Re: Video Surveillance with YOLO+llava

#59
post #54
post #46

Earlier quoted context omitted.

You can use a Coral USB Accelerator, doesn't use more than 10W.

Yeah. But it’s likely it’s an 8-bit quantised, likely very small model with a small number of parameters. Which translates into poor recall and lots of false positives. How many parameters is the model you are using with hailo? And what’s the quantisation and what model is it actually ?

Honestly I have no idea what you are asking about. It's just dedicated hardware to a yolo-like object detection model

Re: Video Surveillance with YOLO+llava

#60
post #46

Earlier quoted context omitted.

You can use a Coral USB Accelerator, doesn't use more than 10W.

You can see here: res = rest(ollama, { "model": "llava", "prompt": genprompt(box.name), "images": [box.export()], "stream": False }) They are calling the ollama API to run Llava. Llava is a combo of an LLM base model and + vision projector (clip or ViT), and is usually around 4 - 8GB. Since every token generated needs access to all of the model weights, you would have to send 4 - 8 GB through USB with the Coral. Even…

Hm yeah sure, I didn't think of the llm part. I don't think it's really useful tbh.
Post reply on HN