Video Surveillance with YOLO+llava • Hacker News Board

Re: Video Surveillance with YOLO+llava

#31

Post by jamesbfb » Tue, Oct 08, 2024, 10:45 AM UTC

skirmish wrote: post #21

ferar wrote: post #4

Can you specify ideal hardware (camera, computer) to deploy the solution? Thanks

Here are hardware recommendations from another similar (and well established) project: [1] [2]. Even though they don't recommend Reolink cameras, I have both Amcrest and Reolink cameras working well with Frigate for more than a year now. [1] https://docs.frigate.video/frigate/hardware [2] https://github.com/blakeblackshear/frigate

+1 for Frigate and Reolink. I have it running in a Proxmox VM on an old dell r710 (yes, it’s sucks watts and needs replacing) but all said, Frigate, is, amazing! The ease of integration with HA is equally great.

Re: Video Surveillance with YOLO+llava

#32

Post by formerly_proven » Tue, Oct 08, 2024, 11:26 AM UTC

pmontra wrote: post #20

This runs with a Geforce GTX 1060. By a quick search it's 120 W. Maybe it's only the peak power consumption but it's still a lot. Do commercial products, if there are any, consume that much power?

YOLO is quick enough that you can just run it on a CPU, assuming you don’t want to run it at full resolution (no point) and full frame rate (ditto) for multiple streams. When you run it scaled down at a 2-3 fps you’ll get several streams per CPU core no problem. Energy use can be minimized by running a quick motion detection pass before, but that would obviously make the system miss things creeping through the frame pixel by pixel (very unlikely if you ask me)

Re: Video Surveillance with YOLO+llava

#33

Post by formerly_proven » Tue, Oct 08, 2024, 11:30 AM UTC

hcfman wrote: post #24

sinuhe69 wrote: post #13

Earlier quoted context omitted.

Not OP, but the reason may be: US - FCC Ban The US Federal Communications Commission (FCC) banned Dahua and Hikvision from new equipment authorizations in November 2022. Most products that use electricity require FCC equipment authorizations; otherwise, they are illegal to import, sell, market, or use, even for private individuals. Jul 5, 2024

Shame, they are the best cameras available.

Also it’s not like you stop supporting these OEMs if you buy other made in china cameras. They’re essentially all designed and manufactured by very few of these large OEMs, all of which are implicated in CCP state surveillance.

You’d have to buy from actual Western companies like Axis or Dallmeier.

Re: Video Surveillance with YOLO+llava

#34

Post by netdur » Tue, Oct 08, 2024, 11:36 AM UTC

jerpint wrote: post #26

rocauc wrote: post #16

A suggestion: I'd swap llava for Florence-2 for your open set text description. Florence-2 seems uniformly more descriptive in its outputs.

I found grounding-dino better than Florence and faster

I found YOLOS to be faster and better, bot real time but 22k objects under half second

Re: Video Surveillance with YOLO+llava

#35

Post by xrd » Tue, Oct 08, 2024, 11:53 AM UTC

I'm confused about why you need yolo and llava. Can't you simply use yolo without a multimodal LLM? What does that add? You can use yolo to detect and grab screen coordinates on its own, right?

Re: Video Surveillance with YOLO+llava

#36

Post by mobilemidget » Tue, Oct 08, 2024, 12:38 PM UTC

lelag wrote: post #28

4ggr0 wrote: post #27

Earlier quoted context omitted.

not sure if i'm misunderstanding - you've got a similar GPU to a 1060 hooked up to a pi?

OP is probably using an AI accelerator like this: https://coral.ai/products/accelerator which works great on a PI and uses very little power. It will do the Yolo part, but you can't really expect it to do the multimodal LLM part, although you could try to run Florence directly on the PI too.

This works better in my experience,

https://hailo.ai https://www.raspberrypi.com/news/raspberry-pi-ai-kit-availab...

Re: Video Surveillance with YOLO+llava

#37

Post by michaelt » Tue, Oct 08, 2024, 12:42 PM UTC

xrd wrote: post #35

I'm confused about why you need yolo and llava. Can't you simply use yolo without a multimodal LLM? What does that add? You can use yolo to detect and grab screen coordinates on its own, right?

Almost certainly using yolo to segment the cars, then llava for the more detailed "silver sedan" description

Re: Video Surveillance with YOLO+llava

#38

Post by npteljes » Tue, Oct 08, 2024, 1:13 PM UTC

ferar wrote: post #4

Can you specify ideal hardware (camera, computer) to deploy the solution? Thanks

I can recommend the Axis brand. Very user friendly while power user friendly as well, true local offerings. I personally bought mine used, it's an older model, and even then, it holds up really well.

Re: Video Surveillance with YOLO+llava

#39

Post by anshumankmr » Tue, Oct 08, 2024, 1:28 PM UTC

Could try with Florence by Microsoft instead of Yolo and Llava, though the results are not going to be as great. Florence will do the inference on CPU. This is just for fun.

Re: Video Surveillance with YOLO+llava

#40

Post by [deleted] » Tue, Oct 08, 2024, 1:30 PM UTC

vaylian wrote: post #29

Hello from the privacy crowd! Please use this responsibly. Tech can be a lot of fun and I encourage you to play around with things and I appreciate it when you push the boundaries of what is technically feasible. But please be mindful that surveillance tech can also be used to oppress people and infringe on their freedoms. Use tech for good!

[deleted]