Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Video Surveillance with YOLO+llava

github.com

Post reply on HN
Print view View original

41–50 of 70 posts

Re: Video Surveillance with YOLO+llava

#41

Post by andblac »

xrd wrote: post #35

I'm confused about why you need yolo and llava. Can't you simply use yolo without a multimodal LLM? What does that add? You can use yolo to detect and grab screen coordinates on its own, right?

Skimming through the source it seems to run 'car' and 'person' objects through llava with the following prompt:

- "person": "get gender and age of this person in 5 words or less",

- "car": "get body type and color of this car in 5 words or less".

So YOLO gives the bounding box and rough category, while llava describes the object in more details.

Re: Video Surveillance with YOLO+llava

#42

Post by formerly_proven »

2Gkashmiri wrote: post #17
meow_catrix wrote: post #12

Earlier quoted context omitted.

”Analysts noticed that CCTV cameras in Taiwan and South Korea were digitally talking to crucial parts of the Indian power grid – for no apparent reason. On closer investigation, the strange conversation was the deliberately indirect route by which Chinese spies were interacting with malware they had previously buried deep inside the Indian power grid.”

link? i am close to CCTV retailers and dahua and hikvision are only brands of CCTV widely available with two exceptions of "cp plus" and "hawkvision" which are in all lilkelihood rebranded or made in china products. https://www.amazon.in/s?k=cctv+system+4+channel so what are your options? i have been contemplating getting a door phone + cctv for my home for the past so many years but problems like these prevent me fr…

> are in all lilkelihood rebranded or made in china products

IPVM did all the legwork on this a while ago and unconvered that, not that surprisingly, two and a half OEMs (including Dahua and Hikvision) are manufacturing essentially every not-completely-garbage CCTV camera coming out of china, and a bunch that very explicitly claimed to not come out of china.

Re: Video Surveillance with YOLO+llava

#44

Post by synergy20 »

mobilemidget wrote: post #36
lelag wrote: post #28

Earlier quoted context omitted.

OP is probably using an AI accelerator like this: https://coral.ai/products/accelerator which works great on a PI and uses very little power. It will do the Yolo part, but you can't really expect it to do the multimodal LLM part, although you could try to run Florence directly on the PI too.

This works better in my experience, https://hailo.ai https://www.raspberrypi.com/news/raspberry-pi-ai-kit-availab...

coral has pcie module which is 1/4 to 1/3 of the price

Re: Video Surveillance with YOLO+llava

#45

Post by gh02t »

01100011 wrote: post #43

If you're interested in DIY security+AI, check out Frigate NVR( https://frigate.video/ ), Scrypted( https://www.scrypted.app/ ) and Viseron( https://viseron.netlify.app/ ).

I've been using Frigate for a long time and it's a really cool project that has been quite reliable. The configuration can be a little bit of a headache to learn, but it gets better with every release.

Viserion is new to me though, that looks really cool.

Re: Video Surveillance with YOLO+llava

#46

Post by phito »

pmontra wrote: post #20

This runs with a Geforce GTX 1060. By a quick search it's 120 W. Maybe it's only the peak power consumption but it's still a lot. Do commercial products, if there are any, consume that much power?

You can use a Coral USB Accelerator, doesn't use more than 10W.

Re: Video Surveillance with YOLO+llava

#47

Post by nicholasjarnold »

phito wrote: post #46
pmontra wrote: post #20

This runs with a Geforce GTX 1060. By a quick search it's 120 W. Maybe it's only the peak power consumption but it's still a lot. Do commercial products, if there are any, consume that much power?

You can use a Coral USB Accelerator, doesn't use more than 10W.

Can confirm. The Coral inference accelerator is quite performant with very low power draw. Once I figured out some passthrough and config issues I was able to run Frigate in an LXC container on Proxmox using Coral USB for inference. It's been working reliably 24/7 for months now.

Re: Video Surveillance with YOLO+llava

#48

Post by dfc »

01100011 wrote: post #43

If you're interested in DIY security+AI, check out Frigate NVR( https://frigate.video/ ), Scrypted( https://www.scrypted.app/ ) and Viseron( https://viseron.netlify.app/ ).

I just recently got frigate up and running. How do the other two compare?

Re: Video Surveillance with YOLO+llava

#49

Post by Eisenstein »

phito wrote: post #46
pmontra wrote: post #20

This runs with a Geforce GTX 1060. By a quick search it's 120 W. Maybe it's only the peak power consumption but it's still a lot. Do commercial products, if there are any, consume that much power?

You can use a Coral USB Accelerator, doesn't use more than 10W.

You can see here:

                res = rest(ollama, {

                    "model": "llava",

                    "prompt": genprompt(box.name),

                    "images": [box.export()],

                    "stream": False

                })
They are calling the ollama API to run Llava. Llava is a combo of an LLM base model and + vision projector (clip or ViT), and is usually around 4 - 8GB. Since every token generated needs access to all of the model weights, you would have to send 4 - 8 GB through USB with the Coral. Even at a generous 10gbit/s that is 8GB / 1.25GB = 6.4seconds per token. A 150 (short paragraph) generation would be 16minutes.

Re: Video Surveillance with YOLO+llava

#50

Post by matrik »

MobileNetV3 and EfficientDet are othwr possible alternatives to YOLO. I was able to get higher than 1.5 FPS on Raspberry Pi Zero 2W which draws 1W on average. With efficient queuing approach, one can eliminate all bottlenecks.
Post reply on HN