Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A 30B Qwen model walks into a Raspberry Pi and runs in real time

byteshape.com

Post reply on HN
Print view View original

111–120 of 139 posts

Re: A 30B Qwen model walks into a Raspberry Pi and runs in real time

#111

Post by 6510 »

sdenton4 wrote: post #28
6510 wrote: post #16

Earlier quoted context omitted.

It should participate in all conversations, take initiative and experiment.

"Hey, hey, are you still asleep? Using spare cycles, I have designed an optimal recipe for mashed potatoes, as you mentioned ten days ago. I need you to go get some potatoes."

I ponder the concept in the 90's. Initially I thought it should be an assistant but with age came wisdom and now I think it should be a virtual drill instructor. "Rise and shine $insult $insult, the sun is up, the store is open, we will be getting some potatoes today, $insult $insult, it was all your idea now apply yourself!" Bright lights flashing, loud music, the shower starts running. "Shower time, you have 7 minutes! $insult $insult" 4 minutes in the coffee machine boots up. "You will be wearing the blue pants, top shelve on the left stack, the green shirt, 7th from the left. Faster faster! $insult $insult"

Re: A 30B Qwen model walks into a Raspberry Pi and runs in real time

#112

Post by 6510 »

FeepingCreature wrote: post #54
sdenton4 wrote: post #28

Earlier quoted context omitted.

"Hey, hey, are you still asleep? Using spare cycles, I have designed an optimal recipe for mashed potatoes, as you mentioned ten days ago. I need you to go get some potatoes."

I unironically want this.

I forget who but someone onhere a while back said he made a contraption that listens in and tries to determine the winner of each conversation.

Re: A 30B Qwen model walks into a Raspberry Pi and runs in real time

#114

Post by PaulDavisThe1st »

terribleperson wrote: post #68
sdenton4 wrote: post #28

Earlier quoted context omitted.

"Hey, hey, are you still asleep? Using spare cycles, I have designed an optimal recipe for mashed potatoes, as you mentioned ten days ago. I need you to go get some potatoes."

A local AI system that hears your conversations, identifies problems, and then uses spare cycles to devise solutions for them is actually an incredible idea. I'm never going to give a cloud system the kind of access it would need to do a really good job, but a local one I control? Absolutely. "Hey, are you still having trouble with[succinct summary of a problem it identified]?" "Yes" "I have a solution that meets you…

> A local AI system that hears your conversations, identifies problems, and then uses spare cycles to devise solutions for them is actually an incredible idea.

I call that Dreaming.

(TM)

Re: A 30B Qwen model walks into a Raspberry Pi and runs in real time

#116

Post by nunodonato »

nl wrote: post #88
Aurornis wrote: post #86

Earlier quoted context omitted.

I’ve experimented with several of the really small models. It’s impressive that they can produce anything at all, but in my experience the output is basically useless for anything of value.

Yes, I thought that too! But qwen3:0.6b (and to some extent gemma 1b) has made me reevaluate. They still aren't useful like large LLMs, but for things like summarization, and other tasks where you can give them structure but want the sheen of natural language they are much better than things like the Phi series were.

qwen3 family, mostly 4B and 8B are absolutely amazing. the VL versions even more

Re: A 30B Qwen model walks into a Raspberry Pi and runs in real time

#117

Post by joshstrange »

Normal_gaussian wrote: post #110
joshstrange wrote: post #105

Earlier quoted context omitted.

> but I've have great success with Home Assistant and the Home Assistant Voice Preview edition As compared to Alexa? I bought their preview hardware (and had a home-rolled ESP32 version before that even) and things are getting closer, I can see the future where this works but we aren't there today IMHO. HA Voice (the current hardware) does not do well enough in the mic or speaker [0] department when compared to the E…

I've not had any issues with the audio picking up, but its in the living room rather than the kitchen. I have Alexa's in most rooms. I don't play music through it, which I do from the Alexa. Tbh I think the mic and the speakers will be fine when the rest of the 'product' is sorted. I failed to mention I have Claude connected to it rather than their default assistant. To us, this just beats Alexa hands down. I have th…

Interesting, well I'm glad it's working well for you all. I tested with local, HA Cloud, and ChatGPT/Claude and that wasn't the sticking point, it was getting the hardware to hear me or for me to hear it.

I will say, while it was too slow (today) with the my local inference hardware (CPU, older computer and a little on my newer MBP) it was magically to talk to and hear back from HA all locally. I look forward to a future where I can do that at the same speed/quality as the cloud models. Yes, I know cloud models will continue to get better but turning on/off my fans/lights/etc doesn't need to best model available, just needs to be reliable and fast, I'm even fine with it "shelling out" to the cloud if I ask it for something outside of the basics though I doubt I'll care to do that.

Re: A 30B Qwen model walks into a Raspberry Pi and runs in real time

#118

Post by BizarroLand »

mkul wrote: post #75
jmward01 wrote: post #6

There is a huge market segment waiting here. At least I think there is. Well, at least people like me want this. Ok, tens of dollars can be made at least. It is just missing a critical tipping point. Basically, I want an alexa like device for the home backed by local inference and storage with some standardized components identified: - the interactive devices - all the alexa/google/apple devices out there are this in…

I've just started using it but I'd recommend https://github.com/steipete/clawdis , you need to set it up a bit but it's really cool to just be able to do things on the go by just texting an assistant. You can see all the different ways people are using it @clawdbot on twitter.

Why does it require an online AI service? Why can it not work with ollama or some other locally hosted setup?

Re: A 30B Qwen model walks into a Raspberry Pi and runs in real time

#119

Post by BizarroLand »

PaulDavisThe1st wrote: post #114
terribleperson wrote: post #68

Earlier quoted context omitted.

A local AI system that hears your conversations, identifies problems, and then uses spare cycles to devise solutions for them is actually an incredible idea. I'm never going to give a cloud system the kind of access it would need to do a really good job, but a local one I control? Absolutely. "Hey, are you still having trouble with[succinct summary of a problem it identified]?" "Yes" "I have a solution that meets you…

> A local AI system that hears your conversations, identifies problems, and then uses spare cycles to devise solutions for them is actually an incredible idea. I call that Dreaming. (TM)

If you could get an AI to listen to the conversations that happen in your sphere of influence and simply jot down the problems it identifies over the course of the day/week/month/year, that in itself would be an amazing tool.

Doubly so if you could just talk and brainstorm while it's listening and condensing, so you can circle back later and see what raindrops formed from the brainstorm.

Call that DayDreaming (TM)

Post reply on HN