Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3-Omni: Native Omni AI model for text, image and video

github.com

Post reply on HN
Print view View original

31–40 of 152 posts

Re: Qwen3-Omni: Native Omni AI model for text, image and video

#31

Post by whimsicalism »

bilbo0s wrote: post #29
state_less wrote: post #25

Interesting, the pacing seemed very slow when conversing in english, but when I spoke to it in spanish, it sounded much faster. It's really impressive that these models are going to be able to do real time translation and much more. The Chinese are going to end up owning the AI market if the American labs don't start competing on open weights. Americans may end up in a situation where they have some $1000-2000 device…

Americans may end up in a situation where they have some $1000-2000 device at home with an open Chinese model running on it Wouldn't worry about that, I'm pretty sure the government is going to ban running Chinese tech in this space sooner or later. And we won't even be able to download it. Not saying any of the bans will make any kind of sense, but I'm pretty sure they're gonna say this is a "strategic" space. And e…

government hardly has the capacity to ban foreign weights

Re: Qwen3-Omni: Native Omni AI model for text, image and video

#32

Post by whimsicalism »

neilmovva wrote: post #4

The multilingual example in the launch graphic has Qwen3 producing the text: > "Bonjour, pourriez-vous me dire comment se rendreà la place Tian'anmen?" translation: "Hello, could you tell me how to get to Tiananmen Square?" a bold choice!

only really a reference with the date or at least 89

Re: Qwen3-Omni: Native Omni AI model for text, image and video

#33

Post by tomatoman »

tedivm wrote: post #28
state_less wrote: post #25

Interesting, the pacing seemed very slow when conversing in english, but when I spoke to it in spanish, it sounded much faster. It's really impressive that these models are going to be able to do real time translation and much more. The Chinese are going to end up owning the AI market if the American labs don't start competing on open weights. Americans may end up in a situation where they have some $1000-2000 device…

This is exactly what I do. I have two 3090s at home, with Qwen3 on it. This is tied into my Home Assistant install, and I use esp32 devices as voice satellites. It works shockingly well.

Seems interesting setup, do you have it documented anywhere, thinking of building one!

Re: Qwen3-Omni: Native Omni AI model for text, image and video

#34

Post by data-ottawa »

hadlock wrote: post #19

Speech input + speech output is a big deal. In theory you can talk to it using voice, and it can respond in your language, or translate for someone else, without intermediary technologies. Right now you need wakeword, speech to text, and then text to speech, in addition to your core LLM. A couple can input speech, or output speech, but not both. It looks like they have at least 3 variants in the ~32b range. Depending…

The opportunities of plugging this into your home automation through tool calls is huge.

Ever since ChatGPT added this feature I've been waiting for anyone else to catch up.

They're are tons of hands free situations like cooking where this would be amazing ("read the next step please, my hands are covered in raw pork", "how much flour for the roux", "crap, I don't have any lemons, what can I substitute")

Re: Qwen3-Omni: Native Omni AI model for text, image and video

#35

Post by joshgachnang »

tedivm wrote: post #28
state_less wrote: post #25

Interesting, the pacing seemed very slow when conversing in english, but when I spoke to it in spanish, it sounded much faster. It's really impressive that these models are going to be able to do real time translation and much more. The Chinese are going to end up owning the AI market if the American labs don't start competing on open weights. Americans may end up in a situation where they have some $1000-2000 device…

This is exactly what I do. I have two 3090s at home, with Qwen3 on it. This is tied into my Home Assistant install, and I use esp32 devices as voice satellites. It works shockingly well.

Ooo interesting, I'd love to hear more about the esp32's as voice satellites!

Re: Qwen3-Omni: Native Omni AI model for text, image and video

#36

Post by sho_hn »

joshgachnang wrote: post #35
tedivm wrote: post #28

Earlier quoted context omitted.

This is exactly what I do. I have two 3090s at home, with Qwen3 on it. This is tied into my Home Assistant install, and I use esp32 devices as voice satellites. It works shockingly well.

Ooo interesting, I'd love to hear more about the esp32's as voice satellites!

I assume it's very similar to what Home Assistant's backing commercial entity Nabu Casa sells with the "Home Assistant Voice PE" device, which is also esp32-based. The code is open and uses the esphome framework so it's fairly easy to recreate on custom HW you have laying around.

Re: Qwen3-Omni: Native Omni AI model for text, image and video

#37

Post by pzo »

will be interesting how it will compare with pricing with audio modality comparing to gemini 2.0 flash once many providers offer it.

Even though gemini 2.0 flash is quite old I still like it. Very cheap (each second of audio is just 32 tokens), support even more languages, non-reasoning so very fast, big rate limits.

Re: Qwen3-Omni: Native Omni AI model for text, image and video

#38

Post by mrandish »

tedivm wrote: post #28
state_less wrote: post #25

Interesting, the pacing seemed very slow when conversing in english, but when I spoke to it in spanish, it sounded much faster. It's really impressive that these models are going to be able to do real time translation and much more. The Chinese are going to end up owning the AI market if the American labs don't start competing on open weights. Americans may end up in a situation where they have some $1000-2000 device…

This is exactly what I do. I have two 3090s at home, with Qwen3 on it. This is tied into my Home Assistant install, and I use esp32 devices as voice satellites. It works shockingly well.

I run Home Assistant on an RPi4 and have an ESP32-based Core2 with mic (https://shop.m5stack.com/products/m5stack-core2-esp32-iot-de...), along with a 16GB 4070 Ti Super in an always-on Windows system I only use for occasional gaming and serving media. I'd love to set up something like you have. Can you recommended a starting place, or ideally, a step-by-step tutorial?

I've never set up any AI system. Would you say setting up such a self-hosted AI is at a point now where an AI novice can get an AI system installed and integrated with an existing Home Assistant install in a couple hours?

Re: Qwen3-Omni: Native Omni AI model for text, image and video

#39

Post by richardlblair »

I recently needed to scan hundreds of low quality invoices and run them through OCR for invoice numbers and dates. I really took for granted how seamless this is in some applications, and was shocked how much work went into producing decent results.

I was obviously really naive. Either way, it gets me excited any time I see progress with OCR. I should give this a try against my (small) dataset.

Re: Qwen3-Omni: Native Omni AI model for text, image and video

#40

Post by state_less »

tedivm wrote: post #28
state_less wrote: post #25

Interesting, the pacing seemed very slow when conversing in english, but when I spoke to it in spanish, it sounded much faster. It's really impressive that these models are going to be able to do real time translation and much more. The Chinese are going to end up owning the AI market if the American labs don't start competing on open weights. Americans may end up in a situation where they have some $1000-2000 device…

This is exactly what I do. I have two 3090s at home, with Qwen3 on it. This is tied into my Home Assistant install, and I use esp32 devices as voice satellites. It works shockingly well.

That's great to hear. I was mostly impressed with Qwen3 coder on my 4090, but am hobbled by the small memory footprint of the single card. What motherboard are you using with your 3090s? Like the others, I too am curious about those esp32s and what software you run on them.

Keep up the good hacking - it's been fun to play with this stuff!

Post reply on HN