361–370 of 411 posts
Earlier quoted context omitted.
Do you work in the space?
Nope, not at all. Was just curious what you were using it for that was so unrelated to computers, but what you said makes a lot of sense
Earlier quoted context omitted.
> transcribe or translate anything you say Is it multimodal? How do you do transcription with it?
Gemma 4 E2B, E4B and 12B unified accept audio - here's a recipe using MLX that can use it for transcription: https://simonwillison.net/2026/Apr/12/mlx-audio/ Only up to 30s though, and the larger 26B A4B and 31B models are text and image only.
I forked llama.cpp and added some crude mechanism to keep exactly this behavior under control - essentially guiding the reasoning process by injecting text strategically at specific thresholds. This was mainly put together to rein in Qwen3.6-27B, but I'd imagine 3.8 would react similarly. Fork can be found here - https://github.com/laurencehardman/llama-mindcontrol/tree/ma... Of course hacks like this are not perfect…
“The fact that a 17GB file can do all of this stuff on my home machines is a miracle. Once again, I’m delighted and amazed at how much progress local models have made this year.” I think that should be the blinking headline - this shows what can be done with consumer hardware.
Full agree. I until very recently thought AI tools of today were limited to prohibitively expensive high end hardware hosted in data centers. I was surprised and amazed to get "decent" (with the expectations set right / low) coding performance out of Qwen3.5-9B on a decidedly medium end Radeon 9070 paired with a 5700x3d and 32GB of DDR4 RAM. We can finally reason with and "talk" to our hardware.
The next 10 years of computing is going to be VERY interesting. May be not just computing but everything, an even wider reach and disruption than iPhone.
Unlike iPhone / Smartphone which I thought was obvious, a computer or web browser in your pocket that basically extends the internet to anywhere. Local Model AI in our computer or pocket that controls other things opens seemingly unlimited possibilities.
I initially thought it was going to be 15-20 years time frame, with perhaps a bubble burst in between. But development of both software and hardware is accelerating.
Earlier quoted context omitted.
> One issue I have with Gemma is that they seem to use old architectures that rely on full attention, requiring a lot of RAM for context and quickly degrading speeds as context is filled. Yes, this is something I hope they will change. Gemma 4 31B is much slower on pre-Blackwell GPUs as a result, which is a bit of a shame for local model experimentation.
Even Muse Glimmer (as did GPT-OSS I think) does ~4 sliding window attention layers + 1 full attention layer (like Gemma 4). I’m assuming both labs have good reason to think that gated delta nets are not optimal. Of course it’s possible the labs just stick with the optimal architecture for large models and GDN is best for smaller models.
Earlier quoted context omitted.
Solar power and batteries are getting cheaper and cheaper at the moment. So Watts should become cheaper in the long run. Especially when chips are becoming cheaper (in the capex sense), then you can afford to only run them when power is cheap. Btw, from where do you take the notion that performance per Watt ain't increasing? We are also still using what's more or less general purpose GPU hardware; we could get a lot…
Like I said, just feels I have from the consumer-hardware space. For several generations of GPU now most improvements come from packing more transistors into a larger die than packing more transistors closer to each other. GPUs have been getting physically bigger with huge heatsinks and fans to support those bigger dies power consumption. Just compare the TDPs: 2020 RTX 3090: 350W 2022 RTX 4090: 450W 2025 RTX 5090: 5…
Earlier quoted context omitted.
Does that mean it's intelligent? To me it just means they can brilliantly fake human conversation - the original design goal of Large Language Models. It's really easy to tell if you're talking to an LLM if you ask a question that requires actually knowing things, not going for the first search result of a tool call or whatever most popular answer was embedded in the weights. For this reason even the most sophisticat…
Why does this matter?
My comparison of its reasoning efforts[0] seems to show that it only really supports 3 modes: none, low, xhigh. Low and medium are basically the same. Also, the electricity it costs to run on a 3090 is not negligible, so that it's cheaper to use Luna high via API than Qwen 3.8 27b locally, hardware costs excluding. [0]: https://aibenchy.com/compare/qwen-qwen3-8-27b-high/qwen-qwen...
$0.286/kWh is a ridiculous amount of money to pay for power. That's more than double the regional residental average here! If I ever found myself in this situation I would much rather just rent cards from hotasile and run open models instead of giving OAI money and playing reset bingo
Earlier quoted context omitted.
I love reading Glimmer's "thoughts". Why use many word when few do trick?
It’s also a little bit snarky, almost. The stuff it thought during the car wash puzzle made me laugh.