131–140 of 196 posts
Earlier quoted context omitted.
I think Apple will become increasingly draconian about LLMs. Very soon people won't need to buy many of their apps. They can just make them. This threatens Apple's entire business model.
VibeOS. It’s just an LLM from which all other userspace is vibed.
Where can I get this amazing technology?
Earlier quoted context omitted.
Huh? LLM output doesn't have the variety of human output, since they operate in fixed fashion - statistical inference followed by formulaic sampling. Additionally, the statistics used by LLMs are going be be similar across different LLMs since at scale its just "the statistics of the internet". Human output has much more variety, partly because we're individuals with our own reading/writing histories (which we're dra…
> LLM output doesn't have the variety of human output, since they operate in fixed fashion - statistical inference followed by formulaic sampling. This is the wrong thing to look at; your chess analogy is much stronger, the detection method similar (if you can figure out a prompt that generates something close to the content, it almost certainly isn't human origin). But to why the thing I'm quoting doesn't work: If y…
Earlier quoted context omitted.
I don't care if it's written by an LLM. The problem with the article is the complete lack of details. No benchmarks on the iPhone capable models. No details, whatsoever. Human or LLM - the article is a whole lot of nothing.
This article is all fluff because real benne marketing. If they mentioned that a 4B model on an iPhone 16 drains 15% of the battery for a single long prompt and triggers hard thermal throttling after 20 seconds, nobody would be clicking on headlines about "commercial viability" fwiw
Is the output coherent though? I am yet to see a local model working on consumer grade hardware being actually useful.
I have a 128 GB Strix Halo tablet (same as the other commenter here with the Framework Desktop). I'm using the larger Gemma 4 26B-A4B model (only 28 GB @ Q8) and it's been working great and runs very fast. It's a 100% replacement for free ChatGPT/Gemini. Compared to the paid pro/thinking models... Gemma does have reasoning, and I have used the reasoning mode for some tax & legal/accounting advice recently as well as…
I noticed the inference is routed through the gpu rather than the Apple neural engine. Google’s engineers likely gave up on trying to compile custom attention kernels for Apple’s proprietary tensor blocks iirc. While Metal is predictable and easy to port to, it drains the battery way faster than a dedicated NPU. Until they rewrite the backend for the ANE, this is just a flashy tech demo rather than a production-ready…
The AI Edge Gallery app on Android (which is the officially recommended way to try out Gemma on phones) uses the GPU (lacks NPU support) even on first party Pixel phones. So it's less of "they didn't want to interface with Apple's proprietary tensor blocks" and more of that they just didn't give a f in general. A truly baffling decision.
Is it me, or does the article sound like LLM output? The pattern "It's not mere X — it's Y", occurs like 4 times in the text :v
qwen3-coder-next uses a lot less since it seems to only activate ~3B parameters at a time.
My guess is that this is still close to tech demo, and a lot of performance is left on the table.
Is it me, or does the article sound like LLM output? The pattern "It's not mere X — it's Y", occurs like 4 times in the text :v