51–60 of 114 posts
i am more excited about the ondevice foundation model update that is coming https://developer.apple.com/documentation/updates/foundation... (not much info yet) but i maintain https://github.com/Arthur-Ficial/apfel so i might be biased
This is why the AI companies are rushing to IPO. By the end of next year you’ll be running most of your AI on device. They have no moat, they’ve reached the limits of scaling, most of the magic can be distilled into smaller models, and they know it
Why on earth I should switch from a top tier model to much worse local model ? Why do I need to suffer my battery ?
i am more excited about the ondevice foundation model update that is coming https://developer.apple.com/documentation/updates/foundation... (not much info yet) but i maintain https://github.com/Arthur-Ficial/apfel so i might be biased
Seems like they still won’t let you run models on GPU while the phone is closed or the user switches apps
Earlier quoted context omitted.
Majority of my agentic setup is pi / Claude code where every single Chinese models are not as good except commercial 1T models . Local is a pipe dream . If you can run it cheap occasionally why commercial companies can’t run it cheaper 24/7 and lower the costs ? The answer is simple. Use cases are more demanding and hence you need more from model not less . Sure if you task is to do a narrow labeling task on 1m recor…
This sounds like something someone at IBM in 1986 would say trying to sell their mainframes. "PCs will never be a thing. No one's gonna want a computer." I'm seeing some impressive results from folks that can afford 10k+ GPUs right now. But those GPUs will all be hand me downs in 10 years. So pipe dream? Hmmm...... that's not how this industry works.
But it doesn’t seem anywhere imminent with our current world state.
Earlier quoted context omitted.
Didn't Qwen stop releasing their more powerful models because they're commercializing them?
Yes and no. Qwen 3.5 was released 3/2/2026. It includes models up to a 397B-A17B model https://huggingface.co/collections/Qwen/qwen35 A day afterwards, a high-up technical leader working on Qwen was let go https://techcrunch.com/2026/03/03/alibabas-qwen-tech-lead-st... The more recent Qwen 3.6 was released on 4/16 https://huggingface.co/collections/Qwen/qwen36 This does not include any particularly large models. But…
Earlier quoted context omitted.
This sounds like something someone at IBM in 1986 would say trying to sell their mainframes. "PCs will never be a thing. No one's gonna want a computer." I'm seeing some impressive results from folks that can afford 10k+ GPUs right now. But those GPUs will all be hand me downs in 10 years. So pipe dream? Hmmm...... that's not how this industry works.
Those are not GPUs available on iPhones. Will we get there eventually? Maybe! Maybe we end up with GPU clusters built on the edge (e.g. cell towers) for offloading, maybe it’s never economical, maybe a different model architecture makes it simpler, who knows. But it doesn’t seem anywhere imminent with our current world state.
For whatever reason every generations thinks they are the peak. Naw man. You're just a blip at the bottom of the logarithmic chart.
AI future is clearly local, and my recent pitch has been "infinite tokens." Because that's what my M1 MBP can do; and that's what my RTX3090 can do. I don't need to pay hundreds of dollars a month and no one else does either.
AI future is clearly local, and my recent pitch has been "infinite tokens." Because that's what my M1 MBP can do; and that's what my RTX3090 can do. I don't need to pay hundreds of dollars a month and no one else does either.
Infinite tokens rate-limited to 10 tok/s is 26MTok per month.
Is there something like this on Linux? For example, if I’m an application developer can I assume GNU Core AI (or whatever it is or would be called) will be there if the kernel is >= some particular version?