Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple Core AI Framework

developer.apple.com

Post reply on HN
Print view View original

41–50 of 114 posts

Re: Apple Core AI Framework

#41

Post by horsawlarway »

hadlock wrote: post #15
an0malous wrote: post #9

This is why the AI companies are rushing to IPO. By the end of next year you’ll be running most of your AI on device. They have no moat, they’ve reached the limits of scaling, most of the magic can be distilled into smaller models, and they know it

Qwen's ~30B-class models are genuinely good enough for use if you can find a machine with enough memory bandwidth to run them at 30-90 tokens/second. It's been extremely telling that Qwen stopped releasing 120b class models. At some point in the next 10 years (maybe 3?) someone is going to release an Opus 4.5 class 256B model you can run locally. Right now our engineers use about $800/mo worth of opus tokens; at that…

I want to echo this.

I've been on claude's opus 4.5/6/7 for work for a couple months, and I finally got back to running Qwen A3B 35B... it's incredibly performant and quite capable on semi-reasonable local hardware.

I get ~150 tokens/s on dual nvidia RTX 3090s and can fit the whole 300k context into gpu on a UD-Q4-K-XL quant gguf.

Combined with Pi as a harness, and I'm surprised to find that it feels about as capable as claude did 8 months ago (their 3.x models).

It's not Opus 4.5 levels yet, but it's good enough for a LOT of basic work. I actually downgraded my personal anthropic subscription because Qwen is absolutely fine for implementation work. I still let a better model write a plan, but then I can just switch over to Qwen to implement.

I don't think we're 10 years away from opus 4.5 levels running on cheap consumer hardware. I think we're probably closer to 18 months away, and I suspect it'll be in the 30-60b range, not the 256b range.

PC manufacturers also seem to be betting on local, with a LOT of focus on 64 to 128gb unified RAM machines.

Re: Apple Core AI Framework

#42

Post by jacobr1 »

connectsnk wrote: post #35

Do we know what is the underlying model? Is it a custome model developed by Apple or one of gemma/deepseeks under the hood

The new siri models will be some variant of the gemini models. This framework seems to be more generalized than that though.

Re: Apple Core AI Framework

#43

Post by maxdo »

an0malous wrote: post #9

This is why the AI companies are rushing to IPO. By the end of next year you’ll be running most of your AI on device. They have no moat, they’ve reached the limits of scaling, most of the magic can be distilled into smaller models, and they know it

Why on earth I should switch from a top tier model to much worse local model ? Why do I need to suffer my battery ?

Re: Apple Core AI Framework

#44

Post by karim79 »

strictnein wrote: post #36
fwip wrote: post #29

Earlier quoted context omitted.

Maybe not per watt, but unless you already happen to own a 3900 cited by that post, you'd have to buy that as well, which is currently selling for around $1400 used.

3090s are running $1400 now? Wowsers. I thought I was overspending when I bought 6x of them for around $800 a pop. Might be time to sell, to be honest. It's fun to have that at home, but I can't justify having $10k (with memory, mobo, cpu, etc) sitting in my basement without being fully utilized.

I'll take two of them. A thousand a piece.

Re: Apple Core AI Framework

#45

Post by maxdo »

horsawlarway wrote: post #41
hadlock wrote: post #15

Earlier quoted context omitted.

Qwen's ~30B-class models are genuinely good enough for use if you can find a machine with enough memory bandwidth to run them at 30-90 tokens/second. It's been extremely telling that Qwen stopped releasing 120b class models. At some point in the next 10 years (maybe 3?) someone is going to release an Opus 4.5 class 256B model you can run locally. Right now our engineers use about $800/mo worth of opus tokens; at that…

I want to echo this. I've been on claude's opus 4.5/6/7 for work for a couple months, and I finally got back to running Qwen A3B 35B... it's incredibly performant and quite capable on semi-reasonable local hardware. I get ~150 tokens/s on dual nvidia RTX 3090s and can fit the whole 300k context into gpu on a UD-Q4-K-XL quant gguf. Combined with Pi as a harness, and I'm surprised to find that it feels about as capable…

Majority of my agentic setup is pi / Claude code where every single Chinese models are not as good except commercial 1T models .

Local is a pipe dream . If you can run it cheap occasionally why commercial companies can’t run it cheaper 24/7 and lower the costs ? The answer is simple. Use cases are more demanding and hence you need more from model not less .

Sure if you task is to do a narrow labeling task on 1m records small optimized model is good . If you want to do complex things , it shifts with models advancements

Re: Apple Core AI Framework

#46

Post by adgjlsfhk1 »

sealeck wrote: post #11
an0malous wrote: post #9

This is why the AI companies are rushing to IPO. By the end of next year you’ll be running most of your AI on device. They have no moat, they’ve reached the limits of scaling, most of the magic can be distilled into smaller models, and they know it

Have we reached the limits of scaling? Sadly it appears that larger model still equals better model

yes and no. We've reached the point where larger models are higher quality, but they're also too expensive and slow to be used broadly. The giant models, however are still useful for training smaller models that are actually deployable.

Re: Apple Core AI Framework

#47

Post by hparadiz »

maxdo wrote: post #45
horsawlarway wrote: post #41

Earlier quoted context omitted.

I want to echo this. I've been on claude's opus 4.5/6/7 for work for a couple months, and I finally got back to running Qwen A3B 35B... it's incredibly performant and quite capable on semi-reasonable local hardware. I get ~150 tokens/s on dual nvidia RTX 3090s and can fit the whole 300k context into gpu on a UD-Q4-K-XL quant gguf. Combined with Pi as a harness, and I'm surprised to find that it feels about as capable…

Majority of my agentic setup is pi / Claude code where every single Chinese models are not as good except commercial 1T models . Local is a pipe dream . If you can run it cheap occasionally why commercial companies can’t run it cheaper 24/7 and lower the costs ? The answer is simple. Use cases are more demanding and hence you need more from model not less . Sure if you task is to do a narrow labeling task on 1m recor…

This sounds like something someone at IBM in 1986 would say trying to sell their mainframes. "PCs will never be a thing. No one's gonna want a computer."

I'm seeing some impressive results from folks that can afford 10k+ GPUs right now. But those GPUs will all be hand me downs in 10 years. So pipe dream? Hmmm...... that's not how this industry works.

Re: Apple Core AI Framework

#48

Post by ankit219 »

they are also working on activations (w4a8, w4a16 from what i know). if they deliver (and a big if), it means that given their market reach, they can dictate the way sub 100b parameter models are trained and served to a large extent, given their major usecase would be on device (macos and not ios for most of them).

Re: Apple Core AI Framework

#49

Post by halJordan »

criddell wrote: post #18

Is there something like this on Linux? For example, if I’m an application developer can I assume GNU Core AI (or whatever it is or would be called) will be there if the kernel is >= some particular version?

No there isn't. RedHat and IBM do though, for their distros

Re: Apple Core AI Framework

#50

Post by mswphd »

strictnein wrote: post #34
hadlock wrote: post #15

Earlier quoted context omitted.

Qwen's ~30B-class models are genuinely good enough for use if you can find a machine with enough memory bandwidth to run them at 30-90 tokens/second. It's been extremely telling that Qwen stopped releasing 120b class models. At some point in the next 10 years (maybe 3?) someone is going to release an Opus 4.5 class 256B model you can run locally. Right now our engineers use about $800/mo worth of opus tokens; at that…

Didn't Qwen stop releasing their more powerful models because they're commercializing them?

Yes and no.

Qwen 3.5 was released 3/2/2026. It includes models up to a 397B-A17B model

https://huggingface.co/collections/Qwen/qwen35

A day afterwards, a high-up technical leader working on Qwen was let go

https://techcrunch.com/2026/03/03/alibabas-qwen-tech-lead-st...

The more recent Qwen 3.6 was released on 4/16

https://huggingface.co/collections/Qwen/qwen36

This does not include any particularly large models. But the models it contains (Qwen3.6 27B and Qwen3.6 35B-A3B) are the local models people have been very excited about lately. So they didn't release any larger models, and the models people praise so much are from this most recent release.

Post reply on HN