Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple Core AI Framework

developer.apple.com

Post reply on HN
Print view View original

11–20 of 114 posts

Re: Apple Core AI Framework

#11

Post by sealeck »

an0malous wrote: post #9

This is why the AI companies are rushing to IPO. By the end of next year you’ll be running most of your AI on device. They have no moat, they’ve reached the limits of scaling, most of the magic can be distilled into smaller models, and they know it

Have we reached the limits of scaling? Sadly it appears that larger model still equals better model

Re: Apple Core AI Framework

#12

Post by stogot »

sealeck wrote: post #11
an0malous wrote: post #9

This is why the AI companies are rushing to IPO. By the end of next year you’ll be running most of your AI on device. They have no moat, they’ve reached the limits of scaling, most of the magic can be distilled into smaller models, and they know it

Have we reached the limits of scaling? Sadly it appears that larger model still equals better model

It’s still diminishing returns yes? It isn’t Moore’s Law

Re: Apple Core AI Framework

#13

Post by pixelready »

sealeck wrote: post #11
an0malous wrote: post #9

This is why the AI companies are rushing to IPO. By the end of next year you’ll be running most of your AI on device. They have no moat, they’ve reached the limits of scaling, most of the magic can be distilled into smaller models, and they know it

Have we reached the limits of scaling? Sadly it appears that larger model still equals better model

I think there’s still an open question around are the ultra-large next-gen models worth it? For those of us without early access to Mythos, it’s hard to verify whether it’s been held back from the public due to actually being “too dangerously powerful to release yet” as implied or because the gains aren’t outpacing the costs.

Re: Apple Core AI Framework

#14

Post by trollbridge »

franze wrote: post #7

i am more excited about the ondevice foundation model update that is coming https://developer.apple.com/documentation/updates/foundation... (not much info yet) but i maintain https://github.com/Arthur-Ficial/apfel so i might be biased

Thanks for building this! Something I grab on a regular basis, especially for doing simple education of folks about the basics of using LLMs by showing something that's not just a chatbot.

Re: Apple Core AI Framework

#15

Post by hadlock »

an0malous wrote: post #9

This is why the AI companies are rushing to IPO. By the end of next year you’ll be running most of your AI on device. They have no moat, they’ve reached the limits of scaling, most of the magic can be distilled into smaller models, and they know it

Qwen's ~30B-class models are genuinely good enough for use if you can find a machine with enough memory bandwidth to run them at 30-90 tokens/second. It's been extremely telling that Qwen stopped releasing 120b class models. At some point in the next 10 years (maybe 3?) someone is going to release an Opus 4.5 class 256B model you can run locally. Right now our engineers use about $800/mo worth of opus tokens; at that rate the ROI for local LLM is ~10 months

Re: Apple Core AI Framework

#16

Post by cat5e »

an0malous wrote: post #9

This is why the AI companies are rushing to IPO. By the end of next year you’ll be running most of your AI on device. They have no moat, they’ve reached the limits of scaling, most of the magic can be distilled into smaller models, and they know it

Huzzah, they’ve lost their stranglehold. Viva la revolution!

Re: Apple Core AI Framework

#17

Post by cat5e »

crancher wrote: post #8
franze wrote: post #7

i am more excited about the ondevice foundation model update that is coming https://developer.apple.com/documentation/updates/foundation... (not much info yet) but i maintain https://github.com/Arthur-Ficial/apfel so i might be biased

Apfel is very useful, thanks for the effort.

I second this, I’m more excited about dumb local models than something I could never run locally.

Re: Apple Core AI Framework

#18

Post by criddell »

Is there something like this on Linux? For example, if I’m an application developer can I assume GNU Core AI (or whatever it is or would be called) will be there if the kernel is >= some particular version?

Re: Apple Core AI Framework

#19

Post by mikestorrent »

sealeck wrote: post #11
an0malous wrote: post #9

This is why the AI companies are rushing to IPO. By the end of next year you’ll be running most of your AI on device. They have no moat, they’ve reached the limits of scaling, most of the magic can be distilled into smaller models, and they know it

Have we reached the limits of scaling? Sadly it appears that larger model still equals better model

Well, let's not forget that text models are not the only models! Video models are much slower and need comparatively more resources, and all they can do even at that size is generate videos a few seconds long. Clearly a ton more work is going to go into those, and demand for them will probably increase as more creative tools get authored using them as a central part of the workflow. Low-res local rendering for preview might be a thing, but the lion's share of the work for high-res, near-realtime rendering is going to be done on huge clusters for a long time yet.

Re: Apple Core AI Framework

#20

Post by mindwok »

sealeck wrote: post #11
an0malous wrote: post #9

This is why the AI companies are rushing to IPO. By the end of next year you’ll be running most of your AI on device. They have no moat, they’ve reached the limits of scaling, most of the magic can be distilled into smaller models, and they know it

Have we reached the limits of scaling? Sadly it appears that larger model still equals better model

I think GPT 4.5 showed that there is indeed a practical limit we're close too. That was supposedly a high-trillions of parameter model that was deprecated almost immediately because it was slow, insanely expensive, and had questionable benefits over the smaller models. Though apparently the new Mythos and whatever GPT Spud is (if it wasn't 5.5) are back up in the high trillions.
Post reply on HN