Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple Foundation Models

platform.claude.com

Post reply on HN
Print view View original

111–120 of 244 posts

Re: Apple Foundation Models

#111

Post by WarmWash »

embedding-shape wrote: post #82
alecco wrote: post #79

Earlier quoted context omitted.

In spite of their deeper pockets, massive datacenters, colosal amounts of user data, and hundreds of thousands of top developers, even Amazon, Meta, Microsoft, and Google are well behind. I think Evans is completely wrong. There are only 2 truly frontier models. (at least for now). And Anthropic seems to be leaving OpenAI behind so there might be only 1 in the near future. (which is scary/dangerous)

> I think Evans is completely wrong. There are only 2 truly frontier models. (at least for now). And Anthropic seems to be leaving OpenAI behind so there might be only 1 in the near future. (which is scary/dangerous) Truly fascinating ecosystem and community in general, as experiences differ so wildly. Anthropic's models seems far behind OpenAI to me, especially when you get into "Pro" territory, and there doesn't se…

People use a model as their daily driver, get very familiar with it and it's behavior, and then go and use another model and have a hard time. It's very difficult to separate "the model is bad" from "the model works differently".

Re: Apple Foundation Models

#112

Post by halJordan »

harrouet wrote: post #49

This is Apple commoditizing LLMs while keeping control of the UX. They are a hardware company and will keep selling the best machine for AI use. Well done.

It's been clear for years now that eventually ai will be embedded at the os level. Apple even recognized it way back when they first introduced Apple Intelligence. Yes they're commoditizing llms or whatever. But this has been a user facing feature they've been iterating on for years now

Re: Apple Foundation Models

#113

Post by ABS »

mlpicker wrote: post #17

What I'm curious about is whether this is actually on-device. Apple's framework caps local models around 3B params last I looked, and Claude is way bigger than that. So either there's some hybrid setup I haven't seen documented, or this is mostly a Claude SDK in FM clothing. Anyone tried it on a plane?

it's cloud, the doc is explicit that requests go straight to api.anthropic.com with Apple not in the way.

so Claude via FM dies offline while Apple's on-device SystemLanguageModel (the ~3B one) keeps working. It isn't a hybrid really: the framework just has both implement the same LanguageModelSession protocol so "local 3B" and "remote frontier model" become a one-argument swap.

IMHO what's worth internalising is that the two share an API but nothing else: the on-device path runs on Apple's Neural Engine and costs battery (you can watch ANE power ramp while it works) while the cloud path costs API credits/tokens and does zero local compute. Same code, opposite cost model.

Re: Apple Foundation Models

#114

Post by halJordan »

criddell wrote: post #80
mcintyre1994 wrote: post #60

I think this is just Apple planning for their on-device models getting better, which makes sense given they have access to Gemini now. If developers use this for all their code calling an external LLM, then as Apple's model becomes more capable and covers more use cases it'll be easy to switch to it at individual call sites. That'll give apps better UX and save developers money on a bill that Apple doesn't get a cut…

How does using Gemini lead to better on-device models?

Apple is distilling models from gemini

Re: Apple Foundation Models

#115

Post by ABS »

swingboy wrote: post #77
harrouet wrote: post #49

This is Apple commoditizing LLMs while keeping control of the UX. They are a hardware company and will keep selling the best machine for AI use. Well done.

Does “the best machine for AI use” apply here considering these models are still server-side?

for the on-device model, yes it runs on the Neural Engine (at the moment) so a newer chip means faster, cheaper local inference. For the server side path this Claude package is about your machine is irrelevant since it's a network call. The same API covers both, so "best machine for AI" only bites when the session is actually local.

But we can imagine that the balance of what's on-device vs what's remote will move continuously towards the former as time, improved HW and improved local models keep progressing

Re: Apple Foundation Models

#116

Post by HDThoreaun »

tarcon wrote: post #15
daniel_iversen wrote: post #4

Is this Apple encouraging developers to go through their api abstraction layer to use LLMs so that when they launch their own (which I think we’ve heard they’ve been spending lots of money on training and might be somehow involved with Siri or current Apple AI?) that they can easily help devs make a seamless transition? Or is it just a developer nicety or something else?

Apple has some clever mechanics to protect user data. I had to work with App tracking stuff lately and their approach to keeping user details private with anonymized cohorts (SKAN, Differential Privacy) before reporting tracking events to third party platforms was surprisingly well thought out. There is value in having them in your loop if you care about privacy.

My read of the ATT stuff is basically that it forced all the apps to use meta ad tracking because they’re the only ones who figured out how to serve relevant ads despite it.

Re: Apple Foundation Models

#120

Post by post-it »

> a Swift package that makes Claude available as a server-side language model in Apple's Foundation Models framework

Ahh I was hoping for the opposite: all of the existing features of Claude Code but somehow running locally on my laptop's neural engine. A pipe dream on an M2 with 8 GB of RAM, but I had a flicker of hope there.

Post reply on HN