Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple's On-Device and Server Foundation Models

machinelearning.apple.com

Post reply on HN
Print view View original

181–190 of 562 posts

Re: Apple's On-Device and Server Foundation Models

#181

Post by rishabhjain1198 »

marcellus23 wrote: post #176
rishabhjain1198 wrote: post #166

For people interested in AI research, there's nothing new here. IMO they should do a better job of referencing existing papers and techniques. The way they wrote about "adaptors" can make it seem like it's something novel, but it's actually just re-iterating vanilla LoRA. It was enough to convince one of the top-voted HackerNews comments that this was a "huge development". Benchmarks are nice though.

They refer to LoRA explicitly in the post.

Although I caught that on the first read, I found myself questioning when I read the adaptors part, "is this not just LoRA...".

Maybe it's my fault as a reader, but I think the writing could be clearer. Usually in a research paper you would link to the LoRA paper there too.

Re: Apple's On-Device and Server Foundation Models

#182

Post by gnicholas »

shreezus wrote: post #126

This is great, however Apple needs to be explicit on what it, and what isn't relayed to third party services, and provide the ability to opt-out if desired. It's one thing to run inference on-device, and another to send your data through OpenAI's APIs. The partnership details are not entirely clear to me as a user.

My understanding is that nothing is shared with any non-Apple company except if you specifically authorize it on a per-use basis. Otherwise it just runs locally or in the Apple AI cloud, and is not retained. All of this is subject to verification of Apple’s claims, of course.

Re: Apple's On-Device and Server Foundation Models

#183

Post by gigglesupstairs »

rishabhjain1198 wrote: post #166

For people interested in AI research, there's nothing new here. IMO they should do a better job of referencing existing papers and techniques. The way they wrote about "adaptors" can make it seem like it's something novel, but it's actually just re-iterating vanilla LoRA. It was enough to convince one of the top-voted HackerNews comments that this was a "huge development". Benchmarks are nice though.

Was there anything about searching through our own photos using prompts? I thought this could be pretty amazing and still a natural way to find very specific photos in one’s own photo gallery.

Re: Apple's On-Device and Server Foundation Models

#184

Post by makeitdouble »

zer0zzz wrote: post #168
talldayo wrote: post #147

Earlier quoted context omitted.

The Steam Deck ships with 16 gigs of quad-channel LPDDR5 and it costs $400. Apple knows exaaaactly what they're doing with this sort of pricing. Can't forget about that cozy 256gb SSD either. An AI computer will need more than that, right?

Is steamdeck sold at cost? From what I know Apple has a rule that everything must be sold at 40% margins. That is prob the main reason.

> From what I know Apple has a rule that everything must be sold at 40% margins.

As for all rules, it's a rule except when it's not. On the top of my head Apple TV [0] had a 20% predicted margin presumably because they wanted to actually sell them.

Otherwise 40% margin is usually calculated against the BOM, which doesn't mean 40% of actual profit when the product is sold.

In that respect we have no idea of the actual margin on a macbook air for instance, it could be 10% when including their operating costs and marketing, or it could 60% if they negociated prices way below the estimated BOM for instance.

It's just to say: Apple sells at 8Gb because they want to, at the end of the day nothing is stopping them to play with their margin or the product price.

[0] https://www.reuters.com/article/idUSN06424767/

Re: Apple's On-Device and Server Foundation Models

#185

Post by lolinder »

rishabhjain1198 wrote: post #166

For people interested in AI research, there's nothing new here. IMO they should do a better job of referencing existing papers and techniques. The way they wrote about "adaptors" can make it seem like it's something novel, but it's actually just re-iterating vanilla LoRA. It was enough to convince one of the top-voted HackerNews comments that this was a "huge development". Benchmarks are nice though.

> For people interested in AI research, there's nothing new here.

Was anyone expecting anything new?

Apple has never been big on living at the cutting edge of technology exploring spaces that no one has explored before—from laptops to the iPhone to iPads to watches, every success they've had has come from taking tech that was already prototyped by many other companies and smoothing out the usability kinks to get it ready for the mainstream. Why would deep learning be different?

Re: Apple's On-Device and Server Foundation Models

#186

Post by data-ottawa »

c1sc0 wrote: post #117
meatmanek wrote: post #78

Earlier quoted context omitted.

> I thought we had all agreed years ago that if you didn’t want your content copied, don’t make it freely available and unlicensed on the internet. Until LLMs came along, most large-scale internet scraping was for search engines. Websites benefited from this arrangement because search engines directed users to those websites. LLMs abused this arrangement to scrape content into a local database, compress that into a l…

In my view it’s ethical even if it’s just for taking revenge on the ad-driven model that has caused the enshittification of the web.

I think you mean it’s justified, not ethical.

Re: Apple's On-Device and Server Foundation Models

#187

Post by avereveard »

gigglesupstairs wrote: post #183
rishabhjain1198 wrote: post #166

For people interested in AI research, there's nothing new here. IMO they should do a better job of referencing existing papers and techniques. The way they wrote about "adaptors" can make it seem like it's something novel, but it's actually just re-iterating vanilla LoRA. It was enough to convince one of the top-voted HackerNews comments that this was a "huge development". Benchmarks are nice though.

Was there anything about searching through our own photos using prompts? I thought this could be pretty amazing and still a natural way to find very specific photos in one’s own photo gallery.

Which is in turn just multimodal embedding

Besides I could do "named person on a beach in August" and get the correct thing in photos on Android photos, so I don't get it.

It's amazing for apple users if they didn't have it before. But from a tech stand point people could have had it for a while.

Re: Apple's On-Device and Server Foundation Models

#188

Post by wpm »

l33t7332273 wrote: post #162
woadwarrior01 wrote: post #144

Earlier quoted context omitted.

None of the current iOS and macOS LLM Apps use the Neural Engine. They use the CPU and the GPU. nb: I'm the author of a fairly popular app in that category.

Why do they not?

AFAIK there is no general purpose, "do this on the ANE" API. You have to be using specific higher level APIs like CoreML or VisionKit in order for it to end up on the ANE.

Re: Apple's On-Device and Server Foundation Models

#189

Post by tsunamifury »

frizlab wrote: post #137
tsunamifury wrote: post #129

Earlier quoted context omitted.

[flagged]

They told explicitly there are three things. On device AI for queries that can be done on device, private cloud compute for those that can’t and opt in ChatGPT(-4o) support for more general queries. Cloud compute queries only use the data for answering the queries and are run on an OS where storage is not available along other privacy measures. The builds of the OS will be public and auditable by security researchers…

[flagged]

Re: Apple's On-Device and Server Foundation Models

#190

Post by jjtheblunt »

woadwarrior01 wrote: post #144
brcmthrowaway wrote: post #25

Earlier quoted context omitted.

So iOS LLM Apps dont use the neural engine? Lol

None of the current iOS and macOS LLM Apps use the Neural Engine. They use the CPU and the GPU. nb: I'm the author of a fairly popular app in that category.

How would you know none of the apple apps use the neural engine? Is the key in the statement “LLM”?
Post reply on HN