Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple's On-Device and Server Foundation Models

machinelearning.apple.com

Post reply on HN
Print view View original

161–170 of 562 posts

Re: Apple's On-Device and Server Foundation Models

#161

Post by ww520 »

Isuckatcode wrote: post #38

>By fine-tuning only the adapter layers, the original parameters of the base pre-trained model remain unchanged, preserving the general knowledge of the model while tailoring the adapter layers to support specific tasks. From a ML noob (me) understanding of this, does this mean that the final matrix is regularly fine tuned instead of fine tuning the main model ? Is this similar to how chatGPT now remembers memory[1]…

The base model is frozen. The smaller adaptor matrices which are finetuned with new data. During inference, the weights from the adaptor matrices "shadow" the weights in the base model. Since the adaptor matrices are much smaller, it's quite efficient to finetune them.

The advantage of the adaptor matrices is you can have different sets of adaptor matrices for different tasks, all based of the base model.

Re: Apple's On-Device and Server Foundation Models

#162

Post by l33t7332273 »

woadwarrior01 wrote: post #144
brcmthrowaway wrote: post #25

Earlier quoted context omitted.

So iOS LLM Apps dont use the neural engine? Lol

None of the current iOS and macOS LLM Apps use the Neural Engine. They use the CPU and the GPU. nb: I'm the author of a fairly popular app in that category.

Why do they not?

Re: Apple's On-Device and Server Foundation Models

#163

Post by anvuong »

ra7 wrote: post #34

> Our foundation models are trained on Apple's AXLearn framework, an open-source project we released in 2023. It builds on top of JAX and XLA, and allows us to train the models with high efficiency and scalability on various training hardware and cloud platforms, including TPUs and both cloud and on-premise GPUs. Interesting that they’re using TPUs for training, in addition to GPUs. Is it both a technical decision (J…

Jax was built with TPUs in mind, so it's not surprising that they use TPUs

Re: Apple's On-Device and Server Foundation Models

#164

Post by dghlsakjg »

tonynator wrote: post #155
cube2222 wrote: post #30

Halfway down the article contains some great charts with comparisons to other relevant models, like Mistral-7B for the on-device models, and both gpt-3.5 and 4 for the server-side models. They include data about the ratio of which outputs human graders preferred (for server side it’s better than 3.5, worse than 4). BUT, the interesting chart to me is „Human Evaluation of Output Harmfulness” which is much, much ”bette…

So it's not going to be better than other models, but it will be more censored. I guess that might be a selling point for their customer base?

iPhone share is ~59% of smartphones in the US.

Their customer base is effectively all demographics.

Re: Apple's On-Device and Server Foundation Models

#165

Post by anvuong »

axoltl wrote: post #116
TheRoque wrote: post #33

Why isn't there a comparison with the Llama3 8b in the "benchmarks" ?

The Llama 3 license says: "If, on the Meta Llama 3 version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee’s affiliates, is greater than 700 million monthly active users in the preceding calendar month, you must request a license from Meta, which Meta may grant to you in its sole discretion, and you are not authorized to exercise any of the rights unde…

They can just run the same tests and cite the results from other websites. That has nothing to do with Meta. No companies can force you to not talk about them.

Re: Apple's On-Device and Server Foundation Models

#166

Post by rishabhjain1198 »

For people interested in AI research, there's nothing new here.

IMO they should do a better job of referencing existing papers and techniques. The way they wrote about "adaptors" can make it seem like it's something novel, but it's actually just re-iterating vanilla LoRA. It was enough to convince one of the top-voted HackerNews comments that this was a "huge development".

Benchmarks are nice though.

Re: Apple's On-Device and Server Foundation Models

#167

Post by worstspotgain »

tmpz22 wrote: post #149
vzaliva wrote: post #67

I love that they use machinelearning.apple.com not ai.apple.com

For the majority of the keynote they explicitly avoided the word AI instead substituting the word Intelligence, then Apple Intelligence, and then towards the end they said AI and ChatGPT once or twice. I think they saw the response to all the AI shoveling and Microsoft Recall and executed a fantastic strategy to reposition themselves in industry discussions. I still have tons of reservations about privacy and what th…

> executed a fantastic strategy to reposition themselves in industry discussions

Just the usual marketing angle, IMO. It's not TV, it's HBO.

No one is reluctant to use the word smartphone to include iPhones. I don't think anyone is going to use the Apple Intelligence moniker except in the same cases where they'd say iCloud instead of cloud services.

It's also a little clunky. Maybe they could have gone with... xI? Too close to the Chinese Xi. iAI? Sounds like the Spanish "ay ay ay." Not an easy one I think. The number of person-hours spent on this must have been something.

Re: Apple's On-Device and Server Foundation Models

#168

Post by zer0zzz »

talldayo wrote: post #147
ksec wrote: post #124

I hope, this could mean Apple will push the baseline of ALL Macs to have higher than 8GB of Memory. While I wish we all get 16GB M4 as baseline. Apple being Apple may only give us 12GB, and charges extra $100 for the 16GB option. It will still be a lot better than 8GB though.

The Steam Deck ships with 16 gigs of quad-channel LPDDR5 and it costs $400. Apple knows exaaaactly what they're doing with this sort of pricing. Can't forget about that cozy 256gb SSD either. An AI computer will need more than that, right?

Is steamdeck sold at cost? From what I know Apple has a rule that everything must be sold at 40% margins. That is prob the main reason.

Re: Apple's On-Device and Server Foundation Models

#169

Post by WiSaGaN »

rishabhjain1198 wrote: post #166

For people interested in AI research, there's nothing new here. IMO they should do a better job of referencing existing papers and techniques. The way they wrote about "adaptors" can make it seem like it's something novel, but it's actually just re-iterating vanilla LoRA. It was enough to convince one of the top-voted HackerNews comments that this was a "huge development". Benchmarks are nice though.

This gives me the vibe of calling high resolution screens as "retina" screens.

Re: Apple's On-Device and Server Foundation Models

#170

Post by tmpz22 »

worstspotgain wrote: post #167
tmpz22 wrote: post #149

Earlier quoted context omitted.

For the majority of the keynote they explicitly avoided the word AI instead substituting the word Intelligence, then Apple Intelligence, and then towards the end they said AI and ChatGPT once or twice. I think they saw the response to all the AI shoveling and Microsoft Recall and executed a fantastic strategy to reposition themselves in industry discussions. I still have tons of reservations about privacy and what th…

> executed a fantastic strategy to reposition themselves in industry discussions Just the usual marketing angle, IMO. It's not TV, it's HBO. No one is reluctant to use the word smartphone to include iPhones. I don't think anyone is going to use the Apple Intelligence moniker except in the same cases where they'd say iCloud instead of cloud services. It's also a little clunky. Maybe they could have gone with... xI? To…

I don't think they actually expect "Apple Intelligence" to enter popular vernacular. I think it was more to drive home the distinction between what Apple is doing and what everybody else is doing.
Post reply on HN