Apple's On-Device and Server Foundation Models
machinelearning.apple.com
Apple's On-Device and Server Foundation Models
1–10 of 562 posts
Re: Apple's On-Device and Server Foundation Models
#2It would be cool to understand when the system will use one or the other (the ~3 billion on-device model or the bigger one on Apple servers).
Re: Apple's On-Device and Server Foundation Models
#3> We train our foundation models on licensed data, including data selected to enhance specific features, as well as publicly available data collected by our web-crawler, AppleBot. Web publishers have the option to opt out of the use of their web content for Apple Intelligence training with a data usage control.
And, of course, nobody has known to opt-out by blocking AppleBot-Extended until after the announcement where they've already pirated shittons of data.
In completely unrelated news, I just trained a new OS development AI on every OS Apple has ever written. Don't worry. There's an opt-out, Apple just needed to know to put these magic words in their installer image years ago. I'm sure Apple legal will be OK with this.
Re: Apple's On-Device and Server Foundation Models
#4It would be cool to understand when the system will use one or the other (the ~3 billion on-device model or the bigger one on Apple servers).
Conceivably, they don't have precise answers for that yet, and won't until after they see what real-world usage looks like.
They built out a system that's ready to scale to deliver features that may not work on available hardware, but they're also incentivized to minimize actual reliance on that cloud stuff as it incurs per-use costs that local runs don't.
Re: Apple's On-Device and Server Foundation Models
#5> We train our foundation models on licensed data, including data selected to enhance specific features, as well as publicly available data collected by our web-crawler, AppleBot. Web publishers have the option to opt out of the use of their web content for Apple Intelligence training with a data usage control. And, of course, nobody has known to opt-out by blocking AppleBot-Extended until after the announcement wher…
So built on stolen data essentially.
Re: Apple's On-Device and Server Foundation Models
#6Will these smaller on device models lead to a crash in GPU prices?
Re: Apple's On-Device and Server Foundation Models
#7> We train our foundation models on licensed data, including data selected to enhance specific features, as well as publicly available data collected by our web-crawler, AppleBot. Web publishers have the option to opt out of the use of their web content for Apple Intelligence training with a data usage control. And, of course, nobody has known to opt-out by blocking AppleBot-Extended until after the announcement wher…
> just trained a new OS development AI on every OS Apple has ever written.
…is there publicly visible source code for every OS Apple has ever written?
Re: Apple's On-Device and Server Foundation Models
#8> Our foundation models are fine-tuned for users’ everyday activities, and can dynamically specialize themselves on-the-fly for the task at hand. We utilize adapters, small neural network modules that can be plugged into various layers of the pre-trained model, to fine-tune our models for specific tasks. For our models we adapt the attention matrices, the attention projection matrix, and the fully connected layers in the point-wise feedforward networks for a suitable set of the decoding layers of the transformer architecture.
>We represent the values of the adapter parameters using 16 bits, and for the ~3 billion parameter on-device model, the parameters for a rank 16 adapter typically require 10s of megabytes. The adapter models can be dynamically loaded, temporarily cached in memory, and swapped — giving our foundation model the ability to specialize itself on the fly for the task at hand while efficiently managing memory and guaranteeing the operating system's responsiveness.
This kind of sounds like Loras......
Re: Apple's On-Device and Server Foundation Models
#9Will these smaller on device models lead to a crash in GPU prices?
X to doubt.
Re: Apple's On-Device and Server Foundation Models
#10> We train our foundation models on licensed data, including data selected to enhance specific features, as well as publicly available data collected by our web-crawler, AppleBot. Web publishers have the option to opt out of the use of their web content for Apple Intelligence training with a data usage control. And, of course, nobody has known to opt-out by blocking AppleBot-Extended until after the announcement wher…
Apple just did more to make this a privacy focused feature versus just a data mine than literally anyone else to date and still people complain.
Public content on the internet is public content on the internet - I thought we had all agreed years ago that if you didn’t want your content copied, don’t make it freely available and unlicensed on the internet.