Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple's On-Device and Server Foundation Models

machinelearning.apple.com

Post reply on HN
Print view View original

321–330 of 562 posts

Re: Apple's On-Device and Server Foundation Models

#321

Post by skohan »

throwaway4good wrote: post #290
Hugsun wrote: post #285

Earlier quoted context omitted.

You get considerably more ML FLOPS per dollar in a 4090 than any mac. It seems like the base M2 MAX is at roughly the same price point. It does grant you more RAM. Quadro and Tesla cards might be a different story. I would still like to see concrete FLOPS/$ numbers.

The M2 is a chip designed to be in a laptop (and it is quite powerful given its low power consumption). Presumedly they have a different chip or at least completely different configuration (RAM, network, etc.) in their data centers.

There was a rumor floating around that Apple might try to enter the server chip business with an AI chip, which is an interesting concept. Apple's never really succeeded in the B2B business, but they have proven a lot of competency in the silicon space.

Even their high-end prosumer hardware could be interesting as an AI workstation given the VRAM available if the software support were better.

Re: Apple's On-Device and Server Foundation Models

#322

Post by skohan »

mrweasel wrote: post #296
throwaway4good wrote: post #290

Earlier quoted context omitted.

The M2 is a chip designed to be in a laptop (and it is quite powerful given its low power consumption). Presumedly they have a different chip or at least completely different configuration (RAM, network, etc.) in their data centers.

The interesting point here is that developers targeting the Mac can safely assume that the users will have a processor capable of significant AI/ML workloads. On the Windows (and Linux) side of things, there's no common platform, no assumption that the users will have an NPU or GPU capable of doing what you want. I think that's also why Microsoft was initially going for the ARM laptops, where they'd be sure that the…

I believe MS is trying to standardize this, in the same way as they do with DirectX support levels, but I agree it's probably going to be inherently a bit less consistent than Apple offerings

Re: Apple's On-Device and Server Foundation Models

#323

Post by fauigerzigerk »

theshrike79 wrote: post #243
avereveard wrote: post #187

Earlier quoted context omitted.

Which is in turn just multimodal embedding Besides I could do "named person on a beach in August" and get the correct thing in photos on Android photos, so I don't get it. It's amazing for apple users if they didn't have it before. But from a tech stand point people could have had it for a while.

The difference is that Apple has been doing this on-device for maybe 4-5 years already with the Neural Engine. Every iOS version has brought more stuff you can search for. The current addition is "just" about adding a natural language interface on top of data they already have about your photos (on device, not in the cloud). My iPhone 14 can, for example, detect the breed of my dog correctly from the pictures and it…

They have been trying and failing to do a tiny little bit of this. It's so broken and useless that I've been uploading all my iCloud photos to Google as well, for search and sharing.

Re: Apple's On-Device and Server Foundation Models

#324

Post by sunshinerag »

lolinder wrote: post #252
IOT_Apprentice wrote: post #246

Earlier quoted context omitted.

Apple was first with 64 bit iPhone chips. Remember Qualcomm VP at the time claimed it was nothing. Apple Silicon for M1 was impressive for instant in low power high performance.

Those are both still (major) incremental improvements to known tech, not cutting-edge research. Apple takes what other companies have already done and does it better.

all cutting-edge research other companies are supposedly doing are also incremental. Depends on your vantage point.

Re: Apple's On-Device and Server Foundation Models

#325

Post by theshrike79 »

fauigerzigerk wrote: post #323
theshrike79 wrote: post #243

Earlier quoted context omitted.

The difference is that Apple has been doing this on-device for maybe 4-5 years already with the Neural Engine. Every iOS version has brought more stuff you can search for. The current addition is "just" about adding a natural language interface on top of data they already have about your photos (on device, not in the cloud). My iPhone 14 can, for example, detect the breed of my dog correctly from the pictures and it…

They have been trying and failing to do a tiny little bit of this. It's so broken and useless that I've been uploading all my iCloud photos to Google as well, for search and sharing.

If you like Google using your personal photos for machine learning, that's your option. Now they have your every photo, geotagged and timestamped so they can see where you have been and at what times. Then they of course anonymise that information into an "advertiser id" they tag on to you and a sufficient quantity of other people so they can claim they're not directly targeting anyone.

I prefer Apple's privacy focused option myself.

Re: Apple's On-Device and Server Foundation Models

#326

Post by binkethy »

notJim wrote: post #140
kmeisthax wrote: post #3

> We train our foundation models on licensed data, including data selected to enhance specific features, as well as publicly available data collected by our web-crawler, AppleBot. Web publishers have the option to opt out of the use of their web content for Apple Intelligence training with a data usage control. And, of course, nobody has known to opt-out by blocking AppleBot-Extended until after the announcement wher…

I hate to tell you, but I've been training a neural network on the internet for over a decade now. Specifically the one between my ears. Unfortunately, it seems to be gradually going insane.

Computer systems are not humans and never will be.you should look unto neurology a bit and learn about our current understanding of how neurons work, let alone network. The tech term is a total non sequitur compared to real neurons.

Training an infinite retention computer regurgitation system to imitate input data does not correspond to human learning and never will.

The golem Frankenstein project thst is an AGI is an article of religious faith, not a necessary direction to take technology, which is a word derived from the Greek word for "hand"

Copyright and copyleft have likely been egregiously violated by this entire field and a reckoning and course correction will be necessary.

Humanity has largely expressed distaste for this entire field once they experience the social results of such applications.

The amount of sycophantic adulation in this thread is sickening.

My comment will likely be grayed out soon by insider downclicks.

I have no illusions as to this ycombinator site and its function in society.

Good day

Re: Apple's On-Device and Server Foundation Models

#327

Post by scosman »

lossolo wrote: post #146
scosman wrote: post #57

“We utilize adapters, small neural network modules that can be plugged into various layers of the pre-trained model, to fine-tune our models for specific tasks.” This is huuuuge. I don’t see announcement of 3rd party training support yet, but I imagine/hope it’s planned. One of the hard things about local+private ML is I don’t want every app I download to need GBs of weights, and don’t want a delay when I open a new…

It's LORA, most of the things you saw in Apple Intelligence on device presentation are basically different LORAs.

The article says it’s lora a bunch of times. That’s clear.

My comment above is about dev experience, memory swapping, tuning base models to each HW release, and app size.

Re: Apple's On-Device and Server Foundation Models

#328

Post by treprinum »

mrweasel wrote: post #296
throwaway4good wrote: post #290

Earlier quoted context omitted.

The M2 is a chip designed to be in a laptop (and it is quite powerful given its low power consumption). Presumedly they have a different chip or at least completely different configuration (RAM, network, etc.) in their data centers.

The interesting point here is that developers targeting the Mac can safely assume that the users will have a processor capable of significant AI/ML workloads. On the Windows (and Linux) side of things, there's no common platform, no assumption that the users will have an NPU or GPU capable of doing what you want. I think that's also why Microsoft was initially going for the ARM laptops, where they'd be sure that the…

How does it help me (with maxed out M3 Max) that Apple might have some chip in the future right now? I do DL on A6000 and 4090, not waiting until Apple produces a chip someday that is faster than 1650 in ML...

Re: Apple's On-Device and Server Foundation Models

#329

Post by janekm »

derefr wrote: post #217
wmf wrote: post #211

Earlier quoted context omitted.

For everything other than handwriting I don't think the LoRAs are fine-tuned locally.

Well, here's another one: they promised that your local (non-iCloud) photos don't leave the device. Yet they will now — among many other things they mentioned doing with your photos — allow you to generate "Memoji" that look like the people in your photos. Which includes the non-iCloud photos. I can't picture any way to use a RAG to do that. I can picture a way to do that that doesn't involve any model fine-tuning, b…

That's going to be something similar to IPAdapter FaceID: https://ipadapterfaceid.com Basically you use a facial structure representation that you'd use for face recognition (which of course Apple already compute on all your photos) together with some additional feature representations to guide the image generation. No need for additional fine-tuning. A similar approach could likely be used for handwriting generation.

Re: Apple's On-Device and Server Foundation Models

#330

Post by rvnx »

jahewson wrote: post #267
wkat4242 wrote: post #209

Earlier quoted context omitted.

Except Apple isn't really for the rest of us. Outside of America and a handful wealthy western countries it's for the top 5-20% earners only.

Approximately 33% of all smartphones in the world are iPhones.

60% in the US
Post reply on HN