[deleted]
Apple's On-Device and Server Foundation Models
221–230 of 562 posts
Re: Apple's On-Device and Server Foundation Models
#222The WWDC show got on my nerves with the corpspeak, but this is pretty cool stuff. I’ve been trying to make smaller more efficient models in my own work. I hope Apple publish some actual papers.
Yeah it was close to “infomercial” levels of cheesy.
Re: Apple's On-Device and Server Foundation Models
#223Earlier quoted context omitted.
The Steam Deck ships with 16 gigs of quad-channel LPDDR5 and it costs $400. Apple knows exaaaactly what they're doing with this sort of pricing. Can't forget about that cozy 256gb SSD either. An AI computer will need more than that, right?
Is steamdeck sold at cost? From what I know Apple has a rule that everything must be sold at 40% margins. That is prob the main reason.
As a consumer I really cannot be made to care why it's the case. This artificial price tiering is stupid and everyone has been calling it a scam for years. Apple clearly knows they're in the wrong, but continues because they know nobody can stop them.
Re: Apple's On-Device and Server Foundation Models
#224I hope, this could mean Apple will push the baseline of ALL Macs to have higher than 8GB of Memory. While I wish we all get 16GB M4 as baseline. Apple being Apple may only give us 12GB, and charges extra $100 for the 16GB option. It will still be a lot better than 8GB though.
The Steam Deck ships with 16 gigs of quad-channel LPDDR5 and it costs $400. Apple knows exaaaactly what they're doing with this sort of pricing. Can't forget about that cozy 256gb SSD either. An AI computer will need more than that, right?
RAM is literally the cheapest primary component in a laptop at going rate of 1-4usd/GB. I'd say that shipping 8GB base model in 2024 is clearly manipulation by Apple, i.e. planned obsolescence or a way to moat Apple software. Anyone who doesn't see this is just being delusional.
Same way Apple and Samsung ship 128GB of storage when the production price between 128gb and 1tb is like 10$ (on a 1000$ device). Samsung even got rid of micro sd slot. It's so blatant it's actually depressing.
Re: Apple's On-Device and Server Foundation Models
#225Earlier quoted context omitted.
This gives me the vibe of calling high resolution screens as "retina" screens.
I don't see anything wrong with that at all. They've created a branding term that allows consumers to get an idea of the sort of pixel density they can expect without having to actually check, should they not want to bother.
Except that everyone has different visual acuity and different distance they use the same devices at, and in the end, "retina" means nothing at all.
But this is exactly the type of marketing Apple is good at, though "retina" is probably not the most successful example.
Re: Apple's On-Device and Server Foundation Models
#226> For on-device inference, we use low-bit palletization, a critical optimization technique that achieves the necessary memory, power, and performance requirements.
Did they go over the entire text with a thesaurus? I've never seen "palletization" be used as a viable synonym for "quantization" before, and I've read quite a few papers on LLM quantization
Re: Apple's On-Device and Server Foundation Models
#227Earlier quoted context omitted.
I hate to tell you, but I've been training a neural network on the internet for over a decade now. Specifically the one between my ears. Unfortunately, it seems to be gradually going insane.
If you're selling it to billions of people, and making big bank, I want a cut based on the parts you stole from me. If you're just using it personally, I'm cool with that.
Anyone who sells professional services based on knowledge they learned on the internet (which probably includes most people reading this) is doing that.
Re: Apple's On-Device and Server Foundation Models
#228Earlier quoted context omitted.
The Steam Deck ships with 16 gigs of quad-channel LPDDR5 and it costs $400. Apple knows exaaaactly what they're doing with this sort of pricing. Can't forget about that cozy 256gb SSD either. An AI computer will need more than that, right?
RAM is literally the cheapest primary component in a laptop at going rate of 1-4usd/GB. I'd say that shipping 8GB base model in 2024 is clearly manipulation by Apple, i.e. planned obsolescence or a way to moat Apple software. Anyone who doesn't see this is just being delusional. Same way Apple and Samsung ship 128GB of storage when the production price between 128gb and 1tb is like 10$ (on a 1000$ device). Samsung ev…
> RAM is literally the cheapest primary component
Is that still true for Apple's integrated memory? It might be - I just don't know.
Re: Apple's On-Device and Server Foundation Models
#229Earlier quoted context omitted.
AI noob here. Is every single model in iOS really just a thin adapter on top of one base model? Can everything they announced today really be built on top of one base LLM model with a specific type of architecture? What about image generation? What about text-to-speech? If they’re obviously different models, they can’t load them all at once into RAM. If they have to load from storage every time an app is opened, how…
The main LLM is only 1.5 GB so it should only take a half second to load. Or they could keep it loaded. The other models may be even smaller.
Maybe they use the "Siri is waking up and the screen wabbles" animation time for loading the model. That would be clever.
Re: Apple's On-Device and Server Foundation Models
#230For people interested in AI research, there's nothing new here. IMO they should do a better job of referencing existing papers and techniques. The way they wrote about "adaptors" can make it seem like it's something novel, but it's actually just re-iterating vanilla LoRA. It was enough to convince one of the top-voted HackerNews comments that this was a "huge development". Benchmarks are nice though.
reminds me of Easel on iMessage: https://easelapps.ai/