Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple's On-Device and Server Foundation Models

machinelearning.apple.com

Post reply on HN
Print view View original

491–500 of 562 posts

Re: Apple's On-Device and Server Foundation Models

#491

Post by gumby »

steve1977 wrote: post #428
gumby wrote: post #410

Earlier quoted context omitted.

> It wasn't Wi-Fi, it was AirPort. etc. etc. FWIW the term “airport” predated the name “wifi” — in those days you had to otherwise call it IEEE 802.11. And the name as great: people were buying them like crazy and hiding them in the drop ceiling to get around the corporate IT department. A nice echo of how analysts would buy their own apple II + visicalc to…get around corporate IT. I’m OK with Apple using “apple sili…

As far as I know, both the AirPort trademark and the term Wi-Fi got introduced in 1999 (could be that AirPort was a couple of weeks earlier)

Airport came out in the beginning of the year, maybe January, at Macworld. WECA renamed themselves the WiFi Alliance a few years later, but the name (and trademark) had to exist for it to be worth them doing so.

WECA reminds me of other "memorable" names of the era, my favorite being PCMCIA, though VESA is another fave.

Re: Apple's On-Device and Server Foundation Models

#492

Post by talldayo »

qwytw wrote: post #302
mrweasel wrote: post #296

Earlier quoted context omitted.

The interesting point here is that developers targeting the Mac can safely assume that the users will have a processor capable of significant AI/ML workloads. On the Windows (and Linux) side of things, there's no common platform, no assumption that the users will have an NPU or GPU capable of doing what you want. I think that's also why Microsoft was initially going for the ARM laptops, where they'd be sure that the…

> The interesting point here is that developers targeting the Mac can safely assume that the users will have a processor capable of significant AI/ML workloads Also that a significant proportion (majority?) of them will have just 8 GB of memory which is not exactly sufficient to run any complex AI/ML workloads.

Easy solution; just swap multiple gigabytes of your model to SSD-based ZRAM when you run out of memory. What could possibly go wrong?

Re: Apple's On-Device and Server Foundation Models

#493

Post by ribit »

eek2121 wrote: post #430
hajile wrote: post #399

Earlier quoted context omitted.

They don't need the entire mac. Their cost per Max chip is probably $200-300 which beats the 4090 by a massive margin and each chip can do more than a 4090 because it also has a CPU onboard. 4090 peaks out at around 550w which means they can run 5+ of their Max chips in the same power budget. A 4090 is $2000. Apple can probably get 5 chips on a custom motherboard for that cost. They'll use the same amount of power, b…

The GPU in the M-series is much slower than a 4090. 4060-4070ish performance at best, and it varies quite a bit.

You need to consider this in the context of the relevant task. Nvidia GPUs have extremely high peak performance for GEMM, but when working with LLMs, bandwidth (and RAM capacity) becomes the limiting factor. There is a reason why real ML-focused datacenter Nvidia GPUs use much wider RAM interfaces and a much higher price point. The M2 Ultra might not have the raw compute, but it has a lot of RAM and large caches.

Re: Apple's On-Device and Server Foundation Models

#494

Post by theshrike79 »

titaniumtown wrote: post #156
ndgold wrote: post #73

Absolutely awesome amount of content in these two pages. This was not expected. It is appreciated. I can’t wait to use the server model on a Mac to spin up my own cloud optimized for the Apple stack.

Did it mentioned being able to spin up the server model locally? I must've missed that part in the article.

They didn't but I'll bet it's coming in the next 5 years.

Most likely integrated with an Apple TV or a similar thing. Enough local LLM processing power to handle a family's data all in-house.

Re: Apple's On-Device and Server Foundation Models

#495

Post by AshamedCaptain »

KerrAvon wrote: post #474
caseyy wrote: post #467

Earlier quoted context omitted.

Yes, it's not perfect. About iTunes: I did not know that! Thank you. About iterative/innovative: I considered hardware and software that became household names or general knowledge to be significant innovations. It is not rigorous, I tried to include more rather than less. Still, on some years these companies mostly did version increases for their hardware and software, like new iOS and macOS versions, and that was i…

You're missing Apple Silicon, which has had a huge impact across the entire industry even if random soccer dad doesn't know about it -- if any one thing is responsible for Intel's marketshare collapsing in the future, the M series of processors is it.

> if any one thing is responsible for Intel's marketshare collapsing in the future, the M series of processors is it.

If anything, that would be AMD. But I'd guess they're both more worried about the entire desktop & laptop market shrinking way more than anything Apple does.

Re: Apple's On-Device and Server Foundation Models

#496

Post by viscanti »

prmoustache wrote: post #275
IOT_Apprentice wrote: post #246

Earlier quoted context omitted.

Apple was first with 64 bit iPhone chips. Remember Qualcomm VP at the time claimed it was nothing. Apple Silicon for M1 was impressive for instant in low power high performance.

But last at bringing a calculator on the iPad =)

No one has ever brought a native (not 3rd party) calculator to the iPad before. Apple is the first.

Re: Apple's On-Device and Server Foundation Models

#497

Post by s3p »

w10-1 wrote: post #478

I think we as tech people lost the forest for the trees. Apple (unwisely I think) is allowing UI's to just generate responses. The wow-neat! experience will wear off quickly. Then even as a miss rate of 0.1%, there will be thousands - millions - of cringe-worthy examples that sully the Apple brand for quality. It will be impossible to create quality filter good enough, and there will be no way to back these features…

"that sully the Apple brand for quality."

OK. No one remembers Apple Maps, the CSAM scanning, the crush ad, etc? Companies do embarrassing stuff all the time. At least they're trying.

Re: Apple's On-Device and Server Foundation Models

#498

Post by scottyah »

w10-1 wrote: post #478

I think we as tech people lost the forest for the trees. Apple (unwisely I think) is allowing UI's to just generate responses. The wow-neat! experience will wear off quickly. Then even as a miss rate of 0.1%, there will be thousands - millions - of cringe-worthy examples that sully the Apple brand for quality. It will be impossible to create quality filter good enough, and there will be no way to back these features…

I agree, they're joining in on the slippery slope auto-correct, home assistants, and Self Driving.

I think it's been awhile since consumers have trusted or relied on consumer tech. Browsing the web from a phone can only be described as adversarial. Scrolling down a top google result recipe site is almost impossible. Texts don't always send and you can't keep up with all the cloud backup offerings that it's hard to tell if your photos are actually being saved.

The current political and media scene is often described as post-truth, where accuracy isn't the biggest driving factor. It seems that computation is headed that way as well.

Re: Apple's On-Device and Server Foundation Models

#499

Post by steve1977 »

gumby wrote: post #491
steve1977 wrote: post #428

Earlier quoted context omitted.

As far as I know, both the AirPort trademark and the term Wi-Fi got introduced in 1999 (could be that AirPort was a couple of weeks earlier)

Airport came out in the beginning of the year, maybe January, at Macworld. WECA renamed themselves the WiFi Alliance a few years later, but the name (and trademark) had to exist for it to be worth them doing so. WECA reminds me of other "memorable" names of the era, my favorite being PCMCIA, though VESA is another fave.

Thanks for the info. That was pretty much around the time when I started out in IT. And yes PCMCIA, what a mouthful! I remember Ethernet cards in this format.

Re: Apple's On-Device and Server Foundation Models

#500

Post by tmpz22 »

dgellow wrote: post #432
tmpz22 wrote: post #149

Earlier quoted context omitted.

For the majority of the keynote they explicitly avoided the word AI instead substituting the word Intelligence, then Apple Intelligence, and then towards the end they said AI and ChatGPT once or twice. I think they saw the response to all the AI shoveling and Microsoft Recall and executed a fantastic strategy to reposition themselves in industry discussions. I still have tons of reservations about privacy and what th…

What excites you specifically as a developer?

For the last 10 years I've been a full stack / devops developer. I think that ecosystem is in a very bad place right now and has failed to efficiently modernize. The tools that people are adopting in an attempt to mitigate this such as NextJS are still grounded in the complexities of the past like Node/React/Express/Serverless and not good enough.

These troubles metastasize to subpar SaaS products, low efficiency, bad company cultures, layoffs, bad hiring practices, management instead of leadership, salary stagnation, dark patterns, you name it.

So to see Apple with a laser focus on tooling, quality of life, privacy, in this WWDC while everyone else runs around like a headless chicken suggests to me that their platform might be the more lucrative path to follow. I think it'll be faster, better, and more enjoyable, to develop consumer and business applications for fun and profit.

Don't get be wrong its far from a silver bullet. Many Apple Platform APIs like CloudKit and Server-side-Swift have a LONG way to go. But Im seeing the right steps to address these issues and at the end of the day it feels a whole lot better then what I've been doing in the past and produces better end products IMO.

Post reply on HN