Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

iPhone 17 Pro Demonstrated Running a 400B LLM

twitter.com

Post reply on HN
Print view View original

41–50 of 362 posts

Re: iPhone 17 Pro Demonstrated Running a 400B LLM

#41

Post by icedchai »

qingcharles wrote: post #25
raw_anon_1111 wrote: post #14

Earlier quoted context omitted.

Apple is already one of the winners of the AI race. It’s making much more profit (ie it ain’t losing money) on AI off of ChatGPT, Claude, Grok (you would be surprised at how many incels pay to make AI generated porn videos) subscriptions through the App Store. It’s only paying Google $1 billion a year for access to Gemini for Siri

Plus all those pricey 512GB Mac Studios they are selling to YouTubers.

They don't offer the 512 gig RAM variant anymore. Outside of social media influencers and the occasional AI researcher, the market for $10K desktops is vanishingly small.

Re: iPhone 17 Pro Demonstrated Running a 400B LLM

#42

Post by timcobb »

WarmWash wrote: post #35
baal80spam wrote: post #30

Earlier quoted context omitted.

> The general population loves being glazed way too much. This is 100% correct!

Thanks for short warm blast of dopamine, no one else ever seems to grasp how smart I truly am!

That is an excellent observation.

Re: iPhone 17 Pro Demonstrated Running a 400B LLM

#43

Post by tombert »

WarmWash wrote: post #26
causal wrote: post #15

Run an incredible 400B parameters on a handheld device. 0.6 t/s, wait 30 seconds to see what these billions of calculations get us: "That is a profound observation, and you are absolutely right ..."

I don't think we are ever going to win this. The general population loves being glazed way too much.

That's an astute point, and you're right to point it out.

Re: iPhone 17 Pro Demonstrated Running a 400B LLM

#44

Post by [deleted] »

intrasight wrote: post #28
causal wrote: post #15

Run an incredible 400B parameters on a handheld device. 0.6 t/s, wait 30 seconds to see what these billions of calculations get us: "That is a profound observation, and you are absolutely right ..."

Better than waiting 7.5 million years to have a tell you the answer is 42.

[deleted]

Re: iPhone 17 Pro Demonstrated Running a 400B LLM

#45

Post by actusual »

tombert wrote: post #43
WarmWash wrote: post #26

Earlier quoted context omitted.

I don't think we are ever going to win this. The general population loves being glazed way too much.

That's an astute point, and you're right to point it out.

You are thinking about this exactly the right way.

Re: iPhone 17 Pro Demonstrated Running a 400B LLM

#47

Post by volemo »

simopa wrote: post #4

It's crazy to see a 400B model running on an iPhone. But moving forward, as the information density and architectural efficiency of smaller models continue to increase, getting high-quality, real-time inference on mobile is going to become trivial.

> moving forward, as the information density and architectural efficiency of smaller models continue to increase

If they continue to increase.

Re: iPhone 17 Pro Demonstrated Running a 400B LLM

#48

Post by amelius »

causal wrote: post #15

Run an incredible 400B parameters on a handheld device. 0.6 t/s, wait 30 seconds to see what these billions of calculations get us: "That is a profound observation, and you are absolutely right ..."

I mean size says nothing, you could do it on a Pi Zero with sufficient storage attached.

So this post is like saying that yes an iPhone is Turing complete. Or at least not locked down so far that you're unable to do it.

Re: iPhone 17 Pro Demonstrated Running a 400B LLM

#50

Post by manmal »

foobiekr wrote: post #39
firstbabylonian wrote: post #5

> SSD streaming to GPU Is this solution based on what Apple describes in their 2023 paper 'LLM in a flash' [1]? 1: https://arxiv.org/abs/2312.11514

This is not entirely dissimilar to what Cerebus does with their weights streaming.

And IIRC the Unreal Engine Matrix demo for PS5 was streaming textures directly from SSD to the engine as well?
Post reply on HN