Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My iPhone 16 Pro Max produces garbage output when running MLX LLMs

journal.rafaelcosta.me

Post reply on HN
Print view View original

1–10 of 239 posts

Re: My iPhone 16 Pro Max produces garbage output when running MLX LLMs

#4

Post by bri3d »

the_arun wrote: post #3

[flagged]

Somewhere along the line, the tensor math that runs an LLM became divergent from every other Apple device. My guess is that there's some kind of accumulation issue here (remembering that floating-point accumulation does not usually commute), but it seems genuinely broken in an unexpected way given that Apple's own LLM also doesn't seem to work on this device.

Re: My iPhone 16 Pro Max produces garbage output when running MLX LLMs

#5

Post by ploum »

the_arun wrote: post #3

[flagged]

Well it seems that, those days, instead of SUM(expense1,expense2) you ask an LLM to "make an app that will compute the total of multiple expenses".

If I read most of the news on this very website, this is "way more efficient" and "it saves time" (and those who don’t do it will lose their job)

Then, when it produces wrong output AND it is obvious enough for you to notice, you blame the hardware.

Re: My iPhone 16 Pro Max produces garbage output when running MLX LLMs

#6

Post by bri3d »

ploum wrote: post #5
the_arun wrote: post #3

[flagged]

Well it seems that, those days, instead of SUM(expense1,expense2) you ask an LLM to "make an app that will compute the total of multiple expenses". If I read most of the news on this very website, this is "way more efficient" and "it saves time" (and those who don’t do it will lose their job) Then, when it produces wrong output AND it is obvious enough for you to notice, you blame the hardware.

I mean, Apple's LLM also doesn't work on this device, plus the author compared the outputs from each iterative calculation on this device vs. others and they diverge from every other Apple device. That's a pretty big sign that both, something is different about that device, and this same broken behavior carried across multiple OS versions. Is the hardware or the software "responsible" - who knows, there's no smoking gun there, but it does seem like something is genuinely wrong.

I don't get the snark about LLMs overall in this context; this author uses LLM to help write their code, but is also clearly competent enough to dig in and determine why things don't work when the LLM fails, and performed an LLM-out-of-the-loop debugging session once they decided it wasn't trustworthy. What else could you do in this situation?

Re: My iPhone 16 Pro Max produces garbage output when running MLX LLMs

#10

Post by raincole »

Low level numerical operation optimizations are often not reproduceable. For example: https://www.intel.com/content/dam/develop/external/us/en/doc... (2013)

But it's still surprising that that LLM doesn't work on iPhone 16 at all. After all LLMs are known for their tolerance to quantization.

Post reply on HN