Live data from Hacker News

My iPhone 16 Pro Max produces garbage output when running MLX LLMs

journal.rafaelcosta.me

151–160 of 239 posts

Re: My iPhone 16 Pro Max produces garbage output when running MLX LLMs

#151
post #11

Earlier quoted context omitted.

Yes, "floating point accumulation doesn't commute" is a mantra everyone should have in their head, and when I first read this article, I was jumping at the bit to dismiss it out of hand for that reason. But, what got me about this is that: * every other Apple device delivered the same results * Apple's own LLM silently failed on this device to me that behavior suggests an unexpected failure rather than a fundamental…

I would go even further and state that "you should never assume that floating point functions will evaluate the same on two different computers, or even on two different versions of the same application", as the results of floating point evaluations can differ depending on platform, compiler optimizations, compilation-flags, run-time FPU environment (rounding mode, &c.), and even memory alignment of run-time data. Th…

Even the exact same source code compiled with different compilers, or the same compiler with different compiler options.

Intel Compiler for e.g. uses less than IEEE764 precision for floating point ops by default, for example.

Re: My iPhone 16 Pro Max produces garbage output when running MLX LLMs

#152

Methodology is one thing; I can't really agree that deploying an LLM to do sums is great. Almost as hilarious as asking "What's moon plus sun?" But phenomenon is another thing. Apple's numerical APIs are producing inconsistent results on a minority of devices. This is something worth Apple's attention.

> What's moon plus sun? "Monsoon," says ChatGPT.

"moonsun" says JavaScript, 1-0 to JS I'd say.

Re: My iPhone 16 Pro Max produces garbage output when running MLX LLMs

#153
post #145

Does it bother anyone else that the author drops "MiniMax" there in the article without bothering to explain or footnote what that is? (I could look it up, but I think article authors should call out these things).

There are tons of terms that aren't explained that some people (like me) might not understand. I think it's fine that some articles have a particular audience in mind and write specifically for those, in this case, it seems it's for "Apple mobile developers who make LLM inference engines" so not so unexpected there are terms I (and others) don't understand.

Re: My iPhone 16 Pro Max produces garbage output when running MLX LLMs

#154
post #145

Does it bother anyone else that the author drops "MiniMax" there in the article without bothering to explain or footnote what that is? (I could look it up, but I think article authors should call out these things).

I think articles are worse when they have to explain everything someone off the street might not know.

Re: My iPhone 16 Pro Max produces garbage output when running MLX LLMs

#155

Earlier quoted context omitted.

So true! And as any sane Apple user or the standard template Apple Support person would have suggested (and as they actually suggest) - did they try reinstalling the OS from scratch after having reset the data (of course before backing it up; preferably with a hefty iCloud+ plan)? Because that's the thing to do in such issues and it's very easy.

Reinstalling the OS sucks. I need to pull all my bank cards out of my safe and re-add their CVV's to the wallet, and sometimes authenticate over the phone. And re-register my face. And log back in to all my apps. It can take an hour or so, except it's spread out over weeks as I open an app and realize I need to log in a dozen times.

> And log back in to all my apps

Isn’t this built in when transferring devices? Are backups different?

Re: My iPhone 16 Pro Max produces garbage output when running MLX LLMs

#157
post #156

It is a bug in MLX that has been fixed a few days ago: https://github.com/ml-explore/mlx/pull/3083

Blog post dated 28 Jan 2026, the bug fix posted 29 Jan 2026, so I guess this story had a happy ending :)

Still, sad state of affairs that it seems like Apple is still fixing bugs based on what blog posts gets the most attention on the internet, but I guess once they started that approach, it's hard to stop and go back to figuring out priorities on their own.

Re: My iPhone 16 Pro Max produces garbage output when running MLX LLMs

#158
post #145

Does it bother anyone else that the author drops "MiniMax" there in the article without bothering to explain or footnote what that is? (I could look it up, but I think article authors should call out these things).

I think articles are worse when they have to explain everything someone off the street might not know.

Yes, maybe. But it would be nice if there would be footnotes or tooltips. Putting the explanation in the text itself breaks the flow of the text so that would make it worse indeed.

Re: My iPhone 16 Pro Max produces garbage output when running MLX LLMs

#159
post #156

It is a bug in MLX that has been fixed a few days ago: https://github.com/ml-explore/mlx/pull/3083

Blog post dated 28 Jan 2026, the bug fix posted 29 Jan 2026, so I guess this story had a happy ending :) Still, sad state of affairs that it seems like Apple is still fixing bugs based on what blog posts gets the most attention on the internet, but I guess once they started that approach, it's hard to stop and go back to figuring out priorities on their own.

Just goes to show that attention is all you need.

Re: My iPhone 16 Pro Max produces garbage output when running MLX LLMs

#160
post #156

It is a bug in MLX that has been fixed a few days ago: https://github.com/ml-explore/mlx/pull/3083

So the underlying issue is that the iPhone 16 Pro SKU was misdetected as having Neural Accelerator (nax) support and this caused silently wrong results. Not a problem with the actual hardware.
Post reply on HN