I wish he would have tried on a different iPhone 16 Pro Max to see if the defect was specific to that individual device.
My iPhone 16 Pro Max produces garbage output when running MLX LLMs
51–60 of 239 posts
Re: My iPhone 16 Pro Max produces garbage output when running MLX LLMs
#52Earlier quoted context omitted.
Why shouldn't you? It's your device, it has hardware made specifically for inference. What's to be gained, other than battery life, by offloading inference to someone else? To be lost, at least, is your data ownership and perhaps money.
> What's to be gained... by offloading inference to someone else? Access to models that local hardware can't run. The kind of model that an iphone struggles to run is blown out of the water by most low end hosted models. Its the same reason that most devs opt for claude code, cursor, copilot, etc. instead of using hosted models for coding assistance.
But apparently this model is sufficient for what the OP wants to do. Also apparently it works on iPhone 15 and 17, but not on 16.
Re: My iPhone 16 Pro Max produces garbage output when running MLX LLMs
#53I clicked hoping this would be about how old graphing calculators are generally better math companions than a phone. The best way to do math on my phone I know of is the HP Prime emulator.
I run a TI 83+ emulator on my Android phone when I don't have my physical calculator at hand. Same concept, just learned a different brand of calculators.
built-in calculator apps are surprisingly underbaked... I'm surprised neither of the big two operating systems have elected to ship something comparable to a real calculator built in. It would be nice if we could preview the whole expression as we type it..
I use the NumWorks emulator app whenever I need something more advanced. It's pretty good https://www.numworks.com/simulator/
Re: My iPhone 16 Pro Max produces garbage output when running MLX LLMs
#54Earlier quoted context omitted.
This is a conclusion that comes with some personal baggage you should identify and consider addressing.
Admittedly, I hate companies that live off their marketing. Nintendo, Disney, Apple. I hate that these companies can weaponize psychology against humans. Function > Form. I think its a Hero Complex, if Jung is correct.
I'd almost say most companies live or die off their marketing. One could argue that understanding your customer as well as or better than they understand themselves is a strength.
To wit, some people do value form over function. Some people do prefer a safe, curated walled garden.
I am not among them--I say this as someone who cannot stand using most Apple products for more than a minute. But I respect what they offer(ed) and for some people even recommended them. (Now I'm less sure because it seems like everything tech has gone to shit, but I can't tell if that's just "old man yells at cloud" or what)
Ideally there would be enough competition for us all to find what we're looking for. I think anticompetitive behavior is a worse sin
Re: My iPhone 16 Pro Max produces garbage output when running MLX LLMs
#55Low level numerical operation optimizations are often not reproduceable. For example: https://www.intel.com/content/dam/develop/external/us/en/doc... (2013) But it's still surprising that that LLM doesn't work on iPhone 16 at all. After all LLMs are known for their tolerance to quantization.
Yes, "floating point accumulation doesn't commute" is a mantra everyone should have in their head, and when I first read this article, I was jumping at the bit to dismiss it out of hand for that reason. But, what got me about this is that: * every other Apple device delivered the same results * Apple's own LLM silently failed on this device to me that behavior suggests an unexpected failure rather than a fundamental…
> floating point accumulation doesn't commute
It is commutative (except for NaN). It isn't associative though.
Re: My iPhone 16 Pro Max produces garbage output when running MLX LLMs
#56[flagged]
Can you prove that is still the case with the iPhone SE by showing a comparable hardware with similar long support on software updates and lower price?
Re: My iPhone 16 Pro Max produces garbage output when running MLX LLMs
#57Maybe this is why my damn keyboard predictive text is so gloriously broken
It’s gotten so bad that I’m half convinced it’s either (a) deliberately trolling, or (b) ‘optimising’ for speech to text adoption.
Re: My iPhone 16 Pro Max produces garbage output when running MLX LLMs
#58Maybe this is why my damn keyboard predictive text is so gloriously broken
Oh it's not just me? Typing on my iPhone in the last few months (~6 months?) has been absolutely atrocious. I've tried disabling/enabling every combination of keyboard setting I can thinkj of, but the predictive text just randomly breaks or it just gives up and stops correcting anything at all.
It’s not just you, and it got bad on my work iPhone at the same time so I know it’s not failing hardware or some customization since I keep that quite vanilla.
Re: My iPhone 16 Pro Max produces garbage output when running MLX LLMs
#59Methodology is one thing; I can't really agree that deploying an LLM to do sums is great. Almost as hilarious as asking "What's moon plus sun?" But phenomenon is another thing. Apple's numerical APIs are producing inconsistent results on a minority of devices. This is something worth Apple's attention.
[dead]
Re: My iPhone 16 Pro Max produces garbage output when running MLX LLMs
#60[flagged]
Well it seems that, those days, instead of SUM(expense1,expense2) you ask an LLM to "make an app that will compute the total of multiple expenses". If I read most of the news on this very website, this is "way more efficient" and "it saves time" (and those who don’t do it will lose their job) Then, when it produces wrong output AND it is obvious enough for you to notice, you blame the hardware.
The author is debugging the tensor operations of the on-device model with a simple prompt. They confirmed the discrepancy with other iPhone models.
It’s no different than someone testing a calculator with 2+2. If it gets that wrong, there’s a hardware issue. That doesn’t mean the only purpose of the calculator is to calculate 2+2. It is for debugging.
You could just as uncharitably complain that “these days no one does arithmetic anymore, they use a calculator for 2+2”.