Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My iPhone 16 Pro Max produces garbage output when running MLX LLMs

journal.rafaelcosta.me

Post reply on HN
Print view View original

81–90 of 239 posts

Re: My iPhone 16 Pro Max produces garbage output when running MLX LLMs

#81

Post by Buttons840 »

xp84 wrote: post #69
Buttons840 wrote: post #7

I clicked hoping this would be about how old graphing calculators are generally better math companions than a phone. The best way to do math on my phone I know of is the HP Prime emulator.

I was pretty delighted to realize I could now delete the lame Calculator.app from my iPhone and replace it with something of my choice. For now I've settled on NumWorks, which is apparently an emulator of a modern upstart physical graphing calc that has made some inroads into schools. And of course, you can make a Control Center button to launch an app, so that's what I did. Honestly, the main beef I have with Calcul…

I looked at that calculator. But HP Prime and TI-89 have CAS systems that can do symbolic math, so I prefer to emulate them.

Re: My iPhone 16 Pro Max produces garbage output when running MLX LLMs

#82

Post by ekelsen »

sva_ wrote: post #55
bri3d wrote: post #11

Earlier quoted context omitted.

Yes, "floating point accumulation doesn't commute" is a mantra everyone should have in their head, and when I first read this article, I was jumping at the bit to dismiss it out of hand for that reason. But, what got me about this is that: * every other Apple device delivered the same results * Apple's own LLM silently failed on this device to me that behavior suggests an unexpected failure rather than a fundamental…

> floating point accumulation doesn't commute It is commutative (except for NaN). It isn't associative though.

I think it commutes even when one or both inputs are NaN? The output is always NaN.

Re: My iPhone 16 Pro Max produces garbage output when running MLX LLMs

#83

Post by LoganDark »

mikestew wrote: post #72
LoganDark wrote: post #44

Earlier quoted context omitted.

> Fascinating the claim is Apple Intelligence doesn't work altogether. Quite a scandal. No, the claim is their particular device has a hardware defect that causes MLX not to work (which includes Apple Intelligence). > EDIT: If you wouldn't mind, could you edit out "AI rage enthusiast" you edited in? I understand it was in good humor, as you describe yourself that way as well. However, I don't want to eat downvotes on…

Then future downvotes won't affect your karma. Who cares? The max amount of karma loss is 4 points, we can afford to eat our downvotes like adults.

Huh. I thought the minimum comment score was -4 (which would make the maximum amount of karma loss 5, since each comment starts at 1 point), but I didn't know if that was a cap on karma loss or just a cap on comment score.

Re: My iPhone 16 Pro Max produces garbage output when running MLX LLMs

#84

Post by addaon »

ekelsen wrote: post #82
sva_ wrote: post #55

Earlier quoted context omitted.

> floating point accumulation doesn't commute It is commutative (except for NaN). It isn't associative though.

I think it commutes even when one or both inputs are NaN? The output is always NaN.

NaNs are distinguishable. /Which/ NaN you get doesn't commute.

Re: My iPhone 16 Pro Max produces garbage output when running MLX LLMs

#85

Post by wolvoleo »

xoa wrote: post #78
Buttons840 wrote: post #7

I clicked hoping this would be about how old graphing calculators are generally better math companions than a phone. The best way to do math on my phone I know of is the HP Prime emulator.

My personal favorite is iHP48 (previously I used m48+ before it died) running an HP 48GX with metakernal installed as I used through college. Still just so intuitive and fast to me.

I still have mine. Never use it though as I'm not handy with RPN anymore. :'(

Re: My iPhone 16 Pro Max produces garbage output when running MLX LLMs

#86

Post by wolvoleo »

DJBunnies wrote: post #26
PlatoIsADisease wrote: post #20

Earlier quoted context omitted.

Admittedly, I hate companies that live off their marketing. Nintendo, Disney, Apple. I hate that these companies can weaponize psychology against humans. Function > Form. I think its a Hero Complex, if Jung is correct.

Macbooks and iPhones are good devices though, saying this as a primarily linux user. There is no way a company could exist purely on marketing, Apple backs it up with tech.

Some companies definitely do just exist on marketing. Some clothing brands are objectively overpriced crap and pure wealth signalling. Or something like a juicero.

But I agree Apple doesn't even though they've gone into a direction I couldn't follow them in.

Re: My iPhone 16 Pro Max produces garbage output when running MLX LLMs

#88

Post by wolvoleo »

decimalenough wrote: post #19
lionkor wrote: post #13

[flagged]

My TL;DR is that they tried to run an on-device model to classify expenses, it didn't work even for simple cases ("Kasai Kitchin" -> "unknown"), they went deeeeeep down the rabbit hole to figure out why and concluded that inference on their particular model/phone is borked at the hardware level. Whether you should do this on device is another story entirely.

I would really not want to upload my expense data to some random cloud server, nope. On device is really a benefit even if it's not quite as comprehensive. And really in line with apple's privacy focus so it's very imaginable that many of their customers agree.

Re: My iPhone 16 Pro Max produces garbage output when running MLX LLMs

#89

Post by geuis »

CrispinS wrote: post #73
csmantle wrote: post #46

Methodology is one thing; I can't really agree that deploying an LLM to do sums is great. Almost as hilarious as asking "What's moon plus sun?" But phenomenon is another thing. Apple's numerical APIs are producing inconsistent results on a minority of devices. This is something worth Apple's attention.

> What's moon plus sun? Eclipse, obviously.

Not obvious. Astronomers are actively looking for signatures of exomoons around exoplanets. So "sun plus moon" could mean that too.

Re: My iPhone 16 Pro Max produces garbage output when running MLX LLMs

#90

Post by ekelsen »

addaon wrote: post #84
ekelsen wrote: post #82

Earlier quoted context omitted.

I think it commutes even when one or both inputs are NaN? The output is always NaN.

NaNs are distinguishable. /Which/ NaN you get doesn't commute.

I guess at the bit level, but not at the level of computation? Anything that relies on bit patterns of nans behaving in a certain way (like how they propagate) is in dangerous territory.
Post reply on HN