Live data from Hacker News

He asked AI to count carbs 27000 times. It couldn't give the same answer twice

diabettech.com

291–300 of 329 posts

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#291
post #6

It’s just an impossible problem. Photons don’t provide sufficient information to determine calories (at least not in any way they could practically be captured). Inside that sandwich could be drenched with olive oil or it could be hollow cheese with lettuce. It’s impossible to tell.

If the problem is so evidently impossible then the LLM itself should recognise this, state that the problem isn't solvable, not* provide what's certain to be an inaccurate result, and suggest better approaches to arriving at a reasonable answer.

That said, it's notable that diabetes education materials often suggest estimating glycemic loads by rough portion size / plate ratios. Which is to say that absent accurate weight measurements (themselves subject to variations in ingredients, moisture levels, etc.) current clinical recommendations are themselves pretty rough.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#292
post #282

Earlier quoted context omitted.

> But the author just took pictures of food & expected a realistic response? There are very popular apps on the App Store right now that are going viral among non-techie people that do exactly this, and they have no concept of how AI works. My wife was talking about one and I had to give her a reality check that the AI had no idea what ingredients were used to make the food. And she's a licensed nutritionalist. Studi…

To be fair my expectations is that those apps have done the prompt engineering, and schema, and tools (to query nutrition database), etc... and although they're not 100% consistent, the margin of errors should be narrow to the point that barely matter, and they should do a bit better than a random ChatGPT chat session.

the problem isn't one that can be solved with prompts. If I gave a panel of food and nutrition experts (human or machine) a bunch of pictures of food, they still wouldn't be able to tell if, e.g. a slice of cake was made with whole milk or skim.

The "pic of packaged food --> LLM --> nutrition DB call" pipeline is workable, but many users of these apps are using them for fresh prepared foods, which is just an unworkable problem without either an understanding of the preparation process or a bomb calorimeter.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#293

Earlier quoted context omitted.

> But the author just took pictures of food & expected a realistic response? There are very popular apps on the App Store right now that are going viral among non-techie people that do exactly this, and they have no concept of how AI works. My wife was talking about one and I had to give her a reality check that the AI had no idea what ingredients were used to make the food. And she's a licensed nutritionalist. Studi…

That's true - I suppose i'm just dissapointed that this study hasn't seemed to include those within any analysis. Being able to point out that the top 100 calorie counting apps on the app store return similiar results to simple frontier models would be of interest. I think i'm just dissapointed that this study doesn't go deep enough, and stays at a surface level statistical analysis of frontier models.

I think it’s a very useful study specifically to debunk the apps that support this flow.

None of those apps have magic. They cannot do better than the frontier models.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#294

This will surprise nobody here, but it’s important to communicate to audiences that are new to LLMs. This is targeted at people with diabetes because there are AI carb counting apps appearing in app stores > If you’re using AI carb counting in a diabetes app These apps are probably not even using the mainstream models used in the study because they would be too expensive for cheap or free apps, and they’re probably f…

Who in Al Gore's Internet is new to LLM in 2026?

[deleted]

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#296
post #88

"Crema catalana: Three of four models called it “creme brulee” 100% of the time. Only Gemini 3.1 Pro got “crema catalana” — in 3.4% of queries." ---- Wikipedia for Crema catalana: Crema catalana (Catalan for 'Catalan cream'), or crema cremada ('burnt cream'), is a Catalan dessert consisting of a custard topped with a layer of caramelized sugar.[1] It is "virtually identical"[2] to the French crème brûlée. It is made…

The difference between them is that crème brûlée is made heavy cream instead of milk and it tastes better. But my Catalan friends would kill me for this blasphemy… so you didn’t hear that from me.

They are both covered by burned sugar and therefore indistinguishable(!) visually.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#297
post #258

Earlier quoted context omitted.

But why would the same llm give you wildly different answers EACH TIME you ask?

There is a parameter in LLMs called temperature that controls creativity/randomness. If you set it to 0 it makes the model deterministic. I think some LLMs expose this as a tunable parameter.

> If you set it to 0 it makes the model deterministic.

No, it doesn't. It can help make the model more deterministic, but it does not guarantee it.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#298

There's an incredibly serious lack of education with how LLMs & carb-counting works. This entire article would be better suited to astrology.com than hackernews. When I opened it up, I assumed the author would have at least attempted a calculation service, maybe even placed something like the size of the meal into an actual model, using the integration of pre-existing tools that are (slightly more) accurate. Hell - m…

> But the author just took pictures of food & expected a realistic response? There are very popular apps on the App Store right now that are going viral among non-techie people that do exactly this, and they have no concept of how AI works. My wife was talking about one and I had to give her a reality check that the AI had no idea what ingredients were used to make the food. And she's a licensed nutritionalist. Studi…

Even simpler examples make the limitations obvious. Images can't distinguish Diet Coke from Coke.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#299
post #271

Earlier quoted context omitted.

The commercial services likely also have frontier model dependencies... The opening to the actual paper is quite explicit that (i) other studies have already tested commercial apps with with unimpressive results and (ii) a popular open source app for carb counting directly relies on API calls from these frontier models, and this research batch tested the images used the exact same models and prompts as the popular op…

A carb counting app might use API calls to these frontier models and then do some kind of analysis. It could see if different models agree or not, or multiple calls, and with how much variance. So it would be more accurate to test the apps rather than the APIs, unless the goal is to warn people that just open chatgpt and ask there.

The open source app could in theory do that, but the paper's authors would be able to determine whether it did or not by reading its code, which they evidently did to replicate the API calls it made with their own script.

(And of course it would also be far more tedious to submit each picture 500 times manually using an app and manually log the response than using a script which is designed to collect the data automatically as fast as API rate limits permit)

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#300

There's an incredibly serious lack of education with how LLMs & carb-counting works. This entire article would be better suited to astrology.com than hackernews. When I opened it up, I assumed the author would have at least attempted a calculation service, maybe even placed something like the size of the meal into an actual model, using the integration of pre-existing tools that are (slightly more) accurate. Hell - m…

[deleted]
Post reply on HN