Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

He asked AI to count carbs 27000 times. It couldn't give the same answer twice

diabettech.com

Post reply on HN
Print view View original

161–170 of 329 posts

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#161

Post by rcxdude »

renticulous wrote: post #86
2ndorderthought wrote: post #28

Earlier quoted context omitted.

The average person has no idea this is true. And the average person cannot tell when this is the case. So we have a bunch of people, going their way through school, and then when they get stuck relying on AI. The future is gonna be wild.

Here's technical literacy of population on display. I love these prank examples which show the true education of populace. https://www.youtube.com/shorts/B7c9qJcRnVk

Anything like this is going to have a very heavy selection bias, don't take any of this kind of content as a reflection of the average person.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#162

Post by jerkstate »

nextlevelwizard wrote: post #71
edu wrote: post #59

Earlier quoted context omitted.

Curious, why it was easier to use an LLM vs a non-AI app with a DB of foods? Seems that in this case a traditional approach would be more precise and more environmentally efficient to get to the same results.

Any app I have used before has asked me to look up the foods and add them manually and usually there has been ads or subscriptions involved. Much easier for me to take pictures of the packets while making the food, the weight the final bulk product and then when I eat just weight the plate and say “500g of casserole” and the LLM spits out the calories and keeps track of the daily consumption

Nice. I vibe coded a similar kind of system, you can dump a recipe into the chat window and it will use tool-calling to lookup macros for any foods it doesn't have in the DB and put them in, estimate raw -> cooked changes in nutrition and weight (if needed), estimate total weight of the cooked product, and macros per gram (e.g. writes a 100 gram serving to the db, you can scale it up and down and it scales the macros linearly). Similar to you I have used this app to alter my macro mix from high-fat to high-carb (for workout performance) and cut my sodium from ~4g/day to ~2.4g/day by interrogating the DB about what foods I should eat more and less of. Found some surprising wins in my habitual diet that were easy to change to hit my health targets, and looking up and logging these things by hand without LLM assistance would have been too tedious and time-consuming for me to continue to do it for as long as I have been (maybe 3 months now)

Curious, what model are you using? I have found Qwen Flash to be really great for this - tool calling works well, it's smart enough, and very cheap.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#163

Post by professoretc »

Ukv wrote: post #133
p-e-w wrote: post #46

Earlier quoted context omitted.

Then the correct answer is “I can’t tell.” Not “Here’s a random guess that I just pulled out of my ass.” LLMs have picked up the bad habit of trying to give an answer when no answer can be given from scientists, who overall don’t say “I don’t know” nearly as often as they should.

> Then the correct answer is “I can’t tell.” From the paper they're using structured JSON schema mode opposed to freeform answers, so it can't. Models do typically caveat their answer for questions like this, in my experience.

They'll qualify their answers in English but as the article mentions, if your prompt asks for a confidence score, that "uncertainty" doesn't translate into low numerical confidence.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#164

Post by endymion-light »

Aurornis wrote: post #149
endymion-light wrote: post #141

Earlier quoted context omitted.

I don't believe the authors of this study are stupid. If there are apps targeting people with diabetes that claims to count your carbs with AI, why haven't those been analysed? That would be a far more effective claim. I based the study off of the clickbait article that they wrote about the study - i'll read through the study to see whether they analyse that, but it would be far more effective to see if the 'carb-cou…

> If there are apps targeting people with diabetes that claims to count your carbs with AI, why haven't those been analysed? That would be a far more effective claim. Because the apps aren’t going to let you submit 29,000 automated requests for statistical analysis. And if you did, the authors of those apps would just release an update saying they changed models and try to dismiss the study. The vitriol against this…

You can commit statistical analysis on frontier models and still use commercial applications as an identifier & comparison.

Criticism is not vitriol - it's possible to make a wider point about being taken aback by the lack of education within AI to the point that there's a critical mass of people using them for calorie counting; but there are many studies on effects of LLMs on psychology etc that are far more effective.

But for me - this is like creating a study that performing algebra & calculus is innacurate on LLMs. That should be common knowledge

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#165

Post by arjie »

This is pretty interesting. Not the content, but the technique. I suspect this was an entirely automated pipeline with Claude Code or Codex and that the author then just unleashed one of the commercial harnesses on the entire flow of querying the APIs and writing the post, including the headline. We've clearly reached the point in AI writing where a small set of inputs can create content that humans enjoy participating in discussion of. Good show.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#166

Post by coldtea »

endymion-light wrote: post #159
coldtea wrote: post #147

Earlier quoted context omitted.

> But the author just took pictures of food & expected a realistic response? Is this genuinely what amounts to a study in AI? If there are commercial services where you take pictures of food and are promised a realistic (paid for) response, then yes. And there are.

But I don't see them using those commercial services in this study - instead, they're using frontier model companies? Is Gemini advertising that you get a realistic calorie count from a picture? Maybe so - in which case i'd take it back!

Are commercial services anything more than just UI facades on top of frontier model APIs?

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#167

Post by jmyeet »

endymion-light wrote: post #43

There's an incredibly serious lack of education with how LLMs & carb-counting works. This entire article would be better suited to astrology.com than hackernews. When I opened it up, I assumed the author would have at least attempted a calculation service, maybe even placed something like the size of the meal into an actual model, using the integration of pre-existing tools that are (slightly more) accurate. Hell - m…

> But the author just took pictures of food & expected a realistic response?

If someone sent me a picture of a meal and asked me what the macros were or how many carbs this is, I would say "I can't tell from a photo. Nobody can". The problem is that current LLM chatbots don't seem to have a concept of telling you "I don't know", "you can't do that" or even "you're wrong".

You can say that somebody shouldn't trust an LLM for this but it's going to be a problem that LLMs give nonsencial answers. What I find particularly amusing is that there are still technical people (generally, not anyone specifically) who seem unable to acknowledge that LLMs hallucinate and lie.

There was a post on here recently that I couldn't find with some quick searching but the premise basically was that chatbots were trained like neurotypical people: A lot of affirmation and basically lying. Separately someone else characterized this NT style of communication as "tone poems" [1]. I keep thinking about that because to me that's so accurate.

Dunning-Kruger is a common refrain on HN, for good reason. Another way to put this is how often people are confidently wrong. I really wonder if this is an inevitable consequence of NT communication because most neurodivergent ("ND") people I know are incredibly intentional in what they say and mean.

[1]: https://news.ycombinator.com/item?id=47832952

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#168

Post by endymion-light »

kalleboo wrote: post #142
endymion-light wrote: post #43

There's an incredibly serious lack of education with how LLMs & carb-counting works. This entire article would be better suited to astrology.com than hackernews. When I opened it up, I assumed the author would have at least attempted a calculation service, maybe even placed something like the size of the meal into an actual model, using the integration of pre-existing tools that are (slightly more) accurate. Hell - m…

> But the author just took pictures of food & expected a realistic response? There are very popular apps on the App Store right now that are going viral among non-techie people that do exactly this, and they have no concept of how AI works. My wife was talking about one and I had to give her a reality check that the AI had no idea what ingredients were used to make the food. And she's a licensed nutritionalist. Studi…

That's true - I suppose i'm just dissapointed that this study hasn't seemed to include those within any analysis. Being able to point out that the top 100 calorie counting apps on the app store return similiar results to simple frontier models would be of interest.

I think i'm just dissapointed that this study doesn't go deep enough, and stays at a surface level statistical analysis of frontier models.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#169

Post by ozgung »

furyofantares wrote: post #87
endymion-light wrote: post #43

There's an incredibly serious lack of education with how LLMs & carb-counting works. This entire article would be better suited to astrology.com than hackernews. When I opened it up, I assumed the author would have at least attempted a calculation service, maybe even placed something like the size of the meal into an actual model, using the integration of pre-existing tools that are (slightly more) accurate. Hell - m…

From the text of the article I believe the author is implying there are apps doing exactly this, and so this is why it was studied that way. Had the author written the article themselves rather than an LLM their motivation probably would have been clearer.

The author uses the prompts and method from an open-source app that connects to insulin pump, a medical device. I think AI food identification is an experimental feature in the app.

> The prompt was adapted from the one used in the iAPS open-source automated insulin delivery system — it’s a real production prompt, not a toy example.

https://github.com/Artificial-Pancreas/iAPS

I think these are the prompts in the app: https://github.com/Artificial-Pancreas/iAPS/tree/5eabe22e7e2...

Post reply on HN