Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

He asked AI to count carbs 27000 times. It couldn't give the same answer twice

diabettech.com

Post reply on HN
Print view View original

141–150 of 329 posts

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#141

Post by endymion-light »

Aurornis wrote: post #128
endymion-light wrote: post #43

There's an incredibly serious lack of education with how LLMs & carb-counting works. This entire article would be better suited to astrology.com than hackernews. When I opened it up, I assumed the author would have at least attempted a calculation service, maybe even placed something like the size of the meal into an actual model, using the integration of pre-existing tools that are (slightly more) accurate. Hell - m…

> But the author just took pictures of food & expected a realistic response? Is this genuinely what amounts to a study in AI? The article explains this: There are apps targeting people with diabetes that claim to count your carbs with AI. > If you’re using AI carb counting in a diabetes app Before you dismiss a study, try to understand where it’s coming from. The authors of the study weren’t stupid. They knew the LLM…

I don't believe the authors of this study are stupid.

If there are apps targeting people with diabetes that claims to count your carbs with AI, why haven't those been analysed? That would be a far more effective claim.

I based the study off of the clickbait article that they wrote about the study - i'll read through the study to see whether they analyse that, but it would be far more effective to see if the 'carb-counting' AI app is returning similiar results to the frontier model - that's an interesting result that actually can forward discussion.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#142

Post by kalleboo »

endymion-light wrote: post #43

There's an incredibly serious lack of education with how LLMs & carb-counting works. This entire article would be better suited to astrology.com than hackernews. When I opened it up, I assumed the author would have at least attempted a calculation service, maybe even placed something like the size of the meal into an actual model, using the integration of pre-existing tools that are (slightly more) accurate. Hell - m…

> But the author just took pictures of food & expected a realistic response?

There are very popular apps on the App Store right now that are going viral among non-techie people that do exactly this, and they have no concept of how AI works. My wife was talking about one and I had to give her a reality check that the AI had no idea what ingredients were used to make the food. And she's a licensed nutritionalist.

Studies like this create something to point at for people who are confused and serve as a springboard for a conversation in the media.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#143

Post by ambicapter »

rsynnott wrote: post #2

I am... unsure why anyone would think LLMs would be able to do this. They are not magic oracles. Like I think even most humans would be extremely bad at this. Like, are people actually using LLMs for this? Please do not, it won't work.

If the LLM can correctly identify a food item some high percentage of the time, why would it be magic for it to guess the amount of calories in an object? It's perhaps a lookup and some simple math as an extra step.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#144

Post by Aurornis »

fabian2k wrote: post #106
throwaw12 wrote: post #92

Earlier quoted context omitted.

I feel like you didn't understand the goal of this study > The DTN-UK stated earlier this year that generic LLMs must never be used as autonomous advisory calculators for insulin delivery. This data is the quantitative evidence base for that statement. This study is to prove that you should not rely on LLMs

The paper itself is a lot clearer about the purpose. The blog post reads very clickbaity and doesn't really explain the context well.

I disagree, it clearly explains that AI carb counting apps are a problem and shouldn’t be used.

They’re writing in a neutral way that reaches their audience without lecturing or being condescending. They lead the reader to the conclusion rather than shoving it at them. I think that’s why it’s triggering so many angry comments on HN, but it’s effective for the audience they’re writing for (non technical people who may need convincing but don’t like being preached at)

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#145

Post by endymion-light »

InsideOutSanta wrote: post #116
endymion-light wrote: post #43

There's an incredibly serious lack of education with how LLMs & carb-counting works. This entire article would be better suited to astrology.com than hackernews. When I opened it up, I assumed the author would have at least attempted a calculation service, maybe even placed something like the size of the meal into an actual model, using the integration of pre-existing tools that are (slightly more) accurate. Hell - m…

There are apps in the app store right now that pretend to do this kind of thing, so having somebody actually show that it doesn't work is valuable, even if we already knew the outcome ahead of time.

I suppose i'd much rather a study analyse the apps in the app store that are attempting and claiming to do that kind of thing - rather than the base model they might be using.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#146

Post by rsynnott »

kdheiwns wrote: post #95
rsynnott wrote: post #2

I am... unsure why anyone would think LLMs would be able to do this. They are not magic oracles. Like I think even most humans would be extremely bad at this. Like, are people actually using LLMs for this? Please do not, it won't work.

They're marketed as AI. AI has a long standing image built up by movies and other media of being some omniscient computer capable of analyzing the world. These AI companies are very aware of this and leverage it. And a person with sufficient knowledge could easily give a rough estimate of the calories. A slice of store bought sandwich bread of a given thickness generally has calories within a certain range. So do che…

> Packaging on food also always has calories, so clearly people know how to estimate it fairly accurately.

That's not done by looking at it and guessing (or at least it _shouldn't_ be; manufacturers have been known to do this but it's bad practice and may cause them regulatory problems). In an ideal world it's done with one of these: https://en.wikipedia.org/wiki/Calorimeter ; less ideally it can be estimated based on the ingredients.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#147

Post by coldtea »

endymion-light wrote: post #43

There's an incredibly serious lack of education with how LLMs & carb-counting works. This entire article would be better suited to astrology.com than hackernews. When I opened it up, I assumed the author would have at least attempted a calculation service, maybe even placed something like the size of the meal into an actual model, using the integration of pre-existing tools that are (slightly more) accurate. Hell - m…

>But the author just took pictures of food & expected a realistic response? Is this genuinely what amounts to a study in AI?

If there are commercial services where you take pictures of food and are promised a realistic (paid for) response, then yes. And there are.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#148

Post by coldtea »

snapcaster wrote: post #103
throwaw12 wrote: post #92

Earlier quoted context omitted.

I feel like you didn't understand the goal of this study > The DTN-UK stated earlier this year that generic LLMs must never be used as autonomous advisory calculators for insulin delivery. This data is the quantitative evidence base for that statement. This study is to prove that you should not rely on LLMs

But it's stupid. If i smack myself in the head with a hammer is that proof hammers shouldn't be relied on?

No, but it would be proof you didn't get the point of the paper.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#149

Post by Aurornis »

endymion-light wrote: post #141
Aurornis wrote: post #128

Earlier quoted context omitted.

> But the author just took pictures of food & expected a realistic response? Is this genuinely what amounts to a study in AI? The article explains this: There are apps targeting people with diabetes that claim to count your carbs with AI. > If you’re using AI carb counting in a diabetes app Before you dismiss a study, try to understand where it’s coming from. The authors of the study weren’t stupid. They knew the LLM…

I don't believe the authors of this study are stupid. If there are apps targeting people with diabetes that claims to count your carbs with AI, why haven't those been analysed? That would be a far more effective claim. I based the study off of the clickbait article that they wrote about the study - i'll read through the study to see whether they analyse that, but it would be far more effective to see if the 'carb-cou…

> If there are apps targeting people with diabetes that claims to count your carbs with AI, why haven't those been analysed? That would be a far more effective claim.

Because the apps aren’t going to let you submit 29,000 automated requests for statistical analysis.

And if you did, the authors of those apps would just release an update saying they changed models and try to dismiss the study.

The vitriol against this article on HN is sad. Commenters who agree with the article and its conclusions are grasping for reasons to be angry about it anyway

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#150

Post by criley2 »

Brendinooo wrote: post #97
furyofantares wrote: post #87

Earlier quoted context omitted.

From the text of the article I believe the author is implying there are apps doing exactly this, and so this is why it was studied that way. Had the author written the article themselves rather than an LLM their motivation probably would have been clearer.

> there are apps doing exactly this Yeah, for sure there are. And people will just ask ChatGPT as well. The funny thing is that for people who are just trying to lose weight without managing any health issues precisely, this type of extreme variance doesn't really matter, because the mere act of consciously quantifying food consumption is, based on my experience counting calories, the single biggest factor in success…

I actually think "just asking ChatGPT" is fine, because A) the data in these apps is suspect at best and B) the data behind calories is also pretty suspect (but we all play along because we can adjust other variables to make it all "work" well enough).

Once or twice a year I spend a few weeks meticulously measuring ingredients/cooked foods and recording calories and on complex recipes apps are next to useless at getting accurate data. You're trying to input five or ten relevant ingredients, and then weighing your cooked outcome to try and divide the ingredients by proportion. Frankly it's a mess and most people aren't doing it for home cooked meals, and are getting very lossy outcomes (weighing cooked chicken and marking it as raw chicken, etc)

With reasoning and tool calling (combined with me meticulously weighing before and after), it's producing fine data for my purposes.

Post reply on HN