Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

He asked AI to count carbs 27000 times. It couldn't give the same answer twice

diabettech.com

Post reply on HN
Print view View original

251–260 of 329 posts

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#251

Post by lukeschlather »

throwaw12 wrote: post #92
endymion-light wrote: post #43

There's an incredibly serious lack of education with how LLMs & carb-counting works. This entire article would be better suited to astrology.com than hackernews. When I opened it up, I assumed the author would have at least attempted a calculation service, maybe even placed something like the size of the meal into an actual model, using the integration of pre-existing tools that are (slightly more) accurate. Hell - m…

I feel like you didn't understand the goal of this study > The DTN-UK stated earlier this year that generic LLMs must never be used as autonomous advisory calculators for insulin delivery. This data is the quantitative evidence base for that statement. This study is to prove that you should not rely on LLMs

The thing is it doesn't really prove LLMs can't do this, it proves no existing frontier LLMs can do this.

The part where they talk about sampling multiple runs is interesting - it suggests to me that in the next few years as the reasoning process is improved the models may be able to do that autonomously.

My mind really is going to using a dedicated object detection models fine-tuned with nutrition information, but I don't think there's a fundamental reason LLMs can't eventually manage this use case, except perhaps the size of the needed weights being prohibitively large.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#252

Post by dahart »

swiftcoder wrote: post #200
dahart wrote: post #174

Earlier quoted context omitted.

And what’s the variance & accuracy of their responses? Isn’t comparing the models’ variance to baseline human variance what matters here? It seems like they didn’t do that, and I agree with parent’s call for that kind of baseline. Having counted calories for years, I don’t think I could reliably estimate the calories or carbs in the example picture of a cheese sandwich. I can make assumptions about the bread and the…

> Isn’t comparing the models’ variance to baseline human variance what matters here? You seem to be missing the context that this isn't just about diet apps - this is about apps claiming to be able to track carbs sufficiently accurately to be used in a medical context to dose insulin (a substance which can be lethal if incorrectly dosed)

No I understand apps are making dubious claims and implications; obviously claiming LLMs can accurately estimate carbs from a photo is just wrong. But that doesn’t necessarily change my question. Should people use photos to estimate carbs? Can people looking at photos do any better?

The presence of variance in the LLM output doesn’t actually prove anything, in fact I would expect and hope for variance when confidence is less than 1.0. I’m more curious about accuracy of the mean of guesses for different models, for example.

But should any diabetic expect photos to be reliable, regardless of whether it’s an app or an LLM or a human? I know some diabetics, and the people I know do not rely on photos for their safety. They don’t even rely on food labels either (which are far more accurate than photos), they measure their insulin.

It’s probably useful to raise awareness, and useful to scare app makers away from making bogus medical claims - products and scams that make bogus medical claims is of course a practice as old as history. But we can still hold the studies and PR around this up to high standards, right? Even assuming this article & the paper behind it are right, there are reasonable questions here about how to demonstrate the problem and what the baselines are.

It’s worth keeping in mind that trying to prove the bogus apps wrong with a flawed methodology or questionable reasoning or just an overly heavy handed style can cause backlash and do damage to the cause. We’re already seeing that effect play out with respect to vaccinations.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#253

Post by tsimionescu »

sjhatfield wrote: post #221
tsimionescu wrote: post #70

Earlier quoted context omitted.

This is based on a real app that someone is selling to real people. It's not a hit against AI, but it is very much a legitimate hit against uses of LLMs for this purpose. Also, if LLMs worked as they are often advertised, they should have easily been able to answer "there isn't enough information in this picture to give you an accurate estimate. Try taking a picture of the label, or at least of the inside of the sand…

iAPS is not software you pay for. This is open source software to dose insulin where you assume full responsibility for the outcomes.

That's irrelevant to my point. Feel free to replace "selling" with "distributing" - my point stands: using an LLM to estimate carbs from a picture is a real use case that real software developers are suggesting to real people is a real solution. It's not some contrived idea that the researcher concocted to make AI look bad.

The GP, and much of the thread, is basically acting as if it should be obvious to anyone who is not an idiot that this is not possible to do precisely; and that the researcher just made up some use case that LLMs can't do and wrote a paper about it to disparage AI.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#255

Post by johsole »

andrewvc wrote: post #189
endymion-light wrote: post #43

There's an incredibly serious lack of education with how LLMs & carb-counting works. This entire article would be better suited to astrology.com than hackernews. When I opened it up, I assumed the author would have at least attempted a calculation service, maybe even placed something like the size of the meal into an actual model, using the integration of pre-existing tools that are (slightly more) accurate. Hell - m…

One of the biggest gaps is that people don't understand that food labels are allowed by the FDA to be off by up to 20% in terms of the number of actual calories! In the real world you need to calibrate your behavior with the results. Are you gaining weight? You'll need to eat less if you want to lose any. You can do all the math with nutrition labels and macros you want but that's all theoretical. See this study belo…

Thank you for linking the study.

Some good news from it. If you weigh the food instead of depending on the package size then the labels become much more accurate!!

"Serving size, by weight, exceeded label statements by 1.2% [median] (25th percentile −1.4, 75th percentile 4.3, p=0.10). When differences in serving size were accounted for, metabolizable calories were 6.8 kcal (0.5, 23.5, p=0.0003) or 4.3% (0.2, 13.7, p=0.001) higher than the label statement."

If you look at the table "Deviation of metabolizable calories from label calories" [https://pmc.ncbi.nlm.nih.gov/articles/PMC3605747/figure/F1/] you'll see that most labels even for service side are pretty good and there are some that are really bad.

If you'll look at one of the worst offenders Tostitos, the label has "Tostitos Tortilla Chips - serving size 24 chips", but chips vary a lot in size, so you could have a huge variance in weight. If instead you weighed them, which I do with my chips, I bet the calories are much closer to the label.

Body composition comes down to routine. I've found found I love to eat, but I pretty much eat the same meals week over week, that makes it extremely easy for me to lose or gain weight depending on my goals.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#256

Post by devilbunny »

The_Blade wrote: post #175
throwaw12 wrote: post #92

Earlier quoted context omitted.

I feel like you didn't understand the goal of this study > The DTN-UK stated earlier this year that generic LLMs must never be used as autonomous advisory calculators for insulin delivery. This data is the quantitative evidence base for that statement. This study is to prove that you should not rely on LLMs

that is good to know. presented this way i find LLM behavior to be a feature, not a bug. then again i think everything is value add over pen and paper / notepad / spreadsheet and maybe a friend or doctor (or specialized equipment if you need more than calorie in, calorie out). just go exercise and don't be a lard lad

> just go exercise and don't be a lard lad

You can out-exercise almost any diet, but it takes 3-4 hours a day of a hard workout.

If calories in, calories out was useful advice rather than a banal statement of physics, nobody would be fat.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#257

Post by machomaster »

ifwinterco wrote: post #192
thechao wrote: post #188

Earlier quoted context omitted.

About once a week I ask ChatGPT to give me a reasonable diet for recomp with weight loss. It consistently insisted I have at least 7 meals consisting of at least 30g of protein per meal, but the protein source can't be whey or casein. When I ask "why" it cites a bunch of studies ... but most of those "studies" are N=1 of a college or Olympic level athlete. If, instead, I grab a large scale lateral analysis, it says "…

To be fair this is a reflection of the general state of nutritional science and the actual answer seems to be "it depends on your genes". Some people do well on 6 small meals, others do well on no breakfast and two large ones. Studies can't tell you anything useful about that, you have to experiment and find out what works best for you

The answer does not really depend on genes. There are personal preferences, there are sex differences (women prefer more carbs), and the biggest component is where you are and in which direction do you want to go to.

But in terms of physiology the answer is quite clear:

1. The protein is the most important macro to get, no matter if bulking or cutting. It is the building block.

2. Whatever the amounts (0.8g-1.8g/kg of bodyweight, depends a bit on a situation and the willingness to lose some potential marginal gains), try to divide your daily protein somewhat evenly between meals.

3. Pareto principle, you get the most benefits by having 3 meals. 4 if you really care about small differences and want to optimize. 5 meals give negligible additional benefits, for professional athletes who want to be anal.

4. So basically eat at least 3 meals and up to whatever works for you practically speaking.

It's not that difficult or ambiguous.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#258

Post by pkaye »

bcjdjsndon wrote: post #202
ozgung wrote: post #196

Earlier quoted context omitted.

As a human, in the photo of that sandwich I see 4 slices of bread and 4 slices of cheese (distributed unevenly). I have no idea about the weight of the bread, flour type or its sugar content. I don't know the type of the cheese, dimensions of the slices or total amount of cheese inside the bread. I don't know if there is butter or anything else inside. I can guess the size of the plate as a size reference but I can't…

But why would the same llm give you wildly different answers EACH TIME you ask?

There is a parameter in LLMs called temperature that controls creativity/randomness. If you set it to 0 it makes the model deterministic. I think some LLMs expose this as a tunable parameter.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#259

Post by whstl »

endymion-light wrote: post #43

There's an incredibly serious lack of education with how LLMs & carb-counting works. This entire article would be better suited to astrology.com than hackernews. When I opened it up, I assumed the author would have at least attempted a calculation service, maybe even placed something like the size of the meal into an actual model, using the integration of pre-existing tools that are (slightly more) accurate. Hell - m…

> But the author just took pictures of food & expected a realistic response?

You say this (and I agree), but I know of quite a few companies in this area, including a couple accelerated by YCombinator, and that's pretty much 100% of what they do in their backend.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#260

Post by sleepybrett »

endymion-light wrote: post #43

There's an incredibly serious lack of education with how LLMs & carb-counting works. This entire article would be better suited to astrology.com than hackernews. When I opened it up, I assumed the author would have at least attempted a calculation service, maybe even placed something like the size of the meal into an actual model, using the integration of pre-existing tools that are (slightly more) accurate. Hell - m…

> There's an incredibly serious lack of education with how LLMs & carb-counting works. This entire article would be better suited to astrology.com than hackernews.

This is because the people who promote these technologies, and the companies that sell these technologies, engage in a massive amount of puffery (aka hyperbolizing aka just straight telling lies).

These technologies are painted as the magical solution to whatever problem you have (all it costs you is a few tens of thousands of tokens, aka your water supply). There is literally nothing they CAN'T do if you will just let us build these gigantic small town destroying, noise polluting, water and electricity hungry 'AI data-centers'. So that we can use those datacenters to sell you more tokens to put into their slot machines.

Post reply on HN