Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

He asked AI to count carbs 27000 times. It couldn't give the same answer twice

diabettech.com

Post reply on HN
Print view View original

151–160 of 329 posts

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#151

Post by rsynnott »

throwaway260124 wrote: post #25
rsynnott wrote: post #2

I am... unsure why anyone would think LLMs would be able to do this. They are not magic oracles. Like I think even most humans would be extremely bad at this. Like, are people actually using LLMs for this? Please do not, it won't work.

But nothing prevents llms from being RLed to do this right? But does training llms to be better at this, improves their world model or does it only make changes at the surface?

Okay, so take the sandwich. There is no way to know what is in it by looking at it. No amount of optimisation will fix this.

I'm sure one could produce a CV model that was a lot better at guessing here than these LLMs are, but fundamentally it is still guessing.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#152

Post by macleginn »

endymion-light wrote: post #43

There's an incredibly serious lack of education with how LLMs & carb-counting works. This entire article would be better suited to astrology.com than hackernews. When I opened it up, I assumed the author would have at least attempted a calculation service, maybe even placed something like the size of the meal into an actual model, using the integration of pre-existing tools that are (slightly more) accurate. Hell - m…

It doesn't really matter if the model cannot make a good educated guess about calories in the food if it cannot give a consistent response given the same input.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#153

Post by something765478 »

endymion-light wrote: post #43

There's an incredibly serious lack of education with how LLMs & carb-counting works. This entire article would be better suited to astrology.com than hackernews. When I opened it up, I assumed the author would have at least attempted a calculation service, maybe even placed something like the size of the meal into an actual model, using the integration of pre-existing tools that are (slightly more) accurate. Hell - m…

> The prompt I used asks each model to return a confidence score (0 to 1) for every food item it identifies. All four models dutifully returned confidence scores for 100% of items. Surely we can use those to filter out bad estimates?

This is a problem with the companies selling the AI models, not the customers. It is their responsibility to inform consumers about the limits of their services, and to train the models to say "I don't know, there is not enough information".

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#155

Post by endymion-light »

ilivethere wrote: post #129
endymion-light wrote: post #43

There's an incredibly serious lack of education with how LLMs & carb-counting works. This entire article would be better suited to astrology.com than hackernews. When I opened it up, I assumed the author would have at least attempted a calculation service, maybe even placed something like the size of the meal into an actual model, using the integration of pre-existing tools that are (slightly more) accurate. Hell - m…

> But the author just took pictures of food & expected a realistic response? Outside our tech-enabled bubble, there are folks who have been sold the idea that ChatGPT et al is a miracle worker capable of replacing dieticians, gym coaches, psychologists, etc. So it's VERY plausible to believe that there are folks out there snapping pics of their meals and asking GPT to spit out nutritional values.

That's a good point - and I think there's a wider core lack of knowledge outside of the bubble.

I suppose I just expected this study to be a little less 'water is wet' which made me dissapointed, but that may be coming at it from a more technical perspective.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#156

Post by larodi »

endymion-light wrote: post #43

There's an incredibly serious lack of education with how LLMs & carb-counting works. This entire article would be better suited to astrology.com than hackernews. When I opened it up, I assumed the author would have at least attempted a calculation service, maybe even placed something like the size of the meal into an actual model, using the integration of pre-existing tools that are (slightly more) accurate. Hell - m…

> This entire article would be better suited to astrology.com than hackernews.

I laughed, but you nailed it. Sadly so many people lack even basic understanding of LLMs and the ViT tower that makes it vLLM, that I expect a whole industry, similar to fortune telling, to emerge out of it.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#157

Post by lesuorac »

2ndorderthought wrote: post #28
jaccola wrote: post #6

It’s just an impossible problem. Photons don’t provide sufficient information to determine calories (at least not in any way they could practically be captured). Inside that sandwich could be drenched with olive oil or it could be hollow cheese with lettuce. It’s impossible to tell.

The average person has no idea this is true. And the average person cannot tell when this is the case. So we have a bunch of people, going their way through school, and then when they get stuck relying on AI. The future is gonna be wild.

If you're looking for a citation about this, the 1999 Dunning-Kurger paper "Unskilled and unaware" [1] is about this.

People who are unskilled at a task are unaware of what that task performed correctly is. So, somebody who can't count calories is unable to tell that the AI can't perform the task correctly either.

[1]: https://pubmed.ncbi.nlm.nih.gov/10626367/

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#158

Post by drrotmos »

jaccola wrote: post #6

It’s just an impossible problem. Photons don’t provide sufficient information to determine calories (at least not in any way they could practically be captured). Inside that sandwich could be drenched with olive oil or it could be hollow cheese with lettuce. It’s impossible to tell.

It is and it isn't. If you ask a human how many calories (or carbs) are in that sandwich, they can give you a qualified guess based on how a sandwich like that is typically constructed. They may not know the calories for a slice of bread or a slice of cheese by heart, but if you give them a food database, they can look it up.

They absolutely won't be 100% correct (bread sizes e.g. are going to be an estimate), but unless it's a trick sandwich drenched in olive oil or with hollow cheese, they're probably going to be in the right ballpark.

I don't think it's outside the realm of possibility for an LLM to be in the right ballpark as well, but that doesn't seem to be where we're at now.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#159

Post by endymion-light »

coldtea wrote: post #147
endymion-light wrote: post #43

There's an incredibly serious lack of education with how LLMs & carb-counting works. This entire article would be better suited to astrology.com than hackernews. When I opened it up, I assumed the author would have at least attempted a calculation service, maybe even placed something like the size of the meal into an actual model, using the integration of pre-existing tools that are (slightly more) accurate. Hell - m…

> But the author just took pictures of food & expected a realistic response? Is this genuinely what amounts to a study in AI? If there are commercial services where you take pictures of food and are promised a realistic (paid for) response, then yes. And there are.

But I don't see them using those commercial services in this study - instead, they're using frontier model companies? Is Gemini advertising that you get a realistic calorie count from a picture? Maybe so - in which case i'd take it back!

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#160

Post by sluck »

recursivedoubts wrote: post #50

> You’d expect the same answer each time. It’s the same photo, the same model, the same question. But you won’t get the same answer. Not even close — and the differences are large enough to cause a hypoglycaemic emergency. No you wouldn't, not if you have a basic understanding of how LLMs work and what "temperature" is. They are stochastic algorithms picking the next token based on a highly structured (and often very…

just came here to read this thanks faith in discussions restored
Post reply on HN