He asked AI to count carbs 27000 times. It couldn't give the same answer twice • Hacker News Board

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#61

Post by vidarh » Wed, Apr 29, 2026, 1:03 PM UTC

throwaway260124 wrote: post #25

rsynnott wrote: post #2

I am... unsure why anyone would think LLMs would be able to do this. They are not magic oracles. Like I think even most humans would be extremely bad at this. Like, are people actually using LLMs for this? Please do not, it won't work.

But nothing prevents llms from being RLed to do this right? But does training llms to be better at this, improves their world model or does it only make changes at the surface?

Yes, something prevents llms from being RLed to do this: You can't see through something opaque to determine whether there's something high calorie or low calorie out of sight.

The problem itself is unsolvable given the data provided.

You could conceivable make it better at making guesses, but they will inherently always be guesses that will sometimes be wildly off.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#62

Post by unsupp0rted » Wed, Apr 29, 2026, 1:04 PM UTC

jaccola wrote: post #6

It’s just an impossible problem. Photons don’t provide sufficient information to determine calories (at least not in any way they could practically be captured). Inside that sandwich could be drenched with olive oil or it could be hollow cheese with lettuce. It’s impossible to tell.

And what if that guy in the surveillance video is just 2 kids in a trench coat? There's no way for AI to be sure from the photons: we should scrap it.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#63

Post by [deleted] » Wed, Apr 29, 2026, 1:04 PM UTC

jaccola wrote: post #6

It’s just an impossible problem. Photons don’t provide sufficient information to determine calories (at least not in any way they could practically be captured). Inside that sandwich could be drenched with olive oil or it could be hollow cheese with lettuce. It’s impossible to tell.

[deleted]

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#64

Post by 2ndorderthought » Wed, Apr 29, 2026, 1:04 PM UTC

engineer_22 wrote: post #39

2ndorderthought wrote: post #28

Earlier quoted context omitted.

The average person has no idea this is true. And the average person cannot tell when this is the case. So we have a bunch of people, going their way through school, and then when they get stuck relying on AI. The future is gonna be wild.

I am asking a lot here, but school needs to be training people what AI is and what it's weaknesses are and how to use it... My school taught me to use a calculator. It also taught me how to check my work when I relied on the calculator. AI is a very complicated calculator - you give it an input, magic happens, it gives you an output. Really no different, to a layman.

It's more complicated than a calculator. Even researchers who have dedicated their lives to the field don't know all of the limitations of any given model. That fact alone isn't helpful when a model is 80% correct in one area but 2% in another.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#65

Post by jaccola » Wed, Apr 29, 2026, 1:04 PM UTC

engineer_22 wrote: post #39

2ndorderthought wrote: post #28

Earlier quoted context omitted.

The average person has no idea this is true. And the average person cannot tell when this is the case. So we have a bunch of people, going their way through school, and then when they get stuck relying on AI. The future is gonna be wild.

I am asking a lot here, but school needs to be training people what AI is and what it's weaknesses are and how to use it... My school taught me to use a calculator. It also taught me how to check my work when I relied on the calculator. AI is a very complicated calculator - you give it an input, magic happens, it gives you an output. Really no different, to a layman.

To be fair, this should probably be covered by basic physics/maybe cooking classes. “You can’t determine the calories in food by looking at it” isn’t really ML specific.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#66

Post by edu » Wed, Apr 29, 2026, 1:04 PM UTC

mottiden wrote: post #45

I am surprised that people believe that calories can be counted correctly from a single photo

Issue is there are many apps claiming they can do that, and for many people are “magic”.

We should not allow companies to lie blatantly to the customers.

Edit: r/blame/lie/

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#67

Post by ben_w » Wed, Apr 29, 2026, 1:05 PM UTC

throwaway260124 wrote: post #25

rsynnott wrote: post #2

I am... unsure why anyone would think LLMs would be able to do this. They are not magic oracles. Like I think even most humans would be extremely bad at this. Like, are people actually using LLMs for this? Please do not, it won't work.

But nothing prevents llms from being RLed to do this right? But does training llms to be better at this, improves their world model or does it only make changes at the surface?

Estimate the calorie count of this door handle: https://m.youtube.com/watch?v=VDSzY52Mkrw&pp=0gcJCVACo7VqN5t...

Extreme example perhaps, but no, you can't just turn pixels into calories. Right now I'd be impressed if we could reliably estimate volume to within 30% from a photo, but even with that correct the contents of the food can easily be way off without visible sign.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#68

Post by garciasn » Wed, Apr 29, 2026, 1:05 PM UTC

engineer_22 wrote: post #39

2ndorderthought wrote: post #28

Earlier quoted context omitted.

The average person has no idea this is true. And the average person cannot tell when this is the case. So we have a bunch of people, going their way through school, and then when they get stuck relying on AI. The future is gonna be wild.

I am asking a lot here, but school needs to be training people what AI is and what it's weaknesses are and how to use it... My school taught me to use a calculator. It also taught me how to check my work when I relied on the calculator. AI is a very complicated calculator - you give it an input, magic happens, it gives you an output. Really no different, to a layman.

Considering the lack of basic math skills I encounter each and every day, I don't think schools did enough; they certainly aren't going to do enough w/LLMs.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#69

Post by fabian2k » Wed, Apr 29, 2026, 1:06 PM UTC

It does sound like a pretty terrible idea to try to count carbohydrates from an image. There just isn't enough information there to reliably do that. At best you could identify the object in the image and then show reference information on typical nutrition values. But if you need anything more accurate than that, you probably have to read the labels on the ingredients and calculate.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#70

Post by tsimionescu » Wed, Apr 29, 2026, 1:06 PM UTC

dyauspitr wrote: post #32

What a dumb article. The picture of the sandwich is essentially just a picture of bread. You can’t see what’s inside. A human wouldn’t be able to tell you. These are essentially AI hit pieces.

This is based on a real app that someone is selling to real people. It's not a hit against AI, but it is very much a legitimate hit against uses of LLMs for this purpose.

Also, if LLMs worked as they are often advertised, they should have easily been able to answer "there isn't enough information in this picture to give you an accurate estimate. Try taking a picture of the label, or at least of the inside of the sandwich, or list the ingredients used".