Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

He asked AI to count carbs 27000 times. It couldn't give the same answer twice

diabettech.com

Post reply on HN
Print view View original

21–30 of 329 posts

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#21

Post by kioleanu »

rsynnott wrote: post #2

I am... unsure why anyone would think LLMs would be able to do this. They are not magic oracles. Like I think even most humans would be extremely bad at this. Like, are people actually using LLMs for this? Please do not, it won't work.

Yes, people are using LLMs for this because that is how they've been marketed, like being able to solve every day tasks like a personal assistant on one hand, but also like researchers being able to solve old problems that humans couldn't crack.

Does the model say it can't do that when asked? No, it answers confidentely.

Also it's easy to trust it if you don't know how it works

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#23

Post by sjsdaiuasgdia »

rsynnott wrote: post #2

I am... unsure why anyone would think LLMs would be able to do this. They are not magic oracles. Like I think even most humans would be extremely bad at this. Like, are people actually using LLMs for this? Please do not, it won't work.

Some people are asking LLMs what's on the menu of restaurants they are actively sitting in, possibly with a menu on the table in front of them.

Some people have a very poor understanding of what LLMs are good for. Some people do see them as magic oracles.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#24

Post by amazingamazing »

With mass information you could infer much more from pictures. With some sort of standard cube in the picture as well as taking a picture at an angle that emphasizes all three dimensions you could also better estimate the relative volume.

It’s tractable I think, but not from a pic alone.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#25

Post by throwaway260124 »

rsynnott wrote: post #2

I am... unsure why anyone would think LLMs would be able to do this. They are not magic oracles. Like I think even most humans would be extremely bad at this. Like, are people actually using LLMs for this? Please do not, it won't work.

But nothing prevents llms from being RLed to do this right?

But does training llms to be better at this, improves their world model or does it only make changes at the surface?

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#26

Post by voidUpdate »

> "The prompt was adapted from the one used in the iAPS open-source automated insulin delivery system — it’s a real production prompt, not a toy example."

This idea is seriously being implemented in a production app? And people are using that app to make health choices? Oh god...

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#27

Post by jchw »

> 42.9 units of insulin from a single photo. That’s not a rounding error. That’s a potential fatality.

Shit like this is why you shouldn't involve AI output in your writing process. It's especially ironic in an article about LLMs being unreliable... but it's pointless when the pre-print seems just fine at least to my eyes.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#28

Post by 2ndorderthought »

jaccola wrote: post #6

It’s just an impossible problem. Photons don’t provide sufficient information to determine calories (at least not in any way they could practically be captured). Inside that sandwich could be drenched with olive oil or it could be hollow cheese with lettuce. It’s impossible to tell.

The average person has no idea this is true. And the average person cannot tell when this is the case. So we have a bunch of people, going their way through school, and then when they get stuck relying on AI. The future is gonna be wild.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#29

Post by nextlevelwizard »

I used LLMs to count calories, but not based on photos, I mean I also did that, but primarily I fed in my exact ingredients and then used weights to get calorie estimates.

Was it always correct? Certainly not. But it helped me lose 30kg of weight since keeping even some track of calories was so much easier with LLM than any app I had used before.

Also of course it didn’t matter if I was exactly on point since it wasn’t about any kind of medicine

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#30

Post by AndrewKemendo »

rsynnott wrote: post #2

I am... unsure why anyone would think LLMs would be able to do this. They are not magic oracles. Like I think even most humans would be extremely bad at this. Like, are people actually using LLMs for this? Please do not, it won't work.

The vast majority of people using LLMs in my experience use them as though they are Oracles

They are surprised and upset when the Oracle is not perfect

Go ahead and search around on hacker news you’ll see precisely the same pattern with people who are ostensibly engineers and hackers

It’s actually pretty mind boggling but then again humans never fail to surprise and disappoint

Post reply on HN