Live data from Hacker News

He asked AI to count carbs 27000 times. It couldn't give the same answer twice

diabettech.com

121–130 of 329 posts

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#121

Earlier quoted context omitted.

I feel like you didn't understand the goal of this study > The DTN-UK stated earlier this year that generic LLMs must never be used as autonomous advisory calculators for insulin delivery. This data is the quantitative evidence base for that statement. This study is to prove that you should not rely on LLMs

But it's stupid. If i smack myself in the head with a hammer is that proof hammers shouldn't be relied on?

[deleted]

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#123

Earlier quoted context omitted.

I feel like you didn't understand the goal of this study > The DTN-UK stated earlier this year that generic LLMs must never be used as autonomous advisory calculators for insulin delivery. This data is the quantitative evidence base for that statement. This study is to prove that you should not rely on LLMs

But it's stupid. If i smack myself in the head with a hammer is that proof hammers shouldn't be relied on?

Are there start-ups led by idiots suggesting that smacking yourself in the head with a hammer will help treat your diabetes?

If not, then perhaps there's a problem in your analogy.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#124

There's an incredibly serious lack of education with how LLMs & carb-counting works. This entire article would be better suited to astrology.com than hackernews. When I opened it up, I assumed the author would have at least attempted a calculation service, maybe even placed something like the size of the meal into an actual model, using the integration of pre-existing tools that are (slightly more) accurate. Hell - m…

[dead]

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#125

Earlier quoted context omitted.

I feel like you didn't understand the goal of this study > The DTN-UK stated earlier this year that generic LLMs must never be used as autonomous advisory calculators for insulin delivery. This data is the quantitative evidence base for that statement. This study is to prove that you should not rely on LLMs

But it's stupid. If i smack myself in the head with a hammer is that proof hammers shouldn't be relied on?

If you smack yourself in the head with a hammer and it injures you that's evidence that smacking people in the head with hammers is bad and shouldn't be done, right?

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#126

> You’d expect the same answer each time. It’s the same photo, the same model, the same question. But you won’t get the same answer. Not even close — and the differences are large enough to cause a hypoglycaemic emergency. Already the first paragraph highlights the issue; unless you set temperature=0.0 and the model can actually do reproducible inference, none of the "answers" you get are deterministic! But it's a ve…

This is one of the reasons i never used LLMs for anything related to coding. And i never intend to do. If i tell the thing to generate, will it generate the same thing, every time? will it change stuff that is working because the random number generator will conjure a slightly different answer? i'd be ok with it if i was generating a picture of X, or some word salad about Y, but not for code. Never for code.

You'll learn to work around it, just like ML practitioners got used to imprecise math in regards to floats. But that LLMs are using imprecise math and doesn't have 100% reproducible output doesn't make them impossible to work with, just a bit harder.

But, if what you're doing right now works for you, do continue as-is if you so wish, I have no stake in if people use LLMs or not, just hope people make choices based on good information :)

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#127

Earlier quoted context omitted.

I feel like you didn't understand the goal of this study > The DTN-UK stated earlier this year that generic LLMs must never be used as autonomous advisory calculators for insulin delivery. This data is the quantitative evidence base for that statement. This study is to prove that you should not rely on LLMs

But it's stupid. If i smack myself in the head with a hammer is that proof hammers shouldn't be relied on?

Here we’re at the origin of the tool and get to watch how many people hit themselves in the head before we learn this collective wisdom.

There’s a gap between what the tool will allow you to tell it to do, and what it’s good at. The feedback mechanism to tell the difference is deficient compared to a hammer.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#128

There's an incredibly serious lack of education with how LLMs & carb-counting works. This entire article would be better suited to astrology.com than hackernews. When I opened it up, I assumed the author would have at least attempted a calculation service, maybe even placed something like the size of the meal into an actual model, using the integration of pre-existing tools that are (slightly more) accurate. Hell - m…

> But the author just took pictures of food & expected a realistic response? Is this genuinely what amounts to a study in AI?

The article explains this: There are apps targeting people with diabetes that claim to count your carbs with AI.

> If you’re using AI carb counting in a diabetes app

Before you dismiss a study, try to understand where it’s coming from.

The authors of the study weren’t stupid. They knew the LLMs would provide poor results. They ran the study to quantify it and create a resource to spread the information in response to the rise of AI carb counting apps.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#129

There's an incredibly serious lack of education with how LLMs & carb-counting works. This entire article would be better suited to astrology.com than hackernews. When I opened it up, I assumed the author would have at least attempted a calculation service, maybe even placed something like the size of the meal into an actual model, using the integration of pre-existing tools that are (slightly more) accurate. Hell - m…

> But the author just took pictures of food & expected a realistic response?

Outside our tech-enabled bubble, there are folks who have been sold the idea that ChatGPT et al is a miracle worker capable of replacing dieticians, gym coaches, psychologists, etc.

So it's VERY plausible to believe that there are folks out there snapping pics of their meals and asking GPT to spit out nutritional values.

Re: He asked AI to count carbs 27000 times. It couldn't give the same answer twice

#130

I am surprised that people believe that calories can be counted correctly from a single photo

It's like the 'enhance' bs they do in crime shows. All of a sudden the computer can make up a sharp image out of nowhere
Post reply on HN