Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Irrelevant facts about cats added to math problems increase LLM errors by 300%

science.org

Post reply on HN
Print view View original

241–250 of 270 posts

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#241

Post by dotancohen »

orbital-decay wrote: post #154
ants_everywhere wrote: post #150

Earlier quoted context omitted.

> We need to move past the humans vs ai discourse it's getting tired. You want a moratorium on comparing AI to other form of intelligence because you think it's tired? If I'm understanding you correctly, that's one of the worst takes on AI I think I've ever seen. The whole point of AI is to create an intelligence modeled on humans and to compare it to humans. Most people who talk about AI have no idea what the psycho…

>The whole point of AI is to create an intelligence modeled on humans and to compare it to humans. According to who? Everyone who's anyone is trying to create highly autonomous systems that do useful work. That's completely unrelated to modeling them on humans or comparing them to humans.

By whoever coined the term Artificial Intelligence. It's right there in the name.

Backronym it to Advanced Inference and the argument goes away.

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#242

Post by getnormality »

acc_297 wrote: post #80

There is more than one comment here asserting that the authors should have done a parallel comparison study against humans on the same question bank as if the study authors had set out to investigate whether humans or LLMs reason better in this situation. The authors do include the claim that humans would immediately disregard this information and maybe some would and some wouldn't that could be debated and seemingly…

After almost three years, the knee-jerk "I'm sure humans would also screw this up" response has become so tired that it feels AI-generated at this point. (Not saying you're doing this, actually the opposite.)

I think a lot of humans would not just disregard the odd information at the end, but say something about how odd it was, and ask the prompter to clarify their intentions. I don't see any of the AI answers doing that.

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#243

Post by senthe »

ants_everywhere wrote: post #150
acc_297 wrote: post #80

There is more than one comment here asserting that the authors should have done a parallel comparison study against humans on the same question bank as if the study authors had set out to investigate whether humans or LLMs reason better in this situation. The authors do include the claim that humans would immediately disregard this information and maybe some would and some wouldn't that could be debated and seemingly…

> We need to move past the humans vs ai discourse it's getting tired. You want a moratorium on comparing AI to other form of intelligence because you think it's tired? If I'm understanding you correctly, that's one of the worst takes on AI I think I've ever seen. The whole point of AI is to create an intelligence modeled on humans and to compare it to humans. Most people who talk about AI have no idea what the psycho…

> The whole point of AI is to create an intelligence modeled on humans and to compare it to humans.

This is like saying the whole point of aeronautics is to create machines that fly like birds and compare them to how birds fly. Birds might have been the inspiration at some point, but learned how to build flying machines that are not bird-like.

In AI, there *are* people trying to create human-like intelligence but the bulk of the field is basically "statistical analysis at scale". LLMs, for example, just predict the most likely next word given a sequence of words. Researchers in this area are trying to make this predictions more accurate, faster and less computationally- and data- intensive. They are not trying to make the workings of LLMs more human-like.

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#245

Post by getnormality »

userbinator wrote: post #164

This looks like it'll be useful for CAPTCHA purposes. According to the researchers, “the triggers are not contextual so humans ignore them when instructed to solve the problem”—but AIs do not. Not all humans, unfortunately: https://en.wikipedia.org/wiki/Age_of_the_captain

I wonder what the role of RLHF is in this. It seems to be one of the more labor-intensive, proprietary, dark-matter aspects of the LLM training process.

Just like some humans may be conditioned by education to assume that all questions posed in school are answerable, RLHF might focus on "happy path" questions where thinking leads to a useful answer that gets rewarded, and the AI might learn to attempt to provide such an answer no matter what.

What is the relationship between the system prompt and the prompting used during RLHF? Does RLHF use many kinds of prompts, so that the system is more adaptable? Or is the system prompt fixed before RLHF begins and then used in all RLHF fine-tuning, so that RLHF has a more limited scope and is potentially more efficient?

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#246

Post by littlestymaar »

ants_everywhere wrote: post #231
littlestymaar wrote: post #197

Earlier quoted context omitted.

Just because something is named after the name of a biological concept doesn't mean it has anything to do with the original thing the name was taken from.

Name collisions are possible, but in these cases the terms are explicitly modeled on the biological concepts.

It's not name “collision”, they took a biological name that somehow felt apt for what they where doing.

To continue oblios's analogy, when you use the “hibernation mode” of your OS, it only has superficial similarity with how manals hibernate during winter…

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#247

Post by ants_everywhere »

littlestymaar wrote: post #246
ants_everywhere wrote: post #231

Earlier quoted context omitted.

Name collisions are possible, but in these cases the terms are explicitly modeled on the biological concepts.

It's not name “collision”, they took a biological name that somehow felt apt for what they where doing. To continue oblios's analogy, when you use the “hibernation mode” of your OS, it only has superficial similarity with how manals hibernate during winter…

[flagged]

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#248

Post by throwanem »

jdmichal wrote: post #215
throwanem wrote: post #96

Earlier quoted context omitted.

As many as ten hundred thousand billion ducks are known to flock in semiannual migrations, but I think you'll find corpus distortion ineffective at any plausible scale. That egg has long since hatched.

> That egg has long since hatched. I imagine there's entire companies in existence now, whose entire value proposition is clean human-generated data. At this point, the Internet as a data source is entirely and irrevokably polluted by large amounts of ducks and various other waterfowl from the Anseriformes order.

What an astonishing eudystopia this implies, after the soft-takeoff singularity Eliezer has predicted 300 of the last [0, 1) of...

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#249

Post by hyperman1 »

cedws wrote: post #230
hyperman1 wrote: post #218

Earlier quoted context omitted.

Because I want to be a polite person by default. It makes life nicer fot everyone involved and gives extra effect when I (rarely)choose not to be polite. I believe any interaction with anything is a little training, and I want to do it in the right direction.

Do you say “thank you” to a vending machine when it dispenses your can of soda?

I presume I would if it would talk to me (playing ads doesn't count). I am known to absent mindedly apologize to my table if I walk into it(sample size of 1). I also try to be polite to my cats(they don't seem to care either way as long as food appears). Make of all this what you want.

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#250

Post by vikramkr »

afiori wrote: post #237
mminer237 wrote: post #235

Earlier quoted context omitted.

Humans are more than just brains. The average American human costs about $50,000/year to run.

That is how I like to think about human lives, as a cost, to be minimized.

Humans, as resources
Post reply on HN