Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Irrelevant facts about cats added to math problems increase LLM errors by 300%

science.org

Post reply on HN
Print view View original

221–230 of 270 posts

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#221

Post by hansmayer »

astrobe_ wrote: post #211
hansmayer wrote: post #71

Oh no, just when we finally got them to properly count the number of "R"s in "strawberry"...

Hopefully these cases will get viral to the general public, so that everyone becomes more aware that despite the words "intelligence", "reasoning", "inference" being used and misused, in the end it is no more than a magic trick, an illusion of intelligence. That being said, I also have hopes in that same technology for its "correlation engine" aspect. A few decades ago I read an article about expert systems; it menti…

> Hopefully these cases will get viral to the general public, so that everyone becomes more aware that despite the words "intelligence", "reasoning", "inference" being used and misused, in the end it is no more than a magic trick, an illusion of intelligence.

I am not optimistic on that. Having met people from "general public" and in general low-effort-crowd who use them, I am really not optimistic.

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#222

Post by gus_massa »

9991 wrote: post #210

Mirrors how my undergrads solve problems.

I teach math in the first year of the university in Argentina, in one of the midterm of linear algebra curses we have a word problem and three dry problems. A few years ago, I added something like (I don't remember the details, so let's made up a new version):

> *John buys a 25' TV and a 30' TV. They usually in total cost $3000. He has a coupon for a 10% discount on the 25' TV and a 20% discount for the 30' TV so he paid $2500. How much does each of the TV cost without coupons?"

I was wondering how many of them would add the 25' and 30' to the matrix and use the Gauss method to solve it, something like:

  25  1  10% | 3000
  30  1  20% | 2500
I don't remember the numbers, but let's say that 40 solved it correctly, 9 didn't solve it and only 1 put the 25 and 30 in the matrix.

I was very happy that they were able to ignore the irrelevant size of the TV. I wonder what would happens if it's not a topic that is so usual.

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#223

Post by wizzwizz4 »

akoboldfrying wrote: post #204
wizzwizz4 wrote: post #200

Earlier quoted context omitted.

No, we're pretty sure brains don't do backprop. See e.g. https://doi.org/10.1038/s41598-018-35221-w

Do we know that backprop is disjoint from variational free energy minimisation? Or could it be that one is an approximation to or special case of the other? I Ctrl-F'd "backprop" and found nothing, so I think they aren't compared in the paper, but maybe this is common knowledge in the field.

Yeah: and people have made comparisons (which I can't find right now). Free energy minimisation works better for some ML tasks (better fit on less data, with less overfitting) but is computationally-expensive to simulate in digital software. (Quite cheap in a physical model, though: I might recall, or might have made up, that you can build such a system with water.)

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#224

Post by mensetmanusman »

acc_297 wrote: post #80

There is more than one comment here asserting that the authors should have done a parallel comparison study against humans on the same question bank as if the study authors had set out to investigate whether humans or LLMs reason better in this situation. The authors do include the claim that humans would immediately disregard this information and maybe some would and some wouldn't that could be debated and seemingly…

“We need to move past the humans vs ai discourse it's getting tired.”

We can do both, the metaphysics of how different types of intelligence manifest will expand our knowledge of ourselves.

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#225

Post by nisegami »

somenameforme wrote: post #170
saurik wrote: post #159

Earlier quoted context omitted.

But since these things are more like humans than computers, to build these autonomous systems you are going to have think in terms of full industrial engineering, not just software engineering: pretend you are dealing with a surprisingly bright and yet ever distracted employee who doesn't really care about their job and ensure that they are able to provide the structure you place them in value without danger to your…

Their output is in natural language, that's about the end of similarities with humans. They're token prediction algorithms, nothing more and nothing less. This can achieve some absolutely remarkable output, probably because our languages (both formal and linguistic) are absurdly redundant. But the next token being a word, instead of e.g. a ticker price, doesn't suddenly make them more like humans than computers.

I see this "next token predictor" description being used as a justification for drawing a distinction between LLMs and human intelligence. While I agree with that description of LLMs, I think the concept of "next token predictor" is much, much closer to describing human intelligence than most people consider.

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#226

Post by daveguy »

thedanbob wrote: post #213
devoutsalsa wrote: post #209

Earlier quoted context omitted.

Today someone who is routinely drug tested at work is being replaced by a hallucinating LLM.

To be fair, the AI probably hallucinates more efficiently than the human.

Nope. The human neural network runs on about 20 watts of power. The LLM is vastly less efficient than the human version. And that's just the inference -- if you consider training it's much worse.

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#227

Post by ants_everywhere »

janalsncm wrote: post #190
ants_everywhere wrote: post #165

Earlier quoted context omitted.

Go back and look at the history of AI, including current papers from the most advanced research teams. Nearly every component is based on humans - neural net - long/short term memory - attention - reasoning - activation function - learning - hallucination - evolutionary algorithm If you're just consuming an AI to build a React app then you don't have to care. If you are building an artificial intelligence then in pra…

Those terms sound similar to biological concepts but they’re very different. Neural networks are not like brains. They don’t grow new neurons. A “neuron” in an artificial neural net is represented with a single floating point number. Sometimes even quantized down to a 4 bit int. Their degrees of freedom are highly limited compared to a brain. Most importantly, the brain does not do back propagation like an ANN does.…

Neural networks are explicitly modeled on brains.

I don't know where this idea that "the things haves similar names but they're unrelated" trope is coming from. But it's not from people who know what they're talking about.

Like I said, go back and read the research. Look at where it was done. Look at the title of Marvin Minksy's thesis. Look at the research on connectionism from the 40s.

I would wager that every major paper about neuroscience from 1899 to 2020 or so has been thoroughly mined by the AI community for ideas.

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#228

Post by pnt12 »

EmiDub wrote: post #212

Why do we keep having these LLM studies that are completely unsurprising. Yes, the probabilistic text generator is more likely to output a correct answer when the input more closely matches its training sources than when you add random noise to the prompt. They don’t actually “understand” maths. It’s worrying how much research seems to operate from the premise that they do.

"It’s worrying how much research seems to operate from the premise that they do."

They are testing an hypothesis, we don't know if they're optimistic or pessimistic about it. Is it even relevant?

They have studied that LLMs can be easily confused with non-sequitors, and this is interesting. Maybe prompts to LLM should be more direct and foccused. Maybe this indicates a problem with end users interacting with LLMs directly - many people have difficulty on writing in a clear and direct way! Probably even more people when speaking!

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#229

Post by daveguy »

akoboldfrying wrote: post #195
janalsncm wrote: post #190

Earlier quoted context omitted.

Those terms sound similar to biological concepts but they’re very different. Neural networks are not like brains. They don’t grow new neurons. A “neuron” in an artificial neural net is represented with a single floating point number. Sometimes even quantized down to a 4 bit int. Their degrees of freedom are highly limited compared to a brain. Most importantly, the brain does not do back propagation like an ANN does.…

Neural networks are a lot like brains. That they don't generally grow new neurons is something that (a) could be changed with a few lines of code and (b) seems like an insignificant detail anyway. > the brain does not do back propagation Do we know this? Ruling this out is tantamount to claiming that we know how brains do learn. My suspicion is that we don't currently know, and that it will turn out that, e.g., sleep…

Neural networks are barely superficially like brains in that they are both composed of multiple functional units. That is the extent of the similarity.

Re: Irrelevant facts about cats added to math problems increase LLM errors by 300%

#230

Post by cedws »

hyperman1 wrote: post #218
cedws wrote: post #193

Earlier quoted context omitted.

Why be polite to a machine?

Because I want to be a polite person by default. It makes life nicer fot everyone involved and gives extra effect when I (rarely)choose not to be polite. I believe any interaction with anything is a little training, and I want to do it in the right direction.

Do you say “thank you” to a vending machine when it dispenses your can of soda?
Post reply on HN