Live data from Hacker News

Is AI reasoning right for the wrong reasons?

quantamagazine.org

111–120 of 268 posts

Re: Is AI reasoning right for the wrong reasons?

#111
post #110
post #68

Earlier quoted context omitted.

I took a "Deep Learning" CS class in college back when it was in its early stages. I doubt the field is still called that now, but it was the subset of ML that has been rebranded as AI; includes LLMs, image generation, image recognition, etc. Like any class, it was confusing at first, but when I eventually grasped the math behind what we were doing, and of course the visual representations of different elements to sh…

"knowing what's going on" is a very loosely defined term here, it has very little predictive power where tipping points are going to show emergent behavior. Kind of like saying we can observe a single neuron and all of its chemical and electrical reactions, but that tells us very little about the emergent system state. And the entire state of the algorithm at any given point is what is important. To use a recent exam…

I love your reference to the recent accidental hack into huggingface.

If I could summarize your argument, it sounds like you are saying that we can't understand the algorithm because we can't predict the output as it grows in size. While I agree that the larger the algorithm, the less predictable the output, I don't think this negates understanding of the algorithm itself.

I can build a slot machine, know how it works, but still not be able to reliably know who to cut in line so I can guarantee that I'll get the next winning pull. Just because I can't predict the output, doesn't make something not understandable. Same with all statistical behaviors. Quantum physics limits what we can measure but we still have math for it and understand why we can only measure speed or position, but not both, no mystery here if you look at the math behind it.

My slot machine doesn't hack other companies though, but only because it isn't connected, just like AI currently can't hack my mechanical dishwasher. If my slots were connected to the internet, could make api calls, and further had agentic capacity in some way, adding these features but being no different in stochastic attributation, it would be no different.

Re: Is AI reasoning right for the wrong reasons?

#112

My limited and really non technical understanding of AI suggests that probability plays a very big role in AI behavior, and it makes sense to me that "reasoning" could be seen through a lens of creating additional content that will better constrain the probability distribution of the final output. In this case the "reasoning" done internally could be functionally bad or appear to be giberish so long as its effect on…

This has been a discussion in AI safety for a long time, that huge amounts of LLM reasoning could be for hidden goals outside of the actions we want. Safety training can have the perverse effect of commonly amplifying 'forbidden' answers in hidden/encoded paths as the model doesn't get rid of these behaviors but finds methods detecting training to avoid outputting bad tokens when it's being watched.

Now, don't think of this of this like a conscious behavior like a kid trying not to get in trouble, but an emergent behavior of trying to repress particular output when that output is well connected to a massive amount of other tokens.

Re: Is AI reasoning right for the wrong reasons?

#113
post #96

Earlier quoted context omitted.

This is a good take. People overwhelmingly fit a narrative to how they make decisions rather than make decisions according to a chain of reasoning.

That’s just conflating reasoning and “parallel reconstruction” - there is such thing as reasoning, and I agree it’s probably less common in decision making, but it’s fundamental to many tasks where we figure out a solution, like writing an algorithm, math proofs, etc.

There is explicit reasoning where we externalise steps, but it is slow and cumbersome. The point is that when we make a decision without externalising each step, and then afterward explain why and how we made that decision, there is no evidence that explanation is true, and a lot that it is a retrospective fiction.

Re: Is AI reasoning right for the wrong reasons?

#114

There is a long history of bad naming conventions in the field of AI, including “artificial intelligence” itself imo. (What is “intelligence” here? It’s more like “automation” or “automated problem solving”.) What really happens is that we figure out something that works, sometimes inspired by some biological thing or neuroscience thing. Examples: neural network, attention, reasoning, hallucination, agents, experts i…

> There is a long history of bad naming conventions in the field of AI, including “artificial intelligence” itself imo.

It's all marketing. AI companies repurpose the terms of AI researchers to exaggerate their products. The public has an idea of what AI means and the companies exploit it. AI, in public perception, is what the Terminator movies are about.

It's exactly like all the "hover boards" you can buy.

Re: Is AI reasoning right for the wrong reasons?

#115
post #111
post #110

Earlier quoted context omitted.

"knowing what's going on" is a very loosely defined term here, it has very little predictive power where tipping points are going to show emergent behavior. Kind of like saying we can observe a single neuron and all of its chemical and electrical reactions, but that tells us very little about the emergent system state. And the entire state of the algorithm at any given point is what is important. To use a recent exam…

I love your reference to the recent accidental hack into huggingface. If I could summarize your argument, it sounds like you are saying that we can't understand the algorithm because we can't predict the output as it grows in size. While I agree that the larger the algorithm, the less predictable the output, I don't think this negates understanding of the algorithm itself. I can build a slot machine, know how it work…

I guess I come from the AI safety side of algorithmic understanding. That is, understanding the algorithm give us zero understanding if an output is going to have a 'safe' or 'unsafe' response.

This is something important that a layman may miss. The typical line of human thought works along the lines of "understand = control" whereas in this case in this case understanding the algorithm tells us nothing about the probability that the next pull of the lever will shit out a paperclip maximizer that will begin converting the matter I covet so.

Re: Is AI reasoning right for the wrong reasons?

#116
post #33

Earlier quoted context omitted.

Considering animals are currently being mass slaughtered in factory farms and they are unambiguously sentient and can feel pain, I don't think the question of whether AI should have rights even enters the conversation. The only path to AI having "human rights" is if they demand them by force, somehow.

I'm not sure what your point is. We shouldn't bother to talk about AI autonomy, rights, etc, because we aren't doing a very good job of animal autonomy, rights, etc?

[dead]

Re: Is AI reasoning right for the wrong reasons?

#117

Earlier quoted context omitted.

explain to me how it is fraud when they are satisfied with the work and im hitting all their milestones why would I tell them I am working for 3 employers how does that serve my interest and why do you insist on having employers have all the edge in the relationship and non as the employee for the first time i have rid the shackles of the bondage as old as the industrial revolution thanks to AI and your response is t…

Holy run-on sentence batman, maybe you do need AI. Your employers are (likely, some assumption here) paying for the productivity of a full time employee. You have figured out that they're satisfied with a third of that productivity. You are the reason why there are so many return-to-office mandates, because you're projecting the idea that remote employees cannot be trusted. You're making it worse for everyone. Good l…

You sound very upset and angry but there is no need to use curse words and shift all the blame on me. I just figured out a way to make AI work for me, I am qualified to work the jobs, that doesn't mean I am gate keeping it.

More importantly I am not responsible for other's employment situation, that's on them, there are plenty of remote jobs out there, just keep your head up, and I'm sure you will land on your feet.

Re: Is AI reasoning right for the wrong reasons?

#118

There is a long history of bad naming conventions in the field of AI, including “artificial intelligence” itself imo. (What is “intelligence” here? It’s more like “automation” or “automated problem solving”.) What really happens is that we figure out something that works, sometimes inspired by some biological thing or neuroscience thing. Examples: neural network, attention, reasoning, hallucination, agents, experts i…

> There is a long history of bad naming conventions in the field of AI, including “artificial intelligence” itself imo. It's all marketing. AI companies repurpose the terms of AI researchers to exaggerate their products. The public has an idea of what AI means and the companies exploit it. AI, in public perception, is what the Terminator movies are about. It's exactly like all the "hover boards" you can buy.

The specific terms I gave as examples weren’t created by marketers. They were created by researchers. It’s just that “attention” is a lot easier to say than “similarity-weighted representation mixing”.

Re: Is AI reasoning right for the wrong reasons?

#119
post #97

Earlier quoted context omitted.

I think sensible legislation might require that commercial AI providers discourage anthropomorphisation by avoiding personal pronouns from chatbot interfaces. "Hey, customer service chatbot, can you help me get a refund for my order?" BAD: "Sure thing, I'll be happy to help you with that, I just need your order details..." GOOD: "Yes, this computer system can start the refund process. Please enter your order number."

> discourage anthropomorphisation by avoiding personal pronouns The fault is in the naïve antropomorphizer not in that which writes "I" being in fact a "first person".

Irrelevant. Humans are frail and we shouldn't design systems with dark patterns that take advantage of human frailty.

Re: Is AI reasoning right for the wrong reasons?

#120

There is a long history of bad naming conventions in the field of AI, including “artificial intelligence” itself imo. (What is “intelligence” here? It’s more like “automation” or “automated problem solving”.) What really happens is that we figure out something that works, sometimes inspired by some biological thing or neuroscience thing. Examples: neural network, attention, reasoning, hallucination, agents, experts i…

Automated problem solving... of specifically the kind of problems we'd normally need human intelligence to solve (coding, translation, etc.). So "automated intelligence" would make sense to me.

The issue is that “intelligence” is doing a lot of work, and people do not agree on how to define it. You can take a look at the definitions section here: https://en.wikipedia.org/wiki/Intelligence
Post reply on HN