Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mechanistic interpretability researchers applying causality theory to LLMs

cacm.acm.org

Post reply on HN
Print view View original

31–40 of 101 posts

Re: Mechanistic interpretability researchers applying causality theory to LLMs

#31

Post by throw310822 »

dataflow wrote: post #23
blooalien wrote: post #9

Earlier quoted context omitted.

> Do we see signs of reasoning or is it anthropomorphism? This is the part that so many folks just don't seem to understand (probably because it's been labeled as "thinking" or "reasoning" mode, and people assume that words have meaning). It's not reasoning or thought. It's spewing tokens pretending to "think", but it's actually just generating extra "context" to help the final answer be more coherent. The model isn'…

Honestly, people need to get over this debate. It's pretty irrelevant in a lot of cases. When people ask "what is the model thinking?", they're really asking "what caused the model to produce this response (as opposed to a bunch of other plausible ones)?" Whether it's thinking or word prediction or whatever you want to call it, people are trying to understand the causal chain.

It's not just a nominalistic debate though, as the people who are vocal against the idea that LLMs might "understand" or "think" also claim that because of this, they are fundamentally limited in what they can achieve, in contrast to human beings. Therefore any possibility of actual intelligence (or even superintelligence) is, according to them, just a fantasy.

Re: Mechanistic interpretability researchers applying causality theory to LLMs

#33

Post by rvba »

warumdarum wrote: post #18
dang wrote: post #61

[stub for offtopicness] [[All: please don't post shallow-generic reactions to baity titles. Those are basically the same thing, a la https://en.wikipedia.org/wiki/Rubin_vase , and we're trying for something more substantive here.]]

They dont. They have input that runs through a invisible stochastic canyon. As long as there is previous experience the stochastic canyon never ends. If there is none or isignificant one, or it runs out of tokkens, it hallucinates and the illusion falls apart. There is no reasoning, just the invisible grand canyon of all of human experience and knowledge. PS: try to get it to retell you a clichee movie or book and yo…

There is a streamer who plays Diablo 2 by listening to the AI advice and it is quite funny since it is pretty clear that most of the advice is an amalgamation of random, often incorrect advicem

I wonder if it is the same for programming or not, but I vibe coded an android app just to see if I can and it just works. It required a lot of "build the code and correct the errors" pushing though. For example requested code in kotlin but received something else.

Re: Mechanistic interpretability researchers applying causality theory to LLMs

#35

Post by ToValueFunfetti »

3848499449 wrote: post #29
ToValueFunfetti wrote: post #16

Earlier quoted context omitted.

For the love of all that is sacred, please stop doing this. I'm begging you. The whole social media landscape is dying and you are creating a throwaway to participate in ruining this small corner. I assume this is not your first. And no one is convinced by this! The guidelines are there for your benefit as well. You achieve nothing but hastening the destruction of one of the last half-decent communities. Sorry for th…

[flagged]

The top two comments in this thread agree with the point you just made. This is true of essentially any thread on the subject. If this place sucks, it would have to be because of people like you. If not, you in particular may not be very good at noticing.

Re: Mechanistic interpretability researchers applying causality theory to LLMs

#37

Post by [deleted] »

warumdarum wrote: post #18
dang wrote: post #61

[stub for offtopicness] [[All: please don't post shallow-generic reactions to baity titles. Those are basically the same thing, a la https://en.wikipedia.org/wiki/Rubin_vase , and we're trying for something more substantive here.]]

They dont. They have input that runs through a invisible stochastic canyon. As long as there is previous experience the stochastic canyon never ends. If there is none or isignificant one, or it runs out of tokkens, it hallucinates and the illusion falls apart. There is no reasoning, just the invisible grand canyon of all of human experience and knowledge. PS: try to get it to retell you a clichee movie or book and yo…

[deleted]

Re: Mechanistic interpretability researchers applying causality theory to LLMs

#38

Post by Lomlioto »

warumdarum wrote: post #18
dang wrote: post #61

[stub for offtopicness] [[All: please don't post shallow-generic reactions to baity titles. Those are basically the same thing, a la https://en.wikipedia.org/wiki/Rubin_vase , and we're trying for something more substantive here.]]

They dont. They have input that runs through a invisible stochastic canyon. As long as there is previous experience the stochastic canyon never ends. If there is none or isignificant one, or it runs out of tokkens, it hallucinates and the illusion falls apart. There is no reasoning, just the invisible grand canyon of all of human experience and knowledge. PS: try to get it to retell you a clichee movie or book and yo…

Compression is the trick. Its even philosophed about if compression = intelligence.

The LLM has to compress everyy question/prompt into its system. It does so by creating rules and ways of processing data (this can lead to AGI, world models or an architecture of sub architectures like an LLM + something else). So if it should respond in a way that only reasoning people can achieve, it might be able to learn a representation of what we call reasoning.

It read enough text in itself to even know about the concept of reasoning and how you would do that.

Even if this is only stochastic, it shouldn't be so devalued as your comment comes across.

Who says that we are doing anything more magic?

Re: Mechanistic interpretability researchers applying causality theory to LLMs

#39

Post by wizzwizz4 »

emp17344 wrote: post #30
alchemist1e9 wrote: post #28

Earlier quoted context omitted.

It’s curious how they solve unsolved math problems without reasoning. Maybe I have a different definition of reasoning than you.

Guess what? SAT solvers have also solved unsolved math problems. Do you believe they are “reasoning”?

The question of whether a SAT solver can reason is about as interesting as the question of whether a submarine can swim. (EWD867, EWD898)

Re: Mechanistic interpretability researchers applying causality theory to LLMs

#40

Post by antleys »

This article is not about "reasoning" in the abstract, philosophical sense but is talking about "mechanistic interpretability" research. The title is more like, "can we understand if the 'knowledge' encoded into a neural networks actually corresponds to reasoning-like concepts" and doing that with actual experiments like tweaking weights and activations.

There's an interesting example where researchers saw a model approached clock time calculations and calendar month-day calculations using the same methodology. So then is this because an underlying concept of "cyclical measures" has emerged in the network?

Post reply on HN