Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mechanistic interpretability researchers applying causality theory to LLMs

cacm.acm.org

Post reply on HN
Print view View original

21–30 of 101 posts

Re: Mechanistic interpretability researchers applying causality theory to LLMs

#21

Post by [deleted] »

warumdarum wrote: post #18
dang wrote: post #61

[stub for offtopicness] [[All: please don't post shallow-generic reactions to baity titles. Those are basically the same thing, a la https://en.wikipedia.org/wiki/Rubin_vase , and we're trying for something more substantive here.]]

They dont. They have input that runs through a invisible stochastic canyon. As long as there is previous experience the stochastic canyon never ends. If there is none or isignificant one, or it runs out of tokkens, it hallucinates and the illusion falls apart. There is no reasoning, just the invisible grand canyon of all of human experience and knowledge. PS: try to get it to retell you a clichee movie or book and yo…

[deleted]

Re: Mechanistic interpretability researchers applying causality theory to LLMs

#22

Post by throw310822 »

chrisjj wrote: post #11
throw310822 wrote: post #4

Earlier quoted context omitted.

Of course they do, how else do you think they manage to implement new features in large codebases, or to prove new theorems? But you don't even have to assume they do because of the results- you can read their chain of thought.

The Eliza effect.

It's indeed so powerful that even my compiler and my unit tests fell victim of this delusion.

Re: Mechanistic interpretability researchers applying causality theory to LLMs

#23

Post by dataflow »

blooalien wrote: post #9
michaelchisari wrote: post #7

Earlier quoted context omitted.

Do we see actual signs of reasoning or is it anthropomorphism? We have an innate tendency to do so as humans.

> Do we see signs of reasoning or is it anthropomorphism? This is the part that so many folks just don't seem to understand (probably because it's been labeled as "thinking" or "reasoning" mode, and people assume that words have meaning). It's not reasoning or thought. It's spewing tokens pretending to "think", but it's actually just generating extra "context" to help the final answer be more coherent. The model isn'…

Honestly, people need to get over this debate. It's pretty irrelevant in a lot of cases. When people ask "what is the model thinking?", they're really asking "what caused the model to produce this response (as opposed to a bunch of other plausible ones)?"

Whether it's thinking or word prediction or whatever you want to call it, people are trying to understand the causal chain.

Re: Mechanistic interpretability researchers applying causality theory to LLMs

#24

Post by Leonard_of_Q »

blooalien wrote: post #9
michaelchisari wrote: post #7

Earlier quoted context omitted.

Do we see actual signs of reasoning or is it anthropomorphism? We have an innate tendency to do so as humans.

> Do we see signs of reasoning or is it anthropomorphism? This is the part that so many folks just don't seem to understand (probably because it's been labeled as "thinking" or "reasoning" mode, and people assume that words have meaning). It's not reasoning or thought. It's spewing tokens pretending to "think", but it's actually just generating extra "context" to help the final answer be more coherent. The model isn'…

You're describing a process by which a 'thinking' entity uses cognition to refine a solution to a stated problem. That's a lot of words so usually we shorten this to 'reasoning'.

Do LLMs 'think'? I 'think' they do in a way. I don't really know how I think myself but I know I do and therefore I am (thanks, Descartes). I have a somewhat better grasp of the way LLMs 'think'. They do so sequentially, building a chain of descriptors which best fit the problem and the preceding descriptors. I suspect I do something not entirely dissimilar- i.e. I imagine 'worlds' which are like the current one changed in some way so they the problem I'm working on is reduced, then refine those until it is resolved - but in a massively parallel way.

Re: Mechanistic interpretability researchers applying causality theory to LLMs

#25

Post by red75prime »

warumdarum wrote: post #18
dang wrote: post #61

[stub for offtopicness] [[All: please don't post shallow-generic reactions to baity titles. Those are basically the same thing, a la https://en.wikipedia.org/wiki/Rubin_vase , and we're trying for something more substantive here.]]

They dont. They have input that runs through a invisible stochastic canyon. As long as there is previous experience the stochastic canyon never ends. If there is none or isignificant one, or it runs out of tokkens, it hallucinates and the illusion falls apart. There is no reasoning, just the invisible grand canyon of all of human experience and knowledge. PS: try to get it to retell you a clichee movie or book and yo…

Stochastic gradient descent can be likened to traveling down a billion-dimensional canyon. But inference? Hardly.

Re: Mechanistic interpretability researchers applying causality theory to LLMs

#26

Post by wat10000 »

analog31 wrote: post #3
dang wrote: post #61

[stub for offtopicness] [[All: please don't post shallow-generic reactions to baity titles. Those are basically the same thing, a la https://en.wikipedia.org/wiki/Rubin_vase , and we're trying for something more substantive here.]]

Do LLMs have Qualia?

Do people?

Re: Mechanistic interpretability researchers applying causality theory to LLMs

#27

Post by wat10000 »

dataflow wrote: post #23
blooalien wrote: post #9

Earlier quoted context omitted.

> Do we see signs of reasoning or is it anthropomorphism? This is the part that so many folks just don't seem to understand (probably because it's been labeled as "thinking" or "reasoning" mode, and people assume that words have meaning). It's not reasoning or thought. It's spewing tokens pretending to "think", but it's actually just generating extra "context" to help the final answer be more coherent. The model isn'…

Honestly, people need to get over this debate. It's pretty irrelevant in a lot of cases. When people ask "what is the model thinking?", they're really asking "what caused the model to produce this response (as opposed to a bunch of other plausible ones)?" Whether it's thinking or word prediction or whatever you want to call it, people are trying to understand the causal chain.

Angry diatribes about whether submarines swim or not.

Re: Mechanistic interpretability researchers applying causality theory to LLMs

#28

Post by alchemist1e9 »

warumdarum wrote: post #18
dang wrote: post #61

[stub for offtopicness] [[All: please don't post shallow-generic reactions to baity titles. Those are basically the same thing, a la https://en.wikipedia.org/wiki/Rubin_vase , and we're trying for something more substantive here.]]

They dont. They have input that runs through a invisible stochastic canyon. As long as there is previous experience the stochastic canyon never ends. If there is none or isignificant one, or it runs out of tokkens, it hallucinates and the illusion falls apart. There is no reasoning, just the invisible grand canyon of all of human experience and knowledge. PS: try to get it to retell you a clichee movie or book and yo…

It’s curious how they solve unsolved math problems without reasoning. Maybe I have a different definition of reasoning than you.

Re: Mechanistic interpretability researchers applying causality theory to LLMs

#29

Post by 3848499449 »

ToValueFunfetti wrote: post #16
3848499449 wrote: post #10

Earlier quoted context omitted.

[flagged]

For the love of all that is sacred, please stop doing this. I'm begging you. The whole social media landscape is dying and you are creating a throwaway to participate in ruining this small corner. I assume this is not your first. And no one is convinced by this! The guidelines are there for your benefit as well. You achieve nothing but hastening the destruction of one of the last half-decent communities. Sorry for th…

[flagged]

Re: Mechanistic interpretability researchers applying causality theory to LLMs

#30

Post by emp17344 »

alchemist1e9 wrote: post #28
warumdarum wrote: post #18

Earlier quoted context omitted.

They dont. They have input that runs through a invisible stochastic canyon. As long as there is previous experience the stochastic canyon never ends. If there is none or isignificant one, or it runs out of tokkens, it hallucinates and the illusion falls apart. There is no reasoning, just the invisible grand canyon of all of human experience and knowledge. PS: try to get it to retell you a clichee movie or book and yo…

It’s curious how they solve unsolved math problems without reasoning. Maybe I have a different definition of reasoning than you.

Guess what? SAT solvers have also solved unsolved math problems. Do you believe they are “reasoning”?
Post reply on HN