Talking About Large Language Models • Hacker News Board

Re: Talking About Large Language Models

#61

Post by genidoi » Sat, Dec 10, 2022, 8:55 PM UTC

nathan_compton wrote: post #2

This will hardly seem like a controversial opinion, but LLM are overhyped. Its certainly impressive to see the things people do with them, but they seem pretty cherry-picked to me. When I sat down with ChatGPT for a day to see if it could help me with literally any project I'm currently actually interested in doing it mostly failed or took so much prompting and fiddling that I'd rather have just written the code or d…

The difference in wether you think ChatGPT is game changing or another overhyped LLM seems to come down to:

1) do you acknowledge prompt engineering is a real skill set?

2) are you willing to improve your prompt engineering skill set through research and iteration?

There is much to learn about prompt engineering from that “Linux VM in ChatGPT” post and other impressive examples (where the goal of is to constrain ChatGPT to only engage in a specific task)

Re: Talking About Large Language Models

#62

Post by joe_the_user » Sat, Dec 10, 2022, 9:01 PM UTC

gillesjacobs wrote: post #59

I am NLP researcher who volunteers for peer review often and the anthropomorphisms in papers are indeed very common and very wrong. I have to ask authors to not ascribe cognition to their deep learning approaches in about a third of the papers I review. People do this because mirroring cognition to machine learning lends credence that their specific modeling mechanism mimicks human understanding and so is closer "to…

People do this because mirroring cognition to machine learning lends credence that their specific modeling mechanism mimicks human understanding and so is closer "to the real thing".

Doesn't this also involve people not having another category aside from "cognition" to put natural language processing acts in? How many neural net constructors have a rigorously developed framework describing what "cognition" is?

I mean, there's a common counter argument to the "this is not cognition" position. That is: "you're just using 'cognition' as a placeholder for whatever these systems can't do". I don't think that counter-argument is true or characterizes the position well but it's important to frame one's position so it doesn't seem to be subject to this counter-argument.

Re: Talking About Large Language Models

#63

Post by lambdatronics » Sat, Dec 10, 2022, 9:07 PM UTC

visarga wrote: post #56

nathan_compton wrote: post #10

Earlier quoted context omitted.

It can reproduce a statistically plausible paragraph, certainly. But there is a great deal more to research than producing statistically plausible paragraphs. It doesn't _understand_ anything! I've actually worked on a project where there have been attempts to use GPT like models to summarize scientific results and the problem is it gets shit wrong all the time! You have to be an expert to separate the wheat from the…

The problem is that we need to pair generative models with verification systems. We have the models, but no verification yet. Fortunately code and math are easier to verify. Some things require simulation. In other cases you can substitute an ensemble of solutions & picking the most frequent answer as consistency based verification. But for each domain we need to create verifiers and that will take some time. The goo…

>we need to pair generative models with verification systems >code and math are easier to verify

I would love to see a two-stage pipeline using a LLM to convert natural language specifications into formal specifications for something like Dafny, and then follow up with another model like AlphaZero that would generate code & assertions to help the verifier. This seems like something that a major group like DeepMind or OpenAI could pull off in a few years.

Re: Talking About Large Language Models

#64

Post by lambdatronics » Sat, Dec 10, 2022, 9:14 PM UTC

RosanaAnaDana wrote: post #11

I like the discussion, but this article 'feels' like more Luddite goalpost moving, and is reflective of a continuous sentiment I feel strains so much of the conversation around intelligence, agentism, and ai going on today. I think that because we lack a coherent understanding of what it means to be intelligent at an individual level, as well as what it means to be an individual, we're missing much of the point of wh…

Edsger Dijkstra: "The question of whether Machines Can Think (…) is about as relevant as the question of whether Submarines Can Swim."

Re: Talking About Large Language Models

#65

Post by gillesjacobs » Sat, Dec 10, 2022, 9:40 PM UTC

joe_the_user wrote: post #62

gillesjacobs wrote: post #59

I am NLP researcher who volunteers for peer review often and the anthropomorphisms in papers are indeed very common and very wrong. I have to ask authors to not ascribe cognition to their deep learning approaches in about a third of the papers I review. People do this because mirroring cognition to machine learning lends credence that their specific modeling mechanism mimicks human understanding and so is closer "to…

People do this because mirroring cognition to machine learning lends credence that their specific modeling mechanism mimicks human understanding and so is closer "to the real thing". Doesn't this also involve people not having another category aside from "cognition" to put natural language processing acts in? How many neural net constructors have a rigorously developed framework describing what "cognition" is? I mean…

> Doesn't this also involve people not having another category aside from "cognition" to put natural language processing acts in?

Yes, of course this might be an even more primary reason; do not attribute to malice what can be explained by laziness. However, AI researchers should be wary of their language, that point is hammered in most curricula I have seen. So at the least it is negligence.

> I mean, there's a common counter argument to the "this is not cognition" position. That is: "you're just using 'cognition' as a placeholder for whatever these systems can't do".

Very valid point, but we know current deep learning mechanisms do not mimick human learning, language understanding and production in any way. They are far too simplified and specific for that.

Neural network activation functions are a far cry from neural spiking models and biological neural connectivity is far more complex than the networks used in deep learning. The attention mechanism that drives recent LLMs is also claimed to have some biological similarities, but upon closer inspection drawing strong analogies is not credible [1]. computer vs. human visual recognition tasks it falls apart and higher-level visual concepts. [2]

1. https://www.frontiersin.org/articles/10.3389/fncom.2020.0002...

2. https://arxiv.org/abs/1906.08764

Re: Talking About Large Language Models

#66

Post by schizo89 » Sat, Dec 10, 2022, 9:43 PM UTC

The paper discusses how these models operate and state that they're only predict next series of token while somehow human intelligence works otherwise. The marxist ideology has the law of the transformation of quantity into quality and vice versa — which was formed in 19th century and performance of these models is just another proof of it. I would argue that _emerging_ mechanics in AI models that we see with increased size of models is no different than how our mind works. It's about emergence of intelligence in complex systems — and that a materialist worldview central to the science.

Re: Talking About Large Language Models

#67

Post by hackinthebochs » Sat, Dec 10, 2022, 9:45 PM UTC

goatlover wrote: post #49

canjobear wrote: post #18

I’ll agree to stop saying LM’s “think” and “know” things if you can tell me precisely what those mean for humans.

Maybe there isn't a precise definition, but clearly for humans thinking and knowing is related to having bodies that need to survive in the world with other humans and organisms, which involves communication and references to external and internal things (how your body feels and what not). This is different from pattern matching tokens, even if it reproduces a lot of the same results, because human language creates a…

>This is different from pattern matching tokens

But is it different in essential ways? This is not so clear. Humans developed the capacity to learn, think, and communicate in service to optimizing an objective function, namely fitness in various environments. But there is an analogous process going on with LLMs; they are constructed such that they maximize an objective function, namely predict the next token. But it is plausible that "understanding" and/or "intelligence" is within the solution-space of such an optimization routine. After all, it's not like "intelligence" was explicitly trained for in the case of humans. Nature has already demonstrated emergent function as a side-effect of an unrelated optimizer.

Re: Talking About Large Language Models

#68

Post by sarchertech » Sat, Dec 10, 2022, 9:47 PM UTC

radford-neal wrote: post #54

marcinzm wrote: post #51

Earlier quoted context omitted.

You're basically complaining that a single model doesn't have full knowledge of every single area of all of human knowledge. It's got decent knowledge of most areas including programming with probably better overall knowledge than a high school student. That's downright amazing and probably more knowledge than any single human actually has. The rest is likely a matter of improvement along the same lines versus some r…

Well, I agree that it's amazing - it almost always produces grammatical output, for instance. But it's not a reliable way of obtaining knowledge. One should not, in particular, try to learn about biology by asking ChatGPT questions. It often produces made-up stuff that is just wrong. And it's very confidently wrong, with the output often coming across like someone barely concealing their contempt that you might doubt…

My wife (a physician) asked it multiple medical questions and the majority of the time they were dangerously wrong, but looked perfectly fine to me.

I asked it a series of questions about my area of expertise and they were wrong but looked perfectly fine to my wife.

It even confidently “solved” the 2 generals problem with a solution that looks completely plausible if you don’t already know that it won’t work.

Re: Talking About Large Language Models

#69

Post by fourfivefour » Sat, Dec 10, 2022, 9:50 PM UTC

gillesjacobs wrote: post #59

I am NLP researcher who volunteers for peer review often and the anthropomorphisms in papers are indeed very common and very wrong. I have to ask authors to not ascribe cognition to their deep learning approaches in about a third of the papers I review. People do this because mirroring cognition to machine learning lends credence that their specific modeling mechanism mimicks human understanding and so is closer "to…

bias infests research as well as seen by the replication crisis. So you being a researcher doesn't give more credence to your words especially given that the state of current research cannot fully comprehend what these ML models are doing internally.

I do agree that we can't ascribe cognition to machine learning.

But I also believe that we can't ascribe that it's NOT cognition. Why? Because we don't even truly understand what "Knowing" or cognition is. We can't even ascribe a quantitative similarity metric.

What we are seeing is that those inputs and outputs look remarkably similar to the real thing. How similar it is internally is not a known thing.

That's why even though you're an NLP researcher, I still say your argument here is just as niave as the person who claims these things are sentient. You simply don't know. No one does.

Re: Talking About Large Language Models

#70

Post by gillesjacobs » Sat, Dec 10, 2022, 9:53 PM UTC

fourfivefour wrote: post #69

gillesjacobs wrote: post #59

I am NLP researcher who volunteers for peer review often and the anthropomorphisms in papers are indeed very common and very wrong. I have to ask authors to not ascribe cognition to their deep learning approaches in about a third of the papers I review. People do this because mirroring cognition to machine learning lends credence that their specific modeling mechanism mimicks human understanding and so is closer "to…

bias infests research as well as seen by the replication crisis. So you being a researcher doesn't give more credence to your words especially given that the state of current research cannot fully comprehend what these ML models are doing internally. I do agree that we can't ascribe cognition to machine learning. But I also believe that we can't ascribe that it's NOT cognition. Why? Because we don't even truly unders…

In science, if you don't know, you don't make the claim, that is basic positivism and the scientific method.

So basic in fact, I was thought this in elementary school. So far ad-hominem attributions of naivety.

Anyone that humanises computation is not only committing an A.I. faux-pas but are going against the basic scientific method.