Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Talking About Large Language Models

arxiv.org

Post reply on HN
Print view View original

141–150 of 158 posts

Re: Talking About Large Language Models

#141

Post by mannykannot »

kirkules wrote: post #128
mannykannot wrote: post #120

Earlier quoted context omitted.

People have been asking grand but poorly-posed questions for millennia, and sometimes this leads to well-posed questions that elicit remarkable answers. What is poorly-posed about the question 'could a machine think'? 'Machine' seems acceptably well-defined, and not in a way that rules out, a priori, the possibility of any machine being able to think, so I'm guessing the problem lies in us not having a good definitio…

Yes, the issue is that word, "think", and its variants. But "what is thinking?" isn't poorly posed in the case that "thinking" isn't well defined, because it is about establishing and agreeing on a definition. The approach some people take to answering "could machines think?" is to try to determine an actual subject in the real world to call "thinking" in a way that aligns with common intuitions, which to me seems li…

I certainly agree that there's a lot of low-quality debate around this and similarly open-ended questions. I think your objection can be made even within the philosophy of the mind (though you may not agree with me!) This is a significant branch of philosophy, and within it, hundreds if not thousands of papers have been written in attempting to prove that nothing operating by the laws of physics as we know them today, nor any future extension of them, could have a mind - yet those who have been trying to do this are no better off than anyone else in knowing what a mind is.

Coming back to Dijkstra's aphorism, you will notice that I was objecting to interpreting it as ruling out, a priori, the question of whether machines could ever think. From your latest post, however, you seem to be saying it would be a reasonable question once we are equipped with an established and agreed-upon definition of thinking.

Re: Talking About Large Language Models

#142

Post by jerrygenser »

armoredkitten wrote: post #28
nathan_compton wrote: post #2

This will hardly seem like a controversial opinion, but LLM are overhyped. Its certainly impressive to see the things people do with them, but they seem pretty cherry-picked to me. When I sat down with ChatGPT for a day to see if it could help me with literally any project I'm currently actually interested in doing it mostly failed or took so much prompting and fiddling that I'd rather have just written the code or d…

Please don't reduce LLM down to ChatGPT (or generative models more generally). People are using LLM for real-world problems every day. BERT and its descendants/variants are used all over the place for many different problems in natural language processing. I and my team have used it on dozens of different projects, mainly in classifying text documents and inputs. And it works very well. Multilingual LLMs are responsi…

Do you feel like for some tasks though that people might be too eager to reach for GPU and more expensive inference and training costs?

In some projects I'm working on, I'm seeing close to 95F1 score using deep learning to do token classification based NER. However, using non-deep learning approaches (still training statistical models) I can get to 91F1 on my use case but have a much faster inference and not need to use a GPU.

I'm similarly optimistic about LLM, but fear that a lot of other really useful workhouse algorithms and strategies are going to be pushed aside and people will forget about them.

Back to my use case, some strategies I'm looking at are using cheap and fast CPU powered models/smaller models for inference and then based on certain signals decide whether a particular instance should be passed to a GPU based model for better accuracy.

Re: Talking About Large Language Models

#143

Post by eternalban »

sdenton4 wrote: post #136
gillesjacobs wrote: post #65

Earlier quoted context omitted.

> Doesn't this also involve people not having another category aside from "cognition" to put natural language processing acts in? Yes, of course this might be an even more primary reason; do not attribute to malice what can be explained by laziness. However, AI researchers should be wary of their language, that point is hammered in most curricula I have seen. So at the least it is negligence. > I mean, there's a comm…

Why does nlp need to mimic human brain function to count as cognition? This seems overly reductive; I see no reason to believe that biomimicry is necessary for intelligence. The argument is overly reductive: only brains think, and this is not a brain, ergo it does not think. (To be clear, I don't think this system is an AGI; just making the point that better goal posts are needed...)

Even if matter other than brains were capable of thinking, that does not ipso facto bestow the title 'thinking thing' onto things which are not brains. All we can say is "well, it is certainly not a brain!". That is it.

Also, computation is not cognition. We need to focus on what distinguishes cognition from computation, if indeed these are distinct. I feel we are using the wrong words and thus having unproductive conversations regarding this topic (in general).

Re: Talking About Large Language Models

#144

Post by armoredkitten »

hodgesrm wrote: post #31
armoredkitten wrote: post #28

Earlier quoted context omitted.

Please don't reduce LLM down to ChatGPT (or generative models more generally). People are using LLM for real-world problems every day. BERT and its descendants/variants are used all over the place for many different problems in natural language processing. I and my team have used it on dozens of different projects, mainly in classifying text documents and inputs. And it works very well. Multilingual LLMs are responsi…

I wouldn't undersell ChatGPT. It's like a repl for a particular LLM. Maybe there are others but it's the first time many people have gotten direct access to the technology. Sometimes the medium is the message.

That's fair -- perhaps we could frame it as a large-scale beta test of sorts. Researchers are building LLMs to solve problems, but new technologies can often end up solving problems they were never designed for. Once people get their hands on them, they test and tinker and find new uses for them. Sometimes it turns out not to be a good solution to the initial problem, but a great solution for something completely different.

For instance, while I'm still generally of the opinion that generative models have limited use unless they're grounded to reality...I did see a post on Reddit about someone using ChatGPT to generate story ideas for their D&D game. So yeah...don't need to be tethered to reality to make a fantasy story! That's not something I would have thought of (even though I'm a DM!), and it's still relatively niche, but it's a great story of how getting something into people's hands to play with can generate lots of new ideas.

Re: Talking About Large Language Models

#145

Post by armoredkitten »

jerrygenser wrote: post #142
armoredkitten wrote: post #28

Earlier quoted context omitted.

Please don't reduce LLM down to ChatGPT (or generative models more generally). People are using LLM for real-world problems every day. BERT and its descendants/variants are used all over the place for many different problems in natural language processing. I and my team have used it on dozens of different projects, mainly in classifying text documents and inputs. And it works very well. Multilingual LLMs are responsi…

Do you feel like for some tasks though that people might be too eager to reach for GPU and more expensive inference and training costs? In some projects I'm working on, I'm seeing close to 95F1 score using deep learning to do token classification based NER. However, using non-deep learning approaches (still training statistical models) I can get to 91F1 on my use case but have a much faster inference and not need to…

I don't know if I'm too concerned about that, to be honest. Yeah, there's a huge cost in terms of training the LLMs, and then there can be a cost for downstream inference, but I think it depends on the use case. In some cases, performance is the absolute top priority; in other cases, you might be willing to trade off some performance for better inference time, or model size, etc. If you need to put the model on cell phones or offline low-power devices, that's a key constraint that might make you reach for a different tool.

The nice thing about more "classical" approaches -- a simple BoW random forest or MLP, for example -- is that they're typically quick to train and experiment with, and they make for great baselines, if nothing else. So I doubt that we're in danger of people forgetting about them entirely. If people do, they're leaving quick, easy solutions on the table.

I do like your idea about triaging inference between smaller CPU vs. larger GPU models based on whatever signals. I haven't tried that before, but a project my colleagues worked on did some triaging between regex pattern-matching vs. model inference. Basically, the regex pulled some of the data out first if it matched very specific, known patterns, and then the rest was handled probabilistically. I guess the effectiveness of that sort of triaging approach depends on how strong and clear your signals are that let you choose one path over the other.

Re: Talking About Large Language Models

#146

Post by oneoneonetwo »

gillesjacobs wrote: post #139
oneoneonetwo wrote: post #130

Earlier quoted context omitted.

I also want to mention that, you directly stated in your first sentence in your first post that humanizing was wrong. This in itself is a claim made without evidence. Which is my point. The claim as it stands cannot be made either way. We simply don't know.

The counterargument stems on a fundamental misunderstand of what it means to make assertive claims in science: you always need to prove your positive claim, saying that we do not know that ML models are human-like requires no evidence because this is the zero hypothesis. Using humanising language is equivalent to attributing human-like cognition to ML models. Unless there is very strong evidence that there are analog…

No post body was provided.

Re: Talking About Large Language Models

#147

Post by oneoneonetwo »

hnnotreddit wrote: post #137
oneoneonetwo wrote: post #129

Earlier quoted context omitted.

>while continuously insulting the GP. Read the conversation. You are accusing me of something vile which the GP started. GP Literally said that science was something elementary school level shit implying that I was so damn stupid I was uneducated. You're only taking his side because You agree with him, you are not seeing who drew first blood and you are burning the witch because it serves your own agenda. He insulted…

> Did you not see the quote from Einstein? > The only reason why I made a new account is because I wanted to comment here. Extremely petty, insubstantial and toxic, you have nothing insightful to add.

No post body was provided.

Re: Talking About Large Language Models

#148

Post by wellpast »

doliveira wrote: post #134
wellpast wrote: post #111

Earlier quoted context omitted.

Physics is a set of things we understand about the universe and often used to include some mystifying things, i.e. that we don't quite understand (eg Bell's theorem). To just claim that Physics has to do with _more_ than this - that it has to do with all the things we don't understand is just blind-faith ascription. It's a nearly meaningless statement, really. So it's bizarre or voodoo to say it so casually and assur…

Physics, by definition, has to do with everything, well, physical going on in this Universe. It's not about whether we understand it or not, or whether we're comfortable with it. So yeah, if you claim conscience is not physical, then you're talking about metaphysics. Which is okay to do, but then claiming that those that say that Physics alone is probably enough are saying voodoo... Well, it's ironic.

By whose definition?

You don't _have_ to define it that way. To _choose_ a definition that ropes _everything_ under "Physics" is an unnecessary move, it's more magic trick than grounded derivation.

Re: Talking About Large Language Models

#149

Post by aeternum »

nathan_compton wrote: post #2

This will hardly seem like a controversial opinion, but LLM are overhyped. Its certainly impressive to see the things people do with them, but they seem pretty cherry-picked to me. When I sat down with ChatGPT for a day to see if it could help me with literally any project I'm currently actually interested in doing it mostly failed or took so much prompting and fiddling that I'd rather have just written the code or d…

Understanding the lifetime/scope of variables is not easy, many experienced programmers get it wrong, especially when moving between languages.

Re: Talking About Large Language Models

#150

Post by nathan_compton »

mannykannot wrote: post #119
nathan_compton wrote: post #2

This will hardly seem like a controversial opinion, but LLM are overhyped. Its certainly impressive to see the things people do with them, but they seem pretty cherry-picked to me. When I sat down with ChatGPT for a day to see if it could help me with literally any project I'm currently actually interested in doing it mostly failed or took so much prompting and fiddling that I'd rather have just written the code or d…

LLMs are polarizing: while there is a lot of hype from some quarters, there's also a faction that seems only interested in dismissing them as the same old something-or-other. What I personally find most interesting about them is what seems to me to be their unreasonable effectiveness, despite their flaws and limitations, and what that might tell us about ourselves. The more one stresses how simple (conceptually) thei…

Well, my take here is that LLMs are do about half of what makes human intelligence work well: which is recall tons of information using relatively broad and imprecise queries or reminders. And even this they don't really do: humans are quite capable of introspecting about what they know and this seems to be a challenge for this class of model. A challenge I don't see this architecture really easily addressing since, in the end, its just a big markov model with chunking.

But anyway, the other part of the equation is missing here. When a human encounters a novel phenomenon about which they know little they can engage an entire separate system: one which _reasons_ about the system, uses principals and intuitions and iteration to produce new knowledge. That is the thing missing from LLMs: they don't ever really produce new knowledge, although they may reveal correlations between texts that people have yet to notice.

I'm not fundamentally skeptical about whether artificial intelligence will ever get there. In fact, the progress of LLMs has me wondering if its not going to be sooner rather than later. But at this moment, I feel quite confident saying that LLMs are just knowledge retrieval systems with some pretty undesirable properties (and some pretty interesting ones).

My _hunch_ is that the next step this is going to be like self driving cars, though: technology which appears stubbornly _just out of reach_ for an indeterminate amount of time.

Post reply on HN