Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Talking About Large Language Models

arxiv.org

Post reply on HN
Print view View original

21–30 of 158 posts

Re: Talking About Large Language Models

#21

Post by lachlan_gray »

gamegoblin wrote: post #5

Everyone pointing out how LLMs fail at some relatively simple tasks are fundamentally misunderstanding the utility of LLMs. Don't think of an LLM as a full "computer" or "brain". Think of it like a CPU. Your CPU can't run whole programs, it runs single instructions. The rest of the computer built around the CPU gives it the ability to run programs. Think of the LLM like a neural CPU whose instructions are relatively…

This makes me wonder if GPT could be any good at defining its own control flow. E.g. asking it to to write a python script that uses control structures along with calls to GPT to synthesize coherent content. Maybe it could give itself a kind of working memory.

Re: Talking About Large Language Models

#22

Post by dragonwriter »

hodgesrm wrote: post #17
nathan_compton wrote: post #2

This will hardly seem like a controversial opinion, but LLM are overhyped. Its certainly impressive to see the things people do with them, but they seem pretty cherry-picked to me. When I sat down with ChatGPT for a day to see if it could help me with literally any project I'm currently actually interested in doing it mostly failed or took so much prompting and fiddling that I'd rather have just written the code or d…

> This will hardly seem like a controversial opinion, but LLM are overhyped. As the [excellent] paper points out, LLMs are complex functions that can be embedded in systems to provide plausible answers to a prompt. Here's the money sentence. LLMs are generative mathematical models of the statistical distribution of tokens in the vast public corpus of humangenerated text, where the tokens in question include words, pa…

> Under what circumstances do LLMs give answers that are reliably equivalent to or better than humans?

Which humans? Humans give a... fairly wide range of responses.

Re: Talking About Large Language Models

#23

Post by solidasparagus »

nathan_compton wrote: post #8
alsodumb wrote: post #6

Earlier quoted context omitted.

I hope ChatGPT in its current form will not be used for search. As my friend says it, ChatGPT is not intelligent, it’s just capable of creating responses like it’s knows everything. The things it hallucinates is likely going to spread misinformation and make it harder for the masses to search for true, factual information. The other part is webtraffic: Google in theory could have created an interactive, conversationa…

I agree that I'd still rather use a search engine over a small set of sites than ChatGPT for exactly the reasons you suggest and others. But I don't see ChatGPT as having a lot of utility beyond functioning as a search interface for credulous dummies. I mean if I were literally developing a chatbot then clearly its a pretty interesting technology (assuming its problems can be tamed or censored somehow), but beyond th…

The problem you are running into is that you are overindexing on the fact that LLMs will sometimes be wrong and you are used to using technology that is basically always right. But we are in the early stages of LLM adoption - correctness will improve (see for example citation driven LLM-search) but more importantly, the set of LLM-driven applications that can be probabilistically correct and still wildly useful will grow.

LLMs like ChatGPT are just so damn cheap for the power they provide, it's inevitable

Re: Talking About Large Language Models

#24

Post by TeMPOraL »

nathan_compton wrote: post #16
TeMPOraL wrote: post #14

Earlier quoted context omitted.

Thing is, ChatGPT is already incredibly useful for searching random things you know enough about you can evaluate responses critically. The alternative here is doing a regular search, and wading through SEO-bloated, ad-laden content marketing "articles". The quality and reliability of information is about the same (or even favoring ChatGPT), but without 90% of the text that's just filler, without bullshit, ads, upsel…

Yeah, but its not as reliable as just restricting your search to Wikipedia or the appropriate academic journals or even chatting with a librarian!

Sure, when the topic matters or I need to study it in depth, I can still go to Wikipedia or PubMed or Arxiv.

But there are plenty of searches one does that are trivial, or serve to illuminate the problem space, and cover topics that in which I can rely on common sense to correct wrong advice. And the issue with non-technical topics, the kind applicable to mass audience - like e.g. cooking or parenting or hygiene - are very hard to search about online, because all results are bullshit pseudo articles written to drive traffic and deliver ads. So it's not that ChatGPT is so good, but more that Internet for normal people is complete trash, and ChatGPT nicely cuts straight through it.

Re: Talking About Large Language Models

#25

Post by TeMPOraL »

dragonwriter wrote: post #22
hodgesrm wrote: post #17

Earlier quoted context omitted.

> This will hardly seem like a controversial opinion, but LLM are overhyped. As the [excellent] paper points out, LLMs are complex functions that can be embedded in systems to provide plausible answers to a prompt. Here's the money sentence. LLMs are generative mathematical models of the statistical distribution of tokens in the vast public corpus of humangenerated text, where the tokens in question include words, pa…

> Under what circumstances do LLMs give answers that are reliably equivalent to or better than humans? Which humans? Humans give a... fairly wide range of responses.

Average humans? Within 1 stdev from the mean?

Re: Talking About Large Language Models

#26

Post by gamegoblin »

lachlan_gray wrote: post #21
gamegoblin wrote: post #5

Everyone pointing out how LLMs fail at some relatively simple tasks are fundamentally misunderstanding the utility of LLMs. Don't think of an LLM as a full "computer" or "brain". Think of it like a CPU. Your CPU can't run whole programs, it runs single instructions. The rest of the computer built around the CPU gives it the ability to run programs. Think of the LLM like a neural CPU whose instructions are relatively…

This makes me wonder if GPT could be any good at defining its own control flow. E.g. asking it to to write a python script that uses control structures along with calls to GPT to synthesize coherent content. Maybe it could give itself a kind of working memory.

Libraries such as https://github.com/hwchase17/langchain allow for easy programmatic pipelines of GPT "programs". So you could imagine taking a few hundred of these programs written by humans for various tasks, as are sure to come into existence in the next year or two, then adding those programs to the training data and training a new GPT that knows how to write programs that call itself.

Re: Talking About Large Language Models

#27

Post by nathan_compton »

hodgesrm wrote: post #17
nathan_compton wrote: post #2

This will hardly seem like a controversial opinion, but LLM are overhyped. Its certainly impressive to see the things people do with them, but they seem pretty cherry-picked to me. When I sat down with ChatGPT for a day to see if it could help me with literally any project I'm currently actually interested in doing it mostly failed or took so much prompting and fiddling that I'd rather have just written the code or d…

> This will hardly seem like a controversial opinion, but LLM are overhyped. As the [excellent] paper points out, LLMs are complex functions that can be embedded in systems to provide plausible answers to a prompt. Here's the money sentence. LLMs are generative mathematical models of the statistical distribution of tokens in the vast public corpus of humangenerated text, where the tokens in question include words, pa…

This reminds me that coding with ChatGPT felt like pair programming with a not super smart person who could google and type really fast. Not really fun!

Re: Talking About Large Language Models

#28

Post by armoredkitten »

nathan_compton wrote: post #2

This will hardly seem like a controversial opinion, but LLM are overhyped. Its certainly impressive to see the things people do with them, but they seem pretty cherry-picked to me. When I sat down with ChatGPT for a day to see if it could help me with literally any project I'm currently actually interested in doing it mostly failed or took so much prompting and fiddling that I'd rather have just written the code or d…

Please don't reduce LLM down to ChatGPT (or generative models more generally). People are using LLM for real-world problems every day. BERT and its descendants/variants are used all over the place for many different problems in natural language processing. I and my team have used it on dozens of different projects, mainly in classifying text documents and inputs. And it works very well. Multilingual LLMs are responsible for the huge improvements in machine translation; my team has to deal with text in multiple languages, and these models are vital there too. We have used LLM on real-world problems that are in production now and are saving hundreds of person-hours of tedious work.

ChatGPT? Yeah, it's neat. I'm sure people will find some useful niche for it. And I do think generative models will eventually have a big impact, once researchers find good ways to ground them to data and facts. This is already an active area of research -- combining generative LLMs with info retrieval methods, or targeting it to a specific context. (Meta just gave a talk last week at the NeurIPS conference about teaching a model to play Diplomacy, a game that mostly involves talking and negotiating deals with the other players. ChatGPT is too broad for that -- they just need a model that can talk about the state of the game board.) So in general, I'm optimistic about generative LLMs. But ChatGPT...is just a toy, really. It's not the solution -- it's one of the signposts along the way toward the real solution. It's a measure of progress.

Re: Talking About Large Language Models

#29

Post by anyonecancode »

RosanaAnaDana wrote: post #11

I like the discussion, but this article 'feels' like more Luddite goalpost moving, and is reflective of a continuous sentiment I feel strains so much of the conversation around intelligence, agentism, and ai going on today. I think that because we lack a coherent understanding of what it means to be intelligent at an individual level, as well as what it means to be an individual, we're missing much of the point of wh…

Increasingly I don't think the question of "what is intelligence" is so useful or relevant here. It feels a bit like arguing over whether the "artificial horse" that started appearing at the end of the 19th/beginning of the 20th C were actually horses. Cars weren't, and still aren't, but that misses the point.

AI isn't intelligent, and never will be, and I don't think that matters all that much.

Re: Talking About Large Language Models

#30

Post by hodgesrm »

dragonwriter wrote: post #22
hodgesrm wrote: post #17

Earlier quoted context omitted.

> This will hardly seem like a controversial opinion, but LLM are overhyped. As the [excellent] paper points out, LLMs are complex functions that can be embedded in systems to provide plausible answers to a prompt. Here's the money sentence. LLMs are generative mathematical models of the statistical distribution of tokens in the vast public corpus of humangenerated text, where the tokens in question include words, pa…

> Under what circumstances do LLMs give answers that are reliably equivalent to or better than humans? Which humans? Humans give a... fairly wide range of responses.

I'm a history major and love classical history. My first question to ChatGPT was:

> Why was Julius Caesar murdered?

The answer was the following, which would pass a standard exam question on the topic. It exhibits [the appearance of] multi-layer reasoning and has a nice conclusion.

> Julius Caesar was assassinated on the Ides of March in 44 BC by a group of Roman senators who believed he had become too powerful and posed a threat to the Roman Republic. The senators were concerned that Caesar had ambitions to become king, which would have ended the Republic and given him absolute power. The senators were also concerned about Caesar's growing popularity with the people of Rome. They believed that assassinating him was the only way to prevent him from becoming too powerful and destroying the Republic.

It's interesting to note that most of the evidence for this answer including 2000 years of interpretation is available in textual form on the Internet. It's easily accessible to LLMs.

Post reply on HN