Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Talking About Large Language Models

arxiv.org

Post reply on HN
Print view View original

31–40 of 158 posts

Re: Talking About Large Language Models

#31

Post by hodgesrm »

armoredkitten wrote: post #28
nathan_compton wrote: post #2

This will hardly seem like a controversial opinion, but LLM are overhyped. Its certainly impressive to see the things people do with them, but they seem pretty cherry-picked to me. When I sat down with ChatGPT for a day to see if it could help me with literally any project I'm currently actually interested in doing it mostly failed or took so much prompting and fiddling that I'd rather have just written the code or d…

Please don't reduce LLM down to ChatGPT (or generative models more generally). People are using LLM for real-world problems every day. BERT and its descendants/variants are used all over the place for many different problems in natural language processing. I and my team have used it on dozens of different projects, mainly in classifying text documents and inputs. And it works very well. Multilingual LLMs are responsi…

I wouldn't undersell ChatGPT. It's like a repl for a particular LLM. Maybe there are others but it's the first time many people have gotten direct access to the technology. Sometimes the medium is the message.

Re: Talking About Large Language Models

#32

Post by macrolocal »

nathan_compton wrote: post #2

This will hardly seem like a controversial opinion, but LLM are overhyped. Its certainly impressive to see the things people do with them, but they seem pretty cherry-picked to me. When I sat down with ChatGPT for a day to see if it could help me with literally any project I'm currently actually interested in doing it mostly failed or took so much prompting and fiddling that I'd rather have just written the code or d…

Points taken, but LLMs are still outpacing expert predictions, so empirically they’re under-hyped.

Re: Talking About Large Language Models

#33

Post by btbuildem »

nathan_compton wrote: post #2

This will hardly seem like a controversial opinion, but LLM are overhyped. Its certainly impressive to see the things people do with them, but they seem pretty cherry-picked to me. When I sat down with ChatGPT for a day to see if it could help me with literally any project I'm currently actually interested in doing it mostly failed or took so much prompting and fiddling that I'd rather have just written the code or d…

It is very, very good with language, and very bad with facts and numbers. That's an oversimplification, but also the gist of it.

You have to recognize how it works, why it works - then you can use it as basically an incredible superpower force multiplier.

Re: Talking About Large Language Models

#34

Post by marcinzm »

nathan_compton wrote: post #15
monkmartinez wrote: post #12

Earlier quoted context omitted.

I disagree and think this is a very controversial opinion. Playing around with it last night convinced me that LLM's are a huge, game changing technology. I was trying to decide which material to use for an upcoming project. The model doesn't use the internet without some hacking, so I had it write a program in python using the tkinter UI kit. I asked it to create a UI with input boxes for material, weight of materia…

I've got a young kid and I'd think twice before letting this model explain any science to him. If your criteria for whether a model is good is "it fooled a 9th grader" well, I don't know what to tell you. I think you have a point about your tkinter example. That kind of stuff _is_ a lot more convenient than googling and copying and pasting code. But if you push it beyond stuff that you could easily find on stack exch…

> If your criteria for whether a model is good is "it fooled a 9th grader" well, I don't know what to tell you.

Fooling a 9th grader is amazing. That's a pretty well formed human being right there except with less life experience. Fundamentally no different from you in general reasoning terms except on a smaller set of information. So fooling you is merely a question of model size.

Re: Talking About Large Language Models

#35

Post by tshaddox »

nathan_compton wrote: post #2

This will hardly seem like a controversial opinion, but LLM are overhyped. Its certainly impressive to see the things people do with them, but they seem pretty cherry-picked to me. When I sat down with ChatGPT for a day to see if it could help me with literally any project I'm currently actually interested in doing it mostly failed or took so much prompting and fiddling that I'd rather have just written the code or d…

Maybe I’m just old, but there just isn’t much that I want to computers to tell me about that they don’t already do a decent job at. Everyone loves to complain about how bad Google search is, but I very rarely find myself desperately looking for something and unable to find it. There’s certainly no normal conversational interactions I can think of that I would love to have with a computer but have been unable to before ChatGPT and similar.

That limits how impressed I can be by ChatGPT and similar beyond just being impressed by it on a purely technical level. And it’s certainly very technically impressive, but not in some transcendental way. It’s also very impressive how could recent video games with ray tracing look, or how good computers are at chess, or how many really cool databases there are these days, or how fast computers can sort data.

Re: Talking About Large Language Models

#37

Post by Jack000 »

nathan_compton wrote: post #2

This will hardly seem like a controversial opinion, but LLM are overhyped. Its certainly impressive to see the things people do with them, but they seem pretty cherry-picked to me. When I sat down with ChatGPT for a day to see if it could help me with literally any project I'm currently actually interested in doing it mostly failed or took so much prompting and fiddling that I'd rather have just written the code or d…

LLMs may be overhyped, but transformers in general are underhyped.

LLMs make a lot of mistakes because they don't actually know what words mean. The key thing is though - it's much harder to generate coherent text when you don't know what the words mean. In a similar vein it's completely unreasonable to expect an LLM to perform visual tasks when it literally has no sense of sight.

The fact that it can kind of sort of do these things at all is evidence of the super-human generalization potential of the transformer architecture.

This isn't very obvious for English because we have prior knowledge of what words mean, but it's a lot more obvious when applied to languages humans don't understand, like DNA and amino acid sequences.

Re: Talking About Large Language Models

#38

Post by btbuildem »

gamegoblin wrote: post #5

Everyone pointing out how LLMs fail at some relatively simple tasks are fundamentally misunderstanding the utility of LLMs. Don't think of an LLM as a full "computer" or "brain". Think of it like a CPU. Your CPU can't run whole programs, it runs single instructions. The rest of the computer built around the CPU gives it the ability to run programs. Think of the LLM like a neural CPU whose instructions are relatively…

Very well put! Having played with it for a week straight, I've come to a similar observation -- it's a generator engine, with a "soft" interface. You still have to have skill and understanding to use it effectively, but it's a great force multiplier, because it removes the friction around the initial interactions.

If you're solving a complex problem, you cannot expect it to "reason" about it. You have to break the problem into simpler pieces, then you can have the LLM do the grunt work for each piece.

Re: Talking About Large Language Models

#39

Post by skybrian »

There's a way to anthropomorphize large language models that I think is less misleading: they are like a well-read actor that always "wants" to play "let's pretend." LLM's are trained on "fill in the blank" which means they follow the "yes, and" rule of improv. They are very willing to follow your lead and to assume whatever role is necessary to play their part.

If you give them hints about what role you want by asking leading questions, they will try to play along and pretend to hold whatever opinions you might want from them.

What are useful applications for this sort of actor? It makes sense that language translation works well because it's pretending to be you, if you could speak a different language. Asking them to pretend to be a Wikipedia article without giving them the text to imitate is going to be hit and miss since they're just as willing to pretend to be a fake Wikipedia article, as they don't know the difference.

Testing an LLM to find out what it believes is unlikely to do anything useful. It's going to pretend to believe whatever is consistent with the role it's currently playing, and that role may be chosen randomly if you don't give it any hints.

It can be helpful to use prompt engineering to try to nail down a particular role, but like in improv, that role is going to drift depending on what happens. You shouldn't forget that whatever the prompt, it's still playing "let's pretend."

Re: Talking About Large Language Models

#40

Post by albystein »

alsodumb wrote: post #6
nathan_compton wrote: post #2

This will hardly seem like a controversial opinion, but LLM are overhyped. Its certainly impressive to see the things people do with them, but they seem pretty cherry-picked to me. When I sat down with ChatGPT for a day to see if it could help me with literally any project I'm currently actually interested in doing it mostly failed or took so much prompting and fiddling that I'd rather have just written the code or d…

I hope ChatGPT in its current form will not be used for search. As my friend says it, ChatGPT is not intelligent, it’s just capable of creating responses like it’s knows everything. The things it hallucinates is likely going to spread misinformation and make it harder for the masses to search for true, factual information. The other part is webtraffic: Google in theory could have created an interactive, conversationa…

The problem of hallucination in LLMs is a well-known and studied problem and solutions have been proposed to counter it. The most promising one is augmenting LLMs with a retrieval system. This involves sourcing a large database of factual information, say journal articles, over which the LLM uses an information retrieval system(search engine) to extract information on which its generated output is conditioned. Recent job postings from OpenAI suggest that’s their next step of development for these LLMs.

I think critics of these LLMs are missing the point about the excitement around them. People are excited because of the rate of progress/improvement from just two years or a year ago. These systems have come a long way, and if you extrapolate that progress into the future, I predict majority of these shortcomings getting resolved

Post reply on HN