Talking About Large Language Models • Hacker News Board

Re: Talking About Large Language Models

#11

Post by RosanaAnaDana » Sat, Dec 10, 2022, 5:24 PM UTC

I like the discussion, but this article 'feels' like more Luddite goalpost moving, and is reflective of a continuous sentiment I feel strains so much of the conversation around intelligence, agentism, and ai going on today.

I think that because we lack a coherent understanding of what it means to be intelligent at an individual level, as well as what it means to be an individual, we're missing much of the point of what's happening right now. The new line in the sand always seems to be justified based on an argument whose lyrics rhyme with identity, individual, self, etc. It seems like there will be no accepting of a thing that may have intelligence if there is no discernable individual involved. Chomsky is basically making the same arguments right now.

I think we'll see something that we can't distinguish from hard advanced general intelligence, prob in the next 3-5 years, and probably still have not made any real advancement into understanding what it means to be intelligent or what it means to be an individual.

Re: Talking About Large Language Models

#12

Post by monkmartinez » Sat, Dec 10, 2022, 5:24 PM UTC

nathan_compton wrote: post #2

This will hardly seem like a controversial opinion, but LLM are overhyped. Its certainly impressive to see the things people do with them, but they seem pretty cherry-picked to me. When I sat down with ChatGPT for a day to see if it could help me with literally any project I'm currently actually interested in doing it mostly failed or took so much prompting and fiddling that I'd rather have just written the code or d…

I disagree and think this is a very controversial opinion.

Playing around with it last night convinced me that LLM's are a huge, game changing technology. I was trying to decide which material to use for an upcoming project. The model doesn't use the internet without some hacking, so I had it write a program in python using the tkinter UI kit.

I asked it to create a UI with input boxes for material, weight of material, price and loss due to wastage. The program takes all of those inputs and converts the material into grams from KG, pounds, ounces. It then calculates the price per gram and takes a loss percentage (estimate given by user). It then writes a text file and saves it to a directory.

I literally pasted the code into VS code and had to change Tkinter to tkinter. Hit run and it worked flawlessly. I have NEVER used tkinter and it took about 30 minutes from start to finish.

This morning, I asked my 9th grade son what he is learning in 9th grade biology. He told me he is learning cellular endocytosis. I asked chapGPT to explain endocytosis like I was a 5 year old and read it to him... he says; "Ask it to explain it like a scientist now." After that he said it was a really good and we started asking it all kinds of biology questions.

I happen to agree that search will be the first thing disrupted. However, I think simply saying "search" doesn't come close to capturing how deep this will change the way we think, use and progress in terms of the way we define "search" right now.

Re: Talking About Large Language Models

#13

Post by sphinxster » Sat, Dec 10, 2022, 5:27 PM UTC

gamegoblin wrote: post #5

Everyone pointing out how LLMs fail at some relatively simple tasks are fundamentally misunderstanding the utility of LLMs. Don't think of an LLM as a full "computer" or "brain". Think of it like a CPU. Your CPU can't run whole programs, it runs single instructions. The rest of the computer built around the CPU gives it the ability to run programs. Think of the LLM like a neural CPU whose instructions are relatively…

Thank you for this interesting insight I haven't seen before.

Are there any datasets out there that provide the full edit stream of a human from idea to final refinement, that a model could be trained on?

Re: Talking About Large Language Models

#14

Post by TeMPOraL » Sat, Dec 10, 2022, 5:27 PM UTC

alsodumb wrote: post #6

nathan_compton wrote: post #2

This will hardly seem like a controversial opinion, but LLM are overhyped. Its certainly impressive to see the things people do with them, but they seem pretty cherry-picked to me. When I sat down with ChatGPT for a day to see if it could help me with literally any project I'm currently actually interested in doing it mostly failed or took so much prompting and fiddling that I'd rather have just written the code or d…

I hope ChatGPT in its current form will not be used for search. As my friend says it, ChatGPT is not intelligent, it’s just capable of creating responses like it’s knows everything. The things it hallucinates is likely going to spread misinformation and make it harder for the masses to search for true, factual information. The other part is webtraffic: Google in theory could have created an interactive, conversationa…

Thing is, ChatGPT is already incredibly useful for searching random things you know enough about you can evaluate responses critically. The alternative here is doing a regular search, and wading through SEO-bloated, ad-laden content marketing "articles". The quality and reliability of information is about the same (or even favoring ChatGPT), but without 90% of the text that's just filler, without bullshit, ads, upsells, tracking scripts, etc. I tried it a few times and it's a much better experience than the web. I'm gonna be using it for as long as it lasts.

Re: Talking About Large Language Models

#15

Post by nathan_compton » Sat, Dec 10, 2022, 5:29 PM UTC

monkmartinez wrote: post #12

nathan_compton wrote: post #2

This will hardly seem like a controversial opinion, but LLM are overhyped. Its certainly impressive to see the things people do with them, but they seem pretty cherry-picked to me. When I sat down with ChatGPT for a day to see if it could help me with literally any project I'm currently actually interested in doing it mostly failed or took so much prompting and fiddling that I'd rather have just written the code or d…

I disagree and think this is a very controversial opinion. Playing around with it last night convinced me that LLM's are a huge, game changing technology. I was trying to decide which material to use for an upcoming project. The model doesn't use the internet without some hacking, so I had it write a program in python using the tkinter UI kit. I asked it to create a UI with input boxes for material, weight of materia…

I've got a young kid and I'd think twice before letting this model explain any science to him. If your criteria for whether a model is good is "it fooled a 9th grader" well, I don't know what to tell you.

I think you have a point about your tkinter example. That kind of stuff _is_ a lot more convenient than googling and copying and pasting code. But if you push it beyond stuff that you could easily find on stack exchange or in documentation somewhere it doesn't work that well. Like I said, its a search engine with a lot of downsides and some upsides.

Re: Talking About Large Language Models

#16

Post by nathan_compton » Sat, Dec 10, 2022, 5:31 PM UTC

TeMPOraL wrote: post #14

alsodumb wrote: post #6

Earlier quoted context omitted.

I hope ChatGPT in its current form will not be used for search. As my friend says it, ChatGPT is not intelligent, it’s just capable of creating responses like it’s knows everything. The things it hallucinates is likely going to spread misinformation and make it harder for the masses to search for true, factual information. The other part is webtraffic: Google in theory could have created an interactive, conversationa…

Thing is, ChatGPT is already incredibly useful for searching random things you know enough about you can evaluate responses critically. The alternative here is doing a regular search, and wading through SEO-bloated, ad-laden content marketing "articles". The quality and reliability of information is about the same (or even favoring ChatGPT), but without 90% of the text that's just filler, without bullshit, ads, upsel…

Yeah, but its not as reliable as just restricting your search to Wikipedia or the appropriate academic journals or even chatting with a librarian!

Re: Talking About Large Language Models

#17

Post by hodgesrm » Sat, Dec 10, 2022, 5:31 PM UTC

nathan_compton wrote: post #2

This will hardly seem like a controversial opinion, but LLM are overhyped. Its certainly impressive to see the things people do with them, but they seem pretty cherry-picked to me. When I sat down with ChatGPT for a day to see if it could help me with literally any project I'm currently actually interested in doing it mostly failed or took so much prompting and fiddling that I'd rather have just written the code or d…

> This will hardly seem like a controversial opinion, but LLM are overhyped.

As the [excellent] paper points out, LLMs are complex functions that can be embedded in systems to provide plausible answers to a prompt. Here's the money sentence.

  LLMs are generative mathematical models of the statistical distribution
  of tokens in the vast public corpus of humangenerated text, where the tokens 
  in question include words, parts of words, or individual characters including 
  punctuation marks.

Rather than focus on the limitations of this approach to answer general queries, which are manifest, it seems more interesting to ask a different question. Under what circumstances do LLMs give answers that are reliably equivalent to or better than humans? The answer would:

1. Illuminate where we can use LLMs safely.

2. Direct work to make them better.

It's already impressive that within certain scopes ChatGPT gives very good answers, indeed better than most humans.

Re: Talking About Large Language Models

#18

Post by canjobear » Sat, Dec 10, 2022, 5:33 PM UTC

I’ll agree to stop saying LM’s “think” and “know” things if you can tell me precisely what those mean for humans.

Re: Talking About Large Language Models

#19

Post by CarbonCycles » Sat, Dec 10, 2022, 5:35 PM UTC

This paper and a recent post by Sebastian Raschka (where he decomposed a Forrester report about the uptake of technologies in industry) is alluding to something I have witnessed in system/control design and applied research.

Both LLMs and massive CV architectures are NOT the holistic solution. Rather, they are the sensors and edge devices that have now improved both the fidelity and reliability to a point where even more interesting things can happen.

I present a relevant use case regarding robotic arm manipulation. Before the latest SOTA CV algorithms were developed, the legacy technology couldn't provide the fidelity and feedback needed. Now, the embedded fusion of control systems, CV models, etc. we are seeing robotic arms that can manipulate and sort items previously deemed to be extremely difficult.

Research appears to follow the same pattern...observations and hypothesis that were once deemed too difficult or impossible at that time to validate are now common (e.g., Einstein's work with relativity).

My head is already spinning on how many companies and non-technical managers/executives are going to be sorely disappointed in the next year or two that Stable Diffusion, Chat GPT, etc. will deliver very little other than massive headaches for the legal, engineering, recruiting teams that will have to deal with this.

Re: Talking About Large Language Models

#20

Post by gamegoblin » Sat, Dec 10, 2022, 5:36 PM UTC

sphinxster wrote: post #13

gamegoblin wrote: post #5

Everyone pointing out how LLMs fail at some relatively simple tasks are fundamentally misunderstanding the utility of LLMs. Don't think of an LLM as a full "computer" or "brain". Think of it like a CPU. Your CPU can't run whole programs, it runs single instructions. The rest of the computer built around the CPU gives it the ability to run programs. Think of the LLM like a neural CPU whose instructions are relatively…

Thank you for this interesting insight I haven't seen before. Are there any datasets out there that provide the full edit stream of a human from idea to final refinement, that a model could be trained on?

REPL transcripts (i.e. bash sessions, python REPL, etc) tend to be pretty good demonstrations of "working up to a conclusion". And, not coincidentally, putting GPT in a REPL environment yields better results.

Other good examples narratives that include a lot of internal monologue. Thing a book written in the form:

> The sphinx asked him, "A ham sandwich costs $1.10. The ham costs $1 more than the bread. How much does the bread cost?"

> He thought carefully. He knew the sphinx asked tricky problems. If the ham costs a dollar more than the bread, the bread couldn't possibly be more than 10 cents. But if the bread was 10 cents, the ham would be $1.10 and the total would be $1.20. That can't be. We need to lose 10 cents, and it has to be divided evenly among the ham and bread to maintain the dollar offset. So the ham must be $1.05 and the bread must be $0.05. He answered the sphinx confidentally "The bread is $0.05!".