Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Simply explained: How does GPT work?

confusedbit.dev

Post reply on HN
Print view View original

331–340 of 392 posts

Re: Simply explained: How does GPT work?

#331

Post by brazzy »

JieJie wrote: post #273
brazzy wrote: post #192

Earlier quoted context omitted.

How can an disagreement over GPT possibly be something to end a friendship over? This sounds like you said "I cannot possibly be friends with someone who does not believe that LLMs are emerging AGI!", and people read it like that and are downvoting you. I'm gonna assume the situation was more complex, but still find it hard to imagine, how a disagreement over such an academic topic could end up destroying a friendshi…

It was a complex situation to be sure. I only shared the story to illustrate how personally people are taking these discussions. I really felt like I was being very neutral and just sharing my enthusiasm. It was entirely unwelcome, apparently. If there's a lesson to be learned it's that people's tempers over these issues may be hotter than they appear. I can barely speak with my artist friends on the issue these days…

If are is a major part of someone's identity (strongly implied by calling them an "artist"), then it's hardly surprising that they would feel threatened and react emotionally to something that massively disrupts the creation of art and the viability of making a living from it.

Re: Simply explained: How does GPT work?

#332

Post by danenania »

staticman2 wrote: post #327
danenania wrote: post #257

Earlier quoted context omitted.

My point is that's not a debate anyone is having. No one claims that ChatGPT is human! The claim is merely that ChatGPT is engaging in (non-human) forms of reasoning, abstraction, creativity, and so on, with varying levels of ability. There's a separate debate on whether the brain produces human thoughts in a similar way to ChatGPT's non-human thought. The question here is whether brains are essentially biological LL…

The word "thought" means something. When you use it to describe ChatGPT, you have in fact argued "there's no fundamental difference between humans and LLMs."

That presupposes that the only thought that exists or can exist is human thought. You can define it that way if you like, but it’s not the only definition.

Re: Simply explained: How does GPT work?

#333

Post by majormajor »

iliane5 wrote: post #320
majormajor wrote: post #258

Earlier quoted context omitted.

We've known for a long time that computers can do calculations far, far, far faster than us. We continue to figure out new ways to make those calculations do more complicated things faster than humans. What is intelligence beyond calculation is an ancient question, but not the one I'm most interested in at the moment, re: today's tools. I'm curious right now about if there's meaning to other people in human creation…

I think as soon as text2video gets really good (like midjourney level), there’s gonna be so much AI generated content that unless it’s all extremely good, human made content will be something people search specifically for. As for curation, I think the success of TikTok proves that you don’t need that much data to pretty preceding pinpoint what someone wants to watch (or what will get them to spend the most time on t…

Do you mean with humans generating the prompts or with some sort of no-human-in-the-loop "generate the text prompt to generate the video" automation?

I think a super accessible animation tool would get a lot of use and result in a lot of cool stuff, but it's the latter that I'm really curious about in terms of how people interact with it.

Re: Simply explained: How does GPT work?

#334

Post by unethical_ban »

EMM_386 wrote: post #118
seydor wrote: post #71

Earlier quoted context omitted.

Indeed the word 'attention' implies some active process, which is also unfortunate, because it is really similarity maps over long ranges. This process apparenty creates a cross-reference maps of items that are commonly co-activated. But then what do the different heads do? Some kind of different "modalities of cross-reference"? I struggle to understand why this thing works the way it does. It's possible that Vaswani…

> I struggle to understand why this thing works the way it does. I'm not in this field but have recently found myself going on the deepest dive possible into it as my small brain can absorb. I now know about (on a surface level) neural networks, transformers, attention mechanisms, vectors, maticies, tokenization, loss functions and all sorts of other crazy stuff. I come out of this realizing that there are some incre…

Got a good Youtube list? Other than the HN threads and submissions I can look up this weekend.

Re: Simply explained: How does GPT work?

#335

Post by giardini »

simonh wrote: post #308
int_19h wrote: post #253

Earlier quoted context omitted.

Thing is, they can still solve the problem , even if the problem was not one from its training set. And, more importantly, they solve the problem much better if you tell them to reason about it in writing first before giving the final answer.

Yes I know, as I said they are very knowledgeable and in some ways very intelligent. We just need to bear in mind their processing architecture is radically different from our. This makes our intuitions about their abilities highly error prone.

simonh says >"We just need to bear in mind their processing architecture is radically different from ours."

The hardware architectures are certainly different but there is a possibility that at least parts of the "software" architectures may be remarkably similar.

Time and research will tell.

Re: Simply explained: How does GPT work?

#336

Post by unethical_ban »

agentultra wrote: post #29

A good article and well articulated! I would change the introduction to be more impartial and not anthropomorphize GPT. It is not smart and it is not skilled in any tasks other than that for which it is designed. I have the same reservations about the conclusion. The whole middle of the article is good. But to then compare the richness of our human experience to an algorithm that was plainly explained? And then to sp…

This site talks *constantly* about the impact of the Internet, the Information Age and social media on society, and its negative effects especially in the case of the third. I think being incredibly concerned about societal stability in the face of such rapid dissolving of our ability to trust our senses, combined with social media, is a valid response.

Re: Simply explained: How does GPT work?

#337

Post by giardini »

simonh wrote: post #314
opportune wrote: post #261

Earlier quoted context omitted.

I don’t think there’s anything making it impossible for actual intelligence to arise from a task as simple as “predicting the next token (to model human thought/speech/writing)” because with enough compute resources, smart AI implementations, and training that task basically would be optimized by becoming a general intelligence. But it’s clear based on current implementations that once you work backwards from the kno…

The problem is such an approach is limited by the content of the training texts. As I mentioned elsewhere, our written texts assume huge swathes of contextual and experiential information and knowledge that LLMs don’t have. It’s possible some of it might be inferred from the texts, but not all of it by a long shot. If somehow you could generate a training text encoding a complete and thorough understanding of the phy…

simonh says >"The problem is such an approach is limited by the content of the training texts."

Aside: I would like to see ChatGPTs with distinct training texts, e.g., a ChatGPTs trained on the "great books" of Western philosophy and science knowledge up to the time of Victorian England.

Re: Simply explained: How does GPT work?

#338

Post by giardini »

JieJie wrote: post #273
brazzy wrote: post #192

Earlier quoted context omitted.

How can an disagreement over GPT possibly be something to end a friendship over? This sounds like you said "I cannot possibly be friends with someone who does not believe that LLMs are emerging AGI!", and people read it like that and are downvoting you. I'm gonna assume the situation was more complex, but still find it hard to imagine, how a disagreement over such an academic topic could end up destroying a friendshi…

It was a complex situation to be sure. I only shared the story to illustrate how personally people are taking these discussions. I really felt like I was being very neutral and just sharing my enthusiasm. It was entirely unwelcome, apparently. If there's a lesson to be learned it's that people's tempers over these issues may be hotter than they appear. I can barely speak with my artist friends on the issue these days…

JeJie says >"Their emotions are completely intractable on the subject: AI art is theft. Period."Then the only true artist would be one who has never seen art before except his/her own. The correction of your friends' belief would be that "ALL art is theft. Period."

Re: Simply explained: How does GPT work?

#339

Post by tornato7 »

tines wrote: post #148
danenania wrote: post #123

Earlier quoted context omitted.

It's all about emergent complexity. While you can reduce it to "just" statistical auto-completion of the next word, we are seeing evidence of abstraction and reasoning produced as a higher-order effect of these simple completions. It's a bit like the Sagan quote: "If you wish to make an apple pie from scratch, you must first invent the universe". Sometimes for GPT to "just" complete the next word in a way that humans…

> Sometimes for GPT to "just" complete the next word in a way that humans find plausible, it must, along the way, develop a model of the world, theory of mind, abstract reasoning. etc. I did an experiment recently where I asked ChatGPT to "tell me an idea [you] have never heard before". ChatGPT replied with what sounded like an idea for a startup, which was delivering farm-fresh vegetables to customers' doors. This i…

For what it's worth, I asked ChatGPT to come up with an original idea but I generated 25 random English words and told it to use them as inspiration. This can help the output be more creative and original. Here's what it came up with:

> The BerryPulse is an innovative, eco-friendly device that captures the energy released during the natural decomposition of berries to produce heat and electricity. The device comprises a closed container, where a cluster of berries is placed in a specially designed compartment, acting as an organic fuel source. The decomposition process is intensified using a biodegradable catalyst, which accelerates the natural breakdown of the berries while ensuring minimal environmental impact.

Re: Simply explained: How does GPT work?

#340

Post by giardini »

tines wrote: post #195
danenania wrote: post #187

Earlier quoted context omitted.

I don't think there's anything wrong with it either. It's an important debate. I just think the arguments usually become very circular and repetitive. If there's nothing an AI could ever do to convince you that it's thinking or reasoning, then really you should be explicit and say "I don't believe an AI can produce human thought or human reasoning" or "an AI is not a human" and nobody will disagree with you on those…

> and nobody will disagree with you on those points But that's the point, they do. Even on HN there are many comments saying that humans are just fancy autocomplete, i.e. there's no fundamental difference between humans and LLMs.

tines says>"Even on HN there are many comments saying that humans are just fancy autocomplete, i.e. there's no fundamental difference between humans and LLMs."

LLMs'may prove a useful analogy as to how parts of human intelligence operate, an analogy that, at the very least, should be thoroughly researched.

Post reply on HN