Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Simply explained: How does GPT work?

confusedbit.dev

Post reply on HN
Print view View original

51–60 of 392 posts

Re: Simply explained: How does GPT work?

#51

Post by rootusrootus »

stareatgoats wrote: post #10

This article seems credible and actually made me feel as if I understood it, i.e. at some depth but not deeper than a relative layperson can grasp. What I can't understand is how the Bing chatbot can give me accurate links to sources but chatGPT4 on request gives me nonsensical URLs in 4 case of 5. It doesn't matter in the cases where I ask it to write a program: the verification is in the running of it. But to have…

> It doesn't matter in the cases where I ask it to write a program: the verification is in the running of it.

Even then. I've had it write programs that were syntactically correct and produced plausible, but incorrect behavior. I'm really careful about what I'll use GPT-generated code for. IMO write the tests yourself, at least.

Re: Simply explained: How does GPT work?

#52

Post by TuringTest »

kingkongjaffa wrote: post #38
EForEndeavour wrote: post #32

Earlier quoted context omitted.

The advanced capabilities of scaled up transformer models fed oodles of training data has burdened me with pseudo-philosophical questions about the nature of cognition that I am not well equipped to articulate, and make me wish I'd studied more neuroscience, philosophy, and comp sci earlier in life. A possibly off-topic thought dump: - What is thinking, exactly? - Does human (or superhuman) thinking require conscious…

I think since the mechanisms are different we should arrive at a distinction between: organic thinking (I.e. the process our squishy human brains do) and mechanical thinking ( the computational and stochastic processes that computers do ).

I don't think the substrate defines the nature of the thinking, but the form of the process does.

It is entirely possible to build mechanical thinking in organic material (think Turing machines built on growing tissue), and it could also be possible to build complex self-referential processes simulated on electronic hardware, of the kind high-level brains do, with their rhythms of alfa and beta waves.

Re: Simply explained: How does GPT work?

#53

Post by camillomiller »

capableweb wrote: post #26
camillomiller wrote: post #18

Earlier quoted context omitted.

Love how you didn’t care about styling this like at all, Lol. Btw, if you ask gpt to make it presentable by using bootstrap 5 for example it can style it for you

One mans "presentable" is another mans bloat. It looks perfectly fine to me, simple, useful and self-explanatory, doesn't need more flash than so.

Sure, but presentation and UX basics are not "bloat".

Re: Simply explained: How does GPT work?

#54

Post by agentultra »

myrmidon wrote: post #48
agentultra wrote: post #37

Earlier quoted context omitted.

It's not thinking, plain and simple. Anything it generates means nothing to the algorithm. When you read it and interpret what was generated you're experiencing something like the Barnum-Forer effect. It's sort of like reading a horoscope and believing it predicted your future.

What gives you any confidence that the way GPT4 comes up with answers is qualitatively different from humans? Why should the emulation of human though, a result of unguided evolution, require anything more than properly wired silicon?

That's highly reductive of our capacities. We are not weighted transformers that can be explained in an arxiv paper. GPT, at the end of the day, is a statistical inference model. That's it.

It's not going to wake up one day, decide it prefers eggs benny and has had enough of your idle chatter because of that sarcastic remark you made last week.

Could we simulate a plausibly realistic human brain on silicon someday? I don't know, maybe? But that's not what GPT is and we're no where near being able to do that.

You can scale up the tokens an LLM can manage and all you get is a more accurate model with more weights and transformers. It's not going to wake up one day, have feelings, religion, decide things for itself, look in a mirror and reflect on its predicament, lament the poor response it gave a user, and decide it doesn't want to live with regret and correct its mistakes.

Re: Simply explained: How does GPT work?

#55

Post by justrealist »

pen2l wrote: post #39
simonw wrote: post #12

Earlier quoted context omitted.

I've started to suspect that generating code is actually one of the easier things for a predictive text completion model to achieve. Programming languages are a whole lot more structured and predictable than human language. In JavaScript the only token that ever comes after "if " is "(" for example.

The more constraints there are (e.g. like your example) the better it should perform. So it disappoints me when copilot, knowing what libraries are available in the IDE it's running in, hallucinates up a method call that doesn't exist. Separately (and apologies for going on a tangent), where do you think we are in the Gartner cycle? Around GPT3 time I was expecting for trough of disillusionment to come, particularly…

> hallucinates up a method call that doesn't exist

I actually think it helps to reframe this. It hallucinates up a method call that predictively should exist.

If you're working with boto3, maybe that's not actually practical. But if it's a method within your codebase, it's actually a helpful suggestion! And if you prompt it with the declaration and signature of the new method, very often it will write the new helper method for you!

Re: Simply explained: How does GPT work?

#56

Post by rootusrootus »

Analog24 wrote: post #24
winternett wrote: post #21

Where is IBM's Watson in all this? It seems as if it never existed? That is just one example of how companies keep making these grand presentations and under-delivering on results... Plain and simple the over-hyped GPT editions are NOT truly AI, it is scripting to assemble coherent looking sentences backed by scripts that parse content off of of stored data and the open web into presented responses.... There is no "a…

What would be the differentiating factor(s) for true AI/intelligence in your opinion?

Intent.

Re: Simply explained: How does GPT work?

#57

Post by masswerk »

seydor wrote: post #17

This is confusing, using the semantic vectors arithmetic of embeddings is not very relevant to transformers and its completely missing the word 'attention'. I don't think transformers are that difficult to explain to people , but it is hard to explain "why" they work. But i think it's important for everyone to look under the hood and know that there are no demons underneath.

> that there are no demons underneath

Given that Lacan already proposed the unconscious as structured language-like more than half a century ago and described attention in his turn on Freud's impulse in favor of his concept of derive, we may say, this is pretty much where our own demons live.

(I actually do think that revisiting Lacan in this context may be productive.)

Re: Simply explained: How does GPT work?

#58

Post by nitnelave »

agentultra wrote: post #29

A good article and well articulated! I would change the introduction to be more impartial and not anthropomorphize GPT. It is not smart and it is not skilled in any tasks other than that for which it is designed. I have the same reservations about the conclusion. The whole middle of the article is good. But to then compare the richness of our human experience to an algorithm that was plainly explained? And then to sp…

I'm planning on continuing this vulgarization series of "Simply explained", for instance to cover how computers communicate, keep an eye out for them!

Regarding the speculation/destroy society, I was directly answering questions that I got from laypeople around me. The consequences on society I don't think are much speculation: it's going to have a big effect on many jobs, just like AI has started to have but much more. For the philosophical questions, I tried to present both sides of the issue to show that it's not just a clear "yes or no": some people will happily argue with you about GPT being smart/skilled/comparable to a human brain. Anyway, it's just an introduction to the questions that you might have about it.

Re: Simply explained: How does GPT work?

#59

Post by TuringTest »

myrmidon wrote: post #48
agentultra wrote: post #37

Earlier quoted context omitted.

It's not thinking, plain and simple. Anything it generates means nothing to the algorithm. When you read it and interpret what was generated you're experiencing something like the Barnum-Forer effect. It's sort of like reading a horoscope and believing it predicted your future.

What gives you any confidence that the way GPT4 comes up with answers is qualitatively different from humans? Why should the emulation of human though, a result of unguided evolution, require anything more than properly wired silicon?

> What gives you any confidence that the way GPT4 comes up with answers is qualitatively different from humans?

For a start, GPT-4 doesn't include in its generation the current state of its internal knowledge used so far; any text built can only use at most the few words already generated in the current session as a kind of short-term memory.

Biological brains OTOH have a rhythm with feedback mechanisms which adapt to the situation where they're doing the thinking.

Re: Simply explained: How does GPT work?

#60

Post by capableweb »

camillomiller wrote: post #53
capableweb wrote: post #26

Earlier quoted context omitted.

One mans "presentable" is another mans bloat. It looks perfectly fine to me, simple, useful and self-explanatory, doesn't need more flash than so.

Sure, but presentation and UX basics are not "bloat".

What "basic UX" principles are being violated here exactly? And how would adding Bootstrap solve those?
Post reply on HN