Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Simply explained: How does GPT work?

confusedbit.dev

Post reply on HN
Print view View original

21–30 of 392 posts

Re: Simply explained: How does GPT work?

#21

Post by winternett »

Where is IBM's Watson in all this? It seems as if it never existed? That is just one example of how companies keep making these grand presentations and under-delivering on results...

Plain and simple the over-hyped GPT editions are NOT truly AI, it is scripting to assemble coherent looking sentences backed by scripts that parse content off of of stored data and the open web into presented responses.... There is no "artificial" nor non-human intelligence backing the process, and if there wasn't human intervention, it wouldn't run on it's own... In a way, it could better replace search engines at this point with even text-to-speech even, if the tech was more geared towards a more basic (and less mystified) reliability and demeanor... It's kind of like the Wizard of OZ, with many humans behind the curtains.

Marketers and companies behind promotion of these infantile technology solutions are being irresponsible in proclaiming that these things represent Ai, and in going as far to claim as they will cost jobs at this point, it will prove costly to repair over zealous moves based on the lie. This is what we do as a planet, we buy Hype, and it costs us a lot. We need a lot more practicality in discussions concerning Ai, because over-assertive and under-accountable marketing is destructive. -- Just look at how much hype and chaos promises of self-driving cars cost many (Not me though thanks). It completely derails tech progress to over promise and under deliver on tech solutions. It creates monopolies that totally destroy other valid research and development efforts. It makes liars profitable, and makes many (less flashy, but actually honest tech and innovation conducted by responsible people) close up shop.

We are far from autonomous and self reliant tech, even power grids across most of the planet aren't reliable enough to support tech being everywhere and replacing jobs.

Just try to hold a conversation with Siri or Google Assistant, which have probably been developed and tested a lot more than GPT, and around for much longer too, and you'll realize why kiosks at the supermarket and CVS are usually out of order, and why articles written by GPT and posted to sites like CNN.Com and Buzz Feed are poorly written and full of filler... We're just not there yet, and there's too many shortcuts, patchwork, human intervention, and failed promises to really say we're even close.

Let's stop making the wrong people rich and popular.

Re: Simply explained: How does GPT work?

#22

Post by stareatgoats »

lm28469 wrote: post #14
stareatgoats wrote: post #10

This article seems credible and actually made me feel as if I understood it, i.e. at some depth but not deeper than a relative layperson can grasp. What I can't understand is how the Bing chatbot can give me accurate links to sources but chatGPT4 on request gives me nonsensical URLs in 4 case of 5. It doesn't matter in the cases where I ask it to write a program: the verification is in the running of it. But to have…

> What I can't understand is how the Bing chatbot can give me accurate links to sources but chatGPT4 on request gives me nonsensical URLs in 4 case of 5 The bing version might run a bing query, fetch the X top pages, run GPT on it, return a response based on what it read, and in the back assign the summary to the source

That might be the reason, probably. I mostly wanted to complain TBH. But I'm assuming it's one of those wrinkles that will get ironed out in subsequent versions.

Re: Simply explained: How does GPT work?

#24

Post by Analog24 »

winternett wrote: post #21

Where is IBM's Watson in all this? It seems as if it never existed? That is just one example of how companies keep making these grand presentations and under-delivering on results... Plain and simple the over-hyped GPT editions are NOT truly AI, it is scripting to assemble coherent looking sentences backed by scripts that parse content off of of stored data and the open web into presented responses.... There is no "a…

What would be the differentiating factor(s) for true AI/intelligence in your opinion?

Re: Simply explained: How does GPT work?

#25

Post by capableweb »

simonw wrote: post #12
onetrickwolf wrote: post #7

I've been using GPT4 to code and these explanations are somewhat unsatisfactory. I have seen it seemingly come up with novel solutions in a way that I can't describe in any other way than it is thinking. It's really difficult for me to imagine how such a seemingly simple predictive algorithm could lead to such complex solutions. I'm not sure even the people building these models really grasp it either.

I've started to suspect that generating code is actually one of the easier things for a predictive text completion model to achieve. Programming languages are a whole lot more structured and predictable than human language. In JavaScript the only token that ever comes after "if " is "(" for example.

> In JavaScript the only token that ever comes after "if " is "(" for example.

I'm pretty sure " " (whitespace) is a token as well, which could come after a `if` as well. I think overall your point is a pretty good one though.

Re: Simply explained: How does GPT work?

#26

Post by capableweb »

camillomiller wrote: post #18
simonw wrote: post #15

Earlier quoted context omitted.

I used GPT-4 to build this tool https://image-to-jpeg.vercel.app using a few prompts the other day - my ChatGPT transcript for that is here: https://gist.github.com/simonw/66918b6cde1f87bf4fc883c677351...

Love how you didn’t care about styling this like at all, Lol. Btw, if you ask gpt to make it presentable by using bootstrap 5 for example it can style it for you

One mans "presentable" is another mans bloat. It looks perfectly fine to me, simple, useful and self-explanatory, doesn't need more flash than so.

Re: Simply explained: How does GPT work?

#27

Post by LeSaucy »

onetrickwolf wrote: post #7

I've been using GPT4 to code and these explanations are somewhat unsatisfactory. I have seen it seemingly come up with novel solutions in a way that I can't describe in any other way than it is thinking. It's really difficult for me to imagine how such a seemingly simple predictive algorithm could lead to such complex solutions. I'm not sure even the people building these models really grasp it either.

I have only seen gpt generate imperative algorithms. Does it have the ability to work with concurrency and asynchrony?

Re: Simply explained: How does GPT work?

#28

Post by Analog24 »

seydor wrote: post #17

This is confusing, using the semantic vectors arithmetic of embeddings is not very relevant to transformers and its completely missing the word 'attention'. I don't think transformers are that difficult to explain to people , but it is hard to explain "why" they work. But i think it's important for everyone to look under the hood and know that there are no demons underneath.

Embeddings and their relationship to each other are definitely relevant to transformers. Why do you think that's not the case?

Re: Simply explained: How does GPT work?

#29

Post by agentultra »

A good article and well articulated!

I would change the introduction to be more impartial and not anthropomorphize GPT. It is not smart and it is not skilled in any tasks other than that for which it is designed.

I have the same reservations about the conclusion. The whole middle of the article is good. But to then compare the richness of our human experience to an algorithm that was plainly explained? And then to speculate on whether an algorithm can "think" and if it will "destroy society," weakens the whole article.

I really would like to see more technical writing of this sort geared towards a general audience without the speculation and science-fiction pontificating.

Good effort!

Re: Simply explained: How does GPT work?

#30

Post by exitb »

simonw wrote: post #12
onetrickwolf wrote: post #7

I've been using GPT4 to code and these explanations are somewhat unsatisfactory. I have seen it seemingly come up with novel solutions in a way that I can't describe in any other way than it is thinking. It's really difficult for me to imagine how such a seemingly simple predictive algorithm could lead to such complex solutions. I'm not sure even the people building these models really grasp it either.

I've started to suspect that generating code is actually one of the easier things for a predictive text completion model to achieve. Programming languages are a whole lot more structured and predictable than human language. In JavaScript the only token that ever comes after "if " is "(" for example.

On the other hand, if you want to use an external library on the line 80, you need to import it at the top.

I once asked it for a short example code of something, no longer than 15 lines and it said "here's a code that's 12 lines long" and then added the code. Did it have the specific code "in mind" already? Or was it just a reasonably-sounding length and it then just came up with code that matched that self-imposed constraint?

Post reply on HN