Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: How does ChatGPT work?

news.ycombinator.com

Post reply on HN
Print view View original

171–180 of 185 posts

Re: Ask HN: How does ChatGPT work?

#171

Post by int_19h »

Scarblac wrote: post #65
int_19h wrote: post #64

Earlier quoted context omitted.

I don't know what it uses to output non-English text, but if you address it in a different language, it'll quite happily speak that back to you. And I'd say it understands it just fine, not really any worse than English - although it sometimes gets the word forms wrong in inflective languages.

But I was asking it for rhyming words in Dutch (because of Sinterklaas), and it suggested words that would rhyme if pronounced as English...

Yup, doesn't rhyme for me, either, if I ask for rhymes or poetry in Russian (but I don't think it'd rhyme even in English).

I think that makes sense if the supermajority of the training data was in English - it'd make it heavily biased towards that by default. So even when it speaks another language, it still "thinks" of the words in terms of questions like "how it sounds". I suspect it might be possible to craft a sufficiently detailed prompt to work around that.

Re: Ask HN: How does ChatGPT work?

#172

Post by chronolitus »

Back when GPT-3 came out, I wanted to understand how it works, so read the papers and made this post:

https://dugas.ch/artificial_curiosity/GPT_architecture.html

I hoped it would be simple enough for anyone who knows a bit of math / algebra to understand. But note that it doesn't go into the difference between GPT-3 and ChatGPT (which adds a RL training objective, among other things).

Re: Ask HN: How does ChatGPT work?

#173

Post by namibj »

Sharlin wrote: post #154
dotancohen wrote: post #145

Earlier quoted context omitted.

> On the other hand, airplanes do fly. In my language, the verb that airplanes do is different than the verb that birds and insects do. As like the submarine, it is based on a different physical mechanism than are the natural animals in that space. For that matter, in English, what is the verb that submarines do? Surely it's not "sail".

Submarines do dive , but only when moving vertically downward. I don't think there is a single verb for what submarines do, they can move , travel , maneuver , navigate , or operate (underwater) depending on context.

Float?

Re: Ask HN: How does ChatGPT work?

#174

Post by 6nf »

dtagames wrote: post #152
6nf wrote: post #33

Earlier quoted context omitted.

> it cannot apply logic Have you tried?

The tool can parrot back to you "logical" arguments which it has already scanned. It cannot figure out the logic behind them, nor seen when it's wrong, nor correct any errors, nor draw new logical conclusions.

Are you sure about that?

Re: Ask HN: How does ChatGPT work?

#175

Post by rvbissell »

dotancohen wrote: post #114
swayson wrote: post #32

Yannic Kilcher did an explainer recently on his YT channel. https://www.youtube.com/watch?v=0A8ljAkdFtg Yannic explains these models pretty well.

I just skimmed the video. It seems full of information, but this video skips. It seems like either the author composed his sentences by slicing together various clips, or a few frames have been removed between some words. I tried opening the video in NewPipe as well, same problem.

I believe this is a stylistic, 'production values' choice. It used to bother me to the same degree that it seems to bother you. I suspect the content creators that do it are clipping out things like "umm" and "uhh". It might also be that some creators are better at this kind of splicing than others, which might explain why this particular example did not bother me so much.

Re: Ask HN: How does ChatGPT work?

#176

Post by dotancohen »

rvbissell wrote: post #175
dotancohen wrote: post #114

Earlier quoted context omitted.

I just skimmed the video. It seems full of information, but this video skips. It seems like either the author composed his sentences by slicing together various clips, or a few frames have been removed between some words. I tried opening the video in NewPipe as well, same problem.

I believe this is a stylistic, 'production values' choice. It used to bother me to the same degree that it seems to bother you. I suspect the content creators that do it are clipping out things like "umm" and "uhh". It might also be that some creators are better at this kind of splicing than others, which might explain why this particular example did not bother me so much.

I see, thank you.

Re: Ask HN: How does ChatGPT work?

#177

Post by jacknews »

hcks wrote: post #146
ofrzeta wrote: post #35

Earlier quoted context omitted.

> Take thousands of prompts, generate several responses for each of them, and have human reviewers rank the responses for each prompt from best to worst Recently I saw an image where Indian women sat in front of computers and the caption said they were classifying "AI" responses. I guess that's true and this kind of work is the new outsourced cheap labour in the AI age.

I personally worked as a « human trainer » for the fine tuning of ChatGPT. The pay was 50$ per hour, which is not bad for a side job as a student.

I'd say. Where do you apply for this kind of work?

Re: Ask HN: How does ChatGPT work?

#178

Post by bozhark »

KaoruAoiShiho wrote: post #143
bozhark wrote: post #125

Earlier quoted context omitted.

7x NVIDIA RTX A6000's so ~$32,550

This is honestly affordable for a lot of upper-middle class people and might well it worth it. It's like the cost of a car. I can seriously see this writing a book for me if I can get it tuned to study only my writing style and remember all of my texts. But it could also only cost $14000 14x RTX 3090s.

5x limit per manufacturer, maybe 3rd party for the additional?

Re: Ask HN: How does ChatGPT work?

#179

Post by jasmer »

heartbreak wrote: post #144
KaoruAoiShiho wrote: post #143

Earlier quoted context omitted.

This is honestly affordable for a lot of upper-middle class people and might well it worth it. It's like the cost of a car. I can seriously see this writing a book for me if I can get it tuned to study only my writing style and remember all of my texts. But it could also only cost $14000 14x RTX 3090s.

I’ll wait a year and buy $2k worth of hardware that runs it.

Or wait a further few years and spend $20.

Or a couple years later for it to be 20 cents.

Or a couple years later for them to give it to you for buying a bottle of Coke.

In the interim, they will find ways to make money from us.

Re: Ask HN: How does ChatGPT work?

#180

Post by dtagames »

dtagames wrote: post #3

ChatGPT is really very simple. Imagine you could analyze a million books and identify all the words within them -- not the meanings of the words, just the actual letters they contain. Now, when someone asks you about the history of France (or why the sky is blue), you could simply pluck out of your library the most common strings of word that seem to follow the words that were in your question! It's like a kid in the…

I wrote this[0] to better explain my wildly unpopular answer above. Everything is absolutely correct here, but it's not a popular view right now. Often very smart people like those on HN can be fooled by the "magic" of tech that might not really exist in the way they think it does.

[0] https://medium.com/gitconnected/behind-the-curtain-understan...

Post reply on HN