Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: How does ChatGPT work?

news.ycombinator.com

Post reply on HN
Print view View original

31–40 of 185 posts

Re: Ask HN: How does ChatGPT work?

#31

Post by dzink »

akelly wrote: post #18

The way they went from GPT-3 to ChatGPT is really quite genius. My understanding is that it's something like this: 1. Start with GPT-3, which predicts the next word in some text and is trained on all the text on the internet 2. Take thousands of prompts, generate several responses for each of them, and have human reviewers rank the responses for each prompt from best to worst 3. The GPT model needs a massive amount o…

ChatGPT seems to be/result in some amount of caching of responses - there is very little variation when to asking the same question multiple times. CharacterAI produces a lot more variety in comparison, making it more helpful for brainstorming. That said ChatGPT is likely closer to the truth, even if not perfect, for searches. The innovation happening lately is incredible.

Re: Ask HN: How does ChatGPT work?

#33

Post by 6nf »

dtagames wrote: post #3

ChatGPT is really very simple. Imagine you could analyze a million books and identify all the words within them -- not the meanings of the words, just the actual letters they contain. Now, when someone asks you about the history of France (or why the sky is blue), you could simply pluck out of your library the most common strings of word that seem to follow the words that were in your question! It's like a kid in the…

> it cannot apply logic

Have you tried?

Re: Ask HN: How does ChatGPT work?

#34

Post by pushedx »

dtagames wrote: post #3

ChatGPT is really very simple. Imagine you could analyze a million books and identify all the words within them -- not the meanings of the words, just the actual letters they contain. Now, when someone asks you about the history of France (or why the sky is blue), you could simply pluck out of your library the most common strings of word that seem to follow the words that were in your question! It's like a kid in the…

Ask ChatGPT to write a song lyric for your favorite band in the style of your favorite poet, with a specific unusual topic.

Then claim that the model is “really very simple”

Re: Ask HN: How does ChatGPT work?

#35

Post by ofrzeta »

akelly wrote: post #18

The way they went from GPT-3 to ChatGPT is really quite genius. My understanding is that it's something like this: 1. Start with GPT-3, which predicts the next word in some text and is trained on all the text on the internet 2. Take thousands of prompts, generate several responses for each of them, and have human reviewers rank the responses for each prompt from best to worst 3. The GPT model needs a massive amount o…

> Take thousands of prompts, generate several responses for each of them, and have human reviewers rank the responses for each prompt from best to worst

Recently I saw an image where Indian women sat in front of computers and the caption said they were classifying "AI" responses. I guess that's true and this kind of work is the new outsourced cheap labour in the AI age.

Re: Ask HN: How does ChatGPT work?

#36

Post by nullish_signal »

dtagames wrote: post #11
nullish_signal wrote: post #6

Earlier quoted context omitted.

OP question mentions specific network types, while reply uses a vague "million-book library" metaphor, but at least mentions LLM? Also arbitrarily sorts human { teacher, journalist, writer } as "people who do custom work with each client" which makes a bit of sense for a Teacher in a small-class environment. But Writers and Journalists write at SCALE, and while each may speak with gusto to individuals just as well as…

I wouldn't deign to have ChatGPT write my comments. They're all me! Obviously, an author's audience can include more than one person. But every good piece of human writing is tailored for some audience. That was my point. You've heard the expression, "Know your audience?" ChatGPT cannot.

ChatGPT's "audience" can range from "hate speech for anti-bias training" to "Teenage Mutant Ninja Turtles Deliver Impassioned Speech to The World"

I would also like to use the No True Scotsman fallacy, and assert that no Human can ever really "know" or "write for" anybody's perfect preferences.

Re: Ask HN: How does ChatGPT work?

#37

Post by sillysaurusx »

soulofmischief wrote: post #25
akelly wrote: post #18

The way they went from GPT-3 to ChatGPT is really quite genius. My understanding is that it's something like this: 1. Start with GPT-3, which predicts the next word in some text and is trained on all the text on the internet 2. Take thousands of prompts, generate several responses for each of them, and have human reviewers rank the responses for each prompt from best to worst 3. The GPT model needs a massive amount o…

I don't mean to downplay how incredible the tech is, but I'm not sure I'd call this approach genius as it's the industry standard.

Then why were they the first ones to exploit it so effectively?

I don’t think it was standard for GPT models.

Re: Ask HN: How does ChatGPT work?

#38

Post by starik36 »

akelly wrote: post #18

The way they went from GPT-3 to ChatGPT is really quite genius. My understanding is that it's something like this: 1. Start with GPT-3, which predicts the next word in some text and is trained on all the text on the internet 2. Take thousands of prompts, generate several responses for each of them, and have human reviewers rank the responses for each prompt from best to worst 3. The GPT model needs a massive amount o…

Ok, but how does it take my code and fixes the bug? It's my own code, no one has seen it besides me and the model wasn't trained on it.

Re: Ask HN: How does ChatGPT work?

#39

Post by [deleted] »

the_gipsy wrote: post #24
dtagames wrote: post #3

ChatGPT is really very simple. Imagine you could analyze a million books and identify all the words within them -- not the meanings of the words, just the actual letters they contain. Now, when someone asks you about the history of France (or why the sky is blue), you could simply pluck out of your library the most common strings of word that seem to follow the words that were in your question! It's like a kid in the…

> So, the big take away needs to be that there is absolutely no understanding, no cognizance of any kind, no language comprehension going on. Okay, but what is understanding, cognizance, comprehension? And so on for further precision of definitions, until you reach that vague nebulous area that is consciousness. We simply haven't figured that out, period. So we cannot really say that this AI does not have those quali…

[deleted]
Post reply on HN