Live data from Hacker News

GPT makes learning fun again

vipshek.com

61–70 of 243 posts

Re: GPT makes learning fun again

#61
post #47
post #41

Earlier quoted context omitted.

That large language models are stochastic parrots is not a bias, it's a fact. You feed it a very large amount of structured data and then put up a question which it'll answer with a series of those which are most likely to follow. There is nothing else .

Do you have any evidence that human minds work substantially different from that?

Yes. Human minds often craft sentences which contain words together which never occurred in the same sentence before, not even close:

> There was a boy called Eustace Clarence Scrubb, and he almost deserved it

> Everything starts somewhere, although many physicists disagree.

> It was a nice day. All the days had been nice. There had been rather more than seven of them so far, and rain hadn't been invented yet

> My father had a face that could stop a clock.

> It is important, when killing a nun, to ensure that you bring an army of sufficient size.

> In the myriadic year of our Lord—the ten thousandth year of the King Undying, the kindly Prince of Death!— Gideon Nav packed her sword, her shoes, and her dirty magazines, and she escaped from the House of the Ninth.

There's just no end to these.

And, of course, science is full of these too, one that jumps to mind is Shinichi Mochizuki's claimed proof of the ABC conjecture which has been proven flawed as it often happens but it was certainly a credible proof despite written in a language no mathematician have ever seen.

Re: GPT makes learning fun again

#62

I noticed a while back that the internet has made me terribly prone to skimming. It got to a point where I could hardly stand to read anything longer than a short news article. To learn anything useful on the internet, you pretty much have to skim. So much of the internet is so loaded with filler and BS that it is hardly worth reading at all. With ChatGPT, it’s incredibly refreshing to be able to ask a question and g…

hmm that makes sense. I too have skimmed way too much because of the BS. and ya it is refreshing now that I think about it

Re: GPT makes learning fun again

#63
post #13

You can downvote me but I think OP hasn't learned how to learn yet. If OP reads Wikipedia by understanding every sentence and clicking on every link, that's deliberately sabotaging his own learning. Attitudes like "I don't really feel like spelunking through a ton more articles" simply shows OP isn't interested in learning per se, just quick answers. OP learns in a way that's very child-like. When you are a five-year…

OP here. I think learning exists on a broad spectrum. On one end, you're just indulging curiosity ("I wonder how...?"). On the other, you're trying to build deep understanding and expertise.

I completely agree that for the latter goal, the approaches in the blog post are insufficient, even undesirable. And I do worry that the way I engage with content on the web is weakening my ability to go deep on a subject I'm interested in.

But I do think there is value in just being able to indulge curiosity quickly and consistently. Not only is it rewarding in its own right, but it also provides the spark that leads you to eventually go deeper.

Lately, I've found myself sitting at a laptop with friends, asking GPT a question, reading and discussing the response, and then coming up with and asking followup questions as a group. I don't think we would've done that in the past, because the interface of search engines and webpages and browser tabs were too unwieldy to engage with collectively. It just feels like a completely new way to learn things, and what's what I'm most excited about.

Re: GPT makes learning fun again

#64
post #7

I'm in two minds about it. On one hand the internet (in general) has become so hard to find information on, and I agree with the author that GPT is a breath of fresh air. On the other hand I've seen GPT fail so miserably at topics that I'm knowledge about that I have a very hard time trusting anything it tells me. I'm not sure what the answer really is, but I'm not sure it's GPT. I wish we could go back to having sea…

I want to use this comment to state one thing, not directly addressed to you.

Stop using GPT as a database! GPT is far more useful a reasoning engine that can accumulate fuzzy data and then provide various views or transformation of that data.

So asking GPT to parse a Wikipedia page and then asking it to teach you from it - this is a much more successful usage than what the author in the original article is doing.

It is not useful as an accurate source of information. It’s inaccurate sometimes, and it’s hard to tell when. OTOH, as a formatted, it has some actual world-changing potential.

Re: GPT makes learning fun again

#65
post #64
post #7

I'm in two minds about it. On one hand the internet (in general) has become so hard to find information on, and I agree with the author that GPT is a breath of fresh air. On the other hand I've seen GPT fail so miserably at topics that I'm knowledge about that I have a very hard time trusting anything it tells me. I'm not sure what the answer really is, but I'm not sure it's GPT. I wish we could go back to having sea…

I want to use this comment to state one thing, not directly addressed to you. Stop using GPT as a database! GPT is far more useful a reasoning engine that can accumulate fuzzy data and then provide various views or transformation of that data. So asking GPT to parse a Wikipedia page and then asking it to teach you from it - this is a much more successful usage than what the author in the original article is doing. It…

Is it? I couldn't even get it to figure out an NGINX config, despite feeding it the documentation and URL rewrites it would need, and the prior Apache .htaccess file. Spent days on it. Consulted with people who know more than me about prompt engineering. Nope.

Reasoning is not its strong point, IMO. It's a next-word prediction model, why would it be? It's doing what an LLM does. Frequently, nonsensically.

Re: GPT makes learning fun again

#66
post #61
post #47

Earlier quoted context omitted.

Do you have any evidence that human minds work substantially different from that?

Yes. Human minds often craft sentences which contain words together which never occurred in the same sentence before, not even close: > There was a boy called Eustace Clarence Scrubb, and he almost deserved it > Everything starts somewhere, although many physicists disagree. > It was a nice day. All the days had been nice. There had been rather more than seven of them so far, and rain hadn't been invented yet > My fa…

"Human minds often craft sentences which contain words together which never occurred in the same sentence before, not even close"

Weirdly, LLMs do that too.

Have you heard of the "Let’s be bear or bunny" pattern?

My iPhone hallucinated that the other day, running a LLM directly on the phone!

https://simonwillison.net/2023/May/1/lets-be-bear-or-bunny/

Re: GPT makes learning fun again

#67
post #64

Earlier quoted context omitted.

I want to use this comment to state one thing, not directly addressed to you. Stop using GPT as a database! GPT is far more useful a reasoning engine that can accumulate fuzzy data and then provide various views or transformation of that data. So asking GPT to parse a Wikipedia page and then asking it to teach you from it - this is a much more successful usage than what the author in the original article is doing. It…

Is it? I couldn't even get it to figure out an NGINX config, despite feeding it the documentation and URL rewrites it would need, and the prior Apache .htaccess file. Spent days on it. Consulted with people who know more than me about prompt engineering. Nope. Reasoning is not its strong point, IMO. It's a next-word prediction model, why would it be? It's doing what an LLM does. Frequently, nonsensically.

Interesting. Curious as to what specific task you gave it and how it failed.

My team has been working on several advanced techniques using reasoning on LLMs. The stacked performance of all of these techniques combined yields is quite impressive.

Re: GPT makes learning fun again

#68
Most people seem to prefer learning by talking and asking questions. At least, that's what I've gathered from Discord servers where 99.9% of people ask questions answered on the first page of the Readme ;)

Re: GPT makes learning fun again

#69
post #64

Earlier quoted context omitted.

I want to use this comment to state one thing, not directly addressed to you. Stop using GPT as a database! GPT is far more useful a reasoning engine that can accumulate fuzzy data and then provide various views or transformation of that data. So asking GPT to parse a Wikipedia page and then asking it to teach you from it - this is a much more successful usage than what the author in the original article is doing. It…

Is it? I couldn't even get it to figure out an NGINX config, despite feeding it the documentation and URL rewrites it would need, and the prior Apache .htaccess file. Spent days on it. Consulted with people who know more than me about prompt engineering. Nope. Reasoning is not its strong point, IMO. It's a next-word prediction model, why would it be? It's doing what an LLM does. Frequently, nonsensically.

GPT-3 or 4? Re: reasoning, apparently it gives much better performance if you ask it to write out its thought process step by step (because it has no "internal" thought process; it must be externalized).

Re: GPT makes learning fun again

#70
post #69

Earlier quoted context omitted.

Is it? I couldn't even get it to figure out an NGINX config, despite feeding it the documentation and URL rewrites it would need, and the prior Apache .htaccess file. Spent days on it. Consulted with people who know more than me about prompt engineering. Nope. Reasoning is not its strong point, IMO. It's a next-word prediction model, why would it be? It's doing what an LLM does. Frequently, nonsensically.

GPT-3 or 4? Re: reasoning, apparently it gives much better performance if you ask it to write out its thought process step by step (because it has no "internal" thought process; it must be externalized).

Yeah, that sounds like a GPT-3.5 problem to me. I wouldn't trust 3.5 to write me an Nginx config, but I'd give 4 a shot at it.

I just tried using GPT-4 to translate a random htaccess file I found to nginx and it seemed to do a good initial job: https://gist.github.com/simonw/8174f653a0c08c120830c56332054...

Post reply on HN