Live data from Hacker News

ChatGPT Explained: A normie's guide to how it works

jonstokes.com

51–60 of 144 posts

Re: ChatGPT Explained: A normie's guide to how it works

#51
Referencing atomic orbitals... for a good time, ask ChatGPT this series of questions:

1. Hello, what can you tell me about the Legendre polynomials, angular momentum, and the three-dimensional visualization of atomic orbitals in the context of quantum mechanics?

2. So, can the three-dimensional representations of atomic orbitals seen in chemistry textbooks be considered to be solutions of the Schrodinger differential equations in the context of Legendre polynomials?

3. What's the nature - mathematically speaking - of the radial and angular combination of the Legendre polynomials and the hydrogenic radial functions that give rise to the textbook probablility density visualizations of atomic orbitals?

4. Tell me about Hartree-Fock representations of the molecular orbitals of the helium atom?

5. Compare and contrast the Hartree-Fock method and the more recently developed Density Functional Theory approach, please, in the context of simple molecular orbital theory.

That's pretty impressive, probably a better and more concise explanation than a lot of professionals in the field could give off the top of their heads. However, it's also interesting to see where ChatGPT is remarkable untrustworthy - Linux terminal command line options, for example, it often gets those wrong (disastrously so in some cases, happily I double-checked first). It also has problems with modular arithmetic... This one always delivers, it seems, and has been doing so for a month at least:

Q: So is seven congruent to four, in the context of modulus three?

> "No, seven is not congruent to four modulo three. To determine whether two integers are congruent modulo a given modulus, we need to compute the difference between them and check if it is divisible by the modulus.

In this case, we have:

7 - 4 = 3

Since 3 is not divisible by 3, we conclude that 7 is not congruent to 4 modulo 3.

We can also verify this by checking the remainders when 7 and 4 are divided by 3:

7 ÷ 3 = 2, remainder 1 4 ÷ 3 = 1, remainder 1

Since the remainders are not the same, we can again conclude that 7 is not congruent to 4 modulo 3."

Hmmm.... this seems wrong...

Re: ChatGPT Explained: A normie's guide to how it works

#54
post #30
post #5

I like the token window bit. I don't really like the probability bit, because it kinda alludes that OpenAI just built a huge probability map of all N-grams (N=8000) and called it a day. Which incidentally would also imply that a lot of N-grams just don't exist in the training data, causing the model to completely halt when someone says something unexpected. But that's not the case - instead we convert words into a lo…

So basically, if a pattern that hasn't been encountered before is seen, it will just try to connect "something" together, which is why it does things like predict today's date being in the future etc? The model says, "I don't have a good enough path forwards here, I'll just make one up given the next best thing I have and serve it back"? Maybe this is why Bing is working differently, they've changed the model or the…

Not just connect something.

For example, at a certain size, GPT models start to "learn" how to do basic arithmetic (addition), even for numbers with multiple digits they've never encountered before.

It might look like a small thing, what with computers being quite able to do arithmetic at the base level. But this is a language model, so its a bit different. It learns how to add numbers without "carrying the 1" first, then at a certain larger size also learns to carry the 1, then when even larger it learns to do that across multiple digits... So its not just blindly guessing, its learning the rules of the game (and in some cases some quite complex rules) by building a model of the world made of words.

And the model of digits and addition is just one small bit most likely, as the training space doesn't contain much of that - writing about adding numbers is pretty boring after all. The full model must encode rules and knowledge about a variety of complex things to be able to make reliable predictions. It probably also contains true generalizations that humanity hasn't thought of before, as well as specializations of those generalizations that could be immensely useful.

Re: ChatGPT Explained: A normie's guide to how it works

#56
post #31

ChatGPT is probably the first software product that I have no idea how I'd go about implementing. I watched a number of YT videos about it including Andrew Carpathy's 2 hour coding session building a mini GPT. I understand the process abstractly, but I am unable to grok the details about how it's able to take my vague interpretation of what I want and then write code and actually give me exactly what I wanted.

The ChatGPT video by Karpathy is the last in a 7 video series. The one that really hit it home for me was the first video on MicroGrad [1]. The second video builds on MicroGrad and is also great for understanding how a basic NN works. [1] https://youtu.be/VMj-3S1tku0

Yeah, unless you're already familiar with the advanced math behind all this don't expect an in-depth understanding of the implementation details the parent comment is concerned about.

Re: ChatGPT Explained: A normie's guide to how it works

#57
post #6

The biggest drawback of LLM is that it never answers with "I don't know" (unless it is some quote) and it just brings bullshit hallucinations which human has to reject as wrong. Thus it is mostly useless for anything serious. Personally I use it to beautify some text, but still have to do a bit of correction to fix b/s or missed context.

It seems to respond better if you include a 'in the context of ' at the end of the prompt. Then keep refining context over a series of questions.

Re: ChatGPT Explained: A normie's guide to how it works

#58
post #6

The biggest drawback of LLM is that it never answers with "I don't know" (unless it is some quote) and it just brings bullshit hallucinations which human has to reject as wrong. Thus it is mostly useless for anything serious. Personally I use it to beautify some text, but still have to do a bit of correction to fix b/s or missed context.

I haven't experienced chatgpt explicitly tell me "I don't know", but it has provided me with factually incorrect answers pertaining to the functionality of an appliance and associated software. Further probing and I was advised to contact IBM support.... lol.

My usage has been confined to specific technical inquiries and chatgpts benefit has been colossal in saving me the time of sifting through decade(s) of documentation, forum posts, bug reports, kb articles, etc.

Re: ChatGPT Explained: A normie's guide to how it works

#59
post #20

I feel like as an intro aimed at "normies" it still manages to communicate in a more abstract and overthinky way than necessary. People often find it difficult to intuit examples from abstract descriptions. BUT, people are great at intuiting abstractions from concrete examples. You rarely need to explicitly mention abstractions, in informal talk. People's minds are always abstracting. > If I’m relating the collection…

> a generative model is a function that can take a structured collection of symbols as input and produce a related structured collection of symbols as output. Yeah that’s exactly the way a nOrMiE would find easy to think about it. Duh. The author probably should dish out that sentence on his grandparents and see how that would work before putting it on the internet and labeling it as “for normies”.

Woah, article is way to TL:DR too! Let me show how it is done:

A generative model is just a computer program. The program is incredibly book smart. No matter how incoherent your ramblings it will find the most likely connection between words, string them into (mostly wrong) candidate sentences with different degrees of certainty, it looks in its database where everything you said is compared to the wrong word combinations assigning scores to each and then it adds up all the points scored and it finds THE most likely correct response: "All gore invented the internet"

There, that is all there is to it.

Re: ChatGPT Explained: A normie's guide to how it works

#60

Referencing atomic orbitals... for a good time, ask ChatGPT this series of questions: 1. Hello, what can you tell me about the Legendre polynomials, angular momentum, and the three-dimensional visualization of atomic orbitals in the context of quantum mechanics? 2. So, can the three-dimensional representations of atomic orbitals seen in chemistry textbooks be considered to be solutions of the Schrodinger differential…

Add “Let’s think step by step” with the math problem and see if you get something different
Post reply on HN