> The ability of complex reasoning with chain-of-thought is likely to be a magical side product of training on code
Seems reasonable, and magical indeed. Can any expert on the topic comment on this hypothesis?
How does GPT obtain its ability? Tracing emergent abilities of language models
51–60 of 205 posts
Re: How does GPT obtain its ability? Tracing emergent abilities of language models
#52Imagine Wolfram's system with GPT integrated. I hope they're working on that.
Current GPT version is not very good at math, and it's not at all clear whether this can be fixed within a language model. Q: I have 2 cats and 1 horse. How many legs do they have in total? A: If you have two cats and one horse, then the total number of legs they have is 8. Cats typically have four legs, while horses have four legs as well. So two cats would have a total of 8 legs, and when you add the four legs of t…
[deleted]
Re: How does GPT obtain its ability? Tracing emergent abilities of language models
#53Re: How does GPT obtain its ability? Tracing emergent abilities of language models
#54Can anyone explain why the GPT response to the same question 4 days ago is different from today's: [ 4 days ago] > Can I learn to sing like Pavarotti? It is possible to learn to sing like Pavarotti, but it will require a lot of practice, dedication, and training. Pavarotti was a renowned opera singer known for his incredible vocal range, powerful voice, and technical mastery. He had many years of training and experie…
Large language models like GPT work by generating the probabilities for the next word in a sequence, given the previous words.
You can make this purely deterministic (same sequence every time) by just selecting the word with the highest probability repeatedly.
Most models will inject some randomness by sampling from the probabilities, to make the generated text more realistic. I don't know what parameters they use for ChatGPT, but they are likely injecting some randomness.
It's also likely that they are continuously training ChatGPT with reinforcement learning, so responses may change over time due to this, also.
Re: How does GPT obtain its ability? Tracing emergent abilities of language models
#55Not sure about all its filters being RL. Sometimes it seems to flag its output as inappropriate because of a single word (or none at all). Also it has asymmetric behavior, e.g. it will make a joke about men but refuse to make one about women
Competitors will remove the political correctness filters and become more prominent.
A Chinese stable diffusion art site is already gaining traction, no filters.
Re: How does GPT obtain its ability? Tracing emergent abilities of language models
#56Earlier quoted context omitted.
The problem with ChatGPT's "knowledge" is that it isn't trustworthy. It will happily output very confident sounding nonsense, or blatantly incorrect statements. We need a way to verify how accurate it's outputs are
One way I tried to do this is by having it write an answer, and a footnote reference at each fact. [1] then list search terms that be used to verify each claim, then I would respond with the url and quotes from found pages for each one, then have it rewrite the answer based on that information and cite the sources. I think something this direction can be automated. I saw someone do this with math and other tasks, tha…
Sounds like an interesting way to reboot Wikipedia.
Re: How does GPT obtain its ability? Tracing emergent abilities of language models
#57Earlier quoted context omitted.
Current GPT version is not very good at math, and it's not at all clear whether this can be fixed within a language model. Q: I have 2 cats and 1 horse. How many legs do they have in total? A: If you have two cats and one horse, then the total number of legs they have is 8. Cats typically have four legs, while horses have four legs as well. So two cats would have a total of 8 legs, and when you add the four legs of t…
I've found it gets the math right most of the time in stories. For example: > Write a story about a veterinarian counting the number of total legs on two cats and a horse. He reports the total number of legs to his assistant so he can write them down. As the veterinarian entered the examination room, he was greeted by two playful cats and a majestic horse. He smiled and began his examination, carefully checking the h…
this is what I got with the same prompt:
The veterinarian, Dr. Smith, walked into the examination room where two cats and a horse were waiting. He greeted the animals and their owners before approaching the first cat. The feline, a fluffy white and orange tabby, was lying on the examination table. Dr. Smith gently lifted one of its hind legs and counted the toes. "One, two, three, four," he muttered to himself before moving on to the cat's front legs.
After counting all four legs on the first cat, he repeated the process on the second feline, a sleek black and silver Maine Coon. Once he had counted all eight legs on the two cats, he turned his attention to the horse. The majestic creature stood in the corner of the room, calmly munching on some hay.
Dr. Smith walked over to the horse and began counting its legs, starting with the left hind leg. "One, two, three, four," he said aloud, moving on to the horse's front legs. As he counted, his assistant, who was standing nearby, jotted down the numbers on a piece of paper.
After counting all four legs on the horse, Dr. Smith stepped back and looked at his assistant. "That's a total of 18 legs," he said. "Can you write that down for me?"
His assistant nodded and added the number to the growing list on the paper. "Got it," she said. "18 legs total."
Dr. Smith smiled and turned back to the animals. "Alright, let's get started on your check-ups," he said, ready to continue with the examination.
Re: How does GPT obtain its ability? Tracing emergent abilities of language models
#58Earlier quoted context omitted.
Not sure which GPT you mean, but if you just run it twice in a row you'll likely get a different response too. There is usually some randomness and how much depends on settings.
Sometimes the meaning of the response is totally different. [ 4 days ago ] The son of my father, but not my brother. Who is he? If a person is the son of the speaker's father but is not the speaker's brother, then that person is the speaker's nephew. A nephew is the son of a person's sibling, so if the speaker's father has a son who is not the speaker's brother, that person is the speaker's nephew. For example, if th…
Yes. GPT doesn't really deal in meanings. Much like autocomplete, it doesn't know what the end of a sentence will be when it starts it. If it randomly chooses different words at the start of a sentence, it may pretend to have a different belief by the end.
In reality it doesn't have beliefs any more than a library does. It implicitly contains beliefs (since it's been trained on them) and it can imitate them, but which one you get is random. (There is likely more than one answer to your question in the training data.)
Re: How does GPT obtain its ability? Tracing emergent abilities of language models
#59Earlier quoted context omitted.
Whose knowledge is trustworthy? We've somehow come to associate certain institutions or scientific authorities with truth when that is about the furthest from real science: "Have no respect whatsoever for authority; forget who said it and instead look what he starts with, where he ends up, and ask yourself, Is it reasonable?" -Richard P. Feynman "One of the great commandments of science is, "Mistrust arguments from a…
There is some kind of recursion in here with authors names and "have no respect..." part:)
Fair point