No post body was provided.
ChatGPT, Rot13, and Daniel Kahneman
51–60 of 213 posts
Re: ChatGPT, Rot13, and Daniel Kahneman
#52Earlier quoted context omitted.
GPT3 can't create ASCII art for shit either. Though it can make little ASCII tables of data.
I asked it to create an ASCII art banana and the result was hilarious. It then tried to explain it by elaborating that the 'O' was a curvy letter and represented the curves of the banana.
I asked it to draw me an ASCII art banana.
It did not go well: https://imgur.com/a/5g2e9Ld
Re: ChatGPT, Rot13, and Daniel Kahneman
#53There's something I don't get about all these models... Why aren't these using external tools, like a calculator, when they "know" they're doing something a tool would solve perfectly? Humans do it all the time now. Engineers aren't designing microchips using pen and papers, doing all the computation in their head. Instead they're using tools (software / calculators) Apparently the model can tell what a multiplicatio…
funny enough i asked it what tool i could use to solve rot13 encryption and it directed me to rot13.com and even explained how to use the site
Re: ChatGPT, Rot13, and Daniel Kahneman
#54A lot of people seem to be overlooking the fact that it's missing a huge piece of the puzzle, and that is being able to learn. This is a model frozen in time, you can explain to it a hundred times why it's wrong and it will learn nothing. Until we have something that learns continuously from more input, I am not impressed
however once it does have the ability to learn from its mistakes, document its millions of simultaneous chats and has the ability to call on the software tools we use its pretty much going to be unstoppable.
Kinda looking forward to the next few years.
Re: ChatGPT, Rot13, and Daniel Kahneman
#55A lot of people seem to be overlooking the fact that it's missing a huge piece of the puzzle, and that is being able to learn. This is a model frozen in time, you can explain to it a hundred times why it's wrong and it will learn nothing. Until we have something that learns continuously from more input, I am not impressed
Yes, this is an attempt to teach ChatGPT how to Rot13 and demonstrates that this isn't possible. The model doesn't learn. It can extend an input to produce a longer input, but its memory is limited. It can find the exact definition of Rot13 because that was in its training data, but it can't apply that definition.
Re: ChatGPT, Rot13, and Daniel Kahneman
#56[deleted]
Re: ChatGPT, Rot13, and Daniel Kahneman
#57There's a way simpler answer than this Type I Type II thinking stuff. Most LLMs like GPT are not trained on the level of individual characters – they process input and outputs on the level of subword units that compose multiple characters to support long context windows (i.e. "door" instead of "d", "o", "o", "r"). As a result, they do poorly on character manipulation tasks. You can get some insight here: https://beta…
Their tokenizer seems very algorithmic and pre-determined. I suppose that is a typical burden of multi-class classifiers. It might be interesting to have a way of dynamically tokenizing words/sentences based on confidence about them. Some unknown words might need a letter-for-letter representation, while some commonly occurring phrases could almost be encoded in a single token. That'd probably require a completely different way of representing stuff and computing a loss as the representations would change throughout training as the confidence about some context changes.
Re: ChatGPT, Rot13, and Daniel Kahneman
#58Earlier quoted context omitted.
It works with your prompt yes, but then I tried a simple example at the end: a= Negvsvpvny vagryyvtrapr => [len(i) for i in a.split()] => The solution should be "Artificial intelligence". It never gets it right.
Yeah check my answer to the other reply to this thread, I stopped when I got the answer I wanted and stupidly forgot to test whether it actually worked with any other sentence. The logic is sound with regards to giving it a few examples and splitting tasks into sub-tasks, that's how they prime their model to evaluate it on all NLP benchmarks in the GPT papers and I've solved many problems like that in ChatGPT. I've r…
It still doesn't really work with "negvsvpvny vagryyvtrapr" :/
Edit: actually it does get the second word vagryyvtrapr->intelligence correct sometimes.
Re: ChatGPT, Rot13, and Daniel Kahneman
#59Related to this: I had fun the other night trying to explain rhymes to ChatGPT. It could ONLY write rhyming couplets, and even when I explained exactly which sentences in a poem I wanted to rhyme, it would write a couplet. (That even happened sometimes when I asked it specifically NOT to rhyme). Eventually I got it to manage ABAB rhymes by:
1. Asking it to generate four sentences on a topic with the same meter and number of syllables.
2. Asking it to come up with two rhyming words that relate to that topic.
3. Asking it to replace the first sentence with a new sentence where the last word is the first of the two rhyming words, and similarly with the other sentence.
4/5. Same as 2/3, but for the other sentences.
6. Asking it to follow all those steps again, explaining each one as it goes along.
The funny thing was that it kept trying to skip steps or simplify what it was doing. It also got completely confused when I asked it to extrapolate the pattern to new rhyme schemes, eg ABA BCB.
Re: ChatGPT, Rot13, and Daniel Kahneman
#60A lot of people seem to be overlooking the fact that it's missing a huge piece of the puzzle, and that is being able to learn. This is a model frozen in time, you can explain to it a hundred times why it's wrong and it will learn nothing. Until we have something that learns continuously from more input, I am not impressed
It is only a limitation of the interface that we're interacting with. There is no reason it couldn't backpropagate towards a better solution when told that it's wrong. OpenAI probably aren't letting it train online lest some jokers try to teach it racism and other bullshit etc.