151–160 of 235 posts
What? You think this is poor performance? This totally blows my mind. I would never have guessed that GPT could get ANY of these right. I mean, is there a data point in the dataset used to train where you can read 2241 + 19873 = 22114? Quite unlikely... And those multiplications. It's consistently getting the number of digits right and the first two numbers correct. How the hell does this happen? Sure, it's sometimes…
If you want to play with the model, you can (with difficulty) for free at https://goose.ai/playground . You have to log in, but thankfully you can via google. The playground crashes every minute, and the defaults ruin your outputs (temperature 1, really? 0.7 to 0.8 is a necessity, with top-k 40), and they turned off autocorrect on mobile, presumably because they hate you and your family for owning an iPad, but you ca…
The latest version of gpt-3 spooked me the other day with: Me: Can here you here still here understand here what here I here am here asking here if here I here put here words here in here between? Gpt: Yes, I can understand what you are asking if you put words in between. Me: What was the word that I put in between? Gpt: The word that you put in between is "here."
[deleted]
Amazing how a very costly to train system using billions of neural nodes on millions of dollars of compute performs more poorly than an 8-bit 1970s pocket calculator. Not sure why people are expecting some sort of "intelligence" to emerge from a text generator model trained on Internet corpus data. GPT-3 doesn't calculate, it pattern matches. I do get why people might be surprised, on the other hand, that it actually…
What's your definition of 'intelligence'? Many of the things GPT-3 does clearly exhibit intelligence (just not human level intelligence).
Thank you for this. Technically it's not GPT-3, but GPT-NeoX-20B, although they are based on a similar architecture. The poor performance is most likely due to not having a large database of math problems to draw from. Github, for example, is part of the dataset that is used to train both GPT-3 and GPT-Neo variants, which is partly why they can generate meaningful code (sometimes). I wonder how a model finetuned for…
When you toss “2241 + 19873 =” into an applet that shows you the default tokenization scheme GPT-3 uses, you get this: (224)(1)( +)( 198)(73)( =) I’ve heard it remarked before that, while tokenization is obviously an unavoidable part of a model with an architecture like GPT, this is a very silly way of tokenizing number strings for the purposes of learning or doing arithmetic. Indeed, I think a lot of GPT-3’s puzzlin…
[deleted]
A really intelligent AI would just use a calculator.
The REPL could also come with libraries for all the important algorithms. Thus the LM could use numerical solvers and other nice toys.
Another excellent module we could augment LM's with is a search engine - already done, proven to improve the efficiency of the LM by 25x (see link). Another one would be a physics simulator.
https://deepmind.com/research/publications/2021/improving-la...
We're just starting with language models. They will improve tremendously.
Earlier quoted context omitted.
That's the astonishing bit
It really isn’t. You see a lot of things when reading 500 billion tokens
I fear the day AI will give superhuman consistent correct answers and nobody will be able to determine why it is right or how the correct answer was found. Maybe someday we'll get an answer from a machine which superhumanly mostly correct and we'll be unable to tell if it is right or wrong. If it is a question whose answer will influence important decisions, considering the machine answer will be close to a form of r…
Like religion, I suspect you will have many different machine answers to choose from.