Detect when humans are lying.
What can LLMs never do?
251–260 of 385 posts
Re: What can LLMs never do?
#252Earlier quoted context omitted.
I think one should feel comfortable arguing that AGI must be stateful and experience continuous time at least. Such that a plain old LLM is definitively not ever going to be AGI; but an LLM called in a do while true for loop might.
I don't understand why you believe it must experience continuous time. If you had a system which clearly could reason, which could learn new tasks on its own, which didn't hallucinate any more than humans do, but it was only active for the period required for it to complete an assigned task, and was completely dormant otherwise, why would that dormant period disqualify it as AGI? I agree that such a system should pro…
I think its note worthy that humans actually fail this test... We have to go dormant for 8 hours every day.
Re: What can LLMs never do?
#253There’s many things they can’t do. Even a simple rule like “ensure that numbers from one to ten are written as words and numbers greater ten as digits in the given text” fails for me for so many examples even if it works for many others; few shot, chain of thought, many versions of the prompt, it doesn’t matter. Sometimes LLMs will even change the number to something else, even with temp set to 0. And then there’s th…
All of these issues are entirely due to the tokenization scheme. Literally all of them You could get this behavior implemented perfectly with constrained text gen techniques like grammars or any of the various libraries implementing constrained text gen (i.e. guidance)
I had briefly looked into Guidance and others (LMQL, Outlines) but I couldn't figure out how to use them for this problem.
I could think of how to use them to prevent the LLM from generating digits for numbers greater than ten by using a regex plus a constraint that forbids digits, but the main problem is the other part of the rule, i.e. numbers above 10 should never be spelled out and should be written as digits instead. For that I presume you need to identify the spelled out numbers first, for which you presumably would need the LLM so you're back to LLM fallibility.
Any pointers would be greatly appreciated.
Re: What can LLMs never do?
#254> If there exist classes of problems that someone in an elementary school can easily solve but a trillion-token billion-dollar sophisticated model cannot solve, what does that tell us about the nature of our cognition? I think what it tells us is that our cognition is capable of more than just language modeling. With LLMs we are discovering (amazing) capabilities and the limits of language models. While language mode…
But even without going this far (with integrating various other specialized or having an LLM use them when required), an LLM is probably able to recognize a sudoku puzzle when it sees one, and even tho it itself can't solve it, I think it can easily write the code that would solve sudoku. So instead of hooking it to a set of pre built models, it might be enough to hook it to a python interpreter
Re: What can LLMs never do?
#255> If there exist classes of problems that someone in an elementary school can easily solve but a trillion-token billion-dollar sophisticated model cannot solve, what does that tell us about the nature of our cognition? I think what it tells us is that our cognition is capable of more than just language modeling. With LLMs we are discovering (amazing) capabilities and the limits of language models. While language mode…
But even without going this far (with integrating various other specialized or having an LLM use them when required), an LLM is probably able to recognize a sudoku puzzle when it sees one, and even tho it itself can't solve it, I think it can easily write the code that would solve sudoku. So instead of hooking it to a set of pre built models, it might be enough to hook it to a python interpreter
Many LLMs are already linked to Python interpreters, but they still need some improvement with recognizing when they need to write some code to solve a problem.
Re: What can LLMs never do?
#256Earlier quoted context omitted.
I am extremely alarmed by the number of HN commenters who apparently confuse "is able to generate text that looks like" and "has a", you guys are going crazy with this anthropomorphization of a token predictor. Doesn't this concern you when it comes to phishing or similar things? I keep hoping it's just short-hand conversation phrases, but the conclusions seem to back the idea that you think it's actually thinking?
Do you have mechanistic model for what it means to think? If not, how do you know thinking isn't equivalent to sophisticated next token prediction?
How do you know my cat isn't constantly solving calculus problems? I also can't come up with a "mechanistic model" for what it means to do that either.
Further, if your rubric for "can reason with intelligence and have an opinion" is "looks like it" (and I certainly hope this isn't the case because woo-boy), then how did you not feel this way about Mark V. Shaney?
Like I understand that people live learning about the Chinese Room thought experiment like it's high school, but we actually know it's a program and how it works. There is no mystery.
Re: What can LLMs never do?
#257Earlier quoted context omitted.
> If a model is trained on a sentence of the form "A is B", it will not automatically generalize to the reverse direction "B is A". This is the Reversal Curse. This is not a tokenization artefact. And furthermore it's a problem for human brains as well. Let's say you get a name, idk, Tom Cruise. You immediately know what his face looks like. Now let's say you get a random face. How quickly would you be able to tell m…
That's not a great example. Remembering a face is memory recall, whereas what's at stake here is LLMs not being able to infer simple relationships - if it learns from data that "John owns the red bicycle", it will succeed at answering "what does John own", but not "who owns the red bicycle". The relationship it learns is unidirectional. Here's the intro to the paper that brought this to light: https://www.lesswrong.c…
If you read the paper again, they deal with pre-training data and fine tuning data specifically. Their test is on information being pulled out zero-shot, which would mean the steps when attention finds associations between tokens are one directional. This is just testing recall as well, as such my example is as apples to apples you can get when comparing systems with such large complexity disparities.
In-context reasoning tends to work a lot more reliably for these examples, if you put any of the test statements into it directly before asking the question, practically any llm can answer correctly. That's why very small models are still useful for RAG use cases.
Re: What can LLMs never do?
#258There’s many things they can’t do. Even a simple rule like “ensure that numbers from one to ten are written as words and numbers greater ten as digits in the given text” fails for me for so many examples even if it works for many others; few shot, chain of thought, many versions of the prompt, it doesn’t matter. Sometimes LLMs will even change the number to something else, even with temp set to 0. And then there’s th…
You constructed a task that no-one understands and then you even admit that it, despite that, actually succeeds most of the times. Sounds like a massive win for the LLMs to me.
Re: What can LLMs never do?
#259The article should be titled " What can LLM never do, yet". By definition, Large Language Models would keep growing larger and larger, to be trained on faster and more advanced hardware, and certain points like "completing complex chains of logical reasoning" tasks, would be just a time hurdle. Only time will tell.
We really don't need time to tell. Just making a transformer bigger and bigger, and feeding it more and more data, will not change it from being a language model into something else, anymore than scaling up an expert system such as Cyc will transform it into something other than an expert system. "Scale it up and it'll become sentient" is one of the recurring myths of AI.. a bit odd that people are falling for it aga…
Thanks for that final paragraph! I'm going to quote you from now on, when trying to explain to someone (for the thousandth time) why ChatGPT isn't about to become super-intelligent and take over the world.
Re: What can LLMs never do?
#260GPTs are trained on natural language.
Why should it surprise anyone that it would fail at cellular automata?