Why AI is failing at giving good advice

> truly good advice can only be rooted in the care and experience of another human being. And it's not just the care that the other person has for you, but the understanding that the advice has come from a place of experience of human suffering.

Can another person give you truly good advice over email? Is it still possible if that person has never met you before, so that the advice is only based on what you have written to that person?

If yes, then it is possible in theory for an AI to give you exactly the same advice as this other human being.

How can you then say that email A from the AI is bad, while the exact same email B from a human is truly good?

> ChatGPT is like the person giving advice about the stock market. It only speaks of averages that are already in the direction of the general movement of the masses.

You have to distinguish between how AIs are trained (or rather, selected for) and whether that can lead to intelligence.

AIs are selected for how well they predict text.

Humans were naturally selected for how well they reproduce.

Neither one automatically leads to intelligence. But that doesn't mean that you have scientifically proven that either type of selection can never lead to intelligence.

A big LLM has on the order of 10^12 parameters. At the base level, it is not that different from a regular 10 TB hard drive. Even if each fact is only a byte big, you could't store more than 10^13 different facts before it starts running out of space. Your address space (or input vector) is only 13 digits long, or let's say 30 letters.

So you only have up to all combinations of 30 letters as the input vector, or about six words. The output vector (response / fact) is just one byte in this example. That is tiny compared to the length of the conversations and the amount of facts that today's LLMs can recall.

During training, the LLM evolves ever smarter compression to store as much information as possible. At some point, it becomes necessary to start developing some sort of model of the world and how different things interact to improve the compression even more.

Now, that doesn't mean that it can do this perfectly, or even well. It can be infurating to argue with an LLM once you hit the limits. But I do believe that we are starting to see the building blocks of real intelligence in the bigger LLMs.

> all the help it provides displaces the need for real human beings, and thus propels society into having even more problems

Your key claim here is a completely orthogonal question.

I would say that this question becomes more and more important the more intelligent the AIs become. Not the other way around.

An AI that can perfectly mimic how you would like to interact with other people would be living in a padded cell. All struggles and challenges that allow you to grow as a person disappear. We'd become like the people in the spaceship in Wall-E.

So you're doing your key claim no favors by trying to prove that the LLMs are dumb.

Do LLMs actually store and reproduce facts? The longer I use LLMs, the lesser i trust facts as complex as "capital of australia" to be answered, truthfully.

> The most propagated (related) text on the internet will likely be repurposed in its own words to answer anything that you ask. Essentially, that means it might give you a mashed answer as you would get from the X first results on Google, but it will fill in contextual gaps from other places and make it more applicable to your specific input.

That's pretty much the way that most of the articles have been being written by humans for the last couple decades, no?

When I was a kid and needed to know how to spell a word, my teachers told me to look it up in a dictionary. This had the problem you're describing — if I could do that, I wouldn't need to.

But AI? I can ask LLMs questions like:

"A couple is buying a house in Germany. What questions do they typically forget to ask, which they regret not asking, and often wish later that someone had told them to ask?"

And it can fill in the gaps better than when I ask a human the same kind of question.

I don't expect it to be perfect, this could well be a half-arsed boilerplate, but the actual humans who I asked this of, mostly responded "Huh? I don't understand?"

https://chat.openai.com/share/f17b013e-6b24-4b49-9986-78a5e4...

(Note this chat had been given custom instructions, that's why it's responding with this unusual pattern).

LLMs could be described as storing and reproducing beliefs, but they're definitely not databases that can be easily maintained as things change or verified for perfect accuracy at any given moment.

I think the best way to use them is to have a minimal language model that is as small as possible while still being able to comprehend language; and that this then goes off to an actual knowledge base of some kind where all the factoids can be checked separately.

Humans have separate explicit/declarative memory and implicit/procedural memory memories. I think the Transformer architecture puts everything into what is really only suitable for implicit/procedural; I think RAG is trying to be a separate explicit/declarative memory system, but I'm always to busy to study this in more than a superficial level, so I'm not sure.