AI Is Smoke and Mirrors

It seems some companies adopting GenAI at this stage are doing so, at least in part, because they want to believe (that they can replace workers), rather than out of any sober analysis of what it can actually do. Maybe there's an element of FOMO too, and companies wanting to use GenAI because they hear everyone talking about it.

No doubt LLMs will continue to improve, but what remains to be seen is if there will be a direct path from LLMs to AGI (which is where the real value gets unlocked), or if we'll just continue to see quantitative improvement in benchmark scores, hallucination reduction, etc, but not much qualitative change in the types of task they are capable of.

I do think that AGI is inevitable, maybe not even that far off (but certainly not next 5 years, probably not next 10 years *), but there's a lot missing from LLMs to get there, and it seems that at least one critical piece, online learning, may require a different approach.

* Note that it's already been 7 years since the transformer paper came out, and all we've really seen since then is a bunch of engineering work in making them more efficient and how best to train them. We haven't yet seen any advances in "cognitive architecture", or even any widespread recognition that there is a need to do so. If all people are doing for the foreseeable future is building pre-trained LLMs, then that is all we will get.

LLMs sound so sure of themselves and people think "well i'm dealign with the most advanced technology ever so it must be right...".

On the implementation side of things, it's hard for me to get the non-deterministic aspect of llms right in my head. I put an LLM and RAG system in prod with a team and went through rounds of the usual testing. 99 times it passed but on test 100 it would fail, so you'd adjust the system prompt. Then it'd pass 500 times and fail at 501. Adjust the system prompt, then it would pass 9 times and fail at 10. That system went to production but there's the low level worry in my mind, when is it going to fail to give the correct output? The fact that you can never guarantee the output of an LLM from a given input severely limits where they should be used IMO. I don't think it's wise to have the output of an LLM be the input to another program, there's no functional relationship between domain and range with an LLM.

That problem is usually met with "well, a human would make the same mistake.." but the reason computers exist is to do long, tedious, lists of tasks/instructions very fast that humans get wrong. Simulating a human, and all those imperfections, with digital logic seems contradictory to me.

edit: Also, just want to point out that the "testing" mentioned in my post was all manually done by humans. You can't automate testing the response of an llm unless you use another model to grade the response as correct or not but then you're right back to not being able to trust that the grader will always act consistently.

So how do you know this exactly?

What the naysayers always miss is exponential growth. They don’t look towards the future.

This is quite a pat response. There are plenty of critics within the AI/ML field of the unbounded growth hypothesis that you are effectively fronting here. I don't think you really believe that they "don't look towards the future", or that they aren't aware of the concept of exponential growth.

I'd be interested in a cohesive response to their arguments. But in the above post at least, I'm not seeing one.