I think many people are really underestimating the "intelligence" of an LLM. People have this misconception that LLMs are just complicated Markoc chains, predicting text based purely on probability. They are not. As it turns out, to accurately predict text, you need to learn a lot about the world. In fact, there's lots of fascinating things hidden in the weights of LLMs, like a geographic model of the world. [1] To m…
The problem is, this sentiment of path towards AGI assumes that everything (i.e our physical reality) can be compressed. Which is highly likely not true.
The future of AI is basically better and better compression with more data sources. You will be able to do things like ask how to build a flying car in your garage, and it will give you step by step instructions on what things you need to order from where, including CAD drawings to get custom made CNC parts, and how to put it all together (including software to run it).
As far as AGI goes, its possible that through AI assisted technology we will be able to measure synapses of a human brain in fine enough detail, or somehow mass surveil the entire population of earth and derive human brain models from that. And then, with optimizations on that, we can potentially arrive at the most efficient human brain in terms of some metric, and we will have very good robotic assistants, but nothing really beyond that.