AGI fantasy is a blocker to actual engineering

The rise of populist polemic as the most effective means for driving public behavior is also downstream from 'neutral technical solutions' designed to 'maximize engagement (anger) to maximize profit'. This is not actually a morally neutral choice and we're all dealing with the consequence. Adding AI is fuel for the fire.

Assumptions you are making:

- AI = transformer ANNs

- People sceptical of transformer ANNs directly leading to AGI within any reasonable period are also sceptical of transformer ANNs directly leading to AGI any time in the far future

This kind of generalisations don't help you as the huge number of comments underneath yours likely shows

Historically software that made humans more efficient resulted in empowerment for the individual, and also created a need for new skilled roles. Efficiency gains were reinvested into the labor market. More people could enter into higher paying work.

With generative AI, if these companies achieve their stated goals, what happens to the wealth generated by the efficiency?

If we automate agriculture and manufacturing, the gain is distributed as post-scarciaty wealth to everyone.

If we automate the last few remaining white-collar jobs that pay a living wage, the gain is captured entirely by the capital owners & investors via elimination of payroll, while society only loses one of its last high-paying ladders for upward mobility.

Nobody lost their career because we built a faster operating system or a better compiler. With generative AI's stated goals, any efficiency gains are exclusively for those at the very top, while everyone else gets screwed.

Now, I'll concede and say, that's not the AI companies' fault. I'm not saying we shouldn't magically stop developing this technology, but we absolutely need our governments to start thinking about the ramifications it can have and start seriously considering things like UBI to be prepared for when the bottom falls out of the labor market.

It’s too easy to dismiss others’ idiosyncrasies and miss the signal. And the story involves a successful and capable person communicating poetically about an area they have a track record in that probably the author of this article and most of us can’t compete with.

I am struck by any technical person that still thinks AGI is any kind of barrier, or what they expect the business plan of a leader in AI, with a global list of competitors, is supposed to look like?

AGI is talked about like a bright line, but it’s more a line of significance to us than any kind of technical barrier.

This isn’t writing. Although that changed everything.

This isn’t the printing press. Although that changed everything.

This isn’t the telegraph. Although that changed everything.

This isn’t the phonograph, radio communication, the Internet, web or mobile phones. Although those changed everything.

This is intelligence. The meta technology of all technology.

And intelligence is the part of the value chain that we currently earn a living at. In the biosphere. In the econosphere.

The artificial kind is moving forward very fast, despite every delay seeming to impress people. “We haven’t achieved X yet” isn’t an argument at any time, but certainly not in the context of today’s accelerated progress.

It is moving forward faster than any single human, growing up from birth, ever has or ever will, if it helps to think of it that way.

Nor is, “they haven’t replaced us yet” an argument. We were always going to be replaced. We didn’t repeal the laws of competition and adaptation “this time”.

Our species was never going to maintain supremacy after we unleashed technology’s ability to accumulate capabilities faster than we or any biological machine could ever hope to evolve.

It isn’t even a race is it? How fast is the Human Bio Intelligence Enhancements Department going? Or the Human Intelligence Breeding Club? Not very fast I think.

Very few AI die hards ever imagined we would be anywhere near this close to AGI today, in 2025, even five years ago, circa Ancient (i.e. January) 2020. There is a dose of singularity.

Yet in retrospect, 99% of AI progress is attributable to faster and more transistors. Today’s architectures fine tune algorithms that existed in the mid-1980’s. Getting here was more about waiting for computer hardware to be ready than anything else. Current investments don’t reflect that main driver stalling, but exploding.

Once we have AGI, we will have already passed it. Or, more accurately, it will have passed us. Don’t spend much time imagining a stable karmic world of parity. Other than as a historically nice trope for some fun science fiction where our continued supremacy made for good story. That’s not what compounding progress looks like.

Chaotically compounding progress has been the story of life. And then tech. It isn’t going to suddenly stop for us.

What an odd and transparently motivated thought.