Gen AI will increase demand for software engineers

Definitions vary, but all definitions I have heard seem to boil down to one thing: AI is little more than a set of fancy programming languages. Which means you still always need a human to be the programmer. Nothing really can change for the white collar so long as you still have to provide instruction. That's what white collar people do.

"AGI" draws a distinction – where the machines start to be able to act as completely independent agents without needing a human to give instruction. Is this, perhaps, what you are envisioning in 10-20 years?

Things that formerly were prohibitively expensive are now cheap enough to be doable. Which increases the market for software engineers that can get things done with the help of AI and other tools. These won't be the type of software engineers that specialize in things that should be automated but the type of engineers that can build a lot of stuff that formerly would have required them to delegate a lot of work to other engineers.

This is how the software developer community has historically grown actually. It used to be that software engineers were faffing about with punch cards, assembly code, etc. Working months to produce a few kb worth of software. No libraries or anything. It all ran on bare metal. These days what engineers build is the tip of the iceberg because it all runs on frameworks, libraries, operating systems, etc. that do all of the heavy lifting. Which makes them way more productive than their colleagues from half a century ago.

Thankfully, AI is still not there yet to make "professional" (as most people would define it) apps—unless maybe when people start feeding my book contents into GenAI, ha! (I guess it's a compliment?)

Also, AI is like DB. Sure, you can use DBeaver or something to access it "raw", but 90% of us programmers are still hired to write a CRUD wrapper. Same thing with AI. It's more useful to have an app supercharged with AI wherever it makes sense.

[1] https://opinionatedlaunch.com

The theoretical research may have improved exponentially. Implementations available for use seem to be on the decline – at best, stagnant. I find less and less utility as time goes on. Things LLMs shined at a couple of years ago now produce garbage. And for creative work, the output is much too formulaic, which was all well and good initially while still novel, but one has to keep pushing new boundaries and the current crop of tools really seems to struggle with that.

I'm guessing that solving run-time learning may well require a different approach, and it's not clear that reasoning (in general form - ability to dynamically synthesize a problem-specific solution) can be just added to LLMs either (e.g. by adding tree search). There are also other missing components such as working memory that seem simpler to solve.

Coming up with brand new architectures and learning approaches is likely to take time. There have been attempts to find alternatives to gradient descent, but none very successful despite a lot of effort.

Perhaps it's just a reflection of people chasing the low hanging fruit (and as Chollet says, LLMs "sucking all the oxygen out of the room"), but architectural advance post-LLM has been minimal. In 7 years we've basically just gone from transformer paper to big pre-trained transformers.

Even when workable architectural approaches to run-time learning and reasoning have been developed, they will also need to be scaled up (another 7 years?), and will also be competing with LLMs for mindshare and dev. resources as long as scaling LLMs continues to be seen as profitable.

The timescale for coming up with new architectures and approaches is hard to predict. AGI prediction timeframes have always been wrong, and the transformer was really one of history's accidental discoveries. Who'd have guessed that a better seq-to-seq model would create such capabilities!

If I had to guess, I'd say human-level AGI (human-level in terms of both capability and generality) is still 15-20 years away at least. 7 years to go from small transformers to big transformers doesn't make me optimistic that architectural innovation is going to happen very quickly, and anyways this is an unpredictable research problem, not an engineering one.