It is superhuman at bullshitting, if it puts bullshitters in the unemployment line I won't cry tears.
It has a hypnotic ability. It's a model for "neurotypical privilege".
There's a certain kind of person who sees a glass that is 70% full and, under the influence of ChatGPT, sees a glass that is 100% full. That is, there is the blog post where the guy thought ChatGPT could play chess (anybody else tries it and it makes invalid moves) or the guy who had it write an essay that they thought was highly accurate (but it was totally wrong.)
Just like I fantasize that NFT pushers will "never work in this global village ever again", I kinda hope these people have their words and actions follow them around. If they can't see the mistakes ChatGPT makes can we trust them to see any mistakes that they make or that other people make? Do we want them writing code, driving cars, or dispensing pills? Do we want them supervising anyone who does the same?
---
Now there is ChatGPT and there are the LLMs that underlie it and the latter can perform much better than ChatGPT at narrowly defined tasks with a certain amount of specialization. Roughly, I see a lot of papers in arXiv that use "zero-shot" (prompted) learning and get 70% accuracy at a task, manage to apply some tricks and maybe get 75-80% but if you break the task into pieces and develop a training set (not so large as you needed five years ago) you might get more like 95% accuracy. Systems like that need a less complex LLM with fewer weights and easier inference because they don't need as much bullshitting capacity.
What excites me (i) are relatively simple systems that do what machine learning systems did 5 years ago but do them much better and (ii) hybrid systems that combine ideas from the new and old AI.
As an example of (ii) I'd call out AlphaGo. You can write a fairly good Go playing program using Monte Carlo Markov Chain search with "lightweight playouts" (sampling the effect of low-effort or random moves) but couple that A.I. to a neural network that makes better "hunchy" moves and now you have a great Go playing program.
Similarly for tasks like automated coding we'll see systems that couple an SMT solver/theorem prover to sequence modellers. The neural network will make better-than-average guesses which will be confirmed, constrained and repaired by the old A.I. part.
So I am excited about (i) and (ii) and think people will be doing great things but somewhere between bemused and depressed about the people who will push bubbles under the rug forever getting ChatGPT-N to almost work. On one hand there is the real promise of LLMs, on the other there are a lot of lazy people who want ChatGPT to write their pitch deck for them. And don't get me started on the general preference people seem to have for lies over truth. (e.g. be you a climate denier or a transsexual maximalist, there will be a model that speaks truthiness to you.)
---
Personally LLMs have revived a project that I first starting thinking about 18 years ago.