> Where on the S-shaped curve are we?
No-one knows.
We can however make educated guesses based on:
- Extrapolating the last couple of years of progress.
- Scientific studies of LLM quality scaling in proportion to input data size, parameter count, and total training compute ops.
- How much more training data is available.
- Reasonable budgets, especially the cost-efficiency of inference, which appears to be more limiting than training, especially for long-term usage and profitability.
- Efficiencies such as 4-bit quantization, algorithmic improvements, etc...
Based on the above, there is at least a factor of 10x scaling available in many directions within a year or two, but not likely all at once. E.g.: 10x context window size at the same intelligence level? Sure! 10x inference speed? Doable! 10x cost reduction? Coming soon!
All at once? Not yet, and not for a while. Everyone is hardware constrained, and demand is pushing up prices and limiting the training scale maximums.
Maximum intelligence is much harder to predict. The current generation of AIs are trained on human-authored text as their input data. They're trained to predict that text. That means that they're "blurry JPEGs of the Internet". More training might make them "sharper JPEGs", but not necessarily smarter because the Internet didn't get smarter.
They'll better model humans, but they'll still be modelling humans, not superhumans.
That can be fixed with self-training, etc... but that will take a lot longer. I'm guessing 5-10 years, which is in line with the predictions of AI futurists that seem to know what they're talking about.
Many of our jobs will remain safe... for now.