> Outside of the most niche domains like frontier AI development, are there software engineering questions that Sol 5.6/Fable 5 can't answer better than the median software engineer if explicitly asked?
"Median" is kinda hard to judge given how rapidly the number of software engineers has been growing, how imprecise our within-industry skill assessments are, and also how fast everything within software was shifting even before agentic coding. Two examples: (1) an old estimate I heard was that 50% of the skills we would put on our CVs would be obsolete every few years, but we'd never know which ones; (2) The reason for the creation of FizzBuzz as an interview question (far too many people are *that* bad).
For the LLMs themselves: the METR time horizons estimates are based on an expert developer's wall-clock times, and the graph on their website only goes as far as Mythos preview which is kinda about this level. Mythos preview shows only 80% success at tasks that would take a human 3h6m on average (95% confidence interval 1h37m-6h39m), which is evidence that yes, there is plenty that the models can't do. I naturally break down larger tasks to smaller ones, the agentic coding models I've used also attempt to do this, but seem to be worse at it than I am.
> It strikes me that models already have superhuman software architecture capabilities, they are simply underelicited.
While they can do better when elicited correctly (and the same is true for humans), they are too "lazy" by themselves to attempt self-elicitation.
The second part, just like the first, is also true for humans; but LLMs are so much cheaper than humans that this is a disappointment for the LLM to stop early and say "[Thought for 35m 24s] Here's your MVP, I've not done subtasks 35 onwards", even though it's an advantage for the humans to say "[Invoice for a 2 week sprint with a team of 5 developers and their manager for a total cost of $23,000] Here's your MVP, we can discuss which of subtasks 35 onwards you want in the next sprint".
I'm not sure where I'd rate LLMs on a strict 0-10 scale from "hack" to "formal methods followed by end-to-end testing", but the examples given by the research companies themselves suggest that even they are only at the "this is starting to get interesting" level for both architecture and their own idea of what "I have now completed this to a good quality" looks like, not at the "actually good" level for complete big projects.
> Furthermore the AI companies are spending gobs of money and compute to address this—harness development, orchestration/agent coordination, computer use are all designed to lower the skill level required to get the best results out of a given model.
The observation is that money and compute spent are not strongly correlated with the general quality of the models. If the correlation was strong then Meta would be near the top, while the multitude of open-weight Chinese models that keep popping to the top of leaderboards every couple of months would be jokes. Grok would have been better from the start and not suddenly gotten good around when they bought up the much smaller and lower-budget Cursor.
https://llm-stats.com/models/muse-spark-1.1
https://llm-stats.com/models/gpt-5.6-sol
https://llm-stats.com/models/kimi-k3
> Soon the models themselves will be more persistent, more observant, and more skilled than any of us at driving models to obtain the "right" solution in a given SWE context.
My experience has been that predictions of "soon" fare about as poorly as predictions of "never" in the context of AI. Regarding the failures of "soon" predictions, I do not know the degree to which this is company leadership BSing how good their stuff is, vs. Goodhart's law where they're fooled by the difference between their metrics and reality.