121–130 of 175 posts
I feel even if the models are stagnating, the tooling around them, and the integrations and harnesses they have are getting significantly more capable (if not always 'better' - the recent vscode update really handicapped them for some reason). Things like the new agent from booking.com or whatever, if it could integrate with all hotels, activities, mapping tools, flight system, etc could be hugely powerful. Assuming…
I feel that two things are true at the same time: 1) Something happened during 2025 that made the models (or crucially, the wrapping terminal-based apps like Claude Code or Codex) much better. I only type in the terminal anymore. 2) The quality of the code is still quite often terrible. Quadruple-nested control flow abounds. Software architecture in rather small scopes is unsound. People say AI is “good at front end”…
Earlier quoted context omitted.
The issue with llm’s is trust. I don’t see that ever going away. Humans have learned to trust other humans over a large time scale with rules in place to control behaviour.
When the dust settles, for example if LLM's were to stop improving today, we would come to learn their exact capabilities, what they can do reliably and what they can't. Once we know what they can do well and how to get them to do it well, and what they can't, you could say we "trust" them to do the first category well and just stop trying to get it to do the second category.
I feel that two things are true at the same time: 1) Something happened during 2025 that made the models (or crucially, the wrapping terminal-based apps like Claude Code or Codex) much better. I only type in the terminal anymore. 2) The quality of the code is still quite often terrible. Quadruple-nested control flow abounds. Software architecture in rather small scopes is unsound. People say AI is “good at front end”…
I have heard say that the change was better context management and compression.
I feel that two things are true at the same time: 1) Something happened during 2025 that made the models (or crucially, the wrapping terminal-based apps like Claude Code or Codex) much better. I only type in the terminal anymore. 2) The quality of the code is still quite often terrible. Quadruple-nested control flow abounds. Software architecture in rather small scopes is unsound. People say AI is “good at front end”…
> 1) Something happened during 2025 that made the models (or crucially, the wrapping terminal-based apps like Claude Code or Codex) much better. I only type in the terminal anymore. I have heard say that the change was better context management and compression.
200k and now 1M contexts. Better context management was enabled by improvements in structured outputs/tool calling at the model level. Also reasoning models really upped the game “plan” mode wouldn’t work well without them.
[flagged]
Earlier quoted context omitted.
Yes, I think this is basically an instance of the "emergent abilities mirage." https://arxiv.org/abs/2304.15004 If you measure completion rate on a task where a single mistake can cause a failure, you won't see noticeable improvements on that metric until all potential sources of error are close to being eliminated, and then if they do get eliminated it causes a sudden large jump in performance. That's fine if you ju…
> until all potential sources of error are close to being eliminated This is what PSP/TSP did - one has to (continually) review its' own work to identify most frequent sources of (user facing) defects. > if you also want to predict future performance, you need to break it down into smaller components and track each of them individually. This is also one of tenets of PSP/TSP. If you have a task with estimate longer th…