I work in DevOps at a firm that has been very enthusiastic about using LLMs (in the good sense).
The phases were basically:
- try out having the LLM do "a lot"
- now even more
- now run multiple agents
- back to single agents but have the agents build tools
- tools that are deterministic AND usable by both the humans (EDIT: and the LLMs)
The reasons:
1. Deterministic tools (for both deployments and testing) get you a binary answer and it's repeatable
2. In the event of an outage, you can always fall back to the tool that a human can run
3. It's faster. A quick script can run in Really, we are back to this article: https://spawn-queue.acm.org/doi/10.1145/3194653.3197520 aka "make a list of tasks, write scripts for each task, combine the scripts into functions, functions become a system"
-- END of original post --
What I would add:
if you let LLMs do whatever they want, they will happily make code. You can add tests to confirm that the tests work (which you used to do with human code, right?). You can also read the code.
When you read the code, you'll find that they sometimes do totally bananas things that still produce working code (I've seen humans do this too but that's another story).
In other words, you still need to make sure the system being built makes sense.
More succinctly:
Coding may be dead but software engineering is alive and kicking.