Okay I think this somewhat answers my question. Is this individual a solo developer? “Triaging GitHub issues” sounds a bit like open source solo developer. Guess I’m just desperate for an article about how organizations are actually speeding up development using agentic AI. Like very practical articles about how existing development processes have been adjusted to facilitate agentic AI. I remain unconvinced that agen…
Can't believe you don't know who the author is my man.
Different folks are interested in different niches. I don't know this author either. I would know many names from other subfields, though.
I once went to a meetup where the host introduced the speaker with "he needs no introduction". Well to this day I've no idea who the speaker was. Familiarity really shouldn't be assumed beyond a very, very small handful of people.
You trust your natural language instructions thousand times a day. If you ask for a large black coffee, you can trust that is more or less what you’ll get. Occasionally you may get something so atrocious that you don’t dare to drink, but generally speaking you trust the coffee shop knows what you want. It you insist on a specific amount of coffee brewed at a specific temperature, however, you need tools to measure. A…
The person I'm replying to believes that there will be a point when you no longer need to test (or review) the output of LLMs, similar to how you don't think about the generated asm/bytecode/etc of a compiler. That's what I disagree with - everything you said is obviously true, but I don't see how it's related to the discussion.
I don't necessarily think we'll ever reach that point and I'm pretty sure we'll never reach that point for some higher risk applications due to natural language being ambiguous.
There are however some applications where ambiguity is fine. For example, I might have a recipe website where I tell a LLM to "add a slider for the user to scale the number of servings". There's a ton of ambiguity there but if you don't care about the exact details then I can see a future where LLMs do something reasonable 99.9999% of the time and no one does more than glance at it and say it looks fine.
How long it is until we reach that point and if we'll ever reach that point is of course still up for debate, but I dnt think it's completely unrealistic.
If natural language is used to specify work to the LLM, how can the output ever be trusted? You'll always need to make sure the program does what you want, rather than what you said.
You trust your natural language instructions thousand times a day. If you ask for a large black coffee, you can trust that is more or less what you’ll get. Occasionally you may get something so atrocious that you don’t dare to drink, but generally speaking you trust the coffee shop knows what you want. It you insist on a specific amount of coffee brewed at a specific temperature, however, you need tools to measure. A…
More analogy.
What’s to stop the barista putting sulphuric acid in your coffee? Well, mainly they don’t because they need a job and don’t want to go to prison. AIs don’t go to prison, so you’re hoping they won’t do it because you’ve promoted them well enough.
The author makes a point that you should redo every manual commit with AI to align you mental model of actions with how models work. This is something that I’m going to need to try. It’s related to my desire to reduce things like “discovery tax” (the phenomenon whereby a 5 minute agent task is 4 minutes of environment exploration and 1 minute of execution) and makes sure that models get things right the first time around, however, my AI improvement plan didn’t really account for how to improve the model in cases where I ended up manually resolving issues or implementing features.
Some arguments are made about retaining focus and single-mindedness while working on AI. I think these points are important. It’s related to the article on cutting out over-eager orchestration and focusing on validation work (https://sibylline.dev/articles/2026-01-27-stop-orchestrating...). There are a few sides to this covered in the article. You should always have high value task to switch to when the agent is working (instead of scrolling tiktok, instagram,X, youtube, facebook, hackernews .etc). In my case I might try start to read some books that I have on the backburner like Ghost in the Wires. You should disable agent notifications and take control of when you return to check the model context to be less ADHD ridden when programming with agents and actually make meaningful progress on the side task since you only context switch when you are satisfied. The final one is to always have at least one agent and preferably only one agent running in the background. The idea is that always having an agent results in a slow burn of productivity improvements and a process where you can slowly improve the background agent performance. Generally, always having some agent running is a good way to stay on top of what current model capabilities are.
I also really liked the idea of overnight agents for library research, redevelopment of projects to test out new skills, tests and AGENTS.md modifications.
What I don't understand about these arguments is that the input to the LLMs is natural language, which is inherently ambiguous. At which point, what does it even mean for an LLM to be reliable? And if you start feeding an unambiguous, formal language to an LLM, couldn't you just write a compiler for that language instead of having the LLM interpret it?
1) Determinism isn't the same as reliability. Compilers are deterministic (modulo bugs), but most things in life are not, but can still be reliable. The opposite also holds: "npm install && npm run build" can work today and fail in a year (due to ecosystem churn) even though every single component in that chain is deterministic. 2) Reliability is a continuum, not a discreet yes/no. In practice, we want things to be r…
> I don't presume this will immediately change your mind
I'm not saying that AI isn't useful. I'm just claiming it's not analogous to a compiler. If it was, you would treat your prompts as source code, and check them into source control. Checking the output of an LLM into source control is analogous to committing the machine code output from a compiler into source control.
My question still stands though. What does it mean for a tool to be reliable when the input language is ambiguous? This isn't just about the LLM being nondeterministic. At some point those ambiguities need to be resolved, either by the prompter, or the LLM. But the resolution to those ambiguities doesn't exist in the original input.
> Don't become famous with illegal stuff. Is this still a constraint? It would seem that money beyond a certain point allows one to wash ones reputation.
For sure. But becoming famous for robbing the bank is not the way.
It worked for most of our current political elites.
You trust your natural language instructions thousand times a day. If you ask for a large black coffee, you can trust that is more or less what you’ll get. Occasionally you may get something so atrocious that you don’t dare to drink, but generally speaking you trust the coffee shop knows what you want. It you insist on a specific amount of coffee brewed at a specific temperature, however, you need tools to measure. A…
More analogy. What’s to stop the barista putting sulphuric acid in your coffee? Well, mainly they don’t because they need a job and don’t want to go to prison. AIs don’t go to prison, so you’re hoping they won’t do it because you’ve promoted them well enough.
The person I'm replying to believes that there will be a point when you no longer need to test (or review) the output of LLMs, similar to how you don't think about the generated asm/bytecode/etc of a compiler. That's what I disagree with - everything you said is obviously true, but I don't see how it's related to the discussion.
I don't necessarily think we'll ever reach that point and I'm pretty sure we'll never reach that point for some higher risk applications due to natural language being ambiguous. There are however some applications where ambiguity is fine. For example, I might have a recipe website where I tell a LLM to "add a slider for the user to scale the number of servings". There's a ton of ambiguity there but if you don't care…
That's true, and I more or less already use it that way for things like one off scripts, mock APIs, etc.
Really appreciated this perspective. Much more tempered and less hype than a lot of other articles I see.
I thought the "dont let agents finishing interrupt you" workflow was an interesting point. I've set up chime hooks to basically do the opposite, and this gives me pause to wonder if I'm underestimating the cost of context switching. I'll give it a go.
This is such a lovely balanced thoughtful refreshingly hype-free post to read. 2025 really was the year when things shifted and many first-rate developers (often previously AI skeptics, as Mitchell was) found the tools had actually got good enough that they could incorporate AI agents into their workflows. It's a shame that AI coding tools have become such a polarizing issue among developers. I understand the reasons…
I skimmed over it, and didn’t find any discussion of: - Pull requests - Merge requests - Code review I feel like I’m taking crazy pills. Are SWE supposed to move away from code review, one of the core activities for the profession? Code review is as fundamental for SWE as double entry is for accounting. Yes, we know that functional code can get generated at incredible speeds. Yes, we know that apps and what not can b…
I've only recently started trying out using LLMs to help me write code (as in, within the last two weeks), and the workflow that makes the most sense to me is to not let the LLM anywhere close to PRs/MRs/CRs, or even version control at all. I've found it useful to give it a fairly constrained task (something that might be a 100-200 line modification of my current code), literally watch the output of Claude Code's "thinking" as it goes to potentially interrupt it if it's going down the wrong path or if it gives me a better idea, wait for it to present the code, and then read through all of it to make sure it's what I want. After making whatever small changes I might want, I commit, and then move onto the next thing. So far, this has pretty much all been for personal side projects outside of work, so there is no code review, but approaching it from the standpoint that the goal is produce the same code and version control history I would want if I created it by hand and just using the LLM as way of automating the typing, I've been pretty surprised that it's already been a net gain in efficiency for a lot of things I've been working on. Ideally, the code I'm generating shouldn't be distinguishable from what I'm already writing, because I would change it if I saw that it was. At that point, either it's high-quality enough to be merged, or it's not and should be rejected, and that's already how things work in the first place. If someone makes an MR that their coworkers find sloppy and annoying to review, there needs to be pushback, and how it was generated should be irrelevant if everyone is on the same page about where the bar for quality is and is acting in good faith. (If you're working in an environment where there's no bandwidth to care about quality or people are acting in bad faith, LLM code will probably not be much of an improvement, but you're also probably going to have a bad time regardless, and unfortunately I don't think there's a magic bullet for fixing that).