Developing with GitHub Copilot Agent Mode and MCP

You pretty much just have to ask and give them access for these things. Talking to a stakeholder and translating jargon and domain terms? Trivial. They can churn through specs and find issues, none of that seems particularly odd to ask of a decent LLM.

> Do they help with testing? Coming up with tests plan, writing test code, running them, analysing the output of the various tools and producing a cohesive report of the defects?

This is pretty standard in agentic coding setups. They'll fix up broken tests, and fix up code when it doesn't pass the test. They can add debug statements & run to find issues, break down code to minimal examples to see what works and then build back up from there.

> Do they help with maintenance? Taking the same software and making changes to keep it churning on new platforms, through dependencies updates and bug fixes?

Yes - dependency updates is probably the easiest. Have it read the changelogs, new api docs and look at failing tests, iterate to have it pass.

These things are progressing surprisingly quickly so if your experience of them is from 2024 then it's quite out of date.

GitHub has gh, there's open source jira-cli, Cloudflare has wrangler and so on. No configuration needed, just mention on the agent doc that this kind of tool is available. Likely it will figure out the rest.

And if you have more complicated needs, then you can combine the commands, add some jq magic, put to package.json and tell agent to use npm run to execute it. Can be faster than doing it via multiple MCP calls.

Selecting one of the implementation over the other is design, aka making decisions. Sometimes you have to prototype it out to where which parameters is the best. And sometimes a decision can revert an earlier one and you have to investigate the impact radius of that change.

But coding is straightforward translation. The illusion of going faster is that we forego making decisions. Instead we're hoping that the agent makes the correct ones based on some general direction, forgetting than an inch deviation can easily turn into a mile error. The hopeful things would have been an iteration, adding the correct decisions and highlighting the bad ones to avoid. But no one have that kind of patience. And those that use LLMs often finish with a "LGTM" patch.

The normal engineering is to attain a critical mass of decisions and turns that immediately to formal notation which is unambiguous. Then we validate with testing if that abrupt transformation was done properly. But all the decisions were made with proper information.

Because we only see very disjointed descriptions, with no attempt to quantify what we're talking about.

For every description of how LLMs work or don't work we know only some, but not all of the following:

- Do we know which projects people work on? No

- Do we know which codebases (greenfield, mature, proprietary etc.) people work on? No

- Do we know the level of expertise the people have? Is the expertise in the same domain, codebase, language that they apply LLMs to?

- How much additional work did they have reviewing, fixing, deploying, finishing etc.?

Even if you have one person describing all of the above, you will not be able to compare their experience to anyone else's because you have no idea what others answer for any of those bullet points.

And that's before we get into how all these systems and agents are completely non-deterministic, and works now may not work even 1 minute from now for the exact same problem.

And that's before we ask the question of how a senior engineer's experience with a greenfield project in React with one agent and model can even be compared to a bon-coding designer in a closed-source proprietary codebase in OCaml with a different agent and model (or even the same, because of non-determinism).

Same way it works with humans: someone tells it what "correct" means until it gets it right.