Live data from Hacker News

After outages, Amazon to make senior engineers sign off on AI-assisted changes

arstechnica.com

331–340 of 510 posts

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#331
post #265

What are we going to do about software for critical infrastructure in the coming decade? Feels inevitable that code for aviation will slowly rot from the same forces at play but with lethal results.

It will be written much the same as now, but using AI to improve quality through code inspection etc.

Just because nearly all software is going to be written by AI, does not mean critical infrastructure will be.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#332

Reviewing AI generated code at PR time is a bottleneck. It cancels most of the benefits senior leadership thinks AI offers (delivery speed). There’s also this implicit imbalance engineers typically don’t like: it takes me 10 min to submit a complete feature thanks to Claude… but for the human reviewing my PR in a manual way it will take them 10-20 times that. Edit: at the end real engineers know that what takes effor…

Most AI advocates I know believe this period, reviewing every line of code, will come to an end when models improve. So there will be no bottleneck. We will simply test and ship, with AI doing all the code and review.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#333
post #40

>Junior and mid-level engineers can no longer push AI-assisted code without a senior signing off Review by a senior is one of the biggest "silver bullet" illusions managers suffer from. For a person (senior or otherwise) to examine code or configuration with the granularity required to verify that it even approximates the result of their own level of experience, even only in terms of security/stability/correctness, r…

> requires an amount of time approaching the time spent if they had just done it themselves It's actually often harder to fix something sloppy than to write it from scratch. To fix it, you need to hold in your head both the original, the new solution, and calculate the difference, which can be very confusing. The original solution can also anchor your thinking to some approach to the problem, which you wouldn't have…

In fairness though, it does give you good practice for the essential skill of maintaining / improving an old codebase.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#334

I think the deeper need is a "self-review" flow. People push AI-reviewed code like they wrote it. In the past, "wrote it" implies "reviewed it." With AI, that's no longer true. I advocate for GitHub and other code review systems to add a "Require self-review" option, where people must attest that they reviewed and approved their own code. This change might seem symbolic, but it clearly sets workflows and expectations…

Self review is #1

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#336

I think the deeper need is a "self-review" flow. People push AI-reviewed code like they wrote it. In the past, "wrote it" implies "reviewed it." With AI, that's no longer true. I advocate for GitHub and other code review systems to add a "Require self-review" option, where people must attest that they reviewed and approved their own code. This change might seem symbolic, but it clearly sets workflows and expectations…

If someone was confident enough to push through an AI change without even reading/reviewing it themselves adding more buttons to the UI isn't going to change anything.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#337

>Junior and mid-level engineers can no longer push AI-assisted code without a senior signing off Review by a senior is one of the biggest "silver bullet" illusions managers suffer from. For a person (senior or otherwise) to examine code or configuration with the granularity required to verify that it even approximates the result of their own level of experience, even only in terms of security/stability/correctness, r…

Going to systemically turn off your senior staff over time also. Most Senior Engineers aren't that interested in doing even more code review.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#338
post #288

Earlier quoted context omitted.

When I was really early in my career, a mentor told me that code review is not about catching bugs but spreading context (i.e. increasing bus factor.) Catching bugs is a side effect, but unless you have a lot of people review each pull request, it's basically just gambling. The more expensive and less sexy option is to actually make testing easier (both programmatically and manually), write more tests and more levels…

I think of code review more about ensuring understandability. When you spend hours gathering context, designing, iterating, debugging, and finally polishing a commit, your ability to judge the readability of your own change has been tainted by your intimate familiarity with it. Getting a fresh pair of eyes to read it and leave comments like "why did you do it this way" or "please refactor to use XYZ for maintainabili…

Alternately, have a small team where you trust everyone.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#339

>Junior and mid-level engineers can no longer push AI-assisted code without a senior signing off Review by a senior is one of the biggest "silver bullet" illusions managers suffer from. For a person (senior or otherwise) to examine code or configuration with the granularity required to verify that it even approximates the result of their own level of experience, even only in terms of security/stability/correctness, r…

Why only AI generated code? I wouldn’t let a junior or mid level developer’s code go into production without at least verifying the known hotspots - concurrency, security, database schema, and various other non functional requirements that only bite you in production. I’m probably not going to review a random website built by someone except for usability, requirements and security.

I’ve seen hundreds of PR’s produced by a junior and reviewed by a mid lvl go into prod. I don’t see any problem with that

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#340
I anticipate they will fix this by adding better AI evaluation tools that work better to test their infra and changes.

In the meantime they will be quite a bit slower I’d imagine.

Also wonder if those seniors will ever get to actually do any engineering themselves now that they’re the bottleneck. :)

Post reply on HN