Live data from Hacker News

After outages, Amazon to make senior engineers sign off on AI-assisted changes

arstechnica.com

401–410 of 510 posts

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#401

Earlier quoted context omitted.

By definition, coding agents are right now the worse they will ever be and the industry as a whole by definition is the least experienced it will ever be at using then. So many people on HN are so insulted that the people who put money in our bank accounts and in some cases stock in our brokerage accounts ever cared about their bespoke clean code, GOF patterns and they never did. LLM just made it more apparent. It’s…

Wow you have completely lost the plot. It’s like you’re a bot that’s mixing up who he’s replying to.

Just maybe you aren’t making the strong argument you think you are making

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#402
One challenge with code review as an antidote to poor quality gen-AI code, is that we largely see only the code itself, not the process or inputs.

In the pre-gen-AI days, if an engineer put up a PR, it implied (somewhat) they wrote their code, reviewed it implicitly as they wrote it, and made choices (ie: why is this the best approach).

If Claude is just the new high level programming language, in terms of prompting in natural language, the challenge is that we're not reviewing the natural language, we're reviewing the machine code without knowing what the inputs were. I'm not sure of a solution to this, but something along the lines of knowing the history of the prompting that ultimately led to the PR, the time/tokens involved, etc. may inform the "quality" or "effort" spent in producing the PR. A one-shotted feature vs. a multi-iteration feature may produce the same lines of code and general shape, but one is likely to be higher "quality" in terms of minimal defects.

Along the same lines, when I review some gen-AI produced PR, it feels like I'm reading assembly and having to reverse how we got here. It may be code that runs and is perfectly fine, but I can't tell what the higher level inputs were that produced it, and if they were sufficient.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#403
We ran into something similar at heyvid.ai — shipped AI-generated code without a proper review gate and ended up with a subtle bug in our rendering pipeline that took the team a week to trace. Not catastrophic, but it seriously eroded trust in the tooling for a while. Amazon's approach makes total sense at their scale. The honest reality is that LLMs are great at producing plausible-looking code and genuinely bad at knowing when they're wrong. Senior sign-off isn't overhead — it's what makes AI-assisted development actually sustainable.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#404
post #364

Someone should teach the decision makers how pipelines work. If AI-created diffs are being churned out at 10x the previous rate but manual reviews are the bottleneck then the overall system is producing at the exact same rate as before. The only thing you have added is cost, uncertainty and engineers being less familiar with the system.

Check back in a year or two. Just the other day we had Claude finding significant bugs in Firefox. https://news.ycombinator.com/item?id=47273854

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#405

From the amazon I know, people only care about a. not getting fired and b. promotions. For devs, the matrix looks like this: 1. Shipping: deliver tickets or be pipped. 2. Having Less comments on their PRs: for some drastically dumb reason, having a PR thoroughly reviewed is a sign of bad quality. L7 and above use this metric to Pip folks. 3. Docs: write docs, get them reviewed to show you're high level. Without AI, a…

> 2. Having Less comments on their PRs: for some drastically dumb reason, having a PR thoroughly reviewed I'm very far away from liking Amazon's engineering culture and general work culture, but having PRs with countless of discussions and feedback on it does signal that you've done a lot of work without collaborating with others before doing the work. Generally in teams that work well together and build great softwa…

If the review tooling is any good, getting the code somewhere it can see it is a convenient way for people to give and receive feedback. As the saying goes, the system is what it does!

(And/but yes/no, I have never worked at NAGFAM...)

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#406

Anyone work with Kiro before? As I understood, it was held as an INTERNAL USE ONLY tool for much longer than expected.

I use Kiro IDE (≠ Kiro CLI) primarily as a spec generator.

In my experience, it's high-quality for creating and iterating on specs. Tools like Cursor are optimized for human-driven vibing -- they have great autocomplete, etc. Kiro, by contrast, is optimized around spec, which ironically has been the most effective approach I've found for driving agents.

I'd argue that Cursor, Antigravity, and similar tools are optimized for human steering, which explains their popularity, while Kiro is optimized for agent harnesses. That's also why it’s underused: it's quite opinionated, but very effective. Vibe-coding culture isn't sold on spec driven development (they think it's waterfall and summarily dismiss it -- even Yegge has this bias), so people tend to underrate it.

Kiro writes specs using structured formats like EARS and INCOSE. It performs automated reasoning to check for consistency, then generates a design document and task list from the spec -- similar to what Beads does. I usually spend a significant amount of time pressure-testing the spec before implementing (often hours to days), and it pays off. Writing a good, consistent spec is essentially the computer equivalent of "writing as a tool of thought" in practice.

Once the spec is tight, implementation tends to follow it closely. Kiro also generates property-based tests (PBTs) using Hypothesis in Python, inspired by Haskell's QuickCheck. These tests sweep the input domain and, when combined with traditional scenario-based unit tests, tend to produce code that adheres closely to the spec. I also add a small instruction "do red/green TDD" (I learned this from Simon Willison) and that one line alone improved the quality of all my tests.

Kiro can technically implement the task list itself, but this is where agents come in. With the spec in hand, I use multiple headless CLI agents in tmux (e.g., Kiro CLI, Claude Code) for implementation. The results have been very good. With a solid Kiro spec and task list, agents usually implement everything end-to-end without stopping -- I haven’t found a need for Ralph loops. (agents sometimes tend to stop mid way on Claude plans, but I've never had that happen with Kiro, not sure why, maybe it's the checklist, which includes PBT tests as gates).

Kiro didn't have the strongest start, but the Kiro IDE is one of the best spec generators I've used, and it integrates extremely well with agent-driven workflows.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#408

Earlier quoted context omitted.

The article claims: >He asked staff to attend the meeting, which is normally optional. Is that false? It also discusses a new policy: >Junior and mid-level engineers will now require more senior engineers to sign off any AI-assisted changes, Treadwell added. Is that inaccurate? It is good context that this is a regularly scheduled meeting. But, regularly scheduled meetings can have newsworthy things happen at them.

When an SVP asks you to do something in a mass email, it's very much optional. Dave Treadwell is an SVP, his org is likely in the 10's of thousands, there is no way to even have a mandatory meeting for that many people. My SVP asks me to do things all the time, indirectly. I do probably 5% of them.

are you saying SVP’s words are not important and should be ignored? This is not what I remember back in the day when Bezos sent his email with a question mark (or maybe !)

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#409

Earlier quoted context omitted.

Wasn't it Shopify who had a system for tracking how much each meeting cost based on attendees? I may be misremembering the company though

I was thinking about this in recent weeks and I think I’ve actually changed my mind on it. It’s not really possible to measure how much it would cost to not have a meeting, and I think it’s pretty obvious that if there were no meetings ever, it would hurt a company a lot

Yeah, I agree it's a silly metric. But it's kinda also a good reminder that meetings do have a cost associated with them, so they should stay short, focused, and held only when necessary.

"This could have been an e-mail" should never need to be said.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#410

>Junior and mid-level engineers can no longer push AI-assisted code without a senior signing off Review by a senior is one of the biggest "silver bullet" illusions managers suffer from. For a person (senior or otherwise) to examine code or configuration with the granularity required to verify that it even approximates the result of their own level of experience, even only in terms of security/stability/correctness, r…

Seniors are going to need to hold Juniors to a high bar for understanding and explaining what they are committing. Otherwise it will become totally soul destroying to have a bunch of juniors submitting piles of nonsense and claiming they are blocked on you all the time.

Make them first go through an AI reviewer that is informed by the code base's standards.
Post reply on HN