Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

After outages, Amazon to make senior engineers sign off on AI-assisted changes

arstechnica.com

Post reply on HN
Print view View original

241–250 of 510 posts

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#241

Post by Barrin92 »

radiator wrote: post #162
sdevonoes wrote: post #57

Reviewing AI generated code at PR time is a bottleneck. It cancels most of the benefits senior leadership thinks AI offers (delivery speed). There’s also this implicit imbalance engineers typically don’t like: it takes me 10 min to submit a complete feature thanks to Claude… but for the human reviewing my PR in a manual way it will take them 10-20 times that. Edit: at the end real engineers know that what takes effor…

> Senior leadership doesn’t know this, though. Well, you'd think senior leadership should know how their business and their people work.

to be fair senior engineering leads in the software world are like Voltaire's joke about the holy roman empire, neither holy, roman or an empire.

Despite the name not a lot of seniority, leadership or engineering going around

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#242

Post by demosito666 »

bonesss wrote: post #139
hard24 wrote: post #72

Earlier quoted context omitted.

Also when you are writing code yourself you are implicitly checking it whilst at the back of your mind retaining some form of the entire system as a whole. People seem to gloss over this... As a CEO if people don't function like this I'd be awake at night sweating.

That’s the reverse-centaur issue I see: humans are not great at repetitive nuanced similar seeming tasks, putting the onus on humans to retroactively approve high volumes of critical code has them managing a critical failure mode at their weakest and worst. Automated reviews should be enhancing known good-faith code, manual reviews of high volume superficially sound but subversive code is begging for issues over time…

> Even if everyone has been 10x’ed, the math still strongly favours not making mistakes in the first place

The math depends on importance of the software. A mistake in a typical CRUD enterprise app with 100 users has zero impact on anything. You will fix it when you have time, the important thing is that the app was delivered in a week a year ago and was solving some problem ever since. It has already made enormous profit if you compare it with today’s (yesterday’s ?) manual development that would take half a year and cost millions.

A mistake in a nuclear reactor control code would be a total different thing. Whatever time savings you made on coding are irrelevant if it allowed for a critical bug to slip through.

Between the two extremes you thus have a whole spectrum of tasks that either benefit or lose from applying coding with LLMs. And there are also more axes than this low to high failure cost, which also affect the math. For example, even non-important but large app will likely soon degrade into unmanageable state if developed with too little human intervention and you will be forced to start from scratch loosing a lot of time.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#243

Post by furyofantares »

cobolcomesback wrote: post #11

This “mandatory meeting” is just the usual weekly company-wide meeting where recent operational issues are discussed. There was a big operational issue last week, so of course this week will have more attendance and discussion. This meeting happens literally every week, and has for years. Feels like the media is making a mountain out of a mole hill here.

This reply chain is confusing but I'm guessing got merged from another thread that had a different title?

Must have as the comments are hours older than OP.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#244

Post by tibbar »

javcasas wrote: post #240
djb_hackernews wrote: post #233

Earlier quoted context omitted.

When an SVP asks you to do something in a mass email, it's very much optional. Dave Treadwell is an SVP, his org is likely in the 10's of thousands, there is no way to even have a mandatory meeting for that many people. My SVP asks me to do things all the time, indirectly. I do probably 5% of them.

With tens of thousands in a meeting, cracking a 30-second stupid joke is probably costing several thousand dollars.

i think closer to tens-of-thousands-of-dollars, by my napkin math!

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#245

Post by malfist »

beardedetim wrote: post #66
sdevonoes wrote: post #57

Reviewing AI generated code at PR time is a bottleneck. It cancels most of the benefits senior leadership thinks AI offers (delivery speed). There’s also this implicit imbalance engineers typically don’t like: it takes me 10 min to submit a complete feature thanks to Claude… but for the human reviewing my PR in a manual way it will take them 10-20 times that. Edit: at the end real engineers know that what takes effor…

This is what I don't understand about this policy. There's no way a senior has enough spare capacity to be the gate keeper on every PR made by AI below them. So now we are just making it so the senior people use more AI to keep up but now they're to blame for letting it happen. It sounds like a piss poor deal for seniors unless senior engineer now means professional code reviewer.

That's amazon in a nutshell though. Create conflicting metrics for performance, push credit up and responsibility down, punish everyone below you for not meeting the double standards

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#246

Post by chatmasta »

malfist wrote: post #93
CoolGuySteve wrote: post #33

Earlier quoted context omitted.

It didn't seem to make the news but at least in NYC the entire Amazon storefront was broken all afternoon on Friday. Items weren't displaying prices and it was impossible to add anything to your cart. It lasted from about 2pm to 5pm. It's especially strange because if a computer glitch brought down a large retail competitor like Walmart I probably would have seen something even though their sales volume is lower.

Over the weekend I was trying to return a pair of shoes and get a different size and I kept getting 500s trying to go to the store page for the shoes.

Funny, I was automatically refunded for a pair of shoes that Amazon thought I never received even though I’m wearing them right now. I couldn’t even find a way to dispute the refund so I just took the win…

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#247

Post by zamalek »

happytoexplain wrote: post #19

>Junior and mid-level engineers can no longer push AI-assisted code without a senior signing off Review by a senior is one of the biggest "silver bullet" illusions managers suffer from. For a person (senior or otherwise) to examine code or configuration with the granularity required to verify that it even approximates the result of their own level of experience, even only in terms of security/stability/correctness, r…

> Review by a senior is one of the biggest "silver bullet" illusions managers suffer from.

My manager has been urging us to truly vibe code, just yesterday saying that "language is irrelevant because we've reached the point where it works - so you don't need to see it." This article is a godsend; I'll take this flawed silver bullet any day of the week.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#248

Post by malfist »

dude250711 wrote: post #236
malfist wrote: post #157

Earlier quoted context omitted.

Not only is having too many comments on your PRs bad for you, but so is not leaving comments on other people's PRs. Both are metrics used

I'd leave lots of comments out of spite whenever I would feel my PRs had been treated unfairly. If I am going down, you all are coming with me.

Welcome to Amazon, you'll fit right in.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#249

Post by philip1209 »

I think the deeper need is a "self-review" flow.

People push AI-reviewed code like they wrote it. In the past, "wrote it" implies "reviewed it." With AI, that's no longer true.

I advocate for GitHub and other code review systems to add a "Require self-review" option, where people must attest that they reviewed and approved their own code. This change might seem symbolic, but it clearly sets workflows and expectations.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#250

Post by inquirerGeneral »

cmiles8 wrote: post #111
cobolcomesback wrote: post #11

This “mandatory meeting” is just the usual weekly company-wide meeting where recent operational issues are discussed. There was a big operational issue last week, so of course this week will have more attendance and discussion. This meeting happens literally every week, and has for years. Feels like the media is making a mountain out of a mole hill here.

The core message of the article is that Amazon has been having issues with AI slop causing operational reliability concerns, and that seems to be 100% accurate.

[dead]
Post reply on HN