Live data from Hacker News

After outages, Amazon to make senior engineers sign off on AI-assisted changes

arstechnica.com

91–100 of 510 posts

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#91
The optics here are really bad for Amazon. The continuing mass departures of long tenured folks, second-rate AI products, and a string of bad outages paints a picture that current leadership is overseeing a once respected engineering train flying off the tracks.

News from the inside makes it sound like things are getting pretty bad.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#92

This “mandatory meeting” is just the usual weekly company-wide meeting where recent operational issues are discussed. There was a big operational issue last week, so of course this week will have more attendance and discussion. This meeting happens literally every week, and has for years. Feels like the media is making a mountain out of a mole hill here.

The article claims: >He asked staff to attend the meeting, which is normally optional. Is that false? It also discusses a new policy: >Junior and mid-level engineers will now require more senior engineers to sign off any AI-assisted changes, Treadwell added. Is that inaccurate? It is good context that this is a regularly scheduled meeting. But, regularly scheduled meetings can have newsworthy things happen at them.

It’s not false. But it’s also weaselly worded.

Note that the article doesn’t say that he told staff they have to attend the meeting. It says he “asked” staff to attend the meeting. Which again, it’s really really normal for there to be an encouragement of “hey, since we just had an operational event, it would be good to prioritize attending this meeting where we discuss how to avoid operational events”.

As for the second quote: senior engineers have always been required to sign off on changes from junior engineers. There’s nothing new there. And there is nothing specific to AI that was announced.

This entire meeting and message is basically just saying “hey we’ve been getting a little sloppy at following our operational best practices, this is a reminder to be less sloppy”. It’s a massive nothingburger.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#93

This “mandatory meeting” is just the usual weekly company-wide meeting where recent operational issues are discussed. There was a big operational issue last week, so of course this week will have more attendance and discussion. This meeting happens literally every week, and has for years. Feels like the media is making a mountain out of a mole hill here.

It didn't seem to make the news but at least in NYC the entire Amazon storefront was broken all afternoon on Friday. Items weren't displaying prices and it was impossible to add anything to your cart. It lasted from about 2pm to 5pm. It's especially strange because if a computer glitch brought down a large retail competitor like Walmart I probably would have seen something even though their sales volume is lower.

Over the weekend I was trying to return a pair of shoes and get a different size and I kept getting 500s trying to go to the store page for the shoes.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#94

This “mandatory meeting” is just the usual weekly company-wide meeting where recent operational issues are discussed. There was a big operational issue last week, so of course this week will have more attendance and discussion. This meeting happens literally every week, and has for years. Feels like the media is making a mountain out of a mole hill here.

> This meeting happens literally every week, and has for years. Feels like the media is making a mountain out of a mole hill here. Are you completely missing the point of the submission? It's not about "Amazon has a mandatory weekly meeting" but about the contents of that specific meeting, about AI-assisted tooling leading to "trends of incidents", having a "large blast radius" and "best practices and safeguards are…

>> Are you completely missing the point of the submission

no, and that's what people are noting: the headline deliberately tries to blow this up into a big deal. When did you last see the HN post about Amazon's mandatory meeting to discuss a human-caused outage, or a post mortem? It's not because they don't happen...

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#95
post #60

Earlier quoted context omitted.

When I was really early in my career, a mentor told me that code review is not about catching bugs but spreading context (i.e. increasing bus factor.) Catching bugs is a side effect, but unless you have a lot of people review each pull request, it's basically just gambling. The more expensive and less sexy option is to actually make testing easier (both programmatically and manually), write more tests and more levels…

> people don't get promoted for preventing issues. they do - but only after a company has been burned hard. They also can be promoted for their area being enough better that everyone notices. still the best way to a promotion is write a major bug that you can come in at the last moment and be the hero for fixing.

That could work but plenty of quiet heros weren’t promoted for fixing critical bugs.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#97

Earlier quoted context omitted.

Expert reviews are just about the only thing that makes AI generated code viable, though doing them after the fact is a bit sketchy, to be efficient you kinda need to keep an eye on what the model is doing as its working. Unchecked, AI models output code that is as buggy as it is inefficient. In smaller green field contexts, it's not so bad, but in a large code base, it's performs much worse as it will not have acces…

In my experience, inefficient code is rarely the issue outside of data engineering type ETL jobs. It’s mostly architectural. Inefficient code isn’t the reason your login is taking 30 seconds. Yes I know at Amazon/AWS scale (former employee) every efficiency matters. But even at Salesforce scale, ringing out every bit of efficiency doesn’t matter. No one cares about handcrafted artisanal code as long as it meets both…

[flagged]

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#99

>Junior and mid-level engineers can no longer push AI-assisted code without a senior signing off Review by a senior is one of the biggest "silver bullet" illusions managers suffer from. For a person (senior or otherwise) to examine code or configuration with the granularity required to verify that it even approximates the result of their own level of experience, even only in terms of security/stability/correctness, r…

Expert reviews are just about the only thing that makes AI generated code viable, though doing them after the fact is a bit sketchy, to be efficient you kinda need to keep an eye on what the model is doing as its working. Unchecked, AI models output code that is as buggy as it is inefficient. In smaller green field contexts, it's not so bad, but in a large code base, it's performs much worse as it will not have acces…

> Expert reviews are just about the only thing that makes AI generated code viable

I disagree, in the sense that an engineer who knows how to work with LLMs can produce code which only needs light review.

* Work in small increments

* Explicitly instruct the LLM to make minimal changes

* Think through possible failure modes

* Build in error-checking and validation for those failure modes

* Write tests which exercise all paths

This is a means to produce "viable" code using an LLM without close review. However, to your point, engineers able to execute this plan are likely to be pretty experienced, so it may not be economically viable.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#100
post #39

>Junior and mid-level engineers can no longer push AI-assisted code without a senior signing off Review by a senior is one of the biggest "silver bullet" illusions managers suffer from. For a person (senior or otherwise) to examine code or configuration with the granularity required to verify that it even approximates the result of their own level of experience, even only in terms of security/stability/correctness, r…

It could create the right sort of incentives though. If I'm a junior and I suddenly have to take my work to a senior every time I use AI, I'm going to be much more selective about how I use it and much more careful when I do use it. AI is dangerous because it is so frictionless and this is a way to add friction. Maybe I don't have the correct mental model for how the typical junior engineer thinks though. I never wan…

What you're actually going to see is seniors inundated by slop and burning out and quitting because what used to be enjoyable solving of problems has become wading through slop that took 10 minutes to generate and submit but 30+ minutes to understand and write up a critique for it.
Post reply on HN