Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

After outages, Amazon to make senior engineers sign off on AI-assisted changes

arstechnica.com

Post reply on HN
Print view View original

111–120 of 510 posts

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#111

Post by cmiles8 »

cobolcomesback wrote: post #11

This “mandatory meeting” is just the usual weekly company-wide meeting where recent operational issues are discussed. There was a big operational issue last week, so of course this week will have more attendance and discussion. This meeting happens literally every week, and has for years. Feels like the media is making a mountain out of a mole hill here.

The core message of the article is that Amazon has been having issues with AI slop causing operational reliability concerns, and that seems to be 100% accurate.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#112

Post by tartoran »

mrbonner wrote: post #53
happytoexplain wrote: post #19

>Junior and mid-level engineers can no longer push AI-assisted code without a senior signing off Review by a senior is one of the biggest "silver bullet" illusions managers suffer from. For a person (senior or otherwise) to examine code or configuration with the granularity required to verify that it even approximates the result of their own level of experience, even only in terms of security/stability/correctness, r…

What stops the senior from using AI to review the AI generated code the junior published?

That’s something that the junior can do. What companies want to do is put responsibility on someone who has more knowledge and skin in the game

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#113

Post by marginalia_nu »

rectang wrote: post #107
marginalia_nu wrote: post #103

Earlier quoted context omitted.

By the time you're working in increments small enough that it doesn't introduce significant issues, you really might as well write the code yourself.

That's not my experience — I'm significantly faster while guiding an LLM using this methodology. The gains are especially notable when working in unfamiliar domains. I can glance over code and know "if this compiles and the tests succeed, it will work", even if I didn't have the knowledge to write it myself.

That's where the Gell-Mann amnesia will get you though. As much it trips up on the domains you're familiar with, it also trips up in unfamiliar domains. You just don't see it.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#114

Post by fredgrott »

Curious question, how many Amazon Engineers flunk basic CS?

If you know CS you know two things:

1. AI can not judge code either noise or signal, AI cannot tell. 2. CS-wise we use statistic analysis to judge good code from bad.

How much time does it take to take AI output and run the basic statistic tools for most computer languages?

Some juniors need firing outright

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#116

Post by hnthrow0287345 »

happytoexplain wrote: post #19

>Junior and mid-level engineers can no longer push AI-assisted code without a senior signing off Review by a senior is one of the biggest "silver bullet" illusions managers suffer from. For a person (senior or otherwise) to examine code or configuration with the granularity required to verify that it even approximates the result of their own level of experience, even only in terms of security/stability/correctness, r…

>requires an amount of time approaching the time spent if they had just done it themselves.

I would actually say having at least 2 people on any given work item should probably be the norm at Amazon's size if you also want to churn through people as Amazon does and also want quality.

Doing code reviews are not as highly valued in terms of incentives to the employees and it blocks them working on things they would get more compensation for.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#117

Post by qnleigh »

happytoexplain wrote: post #19

>Junior and mid-level engineers can no longer push AI-assisted code without a senior signing off Review by a senior is one of the biggest "silver bullet" illusions managers suffer from. For a person (senior or otherwise) to examine code or configuration with the granularity required to verify that it even approximates the result of their own level of experience, even only in terms of security/stability/correctness, r…

I seriously doubt that they think senior reviewers will meticulously hunt down and fix all the AI bugs. Even if they could, they surely don't have the time. But it offers other benefits here:

1. They can assess whether the use of AI is appropriate without looking in detail. E.g. if the AI changed 1000 lines of code to fix a minor bug, or changed code that is essential for security.

2. To discourage AI use, because of the added friction.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#118

Post by raw_anon_1111 »

YCpedohaven wrote: post #97
raw_anon_1111 wrote: post #90

Earlier quoted context omitted.

In my experience, inefficient code is rarely the issue outside of data engineering type ETL jobs. It’s mostly architectural. Inefficient code isn’t the reason your login is taking 30 seconds. Yes I know at Amazon/AWS scale (former employee) every efficiency matters. But even at Salesforce scale, ringing out every bit of efficiency doesn’t matter. No one cares about handcrafted artisanal code as long as it meets both…

[flagged]

Yes because I didn’t check to see if Claude code used a for loop instead of a while loop? Or that it didn’t use my preferred GOF pattern and didn’t use what I read in “Clean Code”?

Guess what? I also stopped caring how registers are used and counting clock cycles in my assembly language code like it’s the 80s and I’m still programming on a 1Mhz 65C02

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#119

Post by rectang »

marginalia_nu wrote: post #113
rectang wrote: post #107

Earlier quoted context omitted.

That's not my experience — I'm significantly faster while guiding an LLM using this methodology. The gains are especially notable when working in unfamiliar domains. I can glance over code and know "if this compiles and the tests succeed, it will work", even if I didn't have the knowledge to write it myself.

That's where the Gell-Mann amnesia will get you though. As much it trips up on the domains you're familiar with, it also trips up in unfamiliar domains. You just don't see it.

You're not telling me anything I don't know already. Only a person who accepts that they're fallible can execute this methodology anyway, because that's the kind of mentality that it takes to think through potential failure modes.

Yes, code produced this way will have bugs, especially of the "unknown unknown" variety — but so would the code that I would have written by hand.

I think a bigger factor contributing to unforeseen bugs is whether the LLM's code is statistically likely to be correct:

* Is this a domain that the LLM has trained on a lot? (i.e. lots of React code out there, not much in your home-grown DSL)

* Is the codebase itself easy to understand, written with best practices, and adhering to popular conventions? Code which is hard for humans to understand is also hard for an LLM to understand.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#120

Post by Clent »

cobolcomesback wrote: post #11

This “mandatory meeting” is just the usual weekly company-wide meeting where recent operational issues are discussed. There was a big operational issue last week, so of course this week will have more attendance and discussion. This meeting happens literally every week, and has for years. Feels like the media is making a mountain out of a mole hill here.

Who is the media you're accusing here? This is a twitter post. As far as I can tell they do not work a media company.

What is worth being pointed out is how quickly people blame "The Media" for how people use, consume and spread information on social networks.

Post reply on HN