Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

After outages, Amazon to make senior engineers sign off on AI-assisted changes

arstechnica.com

Post reply on HN
Print view View original

31–40 of 510 posts

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#31

Post by marginalia_nu »

happytoexplain wrote: post #19

>Junior and mid-level engineers can no longer push AI-assisted code without a senior signing off Review by a senior is one of the biggest "silver bullet" illusions managers suffer from. For a person (senior or otherwise) to examine code or configuration with the granularity required to verify that it even approximates the result of their own level of experience, even only in terms of security/stability/correctness, r…

Expert reviews are just about the only thing that makes AI generated code viable, though doing them after the fact is a bit sketchy, to be efficient you kinda need to keep an eye on what the model is doing as its working.

Unchecked, AI models output code that is as buggy as it is inefficient. In smaller green field contexts, it's not so bad, but in a large code base, it's performs much worse as it will not have access to the bigger picture.

In my experience, you should be spending something like 5-15X the time the model takes to implement a feature on reviewing and making it fix its errors and inefficiencies. If you do that (with an expert's eye), the changes will usually have a high quality and will be correct and good.

If you do not do that due dilligence, the model will produce a staggering amount of low quality code, at a rate that is probably something like 100x what a human could output in a similar timespan. Unchecked, it's like having a small army of the most eager junior devs you can find going completely fucking ape in the codebase.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#32

Post by yifanl »

happytoexplain wrote: post #19

>Junior and mid-level engineers can no longer push AI-assisted code without a senior signing off Review by a senior is one of the biggest "silver bullet" illusions managers suffer from. For a person (senior or otherwise) to examine code or configuration with the granularity required to verify that it even approximates the result of their own level of experience, even only in terms of security/stability/correctness, r…

Senior reviews are useful, but as I understand it, Amazon has a fairly high turnover rate, so I wonder just how many seniors with deep knowledge of the codebase they could possibly have.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#33

Post by CoolGuySteve »

cobolcomesback wrote: post #11

This “mandatory meeting” is just the usual weekly company-wide meeting where recent operational issues are discussed. There was a big operational issue last week, so of course this week will have more attendance and discussion. This meeting happens literally every week, and has for years. Feels like the media is making a mountain out of a mole hill here.

It didn't seem to make the news but at least in NYC the entire Amazon storefront was broken all afternoon on Friday.

Items weren't displaying prices and it was impossible to add anything to your cart. It lasted from about 2pm to 5pm.

It's especially strange because if a computer glitch brought down a large retail competitor like Walmart I probably would have seen something even though their sales volume is lower.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#34

Post by skeledrew »

> the affected tool served customers in mainland China

Thought this blurb most interesting. What's the between-lines subtext here? Are they deliberately serving something they know to be faulty to the Chinese? Or is it the case that the Chinese use it with little to no issue/complaint? Or...?

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#35

Post by altairprime »

dragonelite wrote: post #20
sethops1 wrote: post #18

> The response for now? Junior and mid-level engineers can no longer push AI-assisted code without a senior signing off. So basically, kill the productivity of senior engineers, kill the ability for junior engineers to learn anything, and ensure those senior engineers hate their jobs. Bold move, we'll see how that goes.

Accelerate a person speed toward being burned out..

..and you lower overall engineering salary spend by rotating out seniority-paid engineers for newly-promoted AI reviewers with lower specs

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#36

Post by grvdrm »

happytoexplain wrote: post #19

>Junior and mid-level engineers can no longer push AI-assisted code without a senior signing off Review by a senior is one of the biggest "silver bullet" illusions managers suffer from. For a person (senior or otherwise) to examine code or configuration with the granularity required to verify that it even approximates the result of their own level of experience, even only in terms of security/stability/correctness, r…

What a statement at the end. You are absolutely right.

I hear “x tool doesn’t really work well” and then I immediately ask: “does someone know how to use it well?” The answer “yes” is infrequent. Even a yes is often a maybe.

The problem is pervasive in my world (insurance). Number-producing features need to work in a UX and product sense but also produce the right numbers, and within range of expectations. Just checking the UX does what it’s supposed to do is one job, and checking the numbers an entirely separate task.

I don’t many folks that do both well.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#37

Post by RamblingCTO »

happytoexplain wrote: post #19

>Junior and mid-level engineers can no longer push AI-assisted code without a senior signing off Review by a senior is one of the biggest "silver bullet" illusions managers suffer from. For a person (senior or otherwise) to examine code or configuration with the granularity required to verify that it even approximates the result of their own level of experience, even only in terms of security/stability/correctness, r…

Who said PR reviews need to solve all the things and result in proof against idiots?

So you're saying that peer reviews are a waste of time and only idiots would use/propose them?

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#38

Post by ritlo »

almostdeadguy wrote: post #29
sethops1 wrote: post #18

> The response for now? Junior and mid-level engineers can no longer push AI-assisted code without a senior signing off. So basically, kill the productivity of senior engineers, kill the ability for junior engineers to learn anything, and ensure those senior engineers hate their jobs. Bold move, we'll see how that goes.

I'm sorry what? Junior engineers can't learn anything without using AI assistants (or is the implication that having seniors review their code makes them incapable of learning?) and senior engineer would hate their jobs reviewing more code from their teammates? What reality do people live in now?

> senior engineer would hate their jobs reviewing more code from their teammates

Jesus, yes. Maybe I'm an oddball but there's a limit to how much PR reviewing I could do per week and stay sane. It's not terribly high, either. I'd say like 5 hours per week max, and no more than one hour per half-workday, before my eyes glaze over and my reviews become useless.

Reviewing code is important and is part of the job but if you're asking me to spend far more of my time on it, and across (presumably) a wider set of projects or sections of projects so I've got more context-switching to figure out WTF I'm even looking at, yes, I would hate my job by the end of day 1 of that.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#39

Post by jetrink »

happytoexplain wrote: post #19

>Junior and mid-level engineers can no longer push AI-assisted code without a senior signing off Review by a senior is one of the biggest "silver bullet" illusions managers suffer from. For a person (senior or otherwise) to examine code or configuration with the granularity required to verify that it even approximates the result of their own level of experience, even only in terms of security/stability/correctness, r…

It could create the right sort of incentives though. If I'm a junior and I suddenly have to take my work to a senior every time I use AI, I'm going to be much more selective about how I use it and much more careful when I do use it. AI is dangerous because it is so frictionless and this is a way to add friction.

Maybe I don't have the correct mental model for how the typical junior engineer thinks though. I never wanted to bug senior people and make demands on their time if I could help it.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#40

Post by js8 »

happytoexplain wrote: post #19

>Junior and mid-level engineers can no longer push AI-assisted code without a senior signing off Review by a senior is one of the biggest "silver bullet" illusions managers suffer from. For a person (senior or otherwise) to examine code or configuration with the granularity required to verify that it even approximates the result of their own level of experience, even only in terms of security/stability/correctness, r…

> requires an amount of time approaching the time spent if they had just done it themselves

It's actually often harder to fix something sloppy than to write it from scratch. To fix it, you need to hold in your head both the original, the new solution, and calculate the difference, which can be very confusing. The original solution can also anchor your thinking to some approach to the problem, which you wouldn't have if you solve it from scratch.

Post reply on HN