Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

After outages, Amazon to make senior engineers sign off on AI-assisted changes

arstechnica.com

Post reply on HN
Print view View original

171–180 of 510 posts

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#171

Post by sarchertech »

raw_anon_1111 wrote: post #90
marginalia_nu wrote: post #31

Earlier quoted context omitted.

Expert reviews are just about the only thing that makes AI generated code viable, though doing them after the fact is a bit sketchy, to be efficient you kinda need to keep an eye on what the model is doing as its working. Unchecked, AI models output code that is as buggy as it is inefficient. In smaller green field contexts, it's not so bad, but in a large code base, it's performs much worse as it will not have acces…

In my experience, inefficient code is rarely the issue outside of data engineering type ETL jobs. It’s mostly architectural. Inefficient code isn’t the reason your login is taking 30 seconds. Yes I know at Amazon/AWS scale (former employee) every efficiency matters. But even at Salesforce scale, ringing out every bit of efficiency doesn’t matter. No one cares about handcrafted artisanal code as long as it meets both…

It’s not about hand crafted code or even code performance.

We know from experimentation that agents will change anything that isn’t nailed down. No natural language spec or test suite has ever come close to fully describing all observable behaviors of a non-trivial system.

This means that if no one is reviewing the code, agents adding features will change observable behaviors.

This gets exposed to users as churn, jank, and broken work flows.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#172

Post by dlev_pika »

A few days ago, after some very weird failed purchase attempts I made (payment couldn’t be validated or Smth) I received an even weirder mail from Amazon saying they had detected suspicious activity, all my devices got logged out and I was forced to change my password. I did it, after verifying it was a legit email (even if it looked sketchy af, pure text, unstyled, but sender verified and confirmed with in-app behavior), and next I know all my orders and browsing history had disappeared - +15 yrs of history, done.

Over the next few days my account history came back, except purchases made Q1 2026. Those are still missing. There are a few substantial purchases I made that are nowhere to be found anymore.

I attributed this Iranian missiles hitting some of their infrastructure in EU, as it had been reported.

Now I am not sure if it was blast radius from missiles or AI mishaps. Lmao - couldn’t happen to a worse company…

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#173

Post by lovich »

simplyluke wrote: post #87
czscout wrote: post #83

Earlier quoted context omitted.

In this scenario, how might one become a senior without first being a junior? Seniors just pop into existence?

The bet from various industry leaders appears to be that the current generation of engineers will be the last who will ever need to think about complex systems and engineering, as the AI will just get good enough to do all of that by the time they retire.

I think it’s deeper than that because it’s affected more industries than software and already started pre AI.

American corporate culture has decided that training costs are someone else’s problem. Since every corporation acts this way it means all training costs have been pushed onto the labor market. Combine that with the past few decades of “oops, looks like you picked the wrong career that took years of learning and/or 10 to 100s of thousands of dollars to acquire but we’ve obsoleted that field” and new entrants into the labor market are just choosing not to join.

Take trucking for example. For the past decade I’ve heard logistics companies bemoan the lack of CDL holders, while simultaneously gleefully talk about how the moment self driving is figured out they are going to replace all of them.

We’re going to be outpaced by countries like China at some point because we’re doing the industrial equivalent of eating our seed corn and there is seemingly no will to slow that trend down, much less reverse it.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#174

Post by i_cannot_hack »

cobolcomesback wrote: post #92
davidclark wrote: post #21

Earlier quoted context omitted.

The article claims: >He asked staff to attend the meeting, which is normally optional. Is that false? It also discusses a new policy: >Junior and mid-level engineers will now require more senior engineers to sign off any AI-assisted changes, Treadwell added. Is that inaccurate? It is good context that this is a regularly scheduled meeting. But, regularly scheduled meetings can have newsworthy things happen at them.

It’s not false. But it’s also weaselly worded. Note that the article doesn’t say that he told staff they have to attend the meeting. It says he “asked” staff to attend the meeting. Which again, it’s really really normal for there to be an encouragement of “hey, since we just had an operational event, it would be good to prioritize attending this meeting where we discuss how to avoid operational events”. As for the se…

Your characterization of the event as a simple reminder to follow established best practices is directly contradicted by the briefing note of the meeting, which specifically mentions a lack of best practices related to AI. Which makes me skeptical of your assessment of the situation in general.

> Under “contributing factors” the note included “novel GenAI usage for which best practices and safeguards are not yet fully established”.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#175

Post by raw_anon_1111 »

hard24 wrote: post #72
bluGill wrote: post #61

Earlier quoted context omitted.

Some, but not very much. Writing code is hard. Ai will do a lot of tedious code that you procrastinate writing.

Also when you are writing code yourself you are implicitly checking it whilst at the back of your mind retaining some form of the entire system as a whole. People seem to gloss over this... As a CEO if people don't function like this I'd be awake at night sweating.

No I’m keeping up with the system as a whole because I’m always working at a system level when I’m using AI instead of worrying about the “how”

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#176

Post by misnome »

marginalia_nu wrote: post #76
locusofself wrote: post #50

Earlier quoted context omitted.

If you spend 5-15x the time reviewing what the LLM is doing, are you saving any time by using it?

To be honest, some times it's still beneficial. For fairly straightforward changes it's probably a wash, but ironically enough it's often the trickier jobs where they can be beneficial as it will provide an ansatz that can be refined. It's also very good at tedious chores.

And spotting stuff in review! Sometimes it’s false positives but on several occasions I’ve spent ~15-30 minutes teaching-reviewing a PR in person, checked afterwards and it matched every one of the points.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#177

Post by _wire_ »

hard24 wrote: post #52
ritlo wrote: post #16

The only way to see the kinds of speed-up companies want from these things, right now, is to do way too little review. I think we're going to see a lot of failures in a lot of sectors where companies set goals for reduced hours on various things they do, based on what they expected from LLM speed-ups, and it will have turned out the only way to hit those goals was by spending way too little time reviewing LLM output.…

My prediction is a concorde-like incident is going to shatter trust and make people re-think their expectations of the capabilities of LLMs and their abilities of the present. Essentially something big has to happen that affects the revenue/trust of a large provider of goods, stemming from LLM-use. They wont go away entirely. But this idea that they can displace engineers at a high-rate will.

Like bombing a building full of little kids? Oops too late...

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#178

Post by sethops1 »

AgentOrange1234 wrote: post #143
happytoexplain wrote: post #19

>Junior and mid-level engineers can no longer push AI-assisted code without a senior signing off Review by a senior is one of the biggest "silver bullet" illusions managers suffer from. For a person (senior or otherwise) to examine code or configuration with the granularity required to verify that it even approximates the result of their own level of experience, even only in terms of security/stability/correctness, r…

Seniors are going to need to hold Juniors to a high bar for understanding and explaining what they are committing. Otherwise it will become totally soul destroying to have a bunch of juniors submitting piles of nonsense and claiming they are blocked on you all the time.

This was challenging enough pre AI. Now that everybody has an AI slop button, the life of an effective code reviewer just got so much more miserable.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#179

Post by brandensilva »

happytoexplain wrote: post #67
locusofself wrote: post #50

Earlier quoted context omitted.

If you spend 5-15x the time reviewing what the LLM is doing, are you saving any time by using it?

No, but that's the crux of the AI problem in software. Time to write code was never the bottleneck. AI is most useful for learning, either via conversation or by seeing examples. It makes writing code faster too, but only a little after you take into account review. The cases where it shines are high-profile and exciting to managers, but not common enough to make a big difference in practice. E.g AI can one-shot a sc…

I'm finding it in some cases I'm dealing with even more code given how much code AI outputs. So yeah, for some tasks I find myself extremely fast but for others I find myself spending ungodly amounts of time reviewing the code I never wrote to make sure it doesn't destroy the project from unforseen convincing slop.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#180

Post by zcw100 »

I just met a guy from Amazon this past weekend who was bragging, "We've got unlimited access to LLMs and our developers have 10 agents going at a time.". I tried telling him it wasn't all unicorns and rainbows but I didn't get the impression he cared and just kept crapping out skittles.
Post reply on HN