Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

After outages, Amazon to make senior engineers sign off on AI-assisted changes

arstechnica.com

Post reply on HN
Print view View original

341–350 of 510 posts

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#341

Post by happyghost »

VorpalWay wrote: post #314
ardeaver wrote: post #46

Earlier quoted context omitted.

When I was really early in my career, a mentor told me that code review is not about catching bugs but spreading context (i.e. increasing bus factor.) Catching bugs is a side effect, but unless you have a lot of people review each pull request, it's basically just gambling. The more expensive and less sexy option is to actually make testing easier (both programmatically and manually), write more tests and more levels…

This depends on the industry. I work on industrial machine control software, and we spend a huge amount of time on tests. We have to for some parts (human safety crtitical), but other parts would just be expensive if they failed (loss of income for customers, and possibly damaged equipment). The key to making this scalable is to make as few parts as possible critical, and make the potential bad outcomes as benign as…

> make as few parts as possible critical, and make the potential bad outcomes as benign as possible

This is a good lesson for anyone I think. Definitely something I’m going to think more about. Thanks for sharing!

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#343

Post by [deleted] »

koinedad wrote: post #334
philip1209 wrote: post #249

I think the deeper need is a "self-review" flow. People push AI-reviewed code like they wrote it. In the past, "wrote it" implies "reviewed it." With AI, that's no longer true. I advocate for GitHub and other code review systems to add a "Require self-review" option, where people must attest that they reviewed and approved their own code. This change might seem symbolic, but it clearly sets workflows and expectations…

Self review is #1

[deleted]

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#344

Post by wolvoleo »

hyperpape wrote: post #292
javcasas wrote: post #240

Earlier quoted context omitted.

With tens of thousands in a meeting, cracking a 30-second stupid joke is probably costing several thousand dollars.

Right, but if you say something essential in a meeting with 10 people and it has to percolate through five levels of management to reach the front-lines and gets watered down, that could be much more lost, even millions. Scale cuts both ways. What matters isn't how big the meeting is, it's how important the material is, and how well presented it is.

I don't think I've ever heard a top leader say anything essential in such a meeting. The stuff they work on is not related to my job at all. It's all gartner level strategy stuff. In our company they do take time talking about it in large calls but it's always boring and never relevant. And a lot of political spin you have to poke through to see the real message.

If I ever attend it just put it on mute and look at the slides while I do some real work. That way my attendance gets registered and it doesn't stress me out later with too much stuff left hanging.

That percolation is also translation of what they say to things that are relevant at my level. Like what we will be working on next year, if there's going to be bonus or job losses.

I couldn't give a crap about the company's strategy as a whole and that's not my job anyway. Why should I. I'm not here because I believe in some holy mission. I just wanna do something I like and get paid.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#345

Post by [deleted] »

paxys wrote: post #336
philip1209 wrote: post #249

I think the deeper need is a "self-review" flow. People push AI-reviewed code like they wrote it. In the past, "wrote it" implies "reviewed it." With AI, that's no longer true. I advocate for GitHub and other code review systems to add a "Require self-review" option, where people must attest that they reviewed and approved their own code. This change might seem symbolic, but it clearly sets workflows and expectations…

If someone was confident enough to push through an AI change without even reading/reviewing it themselves adding more buttons to the UI isn't going to change anything.

[deleted]

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#346

Post by tmoertel »

javcasas wrote: post #240
djb_hackernews wrote: post #233

Earlier quoted context omitted.

When an SVP asks you to do something in a mass email, it's very much optional. Dave Treadwell is an SVP, his org is likely in the 10's of thousands, there is no way to even have a mandatory meeting for that many people. My SVP asks me to do things all the time, indirectly. I do probably 5% of them.

With tens of thousands in a meeting, cracking a 30-second stupid joke is probably costing several thousand dollars.

Unless that 30-second stupid joke is what gets the audience to take your request seriously. Sometimes people will help you when you don't come across like a self-interested corporate tool.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#347

Post by desireco42 »

So essentially they will be blamed, everything will stay the same.

I do consulting and use AI a lot. You just have to take responsibility for the code. We are delivering like never before, but have a lot of experience into how to do it as safe as possible. And we are learning along the way. They say you need a year to build up experience fyi.

I feel bad for those engineers who will have to sign off for things they will most likely not have enough time to review. Kiro is nice and all.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#348

Post by ljm »

djb_hackernews wrote: post #233
davidclark wrote: post #21

Earlier quoted context omitted.

The article claims: >He asked staff to attend the meeting, which is normally optional. Is that false? It also discusses a new policy: >Junior and mid-level engineers will now require more senior engineers to sign off any AI-assisted changes, Treadwell added. Is that inaccurate? It is good context that this is a regularly scheduled meeting. But, regularly scheduled meetings can have newsworthy things happen at them.

When an SVP asks you to do something in a mass email, it's very much optional. Dave Treadwell is an SVP, his org is likely in the 10's of thousands, there is no way to even have a mandatory meeting for that many people. My SVP asks me to do things all the time, indirectly. I do probably 5% of them.

Is that because you delegate or descope?

Why is an SVP doing this if it's just gonna be ignored?

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#349

Post by 8note »

coredog64 wrote: post #212
8note wrote: post #136

Earlier quoted context omitted.

id.expect COEs to be coming up with AI code action items though, not to have more thorough human checks

There's an explicit tension: SWEs would love that as a "get out of jail free" card, but their management chain is being evaluated by ajassy on AI/ML adoption. Admitting AI code as the root cause of a CoE is gonna look really bad unless/until your peers are also copping to it.

I think its a question 2 or 3 in a why chain, but 4 and 5 need to be why the agent screwed up, and there needs to be action items that are around giving the ai better guardrails, context, or tooling.

"get a person to look at it" is a cop-out action item, and best intentions only. nothing that you could actually apply to make development better across the whole company

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#350

Post by bravetraveler »

baal80spam wrote: post #225
slopinthebag wrote: post #158

Earlier quoted context omitted.

What's stopping someone from just having the AI churn out garbage all day long? Or like, put your AI into plan mode with extra high reasoning and have it churn for 10 minutes to make a microscopic change in some source file. Repeat ad infinium.

> What's stopping someone from just having the AI churn out garbage all day long? In my case it's morality.

Interesting consideration, 'mandates' and all. Definitely in camp 'toss the output', here. I think I'll see 'morality' leaving when $EMPLOYER fires 'professional discretion'... forcing usage and, ultimately, debasing the position.

edit: Peer said it well, IMO. The consequences aren't really yours. Also: something, something, Goodhart's Law.

Post reply on HN