Getting junior / mid-level people to slop cannon PRs at seniors will just burn out seniors. The team might be better having fewer developers using AI more thoughtfully.
After outages, Amazon to make senior engineers sign off on AI-assisted changes
291–300 of 510 posts
Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes
#292Earlier quoted context omitted.
When an SVP asks you to do something in a mass email, it's very much optional. Dave Treadwell is an SVP, his org is likely in the 10's of thousands, there is no way to even have a mandatory meeting for that many people. My SVP asks me to do things all the time, indirectly. I do probably 5% of them.
With tens of thousands in a meeting, cracking a 30-second stupid joke is probably costing several thousand dollars.
Right, but if you say something essential in a meeting with 10 people and it has to percolate through five levels of management to reach the front-lines and gets watered down, that could be much more lost, even millions.
Scale cuts both ways.
What matters isn't how big the meeting is, it's how important the material is, and how well presented it is.
Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes
#293Earlier quoted context omitted.
I can see the argument both ways. Some code is just not worth looking at... But do you look at any of the AI output? Or is it just "it works, ship it"?
My last project was basically an ETL implementation on AWS starting with an empty AWS account and a internal web admin site that had 10 pages. I am yada yada yadaing over a little bit. What I checked. 1. The bash shell scripts I had it write as my integration test suite 2. To make sure it wasn’t loading the files into Postgres the naive way -loading the file from S3 and doing bulk inserts instead of using the AWS ext…
This all makes sense.
I've witnessed human developers produce incredibly convoluted, slow "ETL pipelines" that took 10+ minutes to load single digit megabytes of data. It could've been reduced to a shell script that called psql \copy.
Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes
#294Earlier quoted context omitted.
When an SVP asks you to do something in a mass email, it's very much optional. Dave Treadwell is an SVP, his org is likely in the 10's of thousands, there is no way to even have a mandatory meeting for that many people. My SVP asks me to do things all the time, indirectly. I do probably 5% of them.
With tens of thousands in a meeting, cracking a 30-second stupid joke is probably costing several thousand dollars.
If you assume everyone is making 100k it only takes 20 people in a meeting for it to cost 1k.
Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes
#295>Junior and mid-level engineers can no longer push AI-assisted code without a senior signing off Review by a senior is one of the biggest "silver bullet" illusions managers suffer from. For a person (senior or otherwise) to examine code or configuration with the granularity required to verify that it even approximates the result of their own level of experience, even only in terms of security/stability/correctness, r…
> For a person (senior or otherwise) to examine code or configuration with the granularity required to verify that it even approximates the result of their own level of experience, even only in terms of security/stability/correctness, requires an amount of time approaching the time spent if they had just done it themselves.
Hell, often it feels slower/worse. Foreign code is easily confusing at first, which slows you down - and bad code quickly gets bewildering and sends you down paths of clarifications that waste time.
Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes
#296Earlier quoted context omitted.
> He asked staff to attend the meeting, which is normally optional. If I get a note from my boss like that, I consider it mandatory.
But it gets less mandatory the more layers up you go. If I get an email from an SVP that is CC: the entire division saying everyone should go to a meeting I will almost certainly be able to ascertain the contents of that meeting in 10 seconds from someone else who did attend
Surely your boss notices your non-attendance.
Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes
#297The only way to see the kinds of speed-up companies want from these things, right now, is to do way too little review. I think we're going to see a lot of failures in a lot of sectors where companies set goals for reduced hours on various things they do, based on what they expected from LLM speed-ups, and it will have turned out the only way to hit those goals was by spending way too little time reviewing LLM output.…
So will it turn out that actually writing code was never the time sink in the first place? That has always been my feeling. Once I really understand what I need to implement, the code is the easy part. Sure it takes some time, but it's not the majority. And for me, actually writing the code will often trigger some additional insight or awareness of edge cases that I hadn't considered.
> actually writing the code will often trigger some additional insight or awareness of edge cases that I hadn't considered.
Thinking through making.
Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes
#298You could create an agent template for each incident you've ever had, with context pre-cached with the postmortem report, full code change, and any other information about the incident. Then for every new PR you could clone agents from all those templates and ask whether the PR could cause something similar to the pre-loaded incident. If any of them say yes, reject the PR unless there's a manual override. You'd never…
You vastly underestimate the complexity of systems in a company like Amazon.
COEs and Operation Readiness Reviews are already the documents that you mention, but they are largely useless in preventing incidents.
Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes
#299I just met a guy from Amazon this past weekend who was bragging, "We've got unlimited access to LLMs and our developers have 10 agents going at a time.". I tried telling him it wasn't all unicorns and rainbows but I didn't get the impression he cared and just kept crapping out skittles.
Because he doesn’t.
The impression I get from SWEs I’ve met throughout my life is that most of them don’t actually care about their job. They got in because it paid well and demand was plentiful.
Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes
#300Yet another example of vibe coding at scale. You'll have to hire a lot of seniors out of retirement to fix that mess of gigantic proportions... and don't blame "the juniors" for that, they didn't make the decision to allow those tools at first place.
A lot of juniors only graduated using these tools. Good luck taking it away from them.
Juniors don't set up these policies or even chose the tools they have to use professionally. If the higher ups are panicking it's fully of their own doing.