Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

After outages, Amazon to make senior engineers sign off on AI-assisted changes

arstechnica.com

Post reply on HN
Print view View original

311–320 of 510 posts

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#311

Post by SchemaLoad »

locusofself wrote: post #50
marginalia_nu wrote: post #31

Earlier quoted context omitted.

Expert reviews are just about the only thing that makes AI generated code viable, though doing them after the fact is a bit sketchy, to be efficient you kinda need to keep an eye on what the model is doing as its working. Unchecked, AI models output code that is as buggy as it is inefficient. In smaller green field contexts, it's not so bad, but in a large code base, it's performs much worse as it will not have acces…

If you spend 5-15x the time reviewing what the LLM is doing, are you saving any time by using it?

The submitter with their name on the Jira ticket saves time, the reviewer who has to actually verify the work loses a lot of time and likely just lets issues slip through.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#313

Post by sizzzzlerz »

Who fixes code that gets rejected? Do you simply try again and hope or does someone go into this computer-generated code that they didn't write and do the equivalent of battlefield triage?

And what are they going to do when they've fired all the senior engineers because they make too much money, leaving just juniors and AI?

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#314

Post by VorpalWay »

ardeaver wrote: post #46
happytoexplain wrote: post #19

>Junior and mid-level engineers can no longer push AI-assisted code without a senior signing off Review by a senior is one of the biggest "silver bullet" illusions managers suffer from. For a person (senior or otherwise) to examine code or configuration with the granularity required to verify that it even approximates the result of their own level of experience, even only in terms of security/stability/correctness, r…

When I was really early in my career, a mentor told me that code review is not about catching bugs but spreading context (i.e. increasing bus factor.) Catching bugs is a side effect, but unless you have a lot of people review each pull request, it's basically just gambling. The more expensive and less sexy option is to actually make testing easier (both programmatically and manually), write more tests and more levels…

This depends on the industry. I work on industrial machine control software, and we spend a huge amount of time on tests. We have to for some parts (human safety crtitical), but other parts would just be expensive if they failed (loss of income for customers, and possibly damaged equipment).

The key to making this scalable is to make as few parts as possible critical, and make the potential bad outcomes as benign as possible. (This lets you go to a lower rating in whatever safety standard applies to your industry.) You still need tests for the less critical parts though, while downtime is better than injury, if you want to sell future machines to your customers you need to have a good track record. At least if you don't want to compete on cost.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#316

Post by messh »

djb_hackernews wrote: post #233
davidclark wrote: post #21

Earlier quoted context omitted.

The article claims: >He asked staff to attend the meeting, which is normally optional. Is that false? It also discusses a new policy: >Junior and mid-level engineers will now require more senior engineers to sign off any AI-assisted changes, Treadwell added. Is that inaccurate? It is good context that this is a regularly scheduled meeting. But, regularly scheduled meetings can have newsworthy things happen at them.

When an SVP asks you to do something in a mass email, it's very much optional. Dave Treadwell is an SVP, his org is likely in the 10's of thousands, there is no way to even have a mandatory meeting for that many people. My SVP asks me to do things all the time, indirectly. I do probably 5% of them.

so.... is RTO optional

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#317

Post by nothrabannosir »

Tyr42 wrote: post #258
philip1209 wrote: post #249

I think the deeper need is a "self-review" flow. People push AI-reviewed code like they wrote it. In the past, "wrote it" implies "reviewed it." With AI, that's no longer true. I advocate for GitHub and other code review systems to add a "Require self-review" option, where people must attest that they reviewed and approved their own code. This change might seem symbolic, but it clearly sets workflows and expectations…

Heck, doing a self review when you wrote the code catches stuff like forgetting debug prints.

(tangent of the decade : prefixing your debug printfs with NOCOMMIT helps catching them before commit :) sample precommit hook and GitHub ci action I wrote is at https://github.com/nobssoftware/nocommit but it’s just a grep)

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#318

Post by yalogin »

happytoexplain wrote: post #19

>Junior and mid-level engineers can no longer push AI-assisted code without a senior signing off Review by a senior is one of the biggest "silver bullet" illusions managers suffer from. For a person (senior or otherwise) to examine code or configuration with the granularity required to verify that it even approximates the result of their own level of experience, even only in terms of security/stability/correctness, r…

Don’t forget that this auto generated code will have subtle bugs and feels complete at the outset

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#319

Post by hibikir »

MikeTheGreat wrote: post #301
djb_hackernews wrote: post #233

Earlier quoted context omitted.

When an SVP asks you to do something in a mass email, it's very much optional. Dave Treadwell is an SVP, his org is likely in the 10's of thousands, there is no way to even have a mandatory meeting for that many people. My SVP asks me to do things all the time, indirectly. I do probably 5% of them.

> org is likely in the 10's of thousands, there is no way to even have a mandatory meeting for that many people. Ok, this is pretty off-topic, but is this still true? I get that you can't have 10K people all actively participate in the meeting at the same time, but doesn't Zoom have a feature where you can broadcast to thousands and thousands? Doesn't X/Twitter have a feature like this? (Although, to be fair, the las…

Yes, but at that point it's an all-hands presentation, and you are basically doing a very careful presentation, thinking about every minute, because of how many hours the "meeting" is costing you.

Very different from the typical weekly/montly outage meeting, where discussion is actually expected, instead of being a ritual.

Re: After outages, Amazon to make senior engineers sign off on AI-assisted changes

#320

Post by bombdailer »

hard24 wrote: post #56
frogperson wrote: post #22

Earlier quoted context omitted.

makes me want to vomit. I am not spending more time reviewing code than the "author" spent creating it. Ill just leave the industry if that happens.

I think as long as having to review code stays around, the 'artistry' of writing code isn't going away. Think about it - how do you increase the speed at which one can review code? Well first it must be attractive to look at - the more attractive the faster you review/understand and move through the review. Now this won't be the case everywhere - e.g. in outsourced regions the conditions will force people to operate…

The better looking the code, the less effort people will put into reviewing it due to the ease of reading it - the assumption being that what is beautiful is good. Just as a beautiful facade of a building can hide a cheap structure behind it, the same is true with code. Beauty itself is not a good signal for goodness as in excess it is in effect a rhetoric device that aims to mislead and draw ones eyes towards itself and away from what lies beneath it.

A beautiful building is only as good as the correctness of its foundation, framework, materials, and construction. Those qualities can only be assessed by those with expertise enough to understand their importance. Beauty in its proper place is the output of the intersection between a craftsman and a engineer. Beauty is optional, but it makes life more worth living. The same is true for code - attractive code is optional, but it makes being a SWE more rewarding.

Post reply on HN