Earlier quoted context omitted.
So far (super early days, I know) the main thing I've used AI for at work is to write tests for code I'm working on. I get to do the fun bit, solving the problem space, and the AI gets to do the tedious bit (and do it faster than I would). I'm liking at least that level of AI integration.
This is the one that is most scary to me as a user of software. Have the AI write some of the code, sure. But tests have to be correct. Autogenerating a mountain of mostly-correct tests strikes me as a great way of ending up with surprising behavior that is a nightmare to untangle.
Or you could have the AI write test cases (writing them out is often the most laborious part) and then validate them by hand. That'd be little different than writing them yourself, though again edge cases may be missed. You just skip the un-fun part of repetitively typing out the code for each case.