Why could we ever assume that?
> but the notion that it could be right took the guesswork out of our next steps,
Devils advocate here. Couldn't this just be a severe case of confirmation bias? You take 100 such cases, ask AI "how does it work?" and in 99 of those, the answer is somewhere on the spectrum between "total nonsense" and "clever formulation but wrong". One turns out to be right. That's the on we are seeing here, getting confirmed in the lab. That doesn't actually mean AI reduced the time by 75%.
A broken clock is also correct twice a day. We wouldn't say we have invented a clock that works without energy, sure it's wrong sometimes, but when it's correct, it's awesome! No, it's just a broken clock that's wrong most of the time.
I would also love to see that with "generative AI" we have discovered some helpful magic, but as long as we are not honest about those details (which would include publishing and owning up to mishaps), this is all just riding a hype train.