Earlier quoted context omitted.
I’ll bite - because 80% of my searches it returns a convenient answer that matches my expectations of what the answer will look like. I still have to validate (there’s still an objective reality to be satisfied) but that’s pretty good. 20% of the time I have to google, but that’s made more efficient by using specific terms mentioned by chatGPT - getting the right terms in a google search is half the battle. It does s…
How do you know what part of the answer is wrong?
So you either know enough to spot the error or you verify all of it.
Another example. I had a long session with it where I was trying to get it to write a gitlab ci config file where the artifacts to be deployed to Netlify were already present at the root of the repo. As an addition it had a requirement to use 'sed' to rewrite a domain name recursively. This is the first and only time chatgpt was useful to me although it made a lot of mistakes that needed correcting.It was more useful than a google search because I wasn't able to find the correct format with it as most examples assumed a build step.
One thing that was frustrating is how I would give it requirements and it would incorporate them only to lose a prior requirement when fixing a latter one. I ended up hand fixing one of the later configs rather than using the final one it gave me.
For example I needed to use a node image because netlify-cli was required (the only custom thing necessary from the image) but it would revert to using an Alpine image and point to a now archived repo with a nonexistent release url for gitlab's cli.
When it fixed something else it would break the find .. sed combination and add nonsensical things like *. html to it rather than working on all files recursively.
In the end, through a combination of AI helping and the user (me) chiming in and trying to reason about it I was able to get somewhere.
Not ideal but not useless.