We have a product that uses ChatGPT via the API, using the 3.5 turbo version. Our query involves some dates. Instead of giving back text like it usually does, today it has been giving errors because it does not think 2024-02-29 is a valid date. This is easy to reproduce with the web interface, at least sometimes [0]. It start out by saying it's not a valid date and then as it's explaining why it isn't it realizes its…
Blows my mind that people consider using ChatGPT for serious applications. I mean it's fine as a code autocorrect/autocomplete tool as in GitHub copilot. But it should not replace the code itself. You encounter a bug in the code, you fix it, you never encounter it again. But ChatGPT will repeat the same mistake sooner or later. That's not how we should engineer solution for critical problems.
I mean, that's how we do it with humans. It's quite a common occurrence to keep a part of a business process human because automating it would we too expense due to edge cases.
Humans make mistakes and are expensive, but are also flexible and usually smartish. ChatGPT makes mistakes and is usually dumbish, but is also flexible and cheap.
Engineering is about picking the right trade-offs in your solution.