I hereby dub this "Baron-von-Munchausen-as-a-Service."
Now you can pay real money for a chatbot to make stuff up about your company and its products.
While it is pretty incredible stuff, until or unless they have a veracity bit — some sort of "please don't lie" flag in it, I'd be wary of what it produces.
Will ChatGPT offer off-the-cuff pricing, discounts, rebates and refunds that are in line with your actual business model? Will it make written offers you will be legally-obligated to adhere to? Will it lie about your features and capabilities, or that of your competitors? Will it invent whole-cloth SLAs and KPIs you'll never be able to adhere to? Will it make up arrival and departure times? Medical conditions and prescribed treatments/therapies? Will it glibly give instructions that can get consumers/users who read these statements fired from their jobs, fined, imprisoned or killed?
There are places where you only want to give a generative AI so much room to color outside the lines. And after that, you are going to want traditional procedural logic to take over. Decision-trees from manually-maintained systems and even multi-decadal-honed expert systems.
There are some things where letting a computer make up a plausible response is ... okay-ish. And there are times where you can cause immense amounts of damage by letting things get outside of your control and human review.
This should come with a consumer warning label that would put your typical pharmaceutical safety disclosure to shame.
Now, with all that said, I am amazed at many of the results these LLM-based systems can generate. My main concern is that they are getting so good — or at least so plausible — that they can even fool domain experts unless they read really, really closely. And the experts won't be able to do that at scales of 10s or 100s of thousands of transactions per second.