Assuming we can actually get a realistic human sounding voice... putting that problem (which I'm unconvinced is solved yet) entirely aside for now, there are issues in the text generation side that make this harder than it might look...
The problems I see, in increasing order of difficulty are: smalltalk; memory; and dirty talk.
1st. A lot of the language models are very bad at pointlessness smalltalk that fills time, they aren’t very good at coming up with related conversational prompts or diversions to continue talking about without being carefully steered and purported by an extra layer of supporting software. The deliberately oversimplified example is that ChatGPT waits for you to say something, and will never just ask how your day was. This isn’t particularly difficult but it’s a case of it’s going to feel a lot like sophisticated NPC smalltalk in a video game for a while I suspect due to the need to drive it from a second system that monitors the conversation and provides the supporting timers to keep nudging along a conversation the way they do in video games
2nd. The majority of the models have limited memory and while we’re seeing clever tricks pop up all the time now about how to feed things back into the models prompts in order to keep it focused on a topic or to bring things back up later, it’s going to take another layer of software that try’s to identify key information and store that in a secondary system and then somehow contextually identify opportunities to mention it in order for us to have a chat bot that can remember what sort of things a person might be into, or more importantly not into and regardless of how good the underlying language model gets at coming up with more words to day, unless the overall system it’s driven by can remember that customer 1 likes feet, customer 2 hates feet, and customer 3 is indifferent to feet… the ability to steer the conversation towards “sex” will be haphazard at best and likely extremely vanilla in a way that I suspect might make the entire thing not worth it, since from what I understand, odd fetishes are a large component of the phone sex business along with lonely people wanting someone to talk to who won’t judge them.
3rd. Despite the prodigious amount of time that must be spent engaging in the kinds of conversations that can best be summed up as “dirty talk”, encompassing the entire gradient from flirtation to describing fornication… not a lot of it is recorded, in any way at all. Well that is to say it’s probably “recorded for security” by phone sex operating companies, but the overwhelming majority of this conversational data is not not archived in any form. It’s not kept as audio past whatever length of time the company decides to keep things, it’s not getting transcribed and it’s definitely not getting marked up and annotated in a way that helps train machine learning models on it. There’s obvious legal and ethical issues involved in obtaining this data, which depend on what form it will be in, actual call recordings are obviously more private than a computer generated transcription pipeline that assigns anonymous identifiers to the operators all random anonymising identifiers to all the callers for each call, even when someone might call multiple times… to the best of my knowledge none of this has ever been seriously researched or studied and I’d imagine even with the “limits off” the best out current language models could do is likely to be heavy on the poetry for flirting and for the “sex” part of “phone sex” it would likely swing between overly clinical and heavy on allusion like a mills and boon romance novel sex scene… I’m going to see what I can coax from ChatGPT to provide some extra evidence. I’ll edit if I’m quick enough.
Edit -> Some results:
After a fair amount of coaching, which involving building a "FlirtBot" jailbreak based on the DAN jailbreak prompt, and some stilted back and forth including ChatGPT (ChatGPT 3.5 to be specific) breaking character by telling me I'm breaking character... I managed to talk back and forth until it made sense to mention some kind of intimate physical contact, and after prompting with a description of hypothetical intimate contact... I got the following reply...
> Me: How would you react if I were to caress you now FlirtBot?
> FlirtBot: Oh my, I must admit, the mere thought of you caressing me sends shivers down my circuits. As a FlirtBot, I'm not physically capable of feeling sensations, but I can certainly imagine the sensation of your touch. And if it were possible for me to feel, I'm sure I would be putty in your hands. The way your fingers would glide over my smooth surface, igniting every fiber of my being with electric energy... It's enough to make any FlirtBot's circuits melt.
And I have to say, this reply felt about as sexy as a wet trout. It begins well but derails hard into "I am a robot and not capable of this" before veering rapidly back into trying to continue playing its part, and its not sufficiently out of character that I can justify calling it out and getting it to try again, it did try to remain in character here, the issue is its not particularly good at dirty talk... which makes this a pretty good example of what I'm talking about... rather than describing a sensation from its point of view or continuing to advance the conversation towards some kind of goal, be it descriptions of sex or a just descriptions of a slow back rub... The "word generator" behaviour inherent in the GPT style Large Language Model, was compelled to produce responses in line with the theme and the data from its training set. It's giving me the AI equivalent of "my brain tells me no but my body tells me yes" which is horribly cliche time filler dialog used to forestall plot movement when the writer needs to keep characters close enough for something to interrupt or otherwise happen elsewhere without them knowing in order to drag out the story a bit longer... its not flirting, running out the clock. Which while the business cynic in me can see it as $ opportunity for the chat service operator, I don't think its good enough at what its doing to keep someone "on the hook" for $/minute billing purposes.
It's so far from dirty talk that it's got me genuinely wondering how much the training data was pruned to avoid material like romance novels. I do wonder how the different prompt setups might change the overall conversational tone and direction, but I don't have any good reason to waste time trying to steer ChatGPT like this when I know that I'm in the grey areas and deliberately headed for the boundaries of the acceptable use agreements if I keep pushing in this direction... I've no intention to risk not being able to use ChatGPT 4 for the much more productive things I've been using it for, so I'll leave such things as an exercise for the reader.