Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PlayAI's new Dialog model achieves 3:1 preference in human evals

play.ht

Post reply on HN
Print view View original

51–59 of 59 posts

Re: PlayAI's new Dialog model achieves 3:1 preference in human evals

#51

Post by crazygringo »

tyre wrote: post #31
crazygringo wrote: post #19

Earlier quoted context omitted.

Yup, and that's going to be the case until AI's can really model human psychology. Speech encodes a gigantic amount of emotion via prosody and rhythm -- how the speaker is feeling, how they feel about each noun and verb, what they're trying to communicate with it. If you try to reproduce all the normal speech prosody, it'll be all over the place and SoUnD bIzArRe and won't make any sense, and be incredibly distractin…

What if you have it read the script, then say, “hey, at this point, what is the character feeling? What are they trying to accomplish? What is there relationship to each person in the scene?” And then you get that and prompt the model to add inflection and pacing and whatever to the text to reflect that. You feed that into the speech model. It seems like it could definitely do the first part (“based on this text, thi…

> What if you have it read the script, then say, “hey, at this point, what is the character feeling?...

Sure, but now how do you make sure all the answers to those questions are consistent? Across clauses, sentences, paragraphs? To do that, you need to have an entire understanding of human psychology.

And I haven't seen any evidence that LLM's possess that kind of knowledge at all, except at the most rudimentary level of narrative.

Just think of how even professional directors struggle to communicate to an actor the emotional and psychological feeling they're looking for. We don't even have words or labels for most of the things, and we say "you know how you feel in a situation when and but ? You know that thing? No, not that, but when . Yeah, that." Most of these things operate on an intuitive, pre-verbal level of thinking in our brain. I don't think LLM's are anywhere close to being able to capture that stuff yet.

Re: PlayAI's new Dialog model achieves 3:1 preference in human evals

#52

Post by munksbeer »

martin-t wrote: post #45
sfink wrote: post #37

Earlier quoted context omitted.

The cat's out of the bag, if someone stops then someone else would start. I would entreat people to consider the net effect of anything they create. Let it at least sway your decisions somewhat. It probably won't be enough to not do it, but I think of it more as the ratio between net positive :: net negative, and paying attention to that ratio should help swing it at least somewhat -- certainly more than giving up an…

The whole idea of developing AGI (even if LLMs are probably the wrong approach) is so strange when you think about it. The smartest people in the world are working very hard in order to make themselves completely redundant and cheaply replaceable. If they succeed, they will turn their main skill and defining characteristic into a meaningless curiosity. And life will not be better, the manual work of today will still…

Do you mean you think we'd all be superfluous for any thinking jobs, and would only be useful for manual labour?

Is there a chance this doom scenario is wrong and we end up with something approaching the idealists benign AGI post scarcity world?

Re: PlayAI's new Dialog model achieves 3:1 preference in human evals

#53

Post by popalchemist »

laurieg wrote: post #46
popalchemist wrote: post #44

Earlier quoted context omitted.

Alternatively, text that is input to these services should be passed through a normalization process, i.e. use LLAMA to convert kanji to hiragana or a romanization. The TTS output is then much better.

Unfortunately, a simple normalization of kanji --> hiragana throws away pronunciation information.

You could just as easily use the LLM to convert the kanji into phonemes.

Re: PlayAI's new Dialog model achieves 3:1 preference in human evals

#54

Post by martin-t »

munksbeer wrote: post #52
martin-t wrote: post #45

Earlier quoted context omitted.

The whole idea of developing AGI (even if LLMs are probably the wrong approach) is so strange when you think about it. The smartest people in the world are working very hard in order to make themselves completely redundant and cheaply replaceable. If they succeed, they will turn their main skill and defining characteristic into a meaningless curiosity. And life will not be better, the manual work of today will still…

Do you mean you think we'd all be superfluous for any thinking jobs, and would only be useful for manual labour? Is there a chance this doom scenario is wrong and we end up with something approaching the idealists benign AGI post scarcity world?

The trend is that once ML models are able to compete with humans on a task, they perform is much faster and cheaply (although very often with a much higher error rate for now).

IMO thinking will be useful to some extent because the bottleneck will be getting instructions from the models into people to perform them. So if the model has to spend less time communicating with you (limited by human cognition, not the model), you will be more productive.

All work that can be done remotely can also be done by AI. Currently there are limitations like you having more context from communicating with your colleagues.

I suspect at some point, companies will try to mandate that all work-related communication (even face to face) has to be made accessible to AI. They will no doubt try to go two steps forward and then back off one step as a compromise so it will be a local model in the name of privacy or something like that.

At that point, AI will have the same work-related knowledge you have. In fact it will be able to cross reference knowledge from all workers in the company. So why would you work from home if it can do everything you could but faster and cheaper?

Re: PlayAI's new Dialog model achieves 3:1 preference in human evals

#55

Post by adriand »

nine_k wrote: post #16
adriand wrote: post #7

Do these services restrict the content that their AIs give voice to? If so, what are the typical restrictions? Like do they seek to prevent their tech being used for scamming, erotica, hate speech, etc? Or is it pretty much anything goes?

How, do you think, can they restrict that? Require that in the EULA, then sue someone who breaks the rules at a scale large enough to be worth the cost of the lawyers? Or do you think they should analyze the text's sentiment and raise a flag if the sentiment is obviously breaking the EULA, e.g. some kind of hate speech? How would you implement that?

Content moderation on text input is how I would assume it would be done, essentially the same way LLMs work now. But to be clear, I am not advocating for it. I’m just asking if they do it.

Re: PlayAI's new Dialog model achieves 3:1 preference in human evals

#56

Post by munksbeer »

martin-t wrote: post #54
munksbeer wrote: post #52

Earlier quoted context omitted.

Do you mean you think we'd all be superfluous for any thinking jobs, and would only be useful for manual labour? Is there a chance this doom scenario is wrong and we end up with something approaching the idealists benign AGI post scarcity world?

The trend is that once ML models are able to compete with humans on a task, they perform is much faster and cheaply (although very often with a much higher error rate for now). IMO thinking will be useful to some extent because the bottleneck will be getting instructions from the models into people to perform them. So if the model has to spend less time communicating with you (limited by human cognition, not the mode…

What is your conclusion? Will it be a benign benefit, bringing us closer to post-scarcity, or will we be doomed to a dystopian existence? Can you elaborate on how you see it?

Re: PlayAI's new Dialog model achieves 3:1 preference in human evals

#57

Post by martin-t »

munksbeer wrote: post #56
martin-t wrote: post #54

Earlier quoted context omitted.

The trend is that once ML models are able to compete with humans on a task, they perform is much faster and cheaply (although very often with a much higher error rate for now). IMO thinking will be useful to some extent because the bottleneck will be getting instructions from the models into people to perform them. So if the model has to spend less time communicating with you (limited by human cognition, not the mode…

What is your conclusion? Will it be a benign benefit, bringing us closer to post-scarcity, or will we be doomed to a dystopian existence? Can you elaborate on how you see it?

Post-scarcity can work if everyone can say "here are me needs, they are now fulfilled, I have a happy and satisfying life". The issue is some people have a deep need to have more than others so they will always strive to have more, including power. AI, if it's ever created will be just another tool they'll use to get it.

Re: PlayAI's new Dialog model achieves 3:1 preference in human evals

#58

Post by xyzhg »

popalchemist wrote: post #53
laurieg wrote: post #46

Earlier quoted context omitted.

Unfortunately, a simple normalization of kanji --> hiragana throws away pronunciation information.

You could just as easily use the LLM to convert the kanji into phonemes.

You can't lose word boundaries and phonemes don't tell you which part of the word is emphasized.

Re: PlayAI's new Dialog model achieves 3:1 preference in human evals

#59

Post by popalchemist »

xyzhg wrote: post #58
popalchemist wrote: post #53

Earlier quoted context omitted.

You could just as easily use the LLM to convert the kanji into phonemes.

You can't lose word boundaries and phonemes don't tell you which part of the word is emphasized.

Modern TTS engines use tokenizers to convert words to phonemes. See: https://github.com/FunAudioLLM/CosyVoice/issues/202
Post reply on HN