Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How AI hears accents: An audible visualization of accent clusters

accent-explorer.boldvoice.com

Post reply on HN
Print view View original

51–60 of 133 posts

Re: How AI hears accents: An audible visualization of accent clusters

#51

Post by hencq »

dwohnitmok wrote: post #47
pinkmuffinere wrote: post #46

Why do the voices all sound so similar? I'm not talking about accent, I'm talking about the pitch, timbre, and other qualities of the voice themselves. For instance, all the phrases I heard sounded like they were said by a medium-set 45 year old man. Nothing from kids, the elderly, or people with lower / higher-pitch voices. I assume this expected from the dataset for some reason, but am really curious about that rea…

From the article: > By clicking or tapping on a point, you will hear a standardized version of the corresponding recording. The reason for voice standardization is two-fold: first, it anonymizes the speaker in the original recordings in order to protect their privacy. Second, it allows us to hear each accent projected onto a neutral voice, making it easier to hear the accent differences and ignore extraneous differen…

I'm kind of curious if it would be possible for it to use my own voice but decoupled from accent. I.e. could it translate a recording from my voice to a different accent but still with my voice. If so, I wonder if that makes it easier for accent training if you can hear yourself say things in a different accent.

Re: How AI hears accents: An audible visualization of accent clusters

#52

Post by pinkmuffinere »

dwohnitmok wrote: post #47
pinkmuffinere wrote: post #46

Why do the voices all sound so similar? I'm not talking about accent, I'm talking about the pitch, timbre, and other qualities of the voice themselves. For instance, all the phrases I heard sounded like they were said by a medium-set 45 year old man. Nothing from kids, the elderly, or people with lower / higher-pitch voices. I assume this expected from the dataset for some reason, but am really curious about that rea…

From the article: > By clicking or tapping on a point, you will hear a standardized version of the corresponding recording. The reason for voice standardization is two-fold: first, it anonymizes the speaker in the original recordings in order to protect their privacy. Second, it allows us to hear each accent projected onto a neutral voice, making it easier to hear the accent differences and ignore extraneous differen…

Ah thanks, missed that somehow

Re: How AI hears accents: An audible visualization of accent clusters

#53

Post by chupchap »

retrac wrote: post #49

I'm deaf. Something close to standard Canadian English is my native language. Most native English speakers claim my speech is unmarked but I think they're being polite; it's slightly marked as unusual and some with a good ear can easily tell it's because of hearing loss. Using the accent guesser, I have a Swedish accent. Danish and Australian English follow as a close tie. It's not just the AI. Non-native speakers of…

To judge local or not, I would consider use of phrases and word as well, and not just the accent. Perhaps, that's what is working for you?

Re: How AI hears accents: An audible visualization of accent clusters

#54

Post by overfeed »

retrac wrote: post #49

I'm deaf. Something close to standard Canadian English is my native language. Most native English speakers claim my speech is unmarked but I think they're being polite; it's slightly marked as unusual and some with a good ear can easily tell it's because of hearing loss. Using the accent guesser, I have a Swedish accent. Danish and Australian English follow as a close tie. It's not just the AI. Non-native speakers of…

> AI also really has trouble with transcribing my speech. I noticed that as early as the '90s with early speech recognition software. It was completely unusable.

I don't know what your transcription use cases are, but you may be able to get an improvement by fine-tuning Whisper. This would require about $4 in training costs[1], and a dataset with 5-10 hours of your labeled (transcribed) speech, which may be the bigger hurdle[2].

1. 2000 steps took me 6 hours on an A100 on Collab, fine-tuning openai/whisper-large-v3 on 12 hours of data. I can shar my notebook/script with you if you'd like.

2. I am working on a PWA that makes it simple for humans to edit initial, automated transcriptions with mistakes for feeding the correct dataset back into the pipeline for fine-tuning, but its not ready yet

Re: How AI hears accents: An audible visualization of accent clusters

#55

Post by TomNomNom »

I tried the oracle and got this:

> Your accent is Dutch, my friend. I identified your accent based on subtle details in your pronunciation. Want to sound like a native English speaker?

I'm British; from Yorkshire.

When letting it know how it got it wrong there's no option more specific than "English - United Kingdom". That's kind of funny, if not absurd, to anyone who knows anything of the incredible range of accents across the UK.

I also think the question "Do you have an accent when speaking English?" is an odd one. Everyone has an accent when speaking any language.

Re: How AI hears accents: An audible visualization of accent clusters

#56

Post by sailingparrot »

> This voice standardization model is an in-house accent-preserving voice conversion model.

Not sure this model works really well. As a french/spanish native speaker, I can immediately recognize an actual French or Spanish person speaking in english, but the examples here are completly foreign to me. If I had to guess where the "french" accent was from I would have guessed something like Nigeria. For example spanish have a very distinct way of pronouncing "r" in english that is just not present here. I would have been unable to correctly guess French or Spanish for the ~10 examples present in each language (mayyybe 1 for French).

Re: How AI hears accents: An audible visualization of accent clusters

#58

Post by david-gpu »

TomNomNom wrote: post #55

I tried the oracle and got this: > Your accent is Dutch, my friend. I identified your accent based on subtle details in your pronunciation. Want to sound like a native English speaker? I'm British; from Yorkshire. When letting it know how it got it wrong there's no option more specific than "English - United Kingdom". That's kind of funny, if not absurd, to anyone who knows anything of the incredible range of accents…

> I also think the question "Do you have an accent when speaking English?" is an odd one. Everyone has an accent when speaking any language.

Sure, I agree. But look at it from the perspective of a foreigner living in an English-speaking country, which is probably their target demographic.

We know that as soon as we open our mouth the locals will instantly pigeonhole us as "a foreigner". No matter how good we might be in other areas, we will never be one of "them". The degree of prejudice that may or may not exist against us doesn't matter as much as the ever present knowledge that the locals know that we are not one of them, and the fear of being dismissed because of that.

Nobody likes to stand out like that, particularly when it so clearly puts you at a disadvantage. That sort of insecurity is what this product is aimed at.

Re: How AI hears accents: An audible visualization of accent clusters

#59

Post by adeptima »

Did research on accent, pronunciation improvement, phoneme recognition, kaldi ecosystem, etc … nothing really changed in the public domain past few years. There’s no even accurate open source dataset. All self claimedccc manually labelled dataset with 10k+ hours was partly done with automation. Next issue, model models operates in different latent space often with 50ms chunks while pronunciation assessment requires much better accuracy. Just try to say B loud - silent part gathering energy in the lips, loud part, and everything what resonates after. Worst part there are too many ml papers from the last year students or junior phd folks claiming success or fake improvements, etc

The article itself is just a vector projection in 3d space … the actual reality is much complex.

Any comments on pronunciation assessment models are greatly appreciated

Re: How AI hears accents: An audible visualization of accent clusters

#60

Post by david-gpu »

djmips wrote: post #40
bikeshaving wrote: post #34

Earlier quoted context omitted.

Yes the Spanish spoken in Spain, especially the one that’s like /ˈɡɾaθjas/ and /baɾθeˈlona/.

But Spanish sounds very different in Spain depending on what region of the country you are talking about.

Yeah, and not all Spaniards have a distinct pronunciation for "c" and "s". For those curious: https://en.wikipedia.org/wiki/Phonological_history_of_Spanis...
Post reply on HN