Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How AI hears accents: An audible visualization of accent clusters

accent-explorer.boldvoice.com

Post reply on HN
Print view View original

121–130 of 133 posts

Re: How AI hears accents: An audible visualization of accent clusters

#121

Post by shade »

retrac wrote: post #49

I'm deaf. Something close to standard Canadian English is my native language. Most native English speakers claim my speech is unmarked but I think they're being polite; it's slightly marked as unusual and some with a good ear can easily tell it's because of hearing loss. Using the accent guesser, I have a Swedish accent. Danish and Australian English follow as a close tie. It's not just the AI. Non-native speakers of…

Yep, I'm also deaf (since age 6), went through a lot of speech therapy, and have a very pronounced deaf accent. I live in the midwestern US (specifically, Ohio) and at least once a year I get asked where I'm from - England being the most common guess, but I've also had folks ask if I'm Scottish or Australian.

AI struggles massively with my accent. I've gotten the best results out of Whisper Large v2 and even that is only perhaps 60% accurate. It's been on my todo list to experiment with using LLMs to try to clean it up further - mostly so I can do things like dictate blog post outlines to my phone on long car rides - but I haven't had as much time as I'd like to mess around with it.

Re: How AI hears accents: An audible visualization of accent clusters

#122

Post by djmips »

djmips wrote: post #38
zman0225 wrote: post #13

Going mono-tonal to that of an expressive ebook increased my "American English" score from a 52% to 92%. I'd suggest training a little less on audio books.

What does it mean mono-tonal and what is an expressive ebook? I assume you are not American born? I had been of the understanding that rythm was more important than the exact sounds in comprehension.

Why do I get downvoted for asking an innocent question? That's weird

Re: How AI hears accents: An audible visualization of accent clusters

#123

Post by Jemaclus »

nocoiner wrote: post #63
Jemaclus wrote: post #61

Earlier quoted context omitted.

I'm also deaf, and I took 14 years of speech therapy. I grew up in Alabama. The only way you would know I'm from the South is because of the pin-pen merger[1]. Otherwise, you'd think I grew up in the American Midwest, due to how my speech therapy went. Almost nobody picks up on it, unless they are linguists that already knew about the pin-pen merger. [1] https://www.acelinguist.com/2020/01/the-pin-pen-merger.html

I’m aware of the merger, but I literally can’t hear a difference between the words. I certainly pronounce them the same way. I also think merry-marry-Mary are all pronounced identically. The only way I can conceive of a difference between them is to think of an exaggerated Long Island accent, which, yeah, I guess is what makes it an accent.

That's exactly what the pin-pen merger is! As you know, it's not limited to pin/pen, and hearing ability (in my case, profound hearing loss) is not related to the ability to hear the difference. I don't understand the linguistics, but my very bad understanding is that there's actual brain chemistry here that means that you _can't_ hear the difference because you never learned it, never spoke it, and you pronounce them the same.

My partner is from the PNW and she pronounces "egg" as "ayg" (like "ayyyy-g") but when I say "egg" she can't hear the difference between what I'm saying and what she says. And she has perfect hearing. But she CAN hear the difference between "pin" and "pen", and she gets upset when i say them the same way. lol

But yeah, that's one of the things that makes accents accents. It's not just the sounds that come out of our mouths but the way we hear things, too. Kinda crazy. :)

Re: How AI hears accents: An audible visualization of accent clusters

#124

Post by hencq »

glandium wrote: post #70
hencq wrote: post #51

Earlier quoted context omitted.

I'm kind of curious if it would be possible for it to use my own voice but decoupled from accent. I.e. could it translate a recording from my voice to a different accent but still with my voice. If so, I wonder if that makes it easier for accent training if you can hear yourself say things in a different accent.

That would be interesting for sure, but considering you don't hear yourself the same way someone else or a mic does, I'm not sure it would have the benefit you're expecting.

Haha yeah not sure how useful it would be in practice, but mostly curious.

Re: How AI hears accents: An audible visualization of accent clusters

#125

Post by nocoiner »

Jemaclus wrote: post #123
nocoiner wrote: post #63

Earlier quoted context omitted.

I’m aware of the merger, but I literally can’t hear a difference between the words. I certainly pronounce them the same way. I also think merry-marry-Mary are all pronounced identically. The only way I can conceive of a difference between them is to think of an exaggerated Long Island accent, which, yeah, I guess is what makes it an accent.

That's exactly what the pin-pen merger is! As you know, it's not limited to pin/pen, and hearing ability (in my case, profound hearing loss) is not related to the ability to hear the difference. I don't understand the linguistics, but my very bad understanding is that there's actual brain chemistry here that means that you _can't_ hear the difference because you never learned it, never spoke it, and you pronounce the…

When I was listening to some of the samples on the page you linked (pronunciation of “when”), it really seemed to me like the difference they were highlighting was how much the “h” was pronounced. Even knowing what I was listening for, it was very like my brain was just refusing to recognize the vowel sound distinction. So I think you must be right about it being a matter of basic brain chemistry.

In the example of the reverse pen/pin merger (HMS Pinafore) on that page, I couldn’t hear “penafore” to save my life. Fascinating stuff.

I used to think of the movie “Fargo” and think “haha comical upper midwestern accents.” And then at some point I realized that the characters in “No Country for Old Men” probably must sound similarly ridiculous to anyone whose grandparents and great grandparents didn’t all speak with a deep, rural West Texas accent - which mine did, so watching the movie it just seemed completely natural for the place and time at a deeply subconscious level.

Re: How AI hears accents: An audible visualization of accent clusters

#126

Post by thebruce87m »

overfeed wrote: post #54
retrac wrote: post #49

I'm deaf. Something close to standard Canadian English is my native language. Most native English speakers claim my speech is unmarked but I think they're being polite; it's slightly marked as unusual and some with a good ear can easily tell it's because of hearing loss. Using the accent guesser, I have a Swedish accent. Danish and Australian English follow as a close tie. It's not just the AI. Non-native speakers of…

> AI also really has trouble with transcribing my speech. I noticed that as early as the '90s with early speech recognition software. It was completely unusable. I don't know what your transcription use cases are, but you may be able to get an improvement by fine-tuning Whisper. This would require about $4 in training costs[1], and a dataset with 5-10 hours of your labeled (transcribed) speech, which may be the bigge…

> with 5-10 hours of your labeled (transcribed) speech, which may be the bigger hurdle[2].

Can’t you just read from a known script?

Re: How AI hears accents: An audible visualization of accent clusters

#127

Post by adeptima »

oezi wrote: post #76
adeptima wrote: post #59

Did research on accent, pronunciation improvement, phoneme recognition, kaldi ecosystem, etc … nothing really changed in the public domain past few years. There’s no even accurate open source dataset. All self claimedccc manually labelled dataset with 10k+ hours was partly done with automation. Next issue, model models operates in different latent space often with 50ms chunks while pronunciation assessment requires m…

You are right and I don't think incentives exist to solve the issues you describe, because currently many of the building blocks people are building are aligned to erase subtleaccent differences: the neural codecs, transcription systems such as whisper want to output clean/compressed representations of their inputs.

100% agree

Re: How AI hears accents: An audible visualization of accent clusters

#128

Post by gucci-on-fleek »

SkyeCA wrote: post #116
gucci-on-fleek wrote: post #68

Earlier quoted context omitted.

Wow, I'm not deaf, but almost everything you mentioned applies to me too. I've never met anyone else who has experienced this before, yet all of your following points apply exactly to me: > standard Canadian English is my native language > Most native English speakers claim my speech is unmarked > Non-native speakers of English often think I have a foreign accent, too. Often they guess at English or Australian. Like…

I'm a maritimer and I'm constantly getting asked if I'm from South England, even by brits them self. More bizarrely? Locals often assume I'm not from around here as well. I actually don't understand it.

> I'm a maritimer and I'm constantly getting asked if I'm from South England, even by brits them self.

I'm assuming that you're from NS/NB? Because it would be pretty fair for someone to mix up a British and a Newfoundland accent. (I'm from Alberta)

Re: How AI hears accents: An audible visualization of accent clusters

#129

Post by f7f3 »

Antibabelic wrote: post #84
f7f3 wrote: post #64

Earlier quoted context omitted.

It's quite offensive. English is my native tongue, I got a perfect IELTS score, and one of my parents was an English professor. But my accent makes me less than "native".

IELTS is a test for non-native English speakers. Why did you have to take it?

Migration purposes. I'm from South Africa, so it was required.

I would have done it regardless, because it got me extra points.

Re: How AI hears accents: An audible visualization of accent clusters

#130

Post by overfeed »

erikerikson wrote: post #118
overfeed wrote: post #54

Earlier quoted context omitted.

> AI also really has trouble with transcribing my speech. I noticed that as early as the '90s with early speech recognition software. It was completely unusable. I don't know what your transcription use cases are, but you may be able to get an improvement by fine-tuning Whisper. This would require about $4 in training costs[1], and a dataset with 5-10 hours of your labeled (transcribed) speech, which may be the bigge…

We have a PWA for this at: https://www.psyome.com/annotator

It is desktop-only - do you have plans to support mobile browsers? My PWA is mobile-first.
Post reply on HN