Live data from Hacker News

Show HN: I trained a 9M speech model to fix my Mandarin tones

simedw.com

41–50 of 166 posts

Re: Show HN: I trained a 9M speech model to fix my Mandarin tones

#41

Longtime lurker, made an account specifically to give feedback here as an intermediate speaker. :) This is a great initiative and I hope to see more come out of this; I am not criticizing, but just want to provide my user experience here so you have data points. In short, my experience lines up with your native speakers. I found that it loses track of the phonemes when speaking quickly, and tones don't seem to line u…

I don't think it takes care of tone transformation (eg 他是 ni3shi4 -> ni2shi4). Or if it does, my tones are just off. But it's a really cool idea!

Re: Show HN: I trained a 9M speech model to fix my Mandarin tones

#42
I wish this had a pinyin mode…! I am learning to speak Mandarin but I am not learning to read/write.

( I’m learning using a flashcards web app I made and continue to update with vocab I encounter or need: https://memalign.github.io/m/mandarin/cards/index.html )

Re: Show HN: I trained a 9M speech model to fix my Mandarin tones

#43

I wish this had a pinyin mode…! I am learning to speak Mandarin but I am not learning to read/write. ( I’m learning using a flashcards web app I made and continue to update with vocab I encounter or need: https://memalign.github.io/m/mandarin/cards/index.html )

same! but if you get it inevitably wrong the first time it gives you the pinyin. but i struggled to get it to transcribe the consonants I was making let alone the tones. i'm pretty sure i'm not as bad as that!

Re: Show HN: I trained a 9M speech model to fix my Mandarin tones

#44

Longtime lurker, made an account specifically to give feedback here as an intermediate speaker. :) This is a great initiative and I hope to see more come out of this; I am not criticizing, but just want to provide my user experience here so you have data points. In short, my experience lines up with your native speakers. I found that it loses track of the phonemes when speaking quickly, and tones don't seem to line u…

The tool definitely needs to address tone transformations, it’s a big part of how the language is spoken. Otherwise it’s mostly useful for a first year student speaking in isolation.

Hoping to see improvements in this area

Re: Show HN: I trained a 9M speech model to fix my Mandarin tones

#45
This is very cool, but from one Mandarin learner to another I’d caution against relying too heavily on any external feedback mechanism for improving your pronunciation.

If you can’t easily hear your pronunciation mistakes so clearly it hurts, consider putting more energy into training your ear. Adult language learners usually have brains that have become resistant to, but not incapable of, changing the parts of the brain responsible for phoneme recognition. The neuroplasticity is still there but it needs some nudging with focused exercises that make it clear to your brain exactly what the problem is. Minimal pair recognition drills, for example, are a great place to start.

It’s not the most fun task, but it’s worth it. You will tighten the pronunciation practice feedback loop much more than is possible with external feedback, so a better accent is the most obvious benefit. But beyond that, it will make a night and day difference for your listening comprehension. And that will get you access to more interesting learning materials sooner. Which hopefully increases your enjoyment and hence your time on task. Plus, more accurate and automatic phoneme recognition leaves more neurological resources free for processing other aspects of your input materials. So it may even help speed things like vocabulary and grammar acquisition.

Re: Show HN: I trained a 9M speech model to fix my Mandarin tones

#46
post #3

When I was living in Taiwan, one of the ways I forced myself to remember to pronounce the tones distinctly was by waving my hand in front of me, tracing the arc of each character’s tone. It helped a lot even if I did look like an insane expat conducting an invisible orchestra. One more thing: there's quite a bit of variation in how regional accents in the mainland can affect tonal pronunciation. It might be worth rea…

You'll love Mike Laoshi: https://youtu.be/cna89A2KAU4?si=SQEZ_0ooO1z119_k

Re: Show HN: I trained a 9M speech model to fix my Mandarin tones

#48
Have you tried the Azure Speech Studio? I wonder how your custom model compares to this solution.

I played around with python scripts for the same purpose. The AI gives feedback that can be transformed to a percentage of correctness. One annoyance is that for Mandarin, the percentage is calculated at the character level, whereas with English, it gives you a more granular score at the phoneme level.

Re: Show HN: I trained a 9M speech model to fix my Mandarin tones

#49

Anyone that is a native European language speaker that hasn't tried to learn Chinese or some other tonal language, its really hard to understand how hard it is. The tones can really be very subtle, and your ear is not fine tuned to them. So you think you are saying it right, but native speakers have no idea what you are saying.

Wholeheartedly (or maybe downheartedly?) agree with this - sometimes I try to say the simplest things and people just stare at me like I'm speaking Martian. Which I suppose I might as well be! One of my big problems is implicit use of tones for things like expressing uncertainty; that's a very difficult habit to get out of.

Another one that I wish I had realized sooner is that, contrary to the impression teachers tend to convey, tones aren’t just a pitch contour thing. There are also intensity and cadence elements. Native speakers can fairly accurately recognize tones in recordings that have had all the pitch contour autotuned out.
Post reply on HN