[dead]
Show HN: I trained a 9M speech model to fix my Mandarin tones
71–80 of 166 posts
Re: Show HN: I trained a 9M speech model to fix my Mandarin tones
#72I'm also working on a Chinese learning app (heyzima.com) and my "solution" to this was to use the TTS token/word log probabilities.
Re: Show HN: I trained a 9M speech model to fix my Mandarin tones
#73Super nice, thanks for sharing!
There's one thing that gave me pause: In the phrase 我想学中文 it identified "wén" as "guó". While my pronunciation isn't perfect, there's no way that what I said is closer to "guó" than to "wén".
This indicates to me that the model learned word structures instead of tones here. "Zhōng guó" probably appears in the training data a lot, so the model has a bias towards recognizing that.
- Edit -
From the blog post:
> If my tone is wrong, I don’t want the model to guess what I meant. I want it to tell me what I actually said.
Your architecture also doesn't tell you what you actually said. It just maps what you said to the likeliest of the 1254 syllables that you allow. For example, it couldn't tell you that you said "wi" or "wr" instead of "wo", because those syllables don't exist in your setup.
Re: Show HN: I trained a 9M speech model to fix my Mandarin tones
#74Longtime lurker, made an account specifically to give feedback here as an intermediate speaker. :) This is a great initiative and I hope to see more come out of this; I am not criticizing, but just want to provide my user experience here so you have data points. In short, my experience lines up with your native speakers. I found that it loses track of the phonemes when speaking quickly, and tones don't seem to line u…
I don't think it takes care of tone transformation (eg 他是 ni3shi4 -> ni2shi4). Or if it does, my tones are just off. But it's a really cool idea!
他是 is tāshì which doesn't transform I think. Did you mean to write 你是 nǐshì? I think that transforms differently though. With the half 3rd tone only dropping.
The classical example is 4/4 不是. Which goes bùshì -> búshì.
Or 3/3 that becomes 2/3. E.g. 你好 nǐhǎo becoming níhǎo.
The 1/4 -> 2/4 transformation I think is specific to one. 一个 yīgè becomes yígè.
Re: Show HN: I trained a 9M speech model to fix my Mandarin tones
#75Very cool ! Will you make the source available as well?
Re: Show HN: I trained a 9M speech model to fix my Mandarin tones
#76How difficult would it be to adapt this to Cantonese? It is a surprisingly difficult language to learn. It has more tones than Mandarin plus comparatively less access to learning resources (in my experience)
Unlike Mandarin and other Chinese languages, Cantonese does not have tone sandhi and has changed tones instead. Cantonese tones are also different from those of Mandarin, so no, it can't be adopted for Cantonese and it would require a complete rework. > It is a surprisingly difficult language to learn. I keep hearing this quite a bit, but I do not find Cantonese to be any more difficult than most languages[0]. Or at…
Cantonese is "hard" mainly for two reasons-
1. tones, and generally the gatekeeping of some Cantonese communities towards people who haven't gotten the tones completely right
2. the lack of learning materials relative to the number of speakers, the confusion between written Chinese and written Cantonese (and also the general lack of the latter)
As they say, "a language is a dialect with an army and navy"... I'll leave it at that.
Re: Show HN: I trained a 9M speech model to fix my Mandarin tones
#77This is incredible. When I was first learning Chinese (casually, ~20 years ago), my teacher used some Windows software that drew a diagram of the shape of my pronunciation, so she could illustrate what I was getting wrong in some objective way. The thing you've built is so good, and I would have loved to have it when I was learning Mandarin. I tried it with a couple of sentences and it did a good job of identifying w…
You're probably thinking of Praat, which is still around. Even has the same UI as 20 years ago.
Re: Show HN: I trained a 9M speech model to fix my Mandarin tones
#78Earlier quoted context omitted.
I don't think it takes care of tone transformation (eg 他是 ni3shi4 -> ni2shi4). Or if it does, my tones are just off. But it's a really cool idea!
The tone sandhi example you just gave looks incorrect to me
Well, OP wrote "he is" but then wrote "you are" in pinyin for one, and that's a bit hard to reconcile.
Re: Show HN: I trained a 9M speech model to fix my Mandarin tones
#79Cool! I'm not great at Chinese but I have to speak slowly for it to recognize the tones/words. I wonder how fast the training data is.
Re: Show HN: I trained a 9M speech model to fix my Mandarin tones
#80Anyone that is a native European language speaker that hasn't tried to learn Chinese or some other tonal language, its really hard to understand how hard it is. The tones can really be very subtle, and your ear is not fine tuned to them. So you think you are saying it right, but native speakers have no idea what you are saying.
The tones are really not as difficult as people make them out to be.
90% of the effort in learning any language is just learning massive amounts of vocabulary.
Things like tone and grammar are the very basics that you learn right at the beginning.‡ Beginners complain about them, but after a few months of studying Chinese, you should be fairly comfortable with the tones. Then, you spend years learning vocabulary.
The two things that make Chinese difficult are:
1. The lack of shared vocabulary with Indo-European languages (this obviously doesn't apply if your native language is something with more shared vocabulary with Chinese).
2. The writing system, which because it's not phonetic requires essentially the same level of effort as learning an entirely new language (beyond spoken Chinese).
‡. The same goes for grammar issues (like declension and conjugation) that people always complain about when learning Indo-European languages. These are the very basics that you learn early on. Most of the real effort is in learning vocab.