Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Neural Japanese Transliteration

github.com

Post reply on HN
Print view View original

21–30 of 55 posts

Re: Show HN: Neural Japanese Transliteration

#21

Post by yomly »

gizmo686 wrote: post #14
derefr wrote: post #12

Earlier quoted context omitted.

Spoken Japanese isn't any more ambiguous than English (for a human, or a speech-to-text AI) because Japanese people pause between spoken words just like anyone else. But a stream of romaji furigana with no spaces is quite ambiguous—since there's nothing to indicate word boundaries, any substring of the input might turn out to have actually intended to be e.g. a katakana spelling of a name. If CJK IMEs expected and re…

>because Japanese people pause between spoken words just like anyone else. This is (surprisingly) not true. People do not pause between words, however when listening to a language that they understand, they do perceive pauses between words; even though such pauses do not exist.

I disagree. There's a subtle difference between breathing cadence and inflection, and a completely monotempo monotonal string of sounds.

Re: Show HN: Neural Japanese Transliteration

#22

Post by [deleted] »

gizmo686 wrote: post #14
derefr wrote: post #12

Earlier quoted context omitted.

Spoken Japanese isn't any more ambiguous than English (for a human, or a speech-to-text AI) because Japanese people pause between spoken words just like anyone else. But a stream of romaji furigana with no spaces is quite ambiguous—since there's nothing to indicate word boundaries, any substring of the input might turn out to have actually intended to be e.g. a katakana spelling of a name. If CJK IMEs expected and re…

>because Japanese people pause between spoken words just like anyone else. This is (surprisingly) not true. People do not pause between words, however when listening to a language that they understand, they do perceive pauses between words; even though such pauses do not exist.

[deleted]

Re: Show HN: Neural Japanese Transliteration

#23

Post by brobinson »

schoen wrote: post #17
blipmusic wrote: post #9

Earlier quoted context omitted.

Well, to be honest my first impression was that this is what I've been using all along. I just liked that someone experimented a bit. :) Perhaps I assumed this would end up with a different predictive model. No idea what kind of corpora have been used for the input methods already available. Edit: your P.S. made me remember the "ma-ma-ma-ma..." mouthful the Chinese language students I studied in parallel with discove…

I don't speak Chinese but I've encountered this series a lot: https://en.wiktionary.org/wiki/%E5%AA%BD#Chinese 媽 mā 'mother' https://en.wiktionary.org/wiki/%E9%BA%BB#Chinese 麻 má 'hemp' (sometimes 'flax') https://en.wiktionary.org/wiki/%E9%A6%AC#Chinese 馬 mǎ 'horse' https://en.wiktionary.org/wiki/%E7%BD%B5#Chinese 罵 mà 'scold' It's also cool that you can see that 媽 is made up of "semantic 女 + phonetic 馬", where 女 mea…

There's one more:

https://en.wiktionary.org/wiki/%E5%97%8E#Chinese 嗎 ma 'question particle'

Re: Show HN: Neural Japanese Transliteration

#24

Post by brobinson »

blipmusic wrote: post #9
microcolonel wrote: post #6

Earlier quoted context omitted.

For what it's worth, this is basically just the same as any popular Japanese (or Chinese) input method. Usually the approach is to greedily form the smallest set of the longest words from the given syllables, because people tend to give inputs where all the words are complete. Sometimes people use markov models to fix situations where that falls over. Not sure how well this model performs, but the task is not novel.…

Well, to be honest my first impression was that this is what I've been using all along. I just liked that someone experimented a bit. :) Perhaps I assumed this would end up with a different predictive model. No idea what kind of corpora have been used for the input methods already available. Edit: your P.S. made me remember the "ma-ma-ma-ma..." mouthful the Chinese language students I studied in parallel with discove…

Was the phrase possibly this: 你敢罵我媽的馬嗎？

(roughly: "you dare to scold my mother's horse?")

Re: Show HN: Neural Japanese Transliteration

#25

Post by snakeboy »

yomly wrote: post #21
gizmo686 wrote: post #14

Earlier quoted context omitted.

>because Japanese people pause between spoken words just like anyone else. This is (surprisingly) not true. People do not pause between words, however when listening to a language that they understand, they do perceive pauses between words; even though such pauses do not exist.

I disagree. There's a subtle difference between breathing cadence and inflection, and a completely monotempo monotonal string of sounds.

It seems like it'd be useful for one or both of you to cite any research than has been done on this.

Seems more productive (and enlightening to all) than the agree/disagree dialogue here.

Re: Show HN: Neural Japanese Transliteration

#26

Post by [deleted] »

yomly wrote: post #21
gizmo686 wrote: post #14

Earlier quoted context omitted.

>because Japanese people pause between spoken words just like anyone else. This is (surprisingly) not true. People do not pause between words, however when listening to a language that they understand, they do perceive pauses between words; even though such pauses do not exist.

I disagree. There's a subtle difference between breathing cadence and inflection, and a completely monotempo monotonal string of sounds.

[deleted]

Re: Show HN: Neural Japanese Transliteration

#27

Post by rcthompson »

yomly wrote: post #21
gizmo686 wrote: post #14

Earlier quoted context omitted.

>because Japanese people pause between spoken words just like anyone else. This is (surprisingly) not true. People do not pause between words, however when listening to a language that they understand, they do perceive pauses between words; even though such pauses do not exist.

I disagree. There's a subtle difference between breathing cadence and inflection, and a completely monotempo monotonal string of sounds.

I think the claim being made is that the pause between consecutive syllables is (on average) the same whether they're part of the same word or not, so inter-syllable time is not a reliable indicator of word boundaries. Anecdotally, I think this is probably true of my speech most of the time (i.e. any time I'm not consciously enunciating). It might be less true for people with accents that have a slower cadence.

Re: Show HN: Neural Japanese Transliteration

#28

Post by blipmusic »

brobinson wrote: post #24
blipmusic wrote: post #9

Earlier quoted context omitted.

Well, to be honest my first impression was that this is what I've been using all along. I just liked that someone experimented a bit. :) Perhaps I assumed this would end up with a different predictive model. No idea what kind of corpora have been used for the input methods already available. Edit: your P.S. made me remember the "ma-ma-ma-ma..." mouthful the Chinese language students I studied in parallel with discove…

Was the phrase possibly this: 你敢罵我媽的馬嗎？ (roughly: "you dare to scold my mother's horse?")

"Scold" or something similar to it was in there, so quite possibly yes. Thanks, for the heads up.

Re: Show HN: Neural Japanese Transliteration

#29

Post by schoen »

brobinson wrote: post #23
schoen wrote: post #17

Earlier quoted context omitted.

I don't speak Chinese but I've encountered this series a lot: https://en.wiktionary.org/wiki/%E5%AA%BD#Chinese 媽 mā 'mother' https://en.wiktionary.org/wiki/%E9%BA%BB#Chinese 麻 má 'hemp' (sometimes 'flax') https://en.wiktionary.org/wiki/%E9%A6%AC#Chinese 馬 mǎ 'horse' https://en.wiktionary.org/wiki/%E7%BD%B5#Chinese 罵 mà 'scold' It's also cool that you can see that 媽 is made up of "semantic 女 + phonetic 馬", where 女 mea…

There's one more: https://en.wiktionary.org/wiki/%E5%97%8E#Chinese 嗎 ma 'question particle'

Oh yeah, thanks!

Apparently the phono-semantic derivation for that is "mouth ma" (maybe because a mouth is used to ask questions?).

Re: Show HN: Neural Japanese Transliteration

#30

Post by gizmo686 »

snakeboy wrote: post #25
yomly wrote: post #21

Earlier quoted context omitted.

I disagree. There's a subtle difference between breathing cadence and inflection, and a completely monotempo monotonal string of sounds.

It seems like it'd be useful for one or both of you to cite any research than has been done on this. Seems more productive (and enlightening to all) than the agree/disagree dialogue here.

Unfourtuantly, this is so well established that it is hard to find research. I did find this paper [1] which looks into how babies acuire word boundries. As you identify, there are probably phonetic cues; but not pauses.

The best way to see this is to try listening to a language you do not understand, and try to identify word boundries.

Indeed, the paper I link argues that some phonetic cue must exist because babies can recognise word boundries.

[1] https://www.ncbi.nlm.nih.gov/pubmed/8176060

Post reply on HN