A simple information-theoretic argument suggests that most of the information is in the remaining article 10%.
The most frequent 777 characters give 90% coverage of Kanji in the wild
171–180 of 214 posts
Re: The most frequent 777 characters give 90% coverage of Kanji in the wild
#172Disclaimer: native Chinese speaker, knows some Japanese, English sufferer Putting aside the argument of whether removing all Hanzi from Japanese text would actually be more efficient or not, the question to me is: why stop at Hanzi? Why not romanizating all the Japanese literature? Surely almost all the reasoning in favor of getting rid of Hanzi can also apply here? edit: grammar
Kana are more directly phonemic than any romanization method, and not difficult for any foreigner to learn. If we want a single universal alphabet then we should be looking to something more like IPA - or at least something that would involve a radically simplified orthography for english. But really the details of which alphabet one chooses are much less important than the decision to use an alphabet/syllabary at all rather than an ideographic system.
Re: The most frequent 777 characters give 90% coverage of Kanji in the wild
#173A simple information-theoretic argument suggests that most of the information is in the remaining article 10%.
Not enough information. For example, in English some of the least common characters are also generally not important for conveying information.
Your argument only holds if Kanji was designed to be optimal in compressing information, which is of course not how Kanji came to be.
That doesn't mean you are wrong, but your argument is invalid.
Re: The most frequent 777 characters give 90% coverage of Kanji in the wild
#174The thing is, 90% coverage is not that great. What happens is that you understand common words that make up for a lot of structure, but when an uncommon word appears it's probably important to the sentence. For example, "son, if you go to the plumbf tomorrow morning don't forget to pick up some zlonks." 98% is closer to what you need in order to read a text and have an idea of what's going on. See this article: https…
This. I peaked at about 1000 kanji in active memory. It was still not enough to read newspapers easily. Joyo kanji ("regular use Chinese characters") have about 2100 characters, which any high school grad should be able to read. Anything much below that level and you are guessing/looking up a lot of content.
You are better off trying to cover specific domains. I was able to read technical articles related to hardware and software topics pretty easily.
Re: The most frequent 777 characters give 90% coverage of Kanji in the wild
#175A few years ago I downloaded several hundreds of megabytes of Japanese subtitles, split into 3 categories: live action/drama, anime and foreign film/tv I’ve listed them in a google sheets together with a few other corpora https://docs.google.com/spreadsheets/d/1yb5dq4ahdwc_g0aQTL3Y... Choose the jimaku tab for subtitles to see how big the variation between corpus can be. According to other comments here, it appears t…
The source links appear to no longer work. Do you know where we can download Japanese subtitles?
I would love to attempt to segment a bunch of Japanese subtitles into words and then do frequency analysis. My interest is in increasing my listening ability, so I want to put the most frequently spoken words into SRS/Anki, and perhaps even break it down by anime.
Alternatively, has anyone already done this?
Re: The most frequent 777 characters give 90% coverage of Kanji in the wild
#176I thought the number of standard kanji was only about 1900 to begin with.
2136. But the list isn’t at all comprehensive. There are a considerable number of kanji in regular use that aren’t on the list, and when you include place/person names, it grows massively. It’s possible to memorize all the standard kanji, but crack open a history book and you won’t recognize half of the words.
Haha, it was 1,945 when I was studying Japanese, times change. Kind of surprised they added characters rather than further shrink the list.
Re: The most frequent 777 characters give 90% coverage of Kanji in the wild
#177The thing is, 90% coverage is not that great. What happens is that you understand common words that make up for a lot of structure, but when an uncommon word appears it's probably important to the sentence. For example, "son, if you go to the plumbf tomorrow morning don't forget to pick up some zlonks." 98% is closer to what you need in order to read a text and have an idea of what's going on. See this article: https…
> son, if you go to the plumbf tomorrow morning don't forget to pick up some zlonks As a counterpoint: “son, if you go to the jawn tomorrow morning, don’t forget to pick up some jawn” https://www.atlasobscura.com/articles/the-enduring-mystery-o...
In fact, we also have another word that serves both as a multipurpose substantive and an multipurpose adjective:
"¿Yo dónde dejé esa chimba?"
"Está muy chimba su carro"
"Esa chimba estaba muy chimba"
Re: The most frequent 777 characters give 90% coverage of Kanji in the wild
#178Wouldn't 777 words in any language give 90% coverage of all the words in a typical newspaper?
One difference here is that Japanese characters can be combined in various ways to create different words, so the 777 characters can be used to create many more than 777 words. Compare with the Simple English Wikipedia which strives to use only the 1000 most commonly used English words. I think you'll find that that experience quite a bit different from reading a typical English language newspaper.
https://simple.wikipedia.org/wiki/Wikipedia:Simple_English_W...
Another example of "using only the ten hundred words people use most often" is Randall Munroe's "Up Goer Five":
It may be helpful to think of "characters" as representing some middle ground between words and alphabetic letters, a little like word stems.
Re: The most frequent 777 characters give 90% coverage of Kanji in the wild
#179If Japanese people just used kana (like Korean people use hangul today), kids in Japan don't have to spend countless hours learning this complex (and often irrational) writing system. http://kanamozi.org/hikari959-04.html > If you really want to be native-level Japanese, kanji are essential There are visually impaired people who have difficulty learning kanji but speak Japanese fluently. Language is not just for peop…
Here we have a native speaker of Japanese being downvoted for giving an opinion on their own writing system that HN readers don't want them to have.
HN is full of the dilettante type who consider themselves an authority on any field based on a few days of wiki-hopping.
I notice this every time I see comments on HN about a topic I actually understand in depth. Quite a few comments are just outright incorrect, most are misguided, but all are confident.
Re: The most frequent 777 characters give 90% coverage of Kanji in the wild
#180Earlier quoted context omitted.
Here we have a native speaker of Japanese being downvoted for giving an opinion on their own writing system that HN readers don't want them to have.
HN is full of the dilettante type who consider themselves an authority on any field based on a few days of wiki-hopping. I notice this every time I see comments on HN about a topic I actually understand in depth. Quite a few comments are just outright incorrect, most are misguided, but all are confident.
Just because someone posts a comment with their understanding of an issue doesn't mean they consider themselves an authority.