Is this a plist?
Reverse-Engineering Apple Dictionary (2020)
31–40 of 61 posts
Re: Reverse-Engineering Apple Dictionary (2020)
#32I assume one reason Apple has made it more challenging to extract the dictionary resources is in order to satisfy licensing constraints with the dictionary authors. I wonder if they'd block an app like this through the App Store submission process, if submitted.
I’d definitely assume this would be a copyright problem if used in an app.
I have a vague recollection of reading somewhere that you're explicitly forbidden from creating a dictionary app using the OS dictionaries. You can do dictionary lookup within apps (so, for example, you're free to use the dictionary to look up definitions in your word processor or ePub app, but not to have an app which lets a user enter a word and get a definition back).
Re: Reverse-Engineering Apple Dictionary (2020)
#33Thank you for posting this code on Github! There has been some reverse-engineering done on the language dictionaries bundled with Mac OS, and it's nice to know that the same model is being used on the Apple Watch! I look forward to seeing your dictionary app. https://josephg.com/blog/reverse-engineering-apple-dictionar... There's also a command-line tool that can query the dictionary: https://github.com/takumakei/osx…
>Apple's word segmentation Unless they changed it, it's probably similar to CFStringTokenizer which used ICU Boundary Analysis (and maybe mecab for Japanese).
Thank you! The ICU Boundary Analysis documentation says it uses a dictionary to split Chinese, Japanese, Thai or Khmer.
https://unicode-org.github.io/icu/userguide/boundaryanalysis...
Is that the same as the macOS dictionary being parsed here? It seems like a pretty big file to grep every time!
Re: Reverse-Engineering Apple Dictionary (2020)
#34Earlier quoted context omitted.
Yes, dictionary content is the revenue source of dictionary vendors, so of course they don't want anyone to use it without permission. On the other hand there are more and more open-data projects (I started one myself), often based on printed dictionaries that felt in the public domain.
What’s the state of Wiktionary like in your opinion?
I don't really use it often as a user nor i my projects to have a definite opinion. There is some pairs of words (about 5K) in Sino-Vietnamese that came with their chu nom writing which was very helpful to one of project. Otherwise I think it lacks structure and can't be harvested automatically easily (I don't think Wikidata integrate it all, and that website is a non-starter for me). Also every language is structured differently so Wiktionary can hardly be commented as a whole.
Re: Reverse-Engineering Apple Dictionary (2020)
#35Earlier quoted context omitted.
Yes, dictionary content is the revenue source of dictionary vendors, so of course they don't want anyone to use it without permission. On the other hand there are more and more open-data projects (I started one myself), often based on printed dictionaries that felt in the public domain.
I started one myself, too, more than twenty years ago, in a naive burst of enthusiasm about the potential of online collaboration. It was an attempt to create a new comprehensive Japanese-English dictionary from scratch, not based on existing dictionaries [1]. Other volunteers were equally enthusiastic, but the immensity of the task before us, and the competing time pressures of paying work, caused us to gradually st…
Did you start it before JEDict from Breen started existing? It's great for English language for other Japanese-* pairs are poorly endowed. There is also focused work on neologisms that could be done with a few people and could really improve existing free solutions.
I'm personally using the route of digitizing an existing dictionary, albeit manually because that can't be automated in this case.
Re: Reverse-Engineering Apple Dictionary (2020)
#36Has anyone reversed-engineered the Apple emoji dictionary that maps some keywords to emojis? Last time I checked, they only shipped binaries on the newest MacOS. Would love to use that mapping to elevate search on my custom emoji picker.
you may try assign text replacements for emojis:
https://www.iphonehacks.com/2016/04/text-replacement-mac.htm...
Re: Reverse-Engineering Apple Dictionary (2020)
#37Earlier quoted context omitted.
I started one myself, too, more than twenty years ago, in a naive burst of enthusiasm about the potential of online collaboration. It was an attempt to create a new comprehensive Japanese-English dictionary from scratch, not based on existing dictionaries [1]. Other volunteers were equally enthusiastic, but the immensity of the task before us, and the competing time pressures of paying work, caused us to gradually st…
Did you start it before JEDict from Breen started existing? It's great for English language for other Japanese-* pairs are poorly endowed. There is also focused work on neologisms that could be done with a few people and could really improve existing free solutions. I'm personally using the route of digitizing an existing dictionary, albeit manually because that can't be automated in this case.
JEDict already existed, but it was just a glossary, i.e., Japanese words with English equivalents. But there aren’t many one-to-one correspondences in meaning between Japanese and English words, and such glossaries, while useful, can also be frustrating and misleading to users. My idea for jeKai was to create a dictionary with explanatory definitions, like those that appear in monolingual dictionaries. That turned out to be much more work than we were ready to do, though.
A paper I wrote nine years ago on related issues is here, in case you are interested:
“Kokugo Dictionaries as Tools for Learners: Problems and Potential”
https://researchmap.jp/multidatabases/multidatabase_contents...
Re: Reverse-Engineering Apple Dictionary (2020)
#38Earlier quoted context omitted.
Yes, dictionary content is the revenue source of dictionary vendors, so of course they don't want anyone to use it without permission. On the other hand there are more and more open-data projects (I started one myself), often based on printed dictionaries that felt in the public domain.
What’s the state of Wiktionary like in your opinion?
I hadn’t used Wiktionary for a few years, so I just spent some time looking through it. It was pretty good a few years ago, and now it looks even better. I’m sure many people find it very useful. The amount of information on each page, though, might make it a bit intimidating to some users.
It also seems to have some unevenness in coverage. For example, the entry for the word “anecdata” (a word discussed recently at [1]), has five illustrative quotations, which are quite handy [2]. The entry for the more mundane “anecdote,” however, has none [3]. Such unevenness might be inevitable in volunteer dictionary projects, as volunteers like to work on the more interesting words.
[1] https://news.ycombinator.com/item?id=28375767
Re: Reverse-Engineering Apple Dictionary (2020)
#39Earlier quoted context omitted.
Did you start it before JEDict from Breen started existing? It's great for English language for other Japanese-* pairs are poorly endowed. There is also focused work on neologisms that could be done with a few people and could really improve existing free solutions. I'm personally using the route of digitizing an existing dictionary, albeit manually because that can't be automated in this case.
JEDict already existed, but it was just a glossary, i.e., Japanese words with English equivalents. But there aren’t many one-to-one correspondences in meaning between Japanese and English words, and such glossaries, while useful, can also be frustrating and misleading to users. My idea for jeKai was to create a dictionary with explanatory definitions, like those that appear in monolingual dictionaries. That turned ou…
Well source language word paired with one or more translations is the minimal structure for a dictionary and some printed dictionaries are indeed like this.
I browsed some entries of your dictionary and indeed their are (sometimes quite elaborate) explanations. I could easily why it ran out of steam, especially even a basic compiling is a daunting task. Also contributors are fews. On the Jibiki.fr (Japanese-French) project, most of the corrections are made by two members. They're starting from an existing dictionary and it still took years to just check the headwords.
Thanks very much for the paper. I'll read it. I actually already had one of your paper on my machine (Asialex 2011 proceedings) but haven't read it yet.
Re: Reverse-Engineering Apple Dictionary (2020)
#40Earlier quoted context omitted.
>Apple's word segmentation Unless they changed it, it's probably similar to CFStringTokenizer which used ICU Boundary Analysis (and maybe mecab for Japanese).
Thank you! The ICU Boundary Analysis documentation says it uses a dictionary to split Chinese, Japanese, Thai or Khmer. https://unicode-org.github.io/icu/userguide/boundaryanalysis... Is that the same as the macOS dictionary being parsed here? It seems like a pretty big file to grep every time!
No, the ICU dictionaries are seen at: https://github.com/unicode-org/icu/tree/main/icu4c/source/da...
I assume at compile time it's converted to a more efficient query format