Reverse-Engineering Apple Dictionary (2020) • Hacker News Board

Re: Reverse-Engineering Apple Dictionary (2020)

#41

Post by peterburkimsher » Mon, Sep 13, 2021, 3:13 AM UTC

simondotau wrote: post #28

tasogare wrote: post #22

Earlier quoted context omitted.

Yes, dictionary content is the revenue source of dictionary vendors, so of course they don't want anyone to use it without permission. On the other hand there are more and more open-data projects (I started one myself), often based on printed dictionaries that felt in the public domain.

What’s the state of Wiktionary like in your opinion?

I use Wiktionary pretty often, and it has come in particularly useful this past week!

We're translating some strings on our software user interface, and checking the abbreviations and acronyms used. Sometimes there are amusing or [nsfw] connotations in other languages! Thank you Wiktionary for warning us about abbreviating "low pressure" as "LP" in Taiwan.

https://en.wiktionary.org/wiki/LP#Noun_2

Re: Reverse-Engineering Apple Dictionary (2020)

#42

Post by tkgally » Mon, Sep 13, 2021, 3:13 AM UTC

tasogare wrote: post #39

tkgally wrote: post #37

Earlier quoted context omitted.

JEDict already existed, but it was just a glossary, i.e., Japanese words with English equivalents. But there aren’t many one-to-one correspondences in meaning between Japanese and English words, and such glossaries, while useful, can also be frustrating and misleading to users. My idea for jeKai was to create a dictionary with explanatory definitions, like those that appear in monolingual dictionaries. That turned ou…

Well source language word paired with one or more translations is the minimal structure for a dictionary and some printed dictionaries are indeed like this. I browsed some entries of your dictionary and indeed their are (sometimes quite elaborate) explanations. I could easily why it ran out of steam, especially even a basic compiling is a daunting task. Also contributors are fews. On the Jibiki.fr (Japanese-French) p…

Thank you. I tried to e-mail you at the address on your profile page to suggest that we continue this discussion privately, but Gmail replied with “Your message wasn't delivered to [that address] because the address couldn't be found, or is unable to receive mail.” If you would like to chat about these issues further, please e-mail me at the address on my paper or at my personal website.

Re: Reverse-Engineering Apple Dictionary (2020)

#43

Post by kccqzy » Mon, Sep 13, 2021, 5:00 AM UTC

btn wrote: post #13

Another approach for this is to explore the format through Apple's tools for building dictionaries – as they provide a "Dictionary Development Kit" in Xcode's downloadable "Additional Tools" package (which has documentation for the XML format and a bunch of scripts/binaries for building the bundle). I wound up doing this a while ago for a similar toy project. After some poking around, it turned out that dictionary bu…

I have memories of using the Dictionary Development Kit to create custom dictionaries (I remember creating one for medical jargon) about ten years ago. (At that time custom dictionaries are placed in ~/Library/Dictionaries, and system dictionaries in /System/Library/Dictionaries, not some obfuscated path like now.)

To find the Kit in question, simply Google "com.apple.TrieAccessMethod" and you should find it online.

Re: Reverse-Engineering Apple Dictionary (2020)

#44

Post by thombles » Mon, Sep 13, 2021, 5:23 AM UTC

tasogare wrote: post #34

simondotau wrote: post #28

Earlier quoted context omitted.

What’s the state of Wiktionary like in your opinion?

I don't really use it often as a user nor i my projects to have a definite opinion. There is some pairs of words (about 5K) in Sino-Vietnamese that came with their chu nom writing which was very helpful to one of project. Otherwise I think it lacks structure and can't be harvested automatically easily (I don't think Wikidata integrate it all, and that website is a non-starter for me). Also every language is structure…

> Otherwise I think it lacks structure and can't be harvested automatically easily

Indeed, it depends on the language and your goals - I had a very high success rate plucking out Russian grammatical tables from English Wiktionary with a few hours of scripting the data cleaning (https://github.com/thombles/declensions). I have a theory that you could get better results using an offline archive of the page sources but haven't tried this yet.

Re: Reverse-Engineering Apple Dictionary (2020)

#45

Post by hellothereworld » Mon, Sep 13, 2021, 6:06 AM UTC

I also found this Dictionary API which imports the dictionaries into NodeJs by utilizing a utility called „dedict“.

https://github.com/nikvdp/dictionary-api/blob/master/convert...

Re: Reverse-Engineering Apple Dictionary (2020)

#46

Post by atorodius » Mon, Sep 13, 2021, 6:12 AM UTC

Author here, funny to see this popping up on HN :) Was definitly a fun ride making this.

Re: Reverse-Engineering Apple Dictionary (2020)

#47

Post by hyperstar » Mon, Sep 13, 2021, 6:30 AM UTC

At the moment, I'm using a slight modification of the gist plus `sed 's/]*>//g'` to look up words from the shell on Linux. It would be nice to have some XML parsing into plain text, but it kind of works.

Re: Reverse-Engineering Apple Dictionary (2020)

#48

Post by bumblebritches5 » Mon, Sep 13, 2021, 6:55 AM UTC

bhl wrote: post #30

Has anyone reversed-engineered the Apple emoji dictionary that maps some keywords to emojis? Last time I checked, they only shipped binaries on the newest MacOS. Would love to use that mapping to elevate search on my custom emoji picker.

That data is actually present in the UCD already.

Re: Reverse-Engineering Apple Dictionary (2020)

#49

Post by cprecioso » Mon, Sep 13, 2021, 7:13 AM UTC

bhl wrote: post #30

Has anyone reversed-engineered the Apple emoji dictionary that maps some keywords to emojis? Last time I checked, they only shipped binaries on the newest MacOS. Would love to use that mapping to elevate search on my custom emoji picker.

I think it’s just the name data from the Unicode standard? (possibly the CLDR data but I might be wrong)

Re: Reverse-Engineering Apple Dictionary (2020)

#50

Post by Someone » Mon, Sep 13, 2021, 7:30 AM UTC

tasogare wrote: post #34

simondotau wrote: post #28

Earlier quoted context omitted.

What’s the state of Wiktionary like in your opinion?

I don't really use it often as a user nor i my projects to have a definite opinion. There is some pairs of words (about 5K) in Sino-Vietnamese that came with their chu nom writing which was very helpful to one of project. Otherwise I think it lacks structure and can't be harvested automatically easily (I don't think Wikidata integrate it all, and that website is a non-starter for me). Also every language is structure…

https://en.wiktionary.org/wiki/Help:FAQ:

Q: Is it possible to download Wiktionary?

A: Yes. https://dumps.wikimedia.org/enwiktionary/ should have the latest copy of the main namespace. The cleanest navigation page is https://dumps.wikimedia.org/. Just download a -articles.xml.bz2 file and some software to read it (for nix, for Windows).

Q: Can I use data from Wiktionary in my program?

A: As long as you meet the conditions of the GNU Free Documentation License or Creative Commons Attribution/Share-Alike License, certainly.

Latest dump for English is from September 1. I wouldn’t know whether it has all the data or how easy it is to parse it.