Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Compressing Icelandic name declension patterns into a 3.27 kB trie

alexharri.com

Post reply on HN
Print view View original

71–80 of 86 posts

Re: Compressing Icelandic name declension patterns into a 3.27 kB trie

#71

Post by pelorat »

ralferoo wrote: post #56

I mean, it's an interesting problem for Icelandic sites, but because he's explaining the basic concepts of how declensions work, it seems like he's aiming this at non-Icelandic developers. If they were to use this, no doubt it'll end up butchering names in some other language and lead to all manner of hard to track down bugs. For example, if an English person called Arthur uses the site in Icelandic, I'm not sure the…

Not sure how it is nowadays, but Iceland used to force anyone immigrating to officialy change or "icelandify" their names.

So if your name was Arthur, and you wanted to emigrate to Iceland you would you change name.

Might still be like this.

Re: Compressing Icelandic name declension patterns into a 3.27 kB trie

#72

Post by Scaevolus »

lifthrasiir wrote: post #6

A possible alternative, especially for beygla/strict, would be perfect hashing.

You can compress even better than standard perfect hashing because not all values are unique, so collisions might be allow you to store multiple name -> suffix combos in the same bucket.

Of course, that would mean you lose the ability to say "name not handled".

Re: Compressing Icelandic name declension patterns into a 3.27 kB trie

#73

Post by xigoi »

jer0me wrote: post #66
esafak wrote: post #41

Earlier quoted context omitted.

I wonder if existing LLMs already know these patterns?

The Icelandic government has been proactive about helping OpenAI train its models on the language to stave off extinction: https://openai.com/index/government-of-iceland/

If they’d rather support open-source models so the future of the language is not in the hands of a single foreign corporation…

Re: Compressing Icelandic name declension patterns into a 3.27 kB trie

#74

Post by necovek »

charcircuit wrote: post #32
sneak wrote: post #25

Earlier quoted context omitted.

Software sold today does not require maintenance. Software to work in the future requires maintenance. I am not buying future software. I am buying today software. Increasingly I am not buying software at all.

Even ignoring security, bug fixes, new features, etc it is also not fair that you can get value from the app every month, but the developer doesn't get to capture a reward for any of this value. Having people pay monthly for value they get monthly seems reasonable.

That's not how markets work (and I disagree that it would be reasonable).

Price is usually established based on how much something cost to make (materials, effort, profit), combined with market conditions (abundance/shortage of products, surplus cash/tough economy...).

If you want to continuously extract profit from consistent use of a hammer or vacuum cleaner, somebody else will trivially make a competing product at a lower price with no subscription.

Re: Compressing Icelandic name declension patterns into a 3.27 kB trie

#75

Post by thaumasiotes »

esafak wrote: post #41
dmurray wrote: post #10

For the 800 names that were missing declension data in the database, it seems like the most straightforward thing to do would be to assign their declensions by hand. It shouldn't take a native speaker more than a couple of hours (if some name they haven't seen before is ambiguous, then whatever they guess at least won't sound obviously wrong to other native speakers). Alternatively, very cheap to ask an LLM to do it.…

I wonder if existing LLMs already know these patterns?

Yes, this is an example of a problem that an LLM is ideally suited to solve.

Re: Compressing Icelandic name declension patterns into a 3.27 kB trie

#76

Post by contravariant »

silvestrov wrote: post #43

One more optimization idea: instead of the trie mapping to the suffix string directly, then instead make an array of unique suffixes and let the trie map to the index into the array, e.g. const suffixes = [",,,", "a,u,u,u", ",,i,s", ",,,s", "i,a,a,a", ...]; and then use the index of this list in the var serializedInput = "{e:{n:{ein:0_r: ...

You could go a step further by putting the suffixes themselves into the trie and then identifying identical subtrees.

If you can use gzip there's bound to be a clever way of using a suffix array as well, that might end up being better unless you can use an optimised binary format for the tree.

Re: Compressing Icelandic name declension patterns into a 3.27 kB trie

#77

Post by adammarples »

jedimastert wrote: post #2

It's like an interview question from hell. Reversing a trie is those things that I might ever use once in my life, but that one time I will look like an absolute wizard.

I don't think they reversed the trie, they just reversed the names before putting them in?

Re: Compressing Icelandic name declension patterns into a 3.27 kB trie

#78

Post by charcircuit »

necovek wrote: post #74
charcircuit wrote: post #32

Earlier quoted context omitted.

Even ignoring security, bug fixes, new features, etc it is also not fair that you can get value from the app every month, but the developer doesn't get to capture a reward for any of this value. Having people pay monthly for value they get monthly seems reasonable.

That's not how markets work (and I disagree that it would be reasonable). Price is usually established based on how much something cost to make (materials, effort, profit), combined with market conditions (abundance/shortage of products, surplus cash/tough economy...). If you want to continuously extract profit from consistent use of a hammer or vacuum cleaner, somebody else will trivially make a competing product at…

>somebody else will trivially make a competing product at a lower price with no subscription.

And software like photoshop is not trivial to copy so it can survive being priced based off of value provided. There exists competitors that don't have a subscription, but they are not good enough to kill it.

Re: Compressing Icelandic name declension patterns into a 3.27 kB trie

#79

Post by wchar_t »

treetalker wrote: post #3

I remember that when I was first learning Spanish in high school, I found a piece of (Windows) software that pelted you with a series of pairs of an infinitive and a tense, and you had to conjugate the infinitive accordingly. (Spanish conjugation typically changes the end of the word; irregular verbs tend to involve stem changes). It was fantastic practice and really ingrained the rules; I became a whiz at it. When I…

Do you happen to still have a link to this software?

Re: Compressing Icelandic name declension patterns into a 3.27 kB trie

#80

Post by johannes1234321 »

3036e4 wrote: post #55
perching_aix wrote: post #26

Earlier quoted context omitted.

This is a good argument in favor of subscriptions not being mandatory, but not in favor of the abolishment of subscriptions overall, which is what they were talking about.

That is the old way. You bought some application and it came with upgrades until next major version release or similar. Then when that release came out you could decide to pay again or just keep using the old (now unsupported) version you already paid for. That solved all the issues with paying for maintenance, but sadly someone must have figured out a mandatory subscription was a better way to make more money.

It's not only a way to make more money, but it also matches better to modern development approaches.

Major versions come from a time where one had to produce physical media. Thus one could do a major release only every few years. Back then features had to be grouped together in a big bang release.

Nowadays one can ship features as they are being developed, with many small features changes all the time.

Post reply on HN