Also, as pointed out in another comment, it is not necessarily balanced (like red-black trees) and there are some real-world datasets where crit-bit (or Patricia) trees behave poorly, for example URLs. I myself have seen Patricia trees with height greater than 100, for sets of URLs where log n ~ 30. According to VTune, most time in searches was spent in cache misses.
Crit-bit Trees: the best way to store sets
21–30 of 42 posts
Re: Crit-bit Trees: the best way to store sets
#22Still, it's a cool data structure, and I respect djb enough to trust that it's at least competitive with hash tables; I'll consider it when I want more operations than offered by a hash table (especially for a set that's very sparse compared to key length).
Re: Crit-bit Trees: the best way to store sets
#23"I have become convinced that this strategy should change. The revised strategy is much simpler: there should be one fundamental set-storage type, namely a crit-bit tree. Here's how a crit-bit tree stacks up against the competition:
* A hash table supports insertion, deletion, and exact searches. A crit-bit tree supports insertion, deletion, exact searches, and ordered operations such as finding the minimum. Another advantage is that a crit-bit tree guarantees good performance: it doesn't have any tricky slowdowns for unusual (or malicious) data."
Oh, what's the big-O of a lookup in a tree vs lookup in a hash-table again? O(lg N) vs O(1) with bad locality of reference to boot, you say?
FAIL.
Re: Crit-bit Trees: the best way to store sets
#24If you don't need the add and remove operations, it's possible to reduce the size significantly by applying automata theory: http://www.n3labs.com/pdf/lexicon-squeeze.pdf It works extremely well "out of the box" for dictionary words, since the automata enables sharing of both prefixes and suffixes.
I only skimmed the paper, but a simple n*log n approach to produce the same result is to first sort the lexicon, then build the trie while canonicalizing the suffix sets depth first (needing to hash/compare only one level deep, by induction).
This guy wrote an entire thesis comparing various methods for the same task - http://www.eti.pg.gda.pl/katedry/kiw/pracownicy/Jan.Daciuk/p...
Re: Crit-bit Trees: the best way to store sets
#25> A crit-bit tree for a nonempty prefix-free set S of bit strings has an external node for each string in S; an internal node for each bit string x such that x0 and x1 are prefixes of strings in S; and ancestors defined by prefixes. And that's a perfect (missing) example for "a picture is worth a thousand words". Then again... this coming from DJB - I should be surprised the description is as verbose as it is.
I don't know if it helps, but here's the picture I drew at the top of my patricia trie code. (Crit-bit trees aren't the same as patricia tries, but they're close enough for expository purposes.)
/**
* Our Patricia tree structure can be thought of as operating on strings of
* 9-bit bytes, where 0x00 -- 0xFF are mapped to 0x100 -- 0x1FF and 0x00
* represents the end-of-string character (note that NUL can occur inside
* keys). The field (struct pnode).mask is either 0 or a power of 2; if 0,
* the left child, if non-NULL, is a pointer to the record associated with
* the key thus far. For example, the strings "hello", "hello colin",
* "hello world", and "wednesday", each associated with pointers to
* themselves, are stored in the following tree:
*
* [0x10, 0x60, 0, ""]
* | |
* [0x00, 0x00, 5, "hello"] [0x00, 0x00, 9, "wednesday"]
* | | | |
* "hello" [0x10, 0x60, 1, " "] "wednesday" NULL
* | |
* [0x00, 0x00, 5, "colin"] [0x00, 0x00, 5, "world"]
* | | | |
* "hello colin" NULL "hello world" NULL
*
*/
Edited to add: And here's the node structure: /* Structure used to store a Patricia tree node. */
struct pnode {
struct pnode * left; /* Left child. */
struct pnode * right; /* Right child. */
uint8_t mask; /* Critical bit mask. */
uint8_t high; /* High bits of this node. */
uint8_t slen; /* Length of s[]. */
uint8_t s[]; /* Bytes since parent's s[]. */
};Re: Crit-bit Trees: the best way to store sets
#26The article puts forward a tree as a better alternative to hashtables: "I have become convinced that this strategy should change. The revised strategy is much simpler: there should be one fundamental set-storage type, namely a crit-bit tree. Here's how a crit-bit tree stacks up against the competition: * A hash table supports insertion, deletion, and exact searches. A crit-bit tree supports insertion, deletion, exact…
When you write an application, you know the performance requirements of the set representation you choose & can pick the most appropriate for your application. A language designer doesn't have this information: their choice of fundamental datatype affects everyone.
Bernstein is arguing that they should choose a set representation with good performance for as wide a range of features as possible rather than one that performs well for a small set of requirements, and very poorly for others.
I can see arguments both ways here, but FAIL is a bit too strong IMO.
Re: Crit-bit Trees: the best way to store sets
#27Re: Crit-bit Trees: the best way to store sets
#28I'm not sure it's the best way to store sets. It's seems like potentially doing a pointer dereference per bit is quite an overhead. It would be 8 times faster (for dense sets) to have a crit-byte tree. Though then obviously if you did that naively, there would be a huge associated storage cost. To mitigate that, you can do what AMTs do and have a bitmap per node followed by an array of pointers.
Crit-bit trees don't branch per bit, they branch per differing bit. Much more efficient for most data sets.
Re: Crit-bit Trees: the best way to store sets
#29> A crit-bit tree for a nonempty prefix-free set S of bit strings has an external node for each string in S; an internal node for each bit string x such that x0 and x1 are prefixes of strings in S; and ancestors defined by prefixes. And that's a perfect (missing) example for "a picture is worth a thousand words". Then again... this coming from DJB - I should be surprised the description is as verbose as it is.
The particular words you've quoted have managed to condense a thousand pictures — an infinite number, really — into 45 words. It's kind of a counterexample. Some examples would certainly help, and probably diagrams would help understanding the examples.
It's 45 words with lots of assumed knowledge that you could write books about ;)
Re: Crit-bit Trees: the best way to store sets
#30If you don't need the add and remove operations, it's possible to reduce the size significantly by applying automata theory: http://www.n3labs.com/pdf/lexicon-squeeze.pdf It works extremely well "out of the box" for dictionary words, since the automata enables sharing of both prefixes and suffixes.
The minimal-acceptor approach is also best when you're considering dynamic programs that effectively involve some kind of FSA-intersection, including best-first lazy versions like "find the closest spelling correction to a sequence of words in your lexicon of some text", because your state (suffix set) representation is canonical. I only skimmed the paper, but a simple n*log n approach to produce the same result is t…
The data structure for the mealy recognizer described in the paper can very easily be changed to support minimal perfect hash numbers for all elements, without using more space (my implementation uses less space because of it, because the same information can be used to calculate the number of strings with a prefix in O(1) time).