Pure Go implementation of D. J. Bernstein's cdb constant database library
1–10 of 21 posts
Re: Pure Go implementation of D. J. Bernstein's cdb constant database library
#2Okay, semi-related:
Why is the speed of the Go compiler so important? Why not just use an incremental compiler? Why the hell would you want to recompile a 100k+ line program when there are known better alternatives?
Just doesn't make sense to me.
Re: Pure Go implementation of D. J. Bernstein's cdb constant database library
#3What are people using this 'cdb' for, irl? Sounds interesting but this is the first i've heard of it.
Re: Pure Go implementation of D. J. Bernstein's cdb constant database library
#4Okay, semi-related: Why is the speed of the Go compiler so important? Why not just use an incremental compiler? Why the hell would you want to recompile a 100k+ line program when there are known better alternatives? Just doesn't make sense to me.
When your compiler is doing interprocedural optimizations like inlining (which Go's does), then changing an upstream dependency generally requires that all downstream dependencies be recompiled as well. So incremental recompilation isn't a panacea, and I think the Go designers made the right choice in striving to make compilation fast.
Of course, you can do something like incremental compilation only at -O0 with no inlining, which is what I suspect we'll end up doing in Rust (the relevant bug is [1]).
Re: Pure Go implementation of D. J. Bernstein's cdb constant database library
#5What are people using this 'cdb' for, irl? Sounds interesting but this is the first i've heard of it.
DJB's qmail (the security-hardened sendmail replacement) uses it.
Re: Pure Go implementation of D. J. Bernstein's cdb constant database library
#6Okay, semi-related: Why is the speed of the Go compiler so important? Why not just use an incremental compiler? Why the hell would you want to recompile a 100k+ line program when there are known better alternatives? Just doesn't make sense to me.
When your compiler is doing interprocedural optimizations like inlining (which Go's does), then changing an upstream dependency generally requires that all downstream dependencies be recompiled as well. So incremental recompilation isn't a panacea, and I think the Go designers made the right choice in striving to make compilation fast. Of course, you can do something like incremental compilation only at -O0 with no i…
[deleted]
Re: Pure Go implementation of D. J. Bernstein's cdb constant database library
#7What are people using this 'cdb' for, irl? Sounds interesting but this is the first i've heard of it.
It's a persistent data structure optimized for static lookup tables. Bernstein's qmail and djbdns use it for mail routing and DNS zone data, respectively.
Re: Pure Go implementation of D. J. Bernstein's cdb constant database library
#8What are people using this 'cdb' for, irl? Sounds interesting but this is the first i've heard of it.
It's useful if you have data that is easy to cache (i.e., rebuilt every 6 hours) but very commonly accessed. Because the lookups are so quick (two seeks) it's almost raw disk speed. But yeah, rebuilding the files is an offline process (build new file and swap it in using a rename), so your data has to be cache-friendly.
It's a good alternative to memcache if your data is larger than what memcached can support in RAM.
In the early 2000s I used it to implement most of the frontend for a PPC marketplace for search engines. Held up well. These days I'd just use memcached or redis.
Re: Pure Go implementation of D. J. Bernstein's cdb constant database library
#9What are people using this 'cdb' for, irl? Sounds interesting but this is the first i've heard of it.
It's used extensively in djb's packages like qmail and ucspi-tcp. In the latter, it's used to maintain a database of blacklisted IPs which are blocked from connecting to a TCP server, among other things. http://cr.yp.to/cdb.html
Re: Pure Go implementation of D. J. Bernstein's cdb constant database library
#10Okay, semi-related: Why is the speed of the Go compiler so important? Why not just use an incremental compiler? Why the hell would you want to recompile a 100k+ line program when there are known better alternatives? Just doesn't make sense to me.
"The Go compiler isn't so fast; other compilers are just slow. Machines are bloody fast. Just don't piss the speed away." -Rob Pike http://twitter.com/rob_pike/status/199620997459087360
Why not do both? I appreciate that Go respects my time.