No random limits: cdb can handle any database up to 4 gigabytes.
Am I the only one who finds it hilarious that the first thing after the "no random limits" heading is a random limitation?
Cdb: a fast, reliable, simple package for creating, reading constant databases
31–40 of 43 posts
Re: Cdb: a fast, reliable, simple package for creating, reading constant databases
#32How does this compare to LMDB (http://symas.com/mdb/)?
Re: Cdb: a fast, reliable, simple package for creating, reading constant databases
#33See also tinycdb, a public domain reimplementation http://www.corpit.ru/mjt/tinycdb.html
The original code is also in the public domain, since 2009: http://cr.yp.to/distributors.html
Re: Cdb: a fast, reliable, simple package for creating, reading constant databases
#34CDB is one of my favorite data structures. When a student wants to learn about databases, I get them to implement cdb. It's easy to implement and really demonstrates some good system engineering tradeoffs.
Any links to theory behind cdb?
Here's an articles I came across that explains the structure:
http://www.unixuser.org/~euske/doc/cdbinternals/
I think the theory is mostly that it's really simple and ends up requiring very few disk reads.
Re: Cdb: a fast, reliable, simple package for creating, reading constant databases
#35No random limits: cdb can handle any database up to 4 gigabytes. Am I the only one who finds it hilarious that the first thing after the "no random limits" heading is a random limitation?
When you have 32-bit pointers, 2^32 isn't a random limit.
Re: Cdb: a fast, reliable, simple package for creating, reading constant databases
#36CDB is awesome for its use case - slow changing, read-heavy workflows that are tolerant of stale data. One limitation of the original implementation is the use of 32-bit keys for addressing, which limit the addressable size to 4gb. There are 64 bit modifications, but I have not used them. Does anyone have an opinion on any of the 64-bit implementations?
Supposedly https://github.com/gstrauss/mcdb is quite good.
This here.....Glenn knows his stuff and the testing used by mcdb is great.
Re: Cdb: a fast, reliable, simple package for creating, reading constant databases
#37CDB is awesome for its use case - slow changing, read-heavy workflows that are tolerant of stale data. One limitation of the original implementation is the use of 32-bit keys for addressing, which limit the addressable size to 4gb. There are 64 bit modifications, but I have not used them. Does anyone have an opinion on any of the 64-bit implementations?
If you're looking for a very straightforward 64-bit port, [I'm the author of] https://github.com/pcarrier/cdb64
Re: Cdb: a fast, reliable, simple package for creating, reading constant databases
#38What advantages does this have over SQLite?
It's extremely fast and it's easy to work with for reading, but it's just a key-value store. Also, it's sort of weird to work with on the write side. You have to do atomic replaces of the read-only data files.
I don't know how many people remember this, but this kind of "weird atomic replaces of the read-only data file" was very common on unix-systems for a long time.
Take sendmail: people would have their email-aliases in a /etc/aliases file, with a simple ascii-format, lines formatted according to: ": ". The "newaliases" command then would convert the content of the ASCII file /etc/aliases into the binary (originally: Berkeley db) /etc/aliases.db.
As far as I know, CDB was developed initially for qmail, DJBs email-daemon, for exactly that purpose.
Another example for this is the "Yellow-Pages" (yp) / NIS directory services to distribute usernames/userids/groups/... in a UNIX cluster or network. ASCII files were converted into e.g. /var/yp/DOMAIN/passwd.byname, a .db-file where keys are the usernames from /etc/passwd and values were the full ASCII lines from /etc/passwd, same goes for passwd.byuid, group.byname, and so on. All for fast indicing, so that the network server didn't have to repeatedly parse /etc/passwd.
Re: Cdb: a fast, reliable, simple package for creating, reading constant databases
#39There is an also sdb - simple string based key-value database with support for arrays and json, and based on CDB https://github.com/radare/sdb
Re: Cdb: a fast, reliable, simple package for creating, reading constant databases
#40That's cool. But I feel like I'll totally forget about it and lose reference to this (in case I have future interest). Where's a github mirror? A google search reveals some entries from the language implementations. Go: https://github.com/jbarham/go-cdb Java: https://github.com/malyn/sg-cdb Haskell: https://github.com/adamsmasher/hs-cdb
Most open source development takes place outside of github. It is a very valley-centric thing. You'll find a ton of life changing stuff on SourceForge and random FTP sites.
That's not entirely fair. Github is not a valley-centric thing, it is used by a large proportion of projects started since 2008 or so.