Sparkey is a simple constant key/value storage library. It is mostly suited for read heavy systems with infrequent large bulk inserts.
Sparkey – Key/value storage by Spotify
github.com
Sparkey – Key/value storage by Spotify
1–10 of 57 posts
Re: Sparkey – Key/value storage by Spotify
#2I was surprised to see LevelDB (https://code.google.com/p/leveldb/) was missing from the list of storage solutions you tried, because it seems optimal for your use-case. Were you aware of it?
Re: Sparkey – Key/value storage by Spotify
#3I'm baffled by the choice of using the GNU autofools chain just to include a Doxygen target in the Makefile. The whole thing is essentially straight up C with just 1 library dependency.
The command line argument processing is also quite haphazardly done, it's not like it using getopt or whatever that poses compatibility issues. Is writing and packaging with a Makefile that difficult?
Re: Sparkey – Key/value storage by Spotify
#4I was surprised to see LevelDB ( https://code.google.com/p/leveldb/ ) was missing from the list of storage solutions you tried, because it seems optimal for your use-case. Were you aware of it?
The problem is, that so many projects for so many things exists. It is hard to find the matching project, thats why many people invent the wheele again.
Re: Sparkey – Key/value storage by Spotify
#5I'm baffled by the choice of using the GNU autofools chain just to include a Doxygen target in the Makefile. The whole thing is essentially straight up C with just 1 library dependency. The command line argument processing is also quite haphazardly done, it's not like it using getopt or whatever that poses compatibility issues. Is writing and packaging with a Makefile that difficult?
Submit a patch. Show them how it could be better.
Re: Sparkey – Key/value storage by Spotify
#6I was surprised to see LevelDB ( https://code.google.com/p/leveldb/ ) was missing from the list of storage solutions you tried, because it seems optimal for your use-case. Were you aware of it?
Why do you think it seems optimal for their use case?
A quick glance over LevelDB's features gives me the impression that its bulk-write performance would not be sufficient.
Re: Sparkey – Key/value storage by Spotify
#7I was surprised to see LevelDB ( https://code.google.com/p/leveldb/ ) was missing from the list of storage solutions you tried, because it seems optimal for your use-case. Were you aware of it?
I'm not sure about the optimal use-case match. Sparkey is for "mostly static" datasets where on disk structures are generated by a batch process and pushed to servers providing read only access to the data to consumers.
leveldb on the other hand, supports concurrent writes and provides features to handle data consistency and cheap gradual reindexing.
Re: Sparkey – Key/value storage by Spotify
#8I'm baffled by the choice of using the GNU autofools chain just to include a Doxygen target in the Makefile. The whole thing is essentially straight up C with just 1 library dependency. The command line argument processing is also quite haphazardly done, it's not like it using getopt or whatever that poses compatibility issues. Is writing and packaging with a Makefile that difficult?
We used to have a "simple" makefile, but once we started to support multiple development environments (OSX, various Linux flavors) it got more and more complex. Autotools actually brings a lot of functionality as part of the package and is what people expect. I'm no fan, but it works for our use case.
Re: Sparkey – Key/value storage by Spotify
#9I'm baffled by the choice of using the GNU autofools chain just to include a Doxygen target in the Makefile. The whole thing is essentially straight up C with just 1 library dependency. The command line argument processing is also quite haphazardly done, it's not like it using getopt or whatever that poses compatibility issues. Is writing and packaging with a Makefile that difficult?
They wrote a database to solve an operational need. From experience I can tell you that's an endeavour you should strife to spend as little time on as possible.
I think it's a miracle they produced something they feel comfortable sharing with the world. If you write a database in house, and the tool chain and the argument processing are the only things done haphazardly, then hats off to you :)
Re: Sparkey – Key/value storage by Spotify
#10I was surprised to see LevelDB ( https://code.google.com/p/leveldb/ ) was missing from the list of storage solutions you tried, because it seems optimal for your use-case. Were you aware of it?
According to GitHub, last commit (which is also initial checkin) is two years old.