Filedb: Disk-based key-value store inspired by Bitcask
1–10 of 16 posts
Re: Filedb: Disk-based key-value store inspired by Bitcask
#2This looks interesting. Maybe I'm not in-the-know, but why would you offload such important aspects like `sync` to the client instead of building in some protocol to ensure that file integrity is maintained? With this kind of design choice, it seems quite easy to lose data, unless I'm missing something.
Re: Filedb: Disk-based key-value store inspired by Bitcask
#3Bitcask, now there's a blast from the Basho past. It always bugged me that no good secondary indexing strategy was built to make using Bitcask viable for more use cases. Everyone always wanted to use the LevelDB backend just to get at secondary indexing features (which also performance scaled inversely relative to cluster size, which was it's own problem). But having Riak exhibit consistent, high-performance was waaaaaaaay easier on Bitcask.
Re: Filedb: Disk-based key-value store inspired by Bitcask
#4This looks interesting. Maybe I'm not in-the-know, but why would you offload such important aspects like `sync` to the client instead of building in some protocol to ensure that file integrity is maintained? With this kind of design choice, it seems quite easy to lose data, unless I'm missing something.
From the README:
A sync process syncs the open disk files once every config.syncInterval. Sync also can be done on every request if config.alwaysFsync is True.
Re: Filedb: Disk-based key-value store inspired by Bitcask
#5[deleted]
Re: Filedb: Disk-based key-value store inspired by Bitcask
#6This is something that sometimes i play with:
https://github.com/lsferreira42/nadb
It is a disk based KV store with tags for search
Re: Filedb: Disk-based key-value store inspired by Bitcask
#7Sorry, maybe I am not in the mood of delving too deep into the project(but I starred it! Amazing job I suppose) and I don't want to ask AI but rather some experts who are surely lurking HN.
Can you guys please explain this to me like I am 5(or maybe 10)? Is this something revolutionary to keep in back of the mind? How does it compare to redis? When should I use it, if any. I always prefer sqlite, then postgresql if scalability and afterwards I am not sure but maybe things like clickhouse. I am also looking more into duckdb but maybe not as a primary database, but rather just in fun. There are also things like turso and cloudflare d1 (if I remember correctly), kinda prefer cloudflare d1 but also like turso or sqlite in general. Still, the database space really piques my interest.
Thanks in advance for helping this young fellow out!
Re: Filedb: Disk-based key-value store inspired by Bitcask
#8Sorry, maybe I am not in the mood of delving too deep into the project(but I starred it! Amazing job I suppose) and I don't want to ask AI but rather some experts who are surely lurking HN. Can you guys please explain this to me like I am 5(or maybe 10)? Is this something revolutionary to keep in back of the mind? How does it compare to redis? When should I use it, if any. I always prefer sqlite, then postgresql if s…
Implementing Bitcask is sorta like a right of passage for people interested in DBs/storage engines. You shouldn't use this in production. SQLite is most likely more flexible, reliable, and ubiquitous for situations where this project would be useful.
Re: Filedb: Disk-based key-value store inspired by Bitcask
#9Sorry, maybe I am not in the mood of delving too deep into the project(but I starred it! Amazing job I suppose) and I don't want to ask AI but rather some experts who are surely lurking HN. Can you guys please explain this to me like I am 5(or maybe 10)? Is this something revolutionary to keep in back of the mind? How does it compare to redis? When should I use it, if any. I always prefer sqlite, then postgresql if s…
Implementing Bitcask is sorta like a right of passage for people interested in DBs/storage engines. You shouldn't use this in production. SQLite is most likely more flexible, reliable, and ubiquitous for situations where this project would be useful.
Gotcha! Thanks a lot mate!
So can I say that this is just a toy project created by the author to learn about DB/storage engines and I should just use sqlite right in prod right?
Re: Filedb: Disk-based key-value store inspired by Bitcask
#10[dead]