31–40 of 71 posts
A database without any test harness? While this could be a good toy or PoC I would never use it in production. Readers should be aware, just because it's on HN doesn't mean it's production ready.
It uses Raft underneath as well which means there's a bunch of non-determinism and hell for anyone who invokes it as well from experience. The thing is cursed. Source: several years dealing with vault and consul.
SET: 253232.12 requests per second GET: 2130875.50 requests per second The 10:1 throughput ratio for GET vs SET is interesting. Redis being in-memory, the rates there are pretty close to the same for read/write. Is a 10:1 ratio typical for a storage backed distributed kv store? Edit: Looks like CockroachDb has roughly a 3:1 ratio, similar for YugabyteDB: https://www.cockroachlabs.com/docs/stable/performance.html http…
Earlier quoted context omitted.
It uses Raft underneath as well which means there's a bunch of non-determinism and hell for anyone who invokes it as well from experience. The thing is cursed. Source: several years dealing with vault and consul.
What is a better consensus protocol to use?
A database without any test harness? While this could be a good toy or PoC I would never use it in production. Readers should be aware, just because it's on HN doesn't mean it's production ready.
A database without any code, actually. It's less than a few hundred lines of Go that just wraps two other databases (syndtr/goleveldb and ledisdb/ledisdb) with a third library (tidwall/uhaha) that provides a Raft API.
Earlier quoted context omitted.
Raft involves waiting for fsync on a majority of nodes, so that's not too surprising. 'Typical' is a matter of what guarantees you want to give.
Typically people use raft for leader election which in turn can coordinate writes. I don't think the writes are being fsync'd in the raft logs here. At least I wouldn't expect that behavior.
Earlier quoted context omitted.
That's a good point and I should have been clearer. I might be off (and probably am) but if I remember correctly Redis persistence is more for disaster recovery - you can create snapshots and recover them or replay a log file. That's very different in terms of performance guarantees from persisting the data itself to disk and reading from it. I was under the impression that's what tools (like this one) and stuff like…
I wouldn't call it disaster recovery per se. It's just that Redis is mostly an in-memory database and if the process is terminated and restarted (for all sorts of reasons) the data can be restored from disk. So what IceFireDB might be good for is data which would not fit easily into the memory of one node. Again, it's really not clear to me.
It's unclear.
Earlier quoted context omitted.
A database without any code, actually. It's less than a few hundred lines of Go that just wraps two other databases (syndtr/goleveldb and ledisdb/ledisdb) with a third library (tidwall/uhaha) that provides a Raft API.
Oooh that means I can form it to do Redis instead right? Because that could be a nice way to to Redis clustering
Here's the LSET code:
https://github.com/gitsrc/IceFireDB/blob/main/lists.go#L232
func cmdLSET(m uhaha.Machine, args []string) (interface{}, error) {
if len(args) != 4 {
return nil, rafthub.ErrWrongNumArgs
}
index, err := ledis.StrInt64([]byte(args[2]), nil)
if err != nil {
return nil, err
}
if err := ldb.LSet([]byte(args[1]), int32(index), []byte(args[3])); err != nil {
return nil, err
}
return redcon.SimpleString("OK"), nil
}
1. tidwall/uhaha - Raft server (m uhaha.Machine, rafthub)
2. tidwall/redcon - Read/write redis protocol (redcon.SimpleString)
3. ledisdb/ledisdb - Redis-compatible with disk persistence via leveldb (ldb.LSet)
4. syndtr/goleveldb/leveldb - Provides snapshots, other scattered references throughout code
It also includes this seemingly random file below, which seems to implement some string slice overloads using unsafe.Pointer:
https://github.com/siddontang/go/blob/master/hack/hack.go
// no copy to change slice to string
// use your own risk
func String(b []byte) (s string) {
pbytes := (*reflect.SliceHeader)(unsafe.Pointer(&b))
pstring := (*reflect.StringHeader)(unsafe.Pointer(&s))
pstring.Data = pbytes.Data
pstring.Len = pbytes.Len
return
}
// no copy to change string to slice
// use your own risk
func Slice(s string) (b []byte) {
pbytes := (*reflect.SliceHeader)(unsafe.Pointer(&b))
pstring := (*reflect.StringHeader)(unsafe.Pointer(&s))
pbytes.Data = pstring.Data
pbytes.Len = pstring.Len
pbytes.Cap = pstring.Len
return
}Earlier quoted context omitted.
What is a better consensus protocol to use?
Look higher up the problem domain and solve it without requiring a consensus protocol.
Earlier quoted context omitted.
Raft involves waiting for fsync on a majority of nodes, so that's not too surprising. 'Typical' is a matter of what guarantees you want to give.
Typically people use raft for leader election which in turn can coordinate writes. I don't think the writes are being fsync'd in the raft logs here. At least I wouldn't expect that behavior.
The reason is that in Raft if a node acknowledges to the leader that it wrote something to the log it must not later accept a different write in the same log position.
This mean if for some reason server rebooted with dirty buffered writes that could not be flushed in time. it’s supposed to forgot everything it know and rejoin the cluster using a brand new node id.