Was anyone else deeply saddened about three words into the headline, on realizing this wasn't a watch? (RIP)
Yes. I have moved on to Amazfit though.
Pebble: A RocksDB Inspired Key-Value Store Written in Go
21–30 of 84 posts
Re: Pebble: A RocksDB Inspired Key-Value Store Written in Go
#22So I understand the rationale for writing your own storage layer and think this is an awesome project, but there's something missing for me. One of the issues Peter brings up is they've come across a number of serious bugs in RocksDB. My question is, why would Pebble have less bugs. In fact, I would expect it to have significantly more bugs because Coackroach is the only company using Pebble. They mention briefly how…
Hi, I'm on the team that works on Pebble. Partially synced WAL records are easier to detect, as they would just appear as corrupt records and we can stop WAL replay at that point. Non-WAL writes are even easier to handle as SSTable files are immutable once fully written and synced. We rely pretty heavily on fsync/fdatasync calls to guarantee that "all" the data in a given range made it. In addition to randomized cras…
> We rely pretty heavily on fsync/fdatasync calls to guarantee that "all" the data in a given range made it.
Reminds me of fsync gate: https://news.ycombinator.com/item?id=20491965 and https://news.ycombinator.com/item?id=19119991 (not implying Pebble uses fysnc incorrectly).
Re: Pebble: A RocksDB Inspired Key-Value Store Written in Go
#23So I understand the rationale for writing your own storage layer and think this is an awesome project, but there's something missing for me. One of the issues Peter brings up is they've come across a number of serious bugs in RocksDB. My question is, why would Pebble have less bugs. In fact, I would expect it to have significantly more bugs because Coackroach is the only company using Pebble. They mention briefly how…
> So I understand the rationale for writing your own storage layer and think this is an awesome project, but there's something missing for me. One of the issues Peter brings up is they've come across a number of serious bugs in RocksDB. My question is, why would Pebble have less bugs. In fact, I would expect it to have significantly more bugs because Cockroach is the only company using Pebble.
We're only worried about functionality in Pebble used by CockroachDB. RocksDB has a huge number of features that sometimes have bugs due to subtle interactions. There is a very stable subset of RocksDB: the configuration and specific API usage patterns used internally by Facebook. That precise combination has seen extreme testing. But that isn't the subset of RocksDB used by CockroachDB. I would guess that the most significant testing of the subset of RocksDB used by CockroachDB is the testing we do at Cockroach Labs. Now that testing is being directed at Pebble along with the Pebble-specific testing detailed in the post.
> For example, it's possible the filesystem had synced some of the buffered data to disk, but not all of it. There's no guarantee about what buffered data was synced to disk. All you know is that some, all, or none of it made it to disk.
The filesystem does provide guarantees when you use fsync() and fdatasync(). Postgres relies on these guarantees. So does RocksDB. Pebble's usage of fsync/fdatasync mirrors RocksDB's. Our crash testing is not testing the filesystem guarantees, only that we're correctly using fsync/fdatasync (which is hard enough to get right).
Re: Pebble: A RocksDB Inspired Key-Value Store Written in Go
#24I’ve run into serious house burning down problems with myrocks too. Simple recipe to crash MySQL in a way that is unrecoverable: do ALTER TABLE on a big table and it runs out of RAM, crashes, and refuses to restart, ever.
Googling and people have been reporting the error on restarting several times on lists and things. What help is it to report to Maria dB or something? But do FB notice? Seems not.
Here’s hoping someone at FB browses HN...
I don’t get why FB don’t have some fuzzing and chaos monkey stress test to find easy stability bugs :(
Re: Pebble: A RocksDB Inspired Key-Value Store Written in Go
#25Was anyone else deeply saddened about three words into the headline, on realizing this wasn't a watch? (RIP)
I definitely was. I still use one now, more than 3 years after their business failure. I really wish someone would make something like the Pebble Time 2.
Re: Pebble: A RocksDB Inspired Key-Value Store Written in Go
#26As a consumer, why would I want something like this written in Go vs. Rust? Is it just that Rust is really good with developer relations? Because it feels like to me that all new foundational technology is safer and faster in a language like Rust, and things written in Go should be higher up the food chain.
In addition to what others mentioned, even if Cockroach is written in Go, they could have used Rust, but the trade-off is that they would need to use cgo which introduces extra complexity for building, debugging, and has performance trade-offs that a pure-Go based solution doesn't necessarily have.
Re: Pebble: A RocksDB Inspired Key-Value Store Written in Go
#27Why would someone remove a non-GC database engine with a database engine with GC?
Has Go evolved better low-GC features? As I understand Go GC vs JVM GC, Go avoids major GC by simply pushing it to the future and consuming memory more readily.
But a database is a long-running program, so you have to pay the piper eventually.
Re: Pebble: A RocksDB Inspired Key-Value Store Written in Go
#28Was anyone else deeply saddened about three words into the headline, on realizing this wasn't a watch? (RIP)
I definitely was. I still use one now, more than 3 years after their business failure. I really wish someone would make something like the Pebble Time 2.
I have an Amazfit Bip, but the UI isn't as good as Pebble sadly.
There is some work in making a similar OS called RebbleOS[1] currently ongoing.
[1]: https://github.com/pebble-dev/RebbleOS
Hopefully it will be portable to other low-end smartwatches.
Re: Pebble: A RocksDB Inspired Key-Value Store Written in Go
#29The name is a little close to this existing LevelDB fork, maybe consider a different name?
https://github.com/utsaslab/pebblesdb
Re: Pebble: A RocksDB Inspired Key-Value Store Written in Go
#30Was anyone else deeply saddened about three words into the headline, on realizing this wasn't a watch? (RIP)
I definitely was. I still use one now, more than 3 years after their business failure. I really wish someone would make something like the Pebble Time 2.
Frankly, I'm satisfied enough with my Pebble 2 that I'm not sure I care whether anyone makes a new one. I just hope I don't end up in a situation where it breaks and I can't get a replacement.