Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pebble: A RocksDB Inspired Key-Value Store Written in Go

cockroachlabs.com

Post reply on HN
Print view View original

61–70 of 84 posts

Re: Pebble: A RocksDB Inspired Key-Value Store Written in Go

#61

Post by dis-sys »

petermattis wrote: post #45
ryanworl wrote: post #40

Earlier quoted context omitted.

Have you considered using direct IO for the log?

Yes. So far performance was worse in experiments, and the durability improvements are questionable because it is extremely difficult to get a clear understanding of the durability semantics of direct IO. If you can find a pointer to clear documentation of what those semantics are I'd be extremely interested in reading it.

RocksDB supports using Direct IO for flush and compaction (use_direct_io_for_flush_and_compaction), enabling that can improve write throughput for my workload in RocksDB. Any plan to do that in pebble?

Re: Pebble: A RocksDB Inspired Key-Value Store Written in Go

#64

Post by m-p-3 »

Polylactic_acid wrote: post #59
m-p-3 wrote: post #28

Earlier quoted context omitted.

I have an Amazfit Bip, but the UI isn't as good as Pebble sadly. There is some work in making a similar OS called RebbleOS[1] currently ongoing. [1]: https://github.com/pebble-dev/RebbleOS Hopefully it will be portable to other low-end smartwatches.

3 years later and it looks like they are just at the trying to get hardware features accessible. At this rate all pebble devices will have died / been discarded before its able to show a notification on your wrist.

Hopefully the OS could run on something newer than a Pebble, like the PineTime.

Re: Pebble: A RocksDB Inspired Key-Value Store Written in Go

#65

Post by willvarfar »

nitrobeast wrote: post #53
willvarfar wrote: post #24

I’ve run into serious house burning down problems with myrocks too. Simple recipe to crash MySQL in a way that is unrecoverable: do ALTER TABLE on a big table and it runs out of RAM, crashes, and refuses to restart, ever. Googling and people have been reporting the error on restarting several times on lists and things. What help is it to report to Maria dB or something? But do FB notice? Seems not. Here’s hoping some…

Likely your db configuration is very different from what FB uses in production, so they have no incentive to investigate or fix.

Its that the 'fix' is so unprofessional.

The problem is that the program runs out of RAM. The challenge is to write the data and metadata in such a way that a program crashing at any point for any reason is recoverable.

This is the basic promise of the Durability in ACID, and people using MyRocks expect it.

Rather than actually making sure that MyRocks is durable, they simply slap on a 'max transaction rows' to make it unlikely you run out of RAM. Instead, you simply get an error, and can't do stuff like ALTER TABLE or UPDATE on large tables.

Of course its easy to run out of RAM despite these thresholds, and its easy to find advice when you google the error messages you get that lead you to up the thresholds and to even set a 'bulk load' flag that disables various checks you probably haven't investigated.

The whole approach is wrongheaded!

A database that crashes should not be corrupt!!! Isn't this reliability 101? Why doesn't myrocks have chaos monkey stress testing etc etc?

Re: Pebble: A RocksDB Inspired Key-Value Store Written in Go

#66

Post by nemothekid »

willvarfar wrote: post #65
nitrobeast wrote: post #53

Earlier quoted context omitted.

Likely your db configuration is very different from what FB uses in production, so they have no incentive to investigate or fix.

Its that the 'fix' is so unprofessional. The problem is that the program runs out of RAM. The challenge is to write the data and metadata in such a way that a program crashing at any point for any reason is recoverable. This is the basic promise of the Durability in ACID, and people using MyRocks expect it. Rather than actually making sure that MyRocks is durable, they simply slap on a 'max transaction rows' to make…

>A database that crashes should not be corrupt!!! Isn't this reliability 101? Why doesn't myrocks have chaos monkey stress testing etc etc?

Because Facebook has little incentive to ensure that RocksDB works well in your use case. MyRocks was built for Facebook and anything that Facebook doesn’t do probably isn’t particularly hardened. They aren’t going to invest time doing chaos monkey stress testing on codepaths they don’t use. Things like durability might not be super important to them because they will make it up in redundancy.

I remember being burned by something similar during the early days of Cassandra. I’m sure Cockroach has hit the same bugs.

Re: Pebble: A RocksDB Inspired Key-Value Store Written in Go

#67

Post by StreamBright »

Few question comes into mind reading this:

- what is the plan to tackle Go's GC?

It seems to me that above a certain scale people run into GC problems with Go.[1]

- have they considered WickedDB?[2]

It appears to be a good candidate for their need.

https://github.com/Fullstop000/wickdb

https://blog.discord.com/why-discord-is-switching-from-go-to...

Re: Pebble: A RocksDB Inspired Key-Value Store Written in Go

#68

Post by throwdbaaway »

petermattis wrote: post #14
tyingq wrote: post #11

Earlier quoted context omitted.

There's some notes on that here: https://github.com/cockroachdb/pebble/blob/c39589c8cb36d95df...

The TLDR is that the GC did cause problems so we had to avoid it for the block cache. Luckily we were able to do so without exposing the complexity in the API. Not for the faint of heart. Don't try this at home kids.

Any reason why the block cache needs to be 10s of GB in size? Cassandra, for example, usually has a rather small key cache on heap, and then just relies on the kernel page cache. I don't have experience with cockroach, it looks like the block cache is similar to cassandra row cache, which can be configured to be on heap or off heap, but usually not beneficial to enable.

Re: Pebble: A RocksDB Inspired Key-Value Store Written in Go

#70

Post by Jnr »

_ea1k wrote: post #25
chaosharmonic wrote: post #2

Was anyone else deeply saddened about three words into the headline, on realizing this wasn't a watch? (RIP)

I definitely was. I still use one now, more than 3 years after their business failure. I really wish someone would make something like the Pebble Time 2.

I still use my Pebble Time Steel from the Kickstarter campaign.

Thinking of switching to Apple watch because I have iPhone and I also want to use Apple Pay without the phone. If they had better battery life, I probably would have done it already.

Post reply on HN