Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

FoundationDB 6.0 released, featuring multi-region support

foundationdb.org

Post reply on HN
Print view View original

41–50 of 72 posts

Re: FoundationDB 6.0 released, featuring multi-region support

#41

Post by ryanworl »

realreality wrote: post #40
ryanworl wrote: post #37

Earlier quoted context omitted.

Regarding your first comment, the reason I’m listed as a contributor for this release is I made a change to the documentation about large range reads. Basically, value sizes are not included in the 10mb limit for reads.

Ah yes, I just noticed that in the docs. That’s a good thing to note, though you could still run into the problem with very large ranges (maybe reading 1 million keys is a rare use case?).

Reading a million individual keys would be quite rare I would guess, but that isn’t really the issue for a large range. The keys at the start and the end of the range are what’s counted in that case. So if you read the range A-Z, the size is only those two keys A and Z, not the size of keys in between.

More relevant for the current storage engines (although changing in a future storage engine from digging through the code and the abstract for an upcoming talk) is the five second transaction duration limit. That’s just because the multi-version data structure only includes the last 5s of versions.

Re: FoundationDB 6.0 released, featuring multi-region support

#42

Post by realreality »

Meai wrote: post #39
realreality wrote: post #36

Earlier quoted context omitted.

- transaction size and duration limitations. I can almost understand the limitation on large write transactions, but the same size limitation applies to read transactions. If you’re doing a large range read, you may not know whether your range will reach the 10MB limit, and thus raise an exception. - the storage backends seem less impressive than the marketing leads you to believe. The default memory backend is obvio…

I havent tested that it does work as described but the docs say that the cluster stops working entirely unless the cluster size corresponds to the configured replication. Seen here: https://apple.github.io/foundationdb/configuration.html#conf... Look under "double" mode or "triple" mode. Is this why it maybe didn't work for you?

I’m not sure if you can switch from single to double, and then back to single. I don’t remember if this configuration was available when I was testing fdb, or if you just increased the number of processes in order to scale the cluster.

I do remember getting into a state where the status said it was migrating data, but there was no available node to migrate it to (because I wanted to shrink the cluster). Effectively, the cluster was deadlocked.

Re: FoundationDB 6.0 released, featuring multi-region support

#43

Post by realreality »

ryanworl wrote: post #41
realreality wrote: post #40

Earlier quoted context omitted.

Ah yes, I just noticed that in the docs. That’s a good thing to note, though you could still run into the problem with very large ranges (maybe reading 1 million keys is a rare use case?).

Reading a million individual keys would be quite rare I would guess, but that isn’t really the issue for a large range. The keys at the start and the end of the range are what’s counted in that case. So if you read the range A-Z, the size is only those two keys A and Z, not the size of keys in between. More relevant for the current storage engines (although changing in a future storage engine from digging through the…

Oh, that’s even less-expected behavior! In that case, one would never run into the size limitation for range reads. I think the docs should clarify that only the first and last keys count toward the transaction size.

Yes, the 5 second limit could be a problem.

Re: FoundationDB 6.0 released, featuring multi-region support

#45

Post by Meai »

realreality wrote: post #42
Meai wrote: post #39

Earlier quoted context omitted.

I havent tested that it does work as described but the docs say that the cluster stops working entirely unless the cluster size corresponds to the configured replication. Seen here: https://apple.github.io/foundationdb/configuration.html#conf... Look under "double" mode or "triple" mode. Is this why it maybe didn't work for you?

I’m not sure if you can switch from single to double, and then back to single. I don’t remember if this configuration was available when I was testing fdb, or if you just increased the number of processes in order to scale the cluster. I do remember getting into a state where the status said it was migrating data, but there was no available node to migrate it to (because I wanted to shrink the cluster). Effectively,…

Supposedly it works, as described a section below: https://apple.github.io/foundationdb/configuration.html#chan...

But yeah, the entire point of my comments in this thread is that the database should be telling me exactly what my options are at any situation if there are any. Judging by your comments it seems that you have also encountered a silent "deadlock" and the database gave no indication of what the hell was going on. That's the key here: The database silently stopped working, right? For something as critical as a database with possibly very important data, this just isn't acceptable to me. I want to be told as if I'm a complete noobie user what I have to do and why and what is going on with my data. The database is not a place where I feel the need to put on my smartypants hat, it's where I want to be taken care of completely.

Re: FoundationDB 6.0 released, featuring multi-region support

#46

Post by ryanworl »

realreality wrote: post #43
ryanworl wrote: post #41

Earlier quoted context omitted.

Reading a million individual keys would be quite rare I would guess, but that isn’t really the issue for a large range. The keys at the start and the end of the range are what’s counted in that case. So if you read the range A-Z, the size is only those two keys A and Z, not the size of keys in between. More relevant for the current storage engines (although changing in a future storage engine from digging through the…

Oh, that’s even less-expected behavior! In that case, one would never run into the size limitation for range reads. I think the docs should clarify that only the first and last keys count toward the transaction size. Yes, the 5 second limit could be a problem.

There are two reasons for this I can see:

1) All mutations and key ranges are stored locally and submitted when the transaction commits. Lots of data to transfer.

2) the optimistic conflict checker can process one range (even if it is a lot of underlying keys) a lot easier than each individual key in that range

Re: FoundationDB 6.0 released, featuring multi-region support

#48

Post by DLA »

zzzcpan wrote: post #19
DLA wrote: post #10

Earlier quoted context omitted.

+1 for a SQL layer/interface. Would massively increase utility of FDB.

Have you tried CockroachDB, TiDB? Or why do you think SQL could increase utility? It would make things slow and unpredictable, something distributed databases already struggle with.

I am a huge fan of CockroachDB. Using it in two projects currently and it’s been fantastic. Having a SQL interface to FDB would be the glue needed to integrate FDB into many domains as a viable solution. FDB has solid k/v interfaces now, handles clustering and failures, adding a SQL layer increases its potential utility and lowers the bar for many users. To your point about slow and unpredictable sure that can be an issue - perhaps a limited SQL interface INSERT, UPDATE, DELETE, SELECT to start. Make it dirt simple to do CRUD ops.

Re: FoundationDB 6.0 released, featuring multi-region support

#49

Post by ddorian43 »

riku_iki wrote: post #27
tensor wrote: post #8

Earlier quoted context omitted.

We use Postgresql heavily and it's not really a viable replacement. Even if there were an SQL layer, I doubt it would be fully featured enough for our needs. We may evaluate FDB as a distributed cache, but haven't had the time.

Curious if it is possible to query fdb as fdw from psql..

Should be able. Yugabyte is doing something similar.

Re: FoundationDB 6.0 released, featuring multi-region support

#50

Post by jared2501 »

realreality wrote: post #36
ryanworl wrote: post #4

To anyone who is on the fence about putting FoundationDB into production (or at least evaluating it for their use cases), what is the number one thing you think is missing or you're worried about? i.e. - a SQL interface, - pre-packaged data structure libraries, - monitoring, - limitations of FoundationDB itself, - etc. I'm working on a talk for the upcoming FoundationDB Summit and I'd love to address some real-world…

- transaction size and duration limitations. I can almost understand the limitation on large write transactions, but the same size limitation applies to read transactions. If you’re doing a large range read, you may not know whether your range will reach the 10MB limit, and thus raise an exception. - the storage backends seem less impressive than the marketing leads you to believe. The default memory backend is obvio…

On shrinking a cluster: you'll want to use the fdbcli to "exclude" nodes. Should be pretty straight forward (search the docs for the word "exclude").

On write amplification: a factor of 3x is not actually that unusual. The default RocksDB size amplification is 2x, and I've seen performant LSM trees with about 3x write amplification.

On the single threaded bottleneck: this is an inherent issue you have when you put your database over a network connection. LMDB can do 10k/100k+ reads/sec on a single thread since it's just doing syscalls. As soon as you start to need to distribute your database across more than 1 machine you start to need to parallelize you work for high throughput.

Post reply on HN