Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

FoundationDB Record Layer

foundationdb.org

Post reply on HN
Print view View original

71–80 of 85 posts

Re: FoundationDB Record Layer

#71

Post by gregwebs »

SloopJon wrote: post #55
wwilson wrote: post #2

This is very cool! FoundationDB excites a lot of people because it's an extremely scalable and extremely reliable distributed database that supports ACID transactions, and which is both open-source and has Apple standing behind it. And yeah, all of that is pretty nice. But arguably the real power comes from the fact that it exposes a relatively low-level data model that can then be wrapped in one or more stateless "l…

Do I understand correctly that the record layer is only usable by Java clients? That strikes me as a drawback of the layer approach: unless you supply a server protocol, like the document layer does, every language essentially reimplements the layer from scratch. That seems to be the case for the tuple layer, for example.

It certainly doesn't have to be the case. The MongoDB layer can be used via the MongoDB protocol. TiDB is an entire MySQL implementation that any MySQL client can connect to which is built on top of TiKV (equivalent to FDB). If the goal is lower overhead of data transfer, another approach is writing layers in C (or Rust exposed as C) and then generating bindings for different languages.

Re: FoundationDB Record Layer

#72

Post by all0c »

continuations wrote: post #38

Does that mean FDB now supports secondary indexes? If that's the case, how does FDB compare to ScyllaDB now that they both have secondary indexes?

(I'm from the FDB team and work on the Record Layer.) As ryanworl's excellent answer suggests, the FoundationDB key value does not support secondary indexing on its own. It is strictly an ordered store mapping byte-array keys to byte-array values.

Secondary indexing is a core feature of the Record Layer, though! It includes a variety of secondary index types. The simplest are implemented using essentially the same strategy as ryanworl outlines (with more details on how that index works available in the key-value store documentation: https://apple.github.io/foundationdb/simple-indexes.html). And index updates are all entirely transactional (i.e., as the index update happens in the same transaction as record insertion, they are always consistent and up-to-date). However, all of that happens behind the scenes. The API presented to the user only asks for what record to save (update or insert), and then the Record Layer updates the appropriate indexes using a user-provided schema. Importantly, the Record Layer also supports handling the various stages of index maintenance (e.g., deleting an index's data after removing it from the schema or filling in data from existing records after an index is added). More can be found within the Record Layer overview: https://foundationdb.github.io/fdb-record-layer/Overview.htm...

Re: FoundationDB Record Layer

#73

Post by all0c »

devj wrote: post #42

Few doubts: 1. Any reason to write it in Java instead of C, C++, Rust, etc? 2. Any reason to use Protobuf instead of Flatbuffers, Avro, etc? 3. Can FoundationdDB be used with Apache Arrow?

The Record Layer is written in Java as it was designed to fit in with an existing stack that was already primarily Java-based. You can read more about how CloudKit uses the Record Layer in the preprint of the Record Layer paper: https://www.foundationdb.org/files/record-layer-paper.pdf

Excellent question regarding the choice to use Protocol Buffers. Firstly, as mentioned in the paper released last year, CloudKit uses Protocol Buffers for client-server intercommunication. As a result, there was already expertise around protobuf, which is a good tie breaker when evaluating alternatives. (Here's that paper, by the way: http://www.vldb.org/pvldb/vol11/p540-shraer.pdf) Secondly, the Record Layer makes heavy use of Protocol Buffer descriptors, which specify the field types and names within protobuf schemata, and dynamic messages. Descriptors are used internally within the Record Layer to do things like schema validation. (For example, if an index is defined on a specific field, the descriptor can be checked to validate that that field exists in the given record type.) Likewise, dynamic messages make it possible for applications using the Record Layer to load their schema at run time by reading it from storage. The FDBMetaDataStore allows the user to do exactly that (while storing the schema persistently in FoundationDB): https://static.javadoc.io/org.foundationdb/fdb-record-layer-...

The Record Layer's data format is not compatible with the specification specified by Apache Arrow, no.

Re: FoundationDB Record Layer

#74

Post by jwr »

aseipp wrote: post #58
jwr wrote: post #33

Earlier quoted context omitted.

Cassandra To be honest, I don't like anything about Cassandra. Beginning with the naming: back when I was trying to learn about Cassandra, I couldn't get past the obscure and bizarre naming (super-columns?). When I dealt with systems using it, I never quite understood how you can keep saying that "the later timestamp wins" and speak of consistency with a straight face: in a distributed system, there is no such thing…

FoundationDB does not support true geo-replicated multi-region distribution the way Cassandra, Spanner, Cockroach, etc do, at least not without paying huge latency/round trip costs. If you want to avoid that, the best you can have is a separate failover region, and, with FoundationDB 6, you can get closer-to-LAN latencies for failover deployments to separate regions (but only one region) while retaining ACID semantic…

That is an excellent summary. There is no silver bullet and you can't have your cake and eat it, too. The approach to multi-region that FoundationDB 6 takes suits my needs (I'm not Google) and I like the compromises they made.

Since most of what I do (or consult with) does not need massive performance, I'd rather pick databases with compromises favoring consistency and correctness. This is why I like what I see in FoundationDB so far.

Re: FoundationDB Record Layer

#75

Post by abalone »

ascagnel_ wrote: post #64
abalone wrote: post #49

Apple low key does some cool server projects with a Java bent. They've contributed to Netty (well, they hired core developers).[1] They've been basically put them to work reimplementing it in Swift.[2] It's open and out there but not a lot of people paying attention. While it's still early days I think there may a year where, suddenly, Swift on the server is a super serious thing and all this work they've been doing…

Apple is probably hoping to run Swift on their servers. I don't foresee them putting in the effort into enterprise sales and service to make Swift overtake Java, though -- it hasn't really been their MO in the past.

They wouldn’t need to. They already partner with IBM for that stuff.

Re: FoundationDB Record Layer

#76

Post by devj »

all0c wrote: post #73
devj wrote: post #42

Few doubts: 1. Any reason to write it in Java instead of C, C++, Rust, etc? 2. Any reason to use Protobuf instead of Flatbuffers, Avro, etc? 3. Can FoundationdDB be used with Apache Arrow?

The Record Layer is written in Java as it was designed to fit in with an existing stack that was already primarily Java-based. You can read more about how CloudKit uses the Record Layer in the preprint of the Record Layer paper: https://www.foundationdb.org/files/record-layer-paper.pdf Excellent question regarding the choice to use Protocol Buffers. Firstly, as mentioned in the paper released last year, CloudKit uses…

Thanks for your reply. Would be really helpful if you can share the following:

1. Size of the CloudKit cluster and the number of RecordLayer instances. A ratio would also be enough to get an approx. idea.

2. How metadata changes involving field data type are being handled?

3. How are relationships and therefore, foreign keys handled? Are any referential actions like cascading deletes supported?

Re: FoundationDB Record Layer

#77

Post by bcx »

ryanworl wrote: post #22
bcx wrote: post #16

I learned that basically all of Imessages and contacts are stored on foundation DB, it's pretty great this is making it into opensource. Thanks Apple!

Are you using FDB at Olark? (Saw it in your profile)

No :) Just off the shelf DBs so far. But the FoundationDB guys are HS friends ;).

Re: FoundationDB Record Layer

#78

Post by nschiefer »

Birch-san wrote: post #59
mbesto wrote: post #35

Has anyone ever used FoundationDB and not found it successful? All I read is "it supports RDMS + NoSQL and can be distributed". So what use cases doesn't it solve?

Even with the Record layer, it doesn't have support for JOINs in the same way as an RDBMS would. Yes, the Record Layer helps you define and index into _hierarchies_ of entities, but I suspect it doesn't have an answer for other access patterns (e.g. producing "report" views that relate or aggregate non-hierarchical data). You could retroactively construct a custom view _after the fact_, but only if you can do so with…

You're right the Record Layer doesn't yet have join support, but note PR #306 [1]. We support aggregate indexes but can't run aggregate “reports” that aren't backed by indexes. The rationale for this is covered in the paper (see [2]), but note that doesn't preclude such support being added into or on top of Record Layer (also discussed in the paper).

The Record Layer does a whole bunch of work to deal with index maintenance (see our docs [3])). Our “index maintainer” abstraction (discussed in the paper) makes maintaining our indexes (including those that are basically materialized views) completely seamless from the user's perspective, even for updates and deletes. We also have a lot of tooling for making efficient schema migrations. For example, schema migrations are performed lazily (when the data is accessed), so they aren't limited by the 5 second transaction limit. If you add/remove/change indexes, they'll be put into a “write-only” mode where they'll keep accepting writes while an “online indexer” builds the index over multiple transactions. We even have fancy logic to automatically adjust the size of the transactions if they start failing due to contention or timeouts!

Basically, the Record Layer solves a lot (but not all) of the pain points that shows up when you don't know your access patterns from the beginning. The paper talks a bit about how CloudKit uses some of those features.

[1] https://github.com/FoundationDB/fdb-record-layer/pull/306

[2] https://foundationdb.github.io/fdb-record-layer/FAQ.html — search for “aggregation”

[3] https://foundationdb.github.io/fdb-record-layer/SchemaEvolut...

Re: FoundationDB Record Layer

#79

Post by spullara »

thejerz wrote: post #45
spullara wrote: post #19

I had built a layer like this one for my startup Bagcheck called Havrobase[1] (it was on top of HBase/Solr, here is the motivating blog post[2]) that ultimately I put on top of MySQL/Solr and other stores. Later, when we started Wavefront, I ported that layer to FDB and that still powers their metadata. Really a good fit and very much like this record layer. I highly recommend this approach for 24/7 services as you n…

What were the pro's and con's of using FDB over HBase?

Several things caused us to move off of HBase:

1) Operationally, HBase is a nightmare whereas FDB is extremely easy to operate. 2) HBase doesn't natively, or efficiently with extensions, support transactions across rows. 3) GC makes HBase performance unpredictable whereas FDB is written in C++. 4) HBase depends on Zookeeper and it is operationally painful to support and we were replacing it with FDB also.

I don't think I will ever again use anything from the Hadoop ecosystem if I can get away with it.

Re: FoundationDB Record Layer

#80

Post by all0c »

devj wrote: post #76
all0c wrote: post #73

Earlier quoted context omitted.

The Record Layer is written in Java as it was designed to fit in with an existing stack that was already primarily Java-based. You can read more about how CloudKit uses the Record Layer in the preprint of the Record Layer paper: https://www.foundationdb.org/files/record-layer-paper.pdf Excellent question regarding the choice to use Protocol Buffers. Firstly, as mentioned in the paper released last year, CloudKit uses…

Thanks for your reply. Would be really helpful if you can share the following: 1. Size of the CloudKit cluster and the number of RecordLayer instances. A ratio would also be enough to get an approx. idea. 2. How metadata changes involving field data type are being handled? 3. How are relationships and therefore, foreign keys handled? Are any referential actions like cascading deletes supported?

The Record Layer doesn't currently support foreign key constraints, so foreign keys are more of an “design pattern” than a first-class feature. For example, in a sample schema in the repository, an “Order” message has have a field called “item_id” that points to the primary key of an “Item” message: https://github.com/FoundationDB/fdb-record-layer/blob/792c95... There isn't an automatic check to make sure the item exists, though, nor are there cascading deletes. That being said, I don't think the architecture is incompatible with that feature, so it would be a reasonable feature request.

There are some guidelines regarding field type changes in the schema evolution guide: https://foundationdb.github.io/fdb-record-layer/SchemaEvolut... Most data type changes are incompatible with either Protobuf's serialization format or the FDB Tuple layer's serialization format (which the Record Layer users for storing secondary indexes and primary keys). The general advice for type changes (if there are existing data in your record stores) would instead be to introduce a new field of the new type and deprecate the old one.

Post reply on HN