Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

FoundationDB Record Layer

foundationdb.org

Post reply on HN
Print view View original

41–50 of 85 posts

Re: FoundationDB Record Layer

#41

Post by mbesto »

ryanworl wrote: post #36
mbesto wrote: post #35

Has anyone ever used FoundationDB and not found it successful? All I read is "it supports RDMS + NoSQL and can be distributed". So what use cases doesn't it solve?

The best way I can describe FoundationDB is it is like a file system. You can do just about whatever you’d like with files and a file system, in theory. You can implement just about any data model you can dream up in FDB. But the current storage engine is not as well optimized as it could be. It does have scalability limits, although they’re not relevant for 99.9% of use cases. Upgrading a cluster to a new non-patch…

Super helpful, thanks for the info.

Re: FoundationDB Record Layer

#42

Post by devj »

Few doubts:

1. Any reason to write it in Java instead of C, C++, Rust, etc?

2. Any reason to use Protobuf instead of Flatbuffers, Avro, etc?

3. Can FoundationdDB be used with Apache Arrow?

Re: FoundationDB Record Layer

#43

Post by Scriptor »

wwilson wrote: post #3
wwilson wrote: post #2

This is very cool! FoundationDB excites a lot of people because it's an extremely scalable and extremely reliable distributed database that supports ACID transactions, and which is both open-source and has Apple standing behind it. And yeah, all of that is pretty nice. But arguably the real power comes from the fact that it exposes a relatively low-level data model that can then be wrapped in one or more stateless "l…

Wow, from the post: "Together, the Record Layer and FoundationDB form the backbone of Apple's CloudKit. We wrote a paper describing how we built the Record Layer to run at massive scale and how CloudKit uses it." I think this is the first time that little detail has been publicly disclosed.

Isn't the code on GitHub?

Re: FoundationDB Record Layer

#44

Post by [deleted] »

PhilippGille wrote: post #23
lima wrote: post #12

This might be the first good alternative to etcd for configuration stores that need real-time updates. Like Kubernetes. Many Kubernetes scaling issues are etcd-related. RethinkDB is dead-ish, and CockroachDB is treating their changefeeds as an enterprise feature that requires a Kafka instance to stream to :(

Is TiKV an alternative? Short overview and maybe good to know it's becoming part of the CNCF: https//www.cncf.io/blog/2018/08/28/cncf-to-host-tikv-in-the-sandbox/ Haven't worked with it myself yet, but maybe others can share their experience? There have also been some HN threads in the past, about TiDB at least.

[deleted]

Re: FoundationDB Record Layer

#45

Post by thejerz »

spullara wrote: post #19

I had built a layer like this one for my startup Bagcheck called Havrobase[1] (it was on top of HBase/Solr, here is the motivating blog post[2]) that ultimately I put on top of MySQL/Solr and other stores. Later, when we started Wavefront, I ported that layer to FDB and that still powers their metadata. Really a good fit and very much like this record layer. I highly recommend this approach for 24/7 services as you n…

What were the pro's and con's of using FDB over HBase?

Re: FoundationDB Record Layer

#47

Post by thanatos_dem »

georgewfraser wrote: post #39

Seeing this soon after the AWS “wire compatible with Mongo” kerfuffle, it makes me think: it would be amazing if the cloud vendors would offer a managed FDB service. An open-source, cloud-agnostic, horizontally scalable, document-oriented transactional database would be an incredible tool. I know AWS is going in the opposite direction these days with proprietary “wire compatible” services but a guy can dream...

Pourque no los dos? Foundation announced a mongo api compatible document layer in November - https://www.foundationdb.org/blog/announcing-document-layer/

Re: FoundationDB Record Layer

#49

Post by abalone »

Apple low key does some cool server projects with a Java bent. They've contributed to Netty (well, they hired core developers).[1]

They've been basically put them to work reimplementing it in Swift.[2] It's open and out there but not a lot of people paying attention. While it's still early days I think there may a year where, suddenly, Swift on the server is a super serious thing and all this work they've been doing on little old CloudKit kind of takes over the world.

Just a fun prediction.. but it wouldn't be the first time Apple pulled something like that.

I do like that Swift's non-tracing garbage collection model is well suited for server apps. Rust is cool too but maybe Swift would be a little friendlier and thus better suited to inherit Java's mantle. I mean can you just imagine if Apple is slowly building up Swift to overtake Java on the server? That that's one of their long game master plans? I know that sounds completely crazy.. It just might work. They do run one of the biggest data center networks in the world so they have a pretty good testbed and can justify a hefty R&D budget.

[1] https://www.infoq.com/presentations/apple-netty

[2] https://github.com/apple/swift-nio

Re: FoundationDB Record Layer

#50

Post by grandinj »

nschiefer wrote: post #24
rdsubhas wrote: post #11

Earlier quoted context omitted.

One could technically write an SQL translation layer on top of it, as a client-side library? Or does it need support on the server-side / record layer?

(I'm from the iCloud team that works on the Record Layer.) Both building a relational database and implementing a proper SQL interface on top of it are huge projects. The SQL spec is large and complicated, so achieving true compatibility (as opposed to superficial compatibility) is challenging. Even worse, once you have a SQL interface users expect to be able to throw any SQL that they give to, say, Postgres, and hav…

Apache Ignite is also using H2 as their "SQL parser and query planner" layer
Post reply on HN