Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple open-sources FoundationDB

foundationdb.org

Post reply on HN
Print view View original

131–140 of 453 posts

Re: Apple open-sources FoundationDB

#131

Post by jtchang »

wwilson wrote: post #98
jtchang wrote: post #93

Earlier quoted context omitted.

I just watched the demo of 5 machines and 2 getting unplugged. The remaining 3 can form a quorum. What happens if it was 3 and 3? Do they both form quorums?

A subset of the processes in a FoundationDB cluster have the job of maintaining coordination state (via disk Paxos). In any partition situation, if one of the partitions contains a majority of the coordinators then it will stay live, while minority partitions become unavailable.

What if the number of boxes are even?

Or do you get around this by not deploying an uneven number of boxes?

Re: Apple open-sources FoundationDB

#133

Post by [deleted] »

wll wrote: post #112
voidmain wrote: post #72

Earlier quoted context omitted.

I think Citus is not really ACID. Spanner (and to an extent its less mature OSS descendants Cockroach and TiKV) has more comparable goals, but is fairly different architecturally. For example, FoundationDB only requires N+1 replicas instead of 2N+1 to achieve N failure tolerance (even lots of databases with much weaker guarantees are in the latter category!), doesn't trust clocks at all, doesn't lose performance when…

The datacenter-aware mode documentation [0] says “Although data will always be triple replicated in this mode, it may not be replicated across all datacenters.” Why is that? [0] https://apple.github.io/foundationdb/configuration.html?data...

[deleted]

Re: Apple open-sources FoundationDB

#134

Post by itp »

wwilson wrote: post #105
X-Istence wrote: post #102

Earlier quoted context omitted.

Is the wire-compatible clone of the MongoDB API available?

It doesn't look like Apple open-sourced the Document Layer, which is a slight bummer. But I echo what Dave said below: what we got is incredible, let's not get greedy! Also TBH now that I don't have commercial reasons to push interop, if I write another document database on top of FDB, I doubt I'd make it Mongo compatible. That API is gnarly.

> That API is gnarly.

The Will Wilson I remember was not prone to such understatement. That API (and the corresponding semantics) was a nightmare.

Re: Apple open-sources FoundationDB

#136

Post by ccifuentes »

wwilson wrote: post #2

This is INCREDIBLE news! FoundationDB is the greatest piece of software I’ve ever worked on or used, and an amazing primitive for anybody who’s building distributed systems. The short version is that FDB is a massively scalable and fast transactional distributed database with some of the best testing and fault-tolerance on earth[1]. It’s in widespread production use at Apple and several other major companies. But the…

Do we really need another rookie database system? To be honest, Postgres and MongoDB is all you need to achieve any project.

I respect hobby projects though, and if that’s the case then great!

Re: Apple open-sources FoundationDB

#137

Post by rthomas900 »

wwilson wrote: post #60
thefounder wrote: post #53

Earlier quoted context omitted.

I'm not familiar with FDB but what you say sounds almost too good to be true. Can I use it to implement the Google Datastore api? I'm trying for years to find a suitable backend so that I can leave the Google land. Everything I tried either required a schema or lacked transactions or key namespaces.

As an existence proof: before the acquisition we built an ANSI SQL database and a wire-compatible clone of the MongoDB API. I see no reason you wouldn't be able to implement Datastore. In fact here's a public source claiming that Firestore (which I believe is its successor) is implemented on top of Spanner: https://www.theregister.co.uk/2017/10/04/google_backs_up_fir...

If I recall correctly, the "SQL layer" you had in FDB before the Apple acquisition was a nice proof of concept, but lacked support for many features (joins in SELECT, for example). Is the SQL layer code from that time available anywhere to the public? (I'm not seeing it in the repo announced by OP.)

Re: Apple open-sources FoundationDB

#138

Post by gregmac »

jtchang wrote: post #131
wwilson wrote: post #98

Earlier quoted context omitted.

A subset of the processes in a FoundationDB cluster have the job of maintaining coordination state (via disk Paxos). In any partition situation, if one of the partitions contains a majority of the coordinators then it will stay live, while minority partitions become unavailable.

What if the number of boxes are even? Or do you get around this by not deploying an uneven number of boxes?

This makes me wonder also what happens with multiple partitions, for example:

5 into 2, 2, and 1

7 into 3, 2, and 2

Re: Apple open-sources FoundationDB

#140

Post by rthomas900 »

rthomas900 wrote: post #137
wwilson wrote: post #60

Earlier quoted context omitted.

As an existence proof: before the acquisition we built an ANSI SQL database and a wire-compatible clone of the MongoDB API. I see no reason you wouldn't be able to implement Datastore. In fact here's a public source claiming that Firestore (which I believe is its successor) is implemented on top of Spanner: https://www.theregister.co.uk/2017/10/04/google_backs_up_fir...

If I recall correctly, the "SQL layer" you had in FDB before the Apple acquisition was a nice proof of concept, but lacked support for many features (joins in SELECT, for example). Is the SQL layer code from that time available anywhere to the public? (I'm not seeing it in the repo announced by OP.)

Perhaps it's this?

https://github.com/jaytaylor/sql-layer

Post reply on HN