Live data from Hacker News

Apple open-sources FoundationDB

foundationdb.org

161–170 of 453 posts

Re: Apple open-sources FoundationDB

#161
We (Wavefront) has been operating petabyte scale clusters for the last 5 years with FoundationDB (we got the source code via escrow) and we are super excited to be involved in the opensourcing of FDB. We have operated over 50 clusters on all kinds of aws instances and I can talk about all the amazing things we have done with it.

https://www.wavefront.com/wavefront-foundationdb-open-source...

Re: Apple open-sources FoundationDB

#162
post #98
post #93

Earlier quoted context omitted.

I just watched the demo of 5 machines and 2 getting unplugged. The remaining 3 can form a quorum. What happens if it was 3 and 3? Do they both form quorums?

A subset of the processes in a FoundationDB cluster have the job of maintaining coordination state (via disk Paxos). In any partition situation, if one of the partitions contains a majority of the coordinators then it will stay live, while minority partitions become unavailable.

Nitpick: To be fully live, a partition needs a majority of the coordinators and at least one replica of each piece of data (if you don't have any replicas of something unimportant, you might be able to get some work done, but if you have coordinators and nothing else in a partition you aren't going to be making progress)

Re: Apple open-sources FoundationDB

#163
post #19

Earlier quoted context omitted.

Sorry if I missed it but can you or someone else please link where the client wire protocol is documented?

The client is complex and needs very sophisticated testing, so there is only one implementation. All the language bindings use the C library.

Curious why the client is complicated compared to other dbs in same space ?

Re: Apple open-sources FoundationDB

#164
post #61

How does it compare to CockroachDB or TiDB?

It's closest to TiDB's key-value layer; a building block for more complex systems. More traditional, monolithic databases like CockroachDB (SQL) or FaunaDB (NoSQL) trade off extensibility for the benefits in performance and operations that come from very tight coupling.

In my understanding, FoundationDB's transaction management is closest to FaunaDB's; read/write sets are linearized in memory in preprocessing nodes and distributed asynchronously to the replicas rather than locked on the replica leaders like Spanner or CockroachDB. This is why FoundationDB doesn't support long-lived transactions.

It's interesting that the FoundationDB team chose to unwind their service architecture (there used to be separate transaction manager and replica processes), I assume in the interests of ease of operations.

It is not clear to me how leader election and failover works for the transaction management role. Maybe somebody from the team can clarify.

Re: Apple open-sources FoundationDB

#165

Earlier quoted context omitted.

~~How is it different from when Apple acquired the then-open-source FoundationDB (and shut down public access)? They could have just kept it open source back then.~~ EDIT: My bad, looks like FoundationDB wasn't fully open-source back then.

From what I recall (and based on some quick retro-googling) I don't believe Foudnation was open-source. One of the complaints about it on HN back in the day was that it was closed...

[deleted]

Re: Apple open-sources FoundationDB

#166

Found this section on performance very interesting. https://apple.github.io/foundationdb/performance.html#throug... I wonder what the SSD engine performance would look with NVMe standard NAND or an Optane SSD instead of SATA. Any FoundationDB guys/gals on this thread able to comment? Another Q: what's more commonly used in current FoundationDB deployments: memory engine or storage engine?

We run both, memory engine hasn't seen too many updates in the last 3 years though. For SSDs, we use i3 as well as EBS-backed volumes.

Re: Apple open-sources FoundationDB

#167
post #77

Earlier quoted context omitted.

Wavefront's co-founder just tweeted this: https://twitter.com/panghy/status/987022825457266689 Finally it's out! @WavefrontHQ managaes petabyte scale clusters with #foundationdb today!

I'm confused. How they could use FoundationDB when Apple aquired it a long time ago? Did apple sell the software to other companies?

We worked on the codebase independently (we have escrow). We run petabyte-scale clusters in aws.

Re: Apple open-sources FoundationDB

#168
post #23

"because it is an ordered key-value store, FoundationDB can use range reads to efficiently scan large swaths of data" https://apple.github.io/foundationdb/features.html I wonder how it compares to MUMPS databases like Intersystems Cache and FIS GtM?

It seems that MUMPS doesn't have serializable isolation.

Re: Apple open-sources FoundationDB

#169
Wow this is some really exciting news! I think it would be amazing to create a GraphQL API for FoundationDB. Therefore i have created a feature request for this in the Prisma repo. For those who don't know Prisma is a GraphQL database mapper for various databases.

https://github.com/graphcool/prisma/issues/2240

Re: Apple open-sources FoundationDB

#170
post #89

Noticed the Visual Studio project files. Was Windows a primary development platform for foundationdb?

Nope, the actor compiler is down with mono and your need windows to compile the windows client. When we were independently working on it, we just ditched the windows code completely (we are a mac/linux shop)
Post reply on HN