Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple open-sources FoundationDB

foundationdb.org

Post reply on HN
Print view View original

111–120 of 453 posts

Re: Apple open-sources FoundationDB

#111

Post by dboreham »

itp wrote: post #40
_xzxj wrote: post #32

Google please take a few notes here: 1. It’s in its own repo 2. The build instructions are concise and clear. Dependencies are listed. You have to follow a total of 0 links. 3. They use a common build system and not an in-house thing.

Speaking as the original author of this monstrosity of a build system, please be careful before offering praise here. To be clear, there is a top-level, non-recursive Makefile that uses the second expansion feature of GNU make, translating Visual Studio project files into generated Makefile inputs that are transformed into targets to power the build. Although it starts by running `make`, it's about as in-house as a t…

This kind of deep-inside-baseball from-the-horses-mouth interaction is what's so awesome about HN!

Re: Apple open-sources FoundationDB

#112

Post by wll »

voidmain wrote: post #72
misterbowfinger wrote: post #43

I hate to be that person, but when I hear "ACID transactions in a distributed database", I hear Citus/Spanner/CockroachDB. I'm positive that Citus & Spanner are quite different from FoundationDB, but I have no idea how. Googling didn't help much. Can someone provide an overview of the differences?

I think Citus is not really ACID. Spanner (and to an extent its less mature OSS descendants Cockroach and TiKV) has more comparable goals, but is fairly different architecturally. For example, FoundationDB only requires N+1 replicas instead of 2N+1 to achieve N failure tolerance (even lots of databases with much weaker guarantees are in the latter category!), doesn't trust clocks at all, doesn't lose performance when…

The datacenter-aware mode documentation [0] says “Although data will always be triple replicated in this mode, it may not be replicated across all datacenters.”

Why is that?

[0] https://apple.github.io/foundationdb/configuration.html?data...

Re: Apple open-sources FoundationDB

#114

Post by jasode »

wwilson wrote: post #2

This is INCREDIBLE news! FoundationDB is the greatest piece of software I’ve ever worked on or used, and an amazing primitive for anybody who’s building distributed systems. The short version is that FDB is a massively scalable and fast transactional distributed database with some of the best testing and fault-tolerance on earth[1]. It’s in widespread production use at Apple and several other major companies. But the…

>It’s in widespread production use at Apple

Anybody know if Apple migrated any projects from Cassandra to FoundationDB?

Or was every project using FoundationDB a greenfield project?

Re: Apple open-sources FoundationDB

#115

Post by woolvalley »

foobaw wrote: post #24
wwilson wrote: post #2

This is INCREDIBLE news! FoundationDB is the greatest piece of software I’ve ever worked on or used, and an amazing primitive for anybody who’s building distributed systems. The short version is that FDB is a massively scalable and fast transactional distributed database with some of the best testing and fault-tolerance on earth[1]. It’s in widespread production use at Apple and several other major companies. But the…

It is a genius move from Apple. I just wish they'd apply this logic to a lot of their other stuff.

They are, slowly. Swift is open source, clang is open source. They are moving parts of the xcode IDE into open source, like with sourcekitd and now recently clangd.

I don't think they will ever move 'secret sauce' into open source, but infrastructural things like DBs and dev tooling seems to be going in that direction.

Re: Apple open-sources FoundationDB

#118

Post by btilly »

PeCaN wrote: post #109
julien_c wrote: post #63

Earlier quoted context omitted.

Honest question, does MongoDB not work for this?

Honest question, does MongoDB work?

Until recently, no. But apparently they got serious about making it work and now it actually does. See https://jepsen.io/analyses/mongodb-3-4-0-rc3 for verification.

I realized this 2 months ago at https://news.ycombinator.com/item?id=16386129

Re: Apple open-sources FoundationDB

#119

Post by infogulch »

voidmain wrote: post #103
infogulch wrote: post #101

Earlier quoted context omitted.

Wait how can it be ACID if it can tolerate N failures from N+1 nodes? Doesn't that kill consistency almost by definition? What level of isolation does it support? Snapshot? Serializable?

It is serializable and totally uncompromising. Philosophically pretty much everything defaults to the safest possible thing. It can't tolerate N failures from N+1 nodes . It can tolerate N failures with N+1 copies of your data . In a big cluster you have plenty of nodes but storing everything 5 times to tolerate 2 failures is really expensive.

> It can't tolerate N failures from N+1 copies of your data

Sorry I got the terminology wrong, but that's a distinction without a difference. If it can tolerate N failures from N+1 copies, that means a network partition would allow any one copy to continue chugging along making changes by itself. You have two options: consistency is dropped and you downgrade to eventually consistent (at best), or availability is dropped meaning a single node can't make changes without a majority, which invalidates the N of N+1 failures claim. (Which is where the N failures of 2N+1 copies claim comes from in the first place: after N failures you still have a majority of copies.)

Or there's some other magic quorum protocol I've never heard of that makes the majority problem disappear.

Re: Apple open-sources FoundationDB

#120

Post by StreamBright »

wwilson wrote: post #2

This is INCREDIBLE news! FoundationDB is the greatest piece of software I’ve ever worked on or used, and an amazing primitive for anybody who’s building distributed systems. The short version is that FDB is a massively scalable and fast transactional distributed database with some of the best testing and fault-tolerance on earth[1]. It’s in widespread production use at Apple and several other major companies. But the…

Fantastic talk! Very engaging and insightful.
Post reply on HN