Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple open-sources FoundationDB

foundationdb.org

Post reply on HN
Print view View original

201–210 of 453 posts

Re: Apple open-sources FoundationDB

#201

Post by qaq »

panghy wrote: post #182
panghy wrote: post #161

We (Wavefront) has been operating petabyte scale clusters for the last 5 years with FoundationDB (we got the source code via escrow) and we are super excited to be involved in the opensourcing of FDB. We have operated over 50 clusters on all kinds of aws instances and I can talk about all the amazing things we have done with it. https://www.wavefront.com/wavefront-foundationdb-open-source...

We basically replaced mySQL, Zookeeper and HBase with a single KV store that supports transactions, watches, and scales. It's not a trivial point that you can just develop code against a single API (finally Java 8 CompletableFutures) and not have to set up a ton of dependencies when you are building on top of FDB. We are (obviously) experts at monitoring FoundationDB with Wavefront and we hope to release the metric h…

"but we have lost machines, connectivity, seen kernel panics, EBS failures, SSD failures, etc., your usual day in AWS " <=== This I wish more people realized that is a day to day reality if you are in AWS at scale.

Re: Apple open-sources FoundationDB

#202

Post by wll »

voidmain wrote: post #187
wll wrote: post #183

Earlier quoted context omitted.

While that’s perfect to shield from orbital laser cannons, is active/active geo-independent replication possibile?

Well, if you want ACID then you are going to have to pay for at least one geographic round trip per committed transaction. (So why not go active/passive, and have at least one of your datacenters be fast?) But what if you have different pieces of data and you want them to be fast in different datacenters? I think a great solution to this can be layered on top of multiple FoundationDB clusters, each using the satellit…

> So why not go active/passive, and have at least one of your datacenters be fast?

While local writes would stay fast, wouldn’t active/passive see higher-latency non-local writes than Spanner or Fauna’s (assuming a NAM-EUR-ASIA topology)?

I agree with and do appreciate the multiple FoundationDB clusters suggestion.

Re: Apple open-sources FoundationDB

#203

Post by voidmain »

monkeydust wrote: post #147

Can someone explain to business / product guy why this is so exciting? (honestly keen to understand)

Well, it's infrastucture. Moreover, it's infrastructure for infrastructure! So if that sounds super boring you don't have to be excited :-)

But I would tell the story something like this: state storage is the root of (almost) all operational evil. It's very easy to make a system reliable if it's totally stateless. Even most bugs can be lived with if the worst you have to do is restart a service and carry on! But to do anything interesting you have to store state somewhere, and you have to modify that state concurrently without screwing it up.

And the many challenges of operating stateful systems are greatly multiplied if you have a lot of different ones. For example, if you have a datacenter outage and some but not all of your stateful systems deal with it correctly, probably your application as a whole is still down.

So as one more stateful system, does FoundationDB just make that worse? Well, FoundationDB is designed specifically to be a foundation for many very different stateful systems - not just different kinds of databases but things like search engines or message queues that you normally don't think of in the same category. So that almost any system can map to it efficiently, it has a lowest common denominator data model (key/value) and the highest possible guarantees in terms of concurrency control. And you can run diverse systems supporting an application on the same FoundationDB cluster, or on different clusters with the same exact operational requirements.

Some few users of FoundationDB have been able to get the benefits of this vision, consolidating lots of different stuff into a single, operationally desirable system. But for more people to be able to, not just does the key/value store have to be available to them, but also lots of stuff has to be built on top of it. By releasing FoundationDB under a very liberal open source license, Apple has hopefully made that possible. In the long run, hopefully it will make all server-side computing more reliable.

Also, it's a really good key/value store, if you happen to need one of those!

Re: Apple open-sources FoundationDB

#205

Post by shawn-butler »

SubuSS wrote: post #110

This is great news, when I was with dynamo, FoundationDB was the other green shore for me :). They did so many things so well. A tiny bit of caution for folks trying to run systems like this though: It is frigging hard at any reasonable scale. The whole thing might be documented / OSS and what not, but very soon you are going to run into deep enough problems that's going to require very core knowledge to debug, energ…

This sounds like a business opportunity to me rather than a cautionary note.

I remember all the people who bashed Apple when they acquired FoundationDB. I hope they are appropriately ashamed now.

Re: Apple open-sources FoundationDB

#206

Post by ece »

wwilson wrote: post #2

This is INCREDIBLE news! FoundationDB is the greatest piece of software I’ve ever worked on or used, and an amazing primitive for anybody who’s building distributed systems. The short version is that FDB is a massively scalable and fast transactional distributed database with some of the best testing and fault-tolerance on earth[1]. It’s in widespread production use at Apple and several other major companies. But the…

Apple also uses HBase for Siri I believe, what are some of the cluster sizes that FoundationDB scales to? Could it be used to replace HBase or Hadoop?

I was in attendance at your talk, and thought it was one of the best at the conference. Apple I think broke some hearts completely going closed-source for a while, but glad to see them open sourcing a promising technology.

Re: Apple open-sources FoundationDB

#207

Post by voidmain »

jamesblonde wrote: post #179
voidmain wrote: post #151

Earlier quoted context omitted.

It's somewhat hard to answer your questions because the architecture (and hence, terminology) of FoundationDB is a little different than I think you are used to. But I will give it a shot. FoundationDB uses optimistic concurrency, so "conflict ranges" rather than "locks". Each range is a (lexicographic) interval of one or more keys read or written by a transaction. The minimum granularity is a single key. FoundationD…

Thanks for the detailed answer. Is it actually serializable isolation - does it handle write skew anomalies ( https://en.wikipedia.org/wiki/Snapshot_isolation )? Most OCC systems I know have only snapshot isolation. Systems that sound closest to FoundationDB's transaction model that i can think of are Omid ( https://omid.incubator.apache.org/ ) and Phoenix ( https://phoenix.apache.org/transactions.html ). They both s…

Yes, it's really serializable isolation. The real kind, not the "doesn't exhibit any of the named anomalies in the ANSI spec" kind. We can selectively relax isolation (to snapshot) on a per-read basis (by just not creating a conflict range for that read).

I tried to explain distributed resolution elsewhere in the thread.

I believe our approach to indices pretty much totally dominates per-partition indexing. You can easily maintain the secondary indexes you describe; I don't understand your objection.

Re: Apple open-sources FoundationDB

#209

Post by ianamartin »

ccifuentes wrote: post #136
wwilson wrote: post #2

This is INCREDIBLE news! FoundationDB is the greatest piece of software I’ve ever worked on or used, and an amazing primitive for anybody who’s building distributed systems. The short version is that FDB is a massively scalable and fast transactional distributed database with some of the best testing and fault-tolerance on earth[1]. It’s in widespread production use at Apple and several other major companies. But the…

Do we really need another rookie database system? To be honest, Postgres and MongoDB is all you need to achieve any project. I respect hobby projects though, and if that’s the case then great!

Postgres operates great as a document store, btw. You don’t really need mongo at all. And if you need to distribute because you’ve outgrown what you can do one a single postgres node, you don’t want to use mongo anyway.

If you’ve read any of the comments or have been following the project, it should be pretty obvious that this is far from rookie.

This is a game changer, not a hobby project. This is the first distributed data store that offers enough safety and a good enough understand of CAP theorem trade offs that it can be safely used as a primary data store.

Re: Apple open-sources FoundationDB

#210

Post by e12e »

aseipp wrote: post #189
wwilson wrote: post #76

Earlier quoted context omitted.

I think my favorite part of the build process was when we frobnicated libstdc++: https://github.com/apple/foundationdb/blob/master/Makefile#L... And then there were the hijinks we went through to build a cross-compiler with modern gcc and ancient libc (plus the steps to make sure no dependency on later glibc symbols snuck in): https://github.com/apple/foundationdb/blob/master/build/link... Ahh... now that was a build…

I saw this in the Makefiles, and -- ah, the life of distributing proprietary Linux software. At one point in a prior job, we just replaced our build system with a wrapper Makefile that 'chroot'ed into a filesystem image that was a snapshot of one of our build machines, since it was so difficult to set up. This meant we (developers) had easier system updates, security upgrades, etc. That was just the tip of the iceber…

Now you can "... be beautiful and terrible as the Morning and the Night! Fair as the Sea and the Sun and the Snow upon the Mountain! Dreadful as the Storm and the Lightning! Stronger than the foundations of the earth. All shall love [you] and despair!" with a build inside docker; the host can run a modern kernel ; your image can run Slackware 0.1 ;-)

(with apologies to Tolkien)

Post reply on HN