May you do good and not evil.
May you find forgiveness for yourself and forgive others.
May you share freely, never taking more than you give.
from their source code blessing. notes.
Apple open-sources FoundationDB
231–240 of 453 posts
Re: Apple open-sources FoundationDB
#232Earlier quoted context omitted.
It is serializable and totally uncompromising. Philosophically pretty much everything defaults to the safest possible thing. It can't tolerate N failures from N+1 nodes . It can tolerate N failures with N+1 copies of your data . In a big cluster you have plenty of nodes but storing everything 5 times to tolerate 2 failures is really expensive.
> It can't tolerate N failures from N+1 copies of your data Sorry I got the terminology wrong, but that's a distinction without a difference. If it can tolerate N failures from N+1 copies, that means a network partition would allow any one copy to continue chugging along making changes by itself. You have two options: consistency is dropped and you downgrade to eventually consistent (at best), or availability is drop…
Maybe not all data is replicated across all zones, so zones provide consensus but replicas provide data. Basically you have witness with enough information to provide consistency but without the full data store.
Re: Apple open-sources FoundationDB
#233This is INCREDIBLE news! FoundationDB is the greatest piece of software I’ve ever worked on or used, and an amazing primitive for anybody who’s building distributed systems. The short version is that FDB is a massively scalable and fast transactional distributed database with some of the best testing and fault-tolerance on earth[1]. It’s in widespread production use at Apple and several other major companies. But the…
Apple also uses HBase for Siri I believe, what are some of the cluster sizes that FoundationDB scales to? Could it be used to replace HBase or Hadoop? I was in attendance at your talk, and thought it was one of the best at the conference. Apple I think broke some hearts completely going closed-source for a while, but glad to see them open sourcing a promising technology.
Re: Apple open-sources FoundationDB
#234May you do good and not evil. May you find forgiveness for yourself and forgive others. May you share freely, never taking more than you give. from their source code blessing. notes.
That's from sqlite. (Awesome tech, awesome license).
Related snippet from the "Distinctive Features Of SQLite" page[1] from the sqlite project:
The source code files for other SQL database engines typically begin with a comment describing your legal rights to view and copy that file. The SQLite source code contains no license since it is not governed by copyright. Instead of a license, the SQLite source code offers a blessing:
May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give.
Re: Apple open-sources FoundationDB
#235This is INCREDIBLE news! FoundationDB is the greatest piece of software I’ve ever worked on or used, and an amazing primitive for anybody who’s building distributed systems. The short version is that FDB is a massively scalable and fast transactional distributed database with some of the best testing and fault-tolerance on earth[1]. It’s in widespread production use at Apple and several other major companies. But the…
I echo what wwilson has said. I work at Snowflake Computing, we're a SQL analytics database in the cloud, and we have been using FoundationDB as our metadata store for over 4 years. It is a truly awesome product and has proven to be rock-solid over this time. It is a core piece in our architecture, and is heavily used by all our services. Some of the layers that wwilson is talking about, we've built them. metadata st…
Where is the object-mapping-layer ? Inside the db in c++ or a service (different process etc) outside of it.
Re: Apple open-sources FoundationDB
#236I went to the same high school as the founders[1]. They were about the 2 best software engineers in a school with a LOT of very smart software engineers. Another pair founded Yext, which went public last year. I still consider that school the group with the highest concentration of raw brain power I've ever been a part of. I'm probably a 1% engineer, been hired by M$, FB, and Google. These guys were light years ahead…
I went to UVA and I think about half of the engineering school was from TJ (including 3 of my roommates). :) I can't think of any superior public high school (and I went to a different Governor's School in Virginia myself) anywhere, due to the amazingly large and deep talent pool TJ pulls from. Nothing like it exists in the Bay Area, that's for sure!
Re: Apple open-sources FoundationDB
#237Earlier quoted context omitted.
Would it be possible to build a tree DB on top of it like MonetDB/Xquery? I always wondered why XML databases never took off, I've never seen anything else quit as powerful. Document databases if du jour seem comparatively lame.
They had a document layer that wasn't released that was MongoDB compatible — so you can do this.
MongoDB is just a database with less features than a SQL database. An XML/XQuery database is fundamentally different, so I figured if FoundationDB layers are really so powerful, they might be able to model a tree DB as well.
Re: Apple open-sources FoundationDB
#238Earlier quoted context omitted.
I echo what wwilson has said. I work at Snowflake Computing, we're a SQL analytics database in the cloud, and we have been using FoundationDB as our metadata store for over 4 years. It is a truly awesome product and has proven to be rock-solid over this time. It is a core piece in our architecture, and is heavily used by all our services. Some of the layers that wwilson is talking about, we've built them. metadata st…
Where is the object-mapping-layer ? Inside the db in c++ or a service (different process etc) outside of it.
Latter. It's a Java library used by our services.
Re: Apple open-sources FoundationDB
#239Re: Apple open-sources FoundationDB
#240This is great news, when I was with dynamo, FoundationDB was the other green shore for me :). They did so many things so well. A tiny bit of caution for folks trying to run systems like this though: It is frigging hard at any reasonable scale. The whole thing might be documented / OSS and what not, but very soon you are going to run into deep enough problems that's going to require very core knowledge to debug, energ…
> IMO In the stateful world, operating/running the damn thing is 85% of the work, 15% being active dev. (Stateless world is a little better, but still painful).
The number of newbie engineers who see docker/kubernetes, and think "let me docker run or helm install" a stateful service in a couple of minutes - is mind boggling.
I'll remember this quote when trying to talk sense to them.