141–150 of 453 posts
This is INCREDIBLE news! FoundationDB is the greatest piece of software I’ve ever worked on or used, and an amazing primitive for anybody who’s building distributed systems. The short version is that FDB is a massively scalable and fast transactional distributed database with some of the best testing and fault-tolerance on earth[1]. It’s in widespread production use at Apple and several other major companies. But the…
Earlier quoted context omitted.
There are several distributed storage systems like Ceph and they all have problems. Ceph is not good because it's an object storage system trying to provide block storage and a filesystem on top, which will never work well.
Not that this really has anything to do with FoundationDB, but why do you say that object storage is a poor substrate for file and block abstractions? There are many high-performance block and file systems built on object storage.
Emulating a low-level layer on a higher-level abstraction (which itself is using this hierarchy) will never match the speed, scale, or reliability of doing it correctly.
Bloomberg’s comdb2 was open sourced recently https://github.com/bloomberg/comdb2 - it seems similar, but would be interesting to see comparison.
Earlier quoted context omitted.
I'm not familiar with FDB but what you say sounds almost too good to be true. Can I use it to implement the Google Datastore api? I'm trying for years to find a suitable backend so that I can leave the Google land. Everything I tried either required a schema or lacked transactions or key namespaces.
Honest question, does MongoDB not work for this?
It was a very bad rdbms (since they marketed as a replacement) and a very bad sharded db (marketed too).
Earlier quoted context omitted.
I just watched the demo of 5 machines and 2 getting unplugged. The remaining 3 can form a quorum. What happens if it was 3 and 3? Do they both form quorums?
A subset of the processes in a FoundationDB cluster have the job of maintaining coordination state (via disk Paxos). In any partition situation, if one of the partitions contains a majority of the coordinators then it will stay live, while minority partitions become unavailable.
I hadn't heard of FoundationDB before, so I did some digging into the features: https://apple.github.io/foundationdb/features.html . It seems to claim ACID transactions with serializable isolation, but also says later on that it uses MVCC, slower clients won't slow down operations, and that it allows true interactive queries. I didn't think an MVCC implementation could provide that level of isolation, and I'm not eve…
I'll try to give you a quick introduction. The architecture talk I recorded for new engineers working on the product ran to four or five hours, I think :-). In short, it is serializable optimistic MVCC concurrency. A FDB transaction roughly works like this, from the client's perspective: 1. Ask the distributed database for an appropriate (externally consistent) read version for the transaction 2. Do reads from a cons…
Is this similar to how Software Transactional Memory (STM) is implemented? It sounds very very similar indeed.
This is INCREDIBLE news! FoundationDB is the greatest piece of software I’ve ever worked on or used, and an amazing primitive for anybody who’s building distributed systems. The short version is that FDB is a massively scalable and fast transactional distributed database with some of the best testing and fault-tolerance on earth[1]. It’s in widespread production use at Apple and several other major companies. But the…
I am one of the designers of probably the best known metadata storage engine for a distributed filesystem, hopsfs - www.hops.io. When I looked at FoundationDB before Apple bought you, you supported transactions - great. But we need much more to scale. Can you tell me which of the following you have: row-level locks partition-pruned index scans non-serialized cross-partition transactions (that is, a transaction coordi…
This is INCREDIBLE news! FoundationDB is the greatest piece of software I’ve ever worked on or used, and an amazing primitive for anybody who’s building distributed systems. The short version is that FDB is a massively scalable and fast transactional distributed database with some of the best testing and fault-tolerance on earth[1]. It’s in widespread production use at Apple and several other major companies. But the…
Do we really need another rookie database system? To be honest, Postgres and MongoDB is all you need to achieve any project. I respect hobby projects though, and if that’s the case then great!
Earlier quoted context omitted.
It is a genius move from Apple. I just wish they'd apply this logic to a lot of their other stuff.
~~How is it different from when Apple acquired the then-open-source FoundationDB (and shut down public access)? They could have just kept it open source back then.~~ EDIT: My bad, looks like FoundationDB wasn't fully open-source back then.