321–330 of 453 posts
This is INCREDIBLE news! FoundationDB is the greatest piece of software I’ve ever worked on or used, and an amazing primitive for anybody who’s building distributed systems. The short version is that FDB is a massively scalable and fast transactional distributed database with some of the best testing and fault-tolerance on earth[1]. It’s in widespread production use at Apple and several other major companies. But the…
I went to the same high school as the founders[1]. They were about the 2 best software engineers in a school with a LOT of very smart software engineers. Another pair founded Yext, which went public last year. I still consider that school the group with the highest concentration of raw brain power I've ever been a part of. I'm probably a 1% engineer, been hired by M$, FB, and Google. These guys were light years ahead…
I can see everyone's extremely happy about this, which is great. As someone who's never used it, I'd like to know more about FoundationDB and how it compares to other offerings such as MySQL or Postgres, and which use cases is it most suited to. I would especially love to hear the thoughts of those with direct experience of using Foundation DB. Thanks!
Earlier quoted context omitted.
"but we have lost machines, connectivity, seen kernel panics, EBS failures, SSD failures, etc., your usual day in AWS " <=== This I wish more people realized that is a day to day reality if you are in AWS at scale.
As I understand it, it's like that everywhere at scale, not just on AWS, it being a property of operating at scale. Or are you saying that AWS is particularly unreliable at scale?
Earlier quoted context omitted.
I am one of the designers of probably the best known metadata storage engine for a distributed filesystem, hopsfs - www.hops.io. When I looked at FoundationDB before Apple bought you, you supported transactions - great. But we need much more to scale. Can you tell me which of the following you have: row-level locks partition-pruned index scans non-serialized cross-partition transactions (that is, a transaction coordi…
> best known metadata storage engine for a distributed filesystem, hopsfs Not even close. I don't even see anything I'd call a filesystem mentioned on your web page. I missed FAST this year, and apparently you had a paper about using Hops as a building block for a non-POSIX filesystem - i.e. not a filesystem in my and many others' opinion - but it's not clear whether it has ever even been used in production anywhere…
Earlier quoted context omitted.
I'm not familiar with FDB but what you say sounds almost too good to be true. Can I use it to implement the Google Datastore api? I'm trying for years to find a suitable backend so that I can leave the Google land. Everything I tried either required a schema or lacked transactions or key namespaces.
Honest question, does MongoDB not work for this?
FoundationDB is now licensed under Apache 2, which is a much more permissive license, so most companies' open source policies allow it.
Earlier quoted context omitted.
It has the ability to watch keys so building a notification system on top of it is pretty easy. Really only limited by your imagination.
In addition to single-key asynchronous watches, there are also versionstamped ops (for maintaining your own, sophisticated log in a layer) and configurable key range transaction logging (but see the caveats in my other post on the topic). I'm not sure it has every feature it will ever need in this area, but it's a pretty good starting point for building "reactive" stuff.
Earlier quoted context omitted.
It's more like what CockroachDB or TiDB use underneath. They all are suited for local clusters, but cannot perform well enough replicating across multiple far away datacenters over public internet, latency trade offs would be unbearable. Spanner is a bit different, with Google's fancy clocks and fancy networks, it can get better latency trade offs that might satisfy more applications. Citus I can't remember, but if C…
CockroachDB (and I assume TiDB as well) can have the same properties and same awesome latencies as Spanner, if you have the same hardware. So if you actually put down your own fiber, and install GPS clocks in each rack, you’ll be able to enjoy the same results.
Noticed the Visual Studio project files. Was Windows a primary development platform for foundationdb?
Nope, the actor compiler is down with mono and your need windows to compile the windows client. When we were independently working on it, we just ditched the windows code completely (we are a mac/linux shop)