Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple open-sources FoundationDB

foundationdb.org

Post reply on HN
Print view View original

341–350 of 453 posts

Re: Apple open-sources FoundationDB

#341

Post by alexis_read »

dbcurtis wrote: post #195
voidmain wrote: post #100

Earlier quoted context omitted.

The fault tolerance is pretty much flawless. You won't be able to get the database "stuck" or see anomalies. But performance is going to suck if you run server nodes over unreliable connections. I have trouble seeing a FoundationDB cluster running on mobile robots as more than a trade show gimmick. Albeit an awesome gimmick. So in summary you should totally do that.

I can see that. There are various vectors to performance. I hear you saying transactions per second would be unimpressive. A couple of less time sensitive applications are: 1. distrubuting information the entire fleet should eventually know, 2. event log aggregation with fine-grained time alignment among nodes. Both are probably silly problems to solve with a database, killing houseflys with sledghammers and all that…

I think Scuttlebutt might be more suitable for you? Extract the text from a conversation on each robot. The offline story is easy here.

The other option would be some sort or CRDT-based system - Antidote perhaps?

Re: Apple open-sources FoundationDB

#342

Post by catwell »

wwilson wrote: post #2

This is INCREDIBLE news! FoundationDB is the greatest piece of software I’ve ever worked on or used, and an amazing primitive for anybody who’s building distributed systems. The short version is that FDB is a massively scalable and fast transactional distributed database with some of the best testing and fault-tolerance on earth[1]. It’s in widespread production use at Apple and several other major companies. But the…

> In fact, our everyday testing was way more brutal than Jepsen, I gave a talk about it here: https://www.youtube.com/watch?v=4fFDFbi3toc

Unrelated to the original topic, but I had never come across that talk and it is great. I use the same basic approach to testing distributed systems (simulating all non-deterministic I/O operations) and that talk is a very good introduction to the principle.

Re: Apple open-sources FoundationDB

#343

Post by cooervo »

rando444 wrote: post #323
osrec wrote: post #227

I can see everyone's extremely happy about this, which is great. As someone who's never used it, I'd like to know more about FoundationDB and how it compares to other offerings such as MySQL or Postgres, and which use cases is it most suited to. I would especially love to hear the thoughts of those with direct experience of using Foundation DB. Thanks!

Personally I'd be more interested in hearing how this compares to other distributed noSQL implementations like Cassandra.

or CockroachDB

Re: Apple open-sources FoundationDB

#344

Post by cooervo »

jaytaylor wrote: post #87

Firstly: Wow! this is amazing news!!! I'm also kind of confused.. is the single repo complete? What about the SQL Layer [0]? Where is all this stuff in the new GH repo? Or is only the KV part be open-source? Looking forward to some CockroachDB vs. FDB benchmark showdowns :) [0] https://github.com/jaytaylor/sql-layer

also super interested in CockroachDB, but I just can't find enough war stories, or stories of people using it in production...

Re: Apple open-sources FoundationDB

#345

Post by cooervo »

evanweaver wrote: post #164
isoos wrote: post #61

How does it compare to CockroachDB or TiDB?

It's closest to TiDB's key-value layer; a building block for more complex systems. More traditional, monolithic databases like CockroachDB (SQL) or FaunaDB (NoSQL) trade off extensibility for the benefits in performance and operations that come from very tight coupling. In my understanding, FoundationDB's transaction management is closest to FaunaDB's; read/write sets are linearized in memory in preprocessing nodes a…

Wait CockroachDB is monolithic? I though it was distributed.

Re: Apple open-sources FoundationDB

#346

Post by pluma »

nlavezzo wrote: post #92
sunw wrote: post #86

Earlier quoted context omitted.

How is FoundationDB's graph performance/feature set compared to those of other graph and multi-model databases?

FoundationDB at its core is not a graph database. You could build a graph database on top of it, using FoundationDB as a very strong and feature rich storage engine, however you'd like. It would be much simpler to do than building a new (especially a distributed) graph database from scratch.

So FDB's role is more comparable to that of, say, RocksDB or LevelDB than an application-level database like Postgres?

I didn't really pay much attention to Foundation before Apple bought them and am unsure how it fits in the wider database ecosystem.

Re: Apple open-sources FoundationDB

#348

Post by catwell »

manigandham wrote: post #142
noahdesu wrote: post #107

Earlier quoted context omitted.

Not that this really has anything to do with FoundationDB, but why do you say that object storage is a poor substrate for file and block abstractions? There are many high-performance block and file systems built on object storage.

Block level is the lowest form of addressing bytes on devices. Filesystems are an abstraction on top of block devices. Object stores are an abstraction on filesystems. Emulating a low-level layer on a higher-level abstraction (which itself is using this hierarchy) will never match the speed, scale, or reliability of doing it correctly.

> Block level is the lowest form of addressing bytes on devices. Filesystems are an abstraction on top of block devices. Object stores are an abstraction on filesystems.

I don't agree with this, but I think you may be confused because "Object Storage" can mean several different things.

"Object Store" in Ceph (as in RADOS - Reliable Autonomous Distributed Object Store) basically means key-value store. I typically say "blob store" instead to avoid the confusion with more sophisticated systems. It is exposed through a S3-like API. As far as I know, this layer of CEPH is pretty good, and you need a layer like this in most distributed systems anyway.

Ceph provides something called RBD, RADOS Block Device, which exposes a Block Device interface and is implemented on top of RADOS blob storage. It is useful for VM disks and has decent performance because it makes heavy use of the cache.

Some people use filesystems on top of RBD, but as far as I know CephFS itself does not sit on top of RBD. It is not as widely used as RBD because it is pretty recent (first release in 2016). The data is stored in RADOS and the metadata (which is the hardest part in a distributed filesystem) is dealt with by a Metadata Server cluster (MDS). This sounds like a typical distributed filesystem architecture to me, similar to GFS (the MDS replaces the GFS master and RADOS is used instead of chunk servers).

People tend to have a lot of issues with Ceph, but I think this is because:

1) It is used in reasonably large scale production settings where you are going to have issues anyway ;

2) It is not as easy to understand and fine-tune as it should be ;

3) Some people expect it to solve all their issues magically with perfect performance...

4) Some people use filesystems on top of RBD when they should have used CephFS or even direct interfaces to RADOS when possible.

But in general, I think Ceph is an example of a decently architectured complex distributed system.

Re: Apple open-sources FoundationDB

#349

Post by kuschku »

polskibus wrote: post #329
kuschku wrote: post #156

Earlier quoted context omitted.

CockroachDB (and I assume TiDB as well) can have the same properties and same awesome latencies as Spanner, if you have the same hardware. So if you actually put down your own fiber, and install GPS clocks in each rack, you’ll be able to enjoy the same results.

Does CockroachDB use TrueTime or a similar technique?

CockroachDB assumes you use NTP on your servers — but if you actually put GPS clocks into your servers, you could adjust the safety interval it assumes for clock drift down, and it'd behave pretty much the same as Google's.

Re: Apple open-sources FoundationDB

#350

Post by manigandham »

catwell wrote: post #348
manigandham wrote: post #142

Earlier quoted context omitted.

Block level is the lowest form of addressing bytes on devices. Filesystems are an abstraction on top of block devices. Object stores are an abstraction on filesystems. Emulating a low-level layer on a higher-level abstraction (which itself is using this hierarchy) will never match the speed, scale, or reliability of doing it correctly.

> Block level is the lowest form of addressing bytes on devices. Filesystems are an abstraction on top of block devices. Object stores are an abstraction on filesystems. I don't agree with this, but I think you may be confused because "Object Storage" can mean several different things. "Object Store" in Ceph (as in RADOS - Reliable Autonomous Distributed Object Store) basically means key-value store. I typically say…

Sure, and key/value systems are at the similar level of object stores, meaning they are abstractions on filesystems (which are abstractions on block devices). This is the hierarchy.

Using Ceph for block and file access is like using AWS S3 to emulate block devices and filesystems. It'll work, and there is software for it, but it will never be very good. And Ceph is far from S3.

Post reply on HN