Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple open-sources FoundationDB

foundationdb.org

Post reply on HN
Print view View original

101–110 of 453 posts

Re: Apple open-sources FoundationDB

#101

Post by infogulch »

voidmain wrote: post #72
misterbowfinger wrote: post #43

I hate to be that person, but when I hear "ACID transactions in a distributed database", I hear Citus/Spanner/CockroachDB. I'm positive that Citus & Spanner are quite different from FoundationDB, but I have no idea how. Googling didn't help much. Can someone provide an overview of the differences?

I think Citus is not really ACID. Spanner (and to an extent its less mature OSS descendants Cockroach and TiKV) has more comparable goals, but is fairly different architecturally. For example, FoundationDB only requires N+1 replicas instead of 2N+1 to achieve N failure tolerance (even lots of databases with much weaker guarantees are in the latter category!), doesn't trust clocks at all, doesn't lose performance when…

Wait how can it be ACID if it can tolerate N failures from N+1 nodes? Doesn't that kill consistency almost by definition? What level of isolation does it support? Snapshot? Serializable?

Re: Apple open-sources FoundationDB

#102

Post by X-Istence »

wwilson wrote: post #60
thefounder wrote: post #53

Earlier quoted context omitted.

I'm not familiar with FDB but what you say sounds almost too good to be true. Can I use it to implement the Google Datastore api? I'm trying for years to find a suitable backend so that I can leave the Google land. Everything I tried either required a schema or lacked transactions or key namespaces.

As an existence proof: before the acquisition we built an ANSI SQL database and a wire-compatible clone of the MongoDB API. I see no reason you wouldn't be able to implement Datastore. In fact here's a public source claiming that Firestore (which I believe is its successor) is implemented on top of Spanner: https://www.theregister.co.uk/2017/10/04/google_backs_up_fir...

Is the wire-compatible clone of the MongoDB API available?

Re: Apple open-sources FoundationDB

#103

Post by voidmain »

infogulch wrote: post #101
voidmain wrote: post #72

Earlier quoted context omitted.

I think Citus is not really ACID. Spanner (and to an extent its less mature OSS descendants Cockroach and TiKV) has more comparable goals, but is fairly different architecturally. For example, FoundationDB only requires N+1 replicas instead of 2N+1 to achieve N failure tolerance (even lots of databases with much weaker guarantees are in the latter category!), doesn't trust clocks at all, doesn't lose performance when…

Wait how can it be ACID if it can tolerate N failures from N+1 nodes? Doesn't that kill consistency almost by definition? What level of isolation does it support? Snapshot? Serializable?

It is serializable and totally uncompromising. Philosophically pretty much everything defaults to the safest possible thing.

It can't tolerate N failures from N+1 nodes. It can tolerate N failures with N+1 copies of your data. In a big cluster you have plenty of nodes but storing everything 5 times to tolerate 2 failures is really expensive.

Re: Apple open-sources FoundationDB

#104

Post by [deleted] »

kbumsik wrote: post #77
nlavezzo wrote: post #69

Earlier quoted context omitted.

Wavefront's co-founder just tweeted this: https://twitter.com/panghy/status/987022825457266689 Finally it's out! @WavefrontHQ managaes petabyte scale clusters with #foundationdb today!

I'm confused. How they could use FoundationDB when Apple aquired it a long time ago? Did apple sell the software to other companies?

[deleted]

Re: Apple open-sources FoundationDB

#105

Post by wwilson »

X-Istence wrote: post #102
wwilson wrote: post #60

Earlier quoted context omitted.

As an existence proof: before the acquisition we built an ANSI SQL database and a wire-compatible clone of the MongoDB API. I see no reason you wouldn't be able to implement Datastore. In fact here's a public source claiming that Firestore (which I believe is its successor) is implemented on top of Spanner: https://www.theregister.co.uk/2017/10/04/google_backs_up_fir...

Is the wire-compatible clone of the MongoDB API available?

It doesn't look like Apple open-sourced the Document Layer, which is a slight bummer. But I echo what Dave said below: what we got is incredible, let's not get greedy!

Also TBH now that I don't have commercial reasons to push interop, if I write another document database on top of FDB, I doubt I'd make it Mongo compatible. That API is gnarly.

Re: Apple open-sources FoundationDB

#106

Post by voidmain »

mzeier wrote: post #99
voidmain wrote: post #59

Earlier quoted context omitted.

The storage engine is and always was a fairly heavily modified asynchronous version of sqlite's btree. It's been extremely reliable, which was always our top priority, and the performance isn't bad. But honestly when there was a problem with it our development velocity improving it wasn't great. It's super easily pluggable[1], so now that it is open source people can experiment with other engines. I think there is a…

Truthfully at Wavefront we've taken the json status directly into telegraf. Plus a bunch of python tooling to massage additional telemetry on a clusters health (coordinator reachability for example). Plus even more tooling (mostly Ansible) for managing large fleets.

I saw @spullara do pretty neat stuff with our log files in Wavefront.

Will you guys think about open sourcing tooling? Apple is realistically never going to do that stuff.

Re: Apple open-sources FoundationDB

#107

Post by noahdesu »

manigandham wrote: post #96
xvilka wrote: post #90

These days most people know Hadoop for a distributed storage. In my opinion, though CEPH [1] has the bigger potential. [1] https://ceph.com/

There are several distributed storage systems like Ceph and they all have problems. Ceph is not good because it's an object storage system trying to provide block storage and a filesystem on top, which will never work well.

Not that this really has anything to do with FoundationDB, but why do you say that object storage is a poor substrate for file and block abstractions? There are many high-performance block and file systems built on object storage.

Re: Apple open-sources FoundationDB

#108

Post by Rafuino »

Found this section on performance very interesting.

https://apple.github.io/foundationdb/performance.html#throug...

I wonder what the SSD engine performance would look with NVMe standard NAND or an Optane SSD instead of SATA. Any FoundationDB guys/gals on this thread able to comment?

Another Q: what's more commonly used in current FoundationDB deployments: memory engine or storage engine?

Re: Apple open-sources FoundationDB

#109

Post by PeCaN »

julien_c wrote: post #63
thefounder wrote: post #53

Earlier quoted context omitted.

I'm not familiar with FDB but what you say sounds almost too good to be true. Can I use it to implement the Google Datastore api? I'm trying for years to find a suitable backend so that I can leave the Google land. Everything I tried either required a schema or lacked transactions or key namespaces.

Honest question, does MongoDB not work for this?

Honest question, does MongoDB work?

Re: Apple open-sources FoundationDB

#110

Post by SubuSS »

This is great news, when I was with dynamo, FoundationDB was the other green shore for me :). They did so many things so well.

A tiny bit of caution for folks trying to run systems like this though: It is frigging hard at any reasonable scale. The whole thing might be documented / OSS and what not, but very soon you are going to run into deep enough problems that's going to require very core knowledge to debug, energy to deep dive. Both of which you probably don't want to invest your time into. Do evaluate the cloud offerings / supported offerings before spinning these up. Else ensure you have hired experts who can keep this going. They are great as a learning tool, pretty hard as an enterprise solution. I have seen the same issue a ton of times with a bunch of software (redis/kafka/cassandra/mongo...) by now. IMO In the stateful world, operating/running the damn thing is 85% of the work, 15% being active dev. (Stateless world is a little better, but still painful).

Post reply on HN