Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

RethinkDB Postmortem

github.com

Post reply on HN
Print view View original

211–220 of 270 posts

Re: RethinkDB Postmortem

#211

Post by pandeiro »

flukus wrote: post #69
yummyfajitas wrote: post #56

I do find it truly sad how anti-intellectualism is pervading our industry and causing good products to die. Nowadays, a significant number of the hottest new technologies offer literally nothing of value beyond "low learning curve". It's my current belief that tech culture has lost nearly everything that once made us great. Think about some examples. MongoDB is nothing but the object databases of the 80's, rebranded.…

Don't forget the proliferation of electron so no one has to learn a new language and desktop toolsets. Just for kicks I wrote a desktop todo list in GTK and C, not tools I had extensive knowledge of ever and nothing recently. It was a breath of fresh air how simple it was compared to doing the same in modern frameworks. A single ~30 line function to create the gui and the rest held together with a couple of function…

Did you blog/gist that example, by any chance?

Re: RethinkDB Postmortem

#212

Post by joshberkus »

Plugawy wrote: post #106
jhchen wrote: post #89

Earlier quoted context omitted.

Why is optimizing for SSDs nonsense? Empirically this seems correct since RethinkDB themselves pivoted away from this but curious about the technical explanation.

coffeemug explained it himself in a podcast where he was interviewed about Rethinkdb (I don't remember the name of the podcast). Basically it boiled down to the fact that all existing databases (at that time) could be tuned with very little effort to take advantage of SSDs so that differentiating factor flew through the window

Exactly.

Re: RethinkDB Postmortem

#213

Post by SamReidHughes »

dkersten wrote: post #168
jhchen wrote: post #89

Earlier quoted context omitted.

Why is optimizing for SSDs nonsense? Empirically this seems correct since RethinkDB themselves pivoted away from this but curious about the technical explanation.

RethinkDB themselves pivoted away from this I believe the storage backend remained largely the same (ie current RethinkDB, IIRC, is still optimised for SSD). Maybe I'm mistaken.

It's the same storage backend (well, modified quite a bit) and optimizing for SSD's isn't really the goal anymore, especially since things are now stored in a file and not the whole block device, and O_DIRECT is turned off.

Re: RethinkDB Postmortem

#214

Post by andrewbarba »

ryanbrunner wrote: post #210
andrewbarba wrote: post #199

Earlier quoted context omitted.

I was a huge advocate of RethinkDB, absolutely love what Slava and the team were able to accomplish, but I am having a really hard time digesting that "Correctness", "Simplicity of the interface" and "Consistency" were the issue here. Example: Realm. I can only imagine what the team at Realm is thinking after going through comments and the post-mortem. Realm was an open-source mobile first database for the past 3 yea…

I'm not familiar with Realm, but after a brief glance at their site: Yes, mobile in this context is very different. Or rather, local is very different. Building a database that resides on a local device and is primarily accessed by a single user is an entirely different class of problem than one that is on a server. Usually most of the hard problems in building a database system involves dealing with multiple people…

Absolutely, in no way am I saying their cloud offering is successful. What I am saying is they have shown you can be successful with the very same ideals and goals that the RethinkDB team had, with a very similar developer-minded audience. The biggest factor here is obviously the platforms their database runs on - Mobile (or local as you put it nicely) vs cloud.

Re: RethinkDB Postmortem

#215

Post by JohnBooty »

andrewbarba wrote: post #199
Perceptes wrote: post #9

I really appreciate the deep introspection in this post. We can all learn a lot from it. I'm one of the (possible minority) of HN users that doesn't care about startups and business, so for me the takeaway was about how choosing quality and correctness over speed works out in the real world. It seems that you really can't take the idealist approach to quality that RethinkDB took. The world just isn't willing to wait,…

I was a huge advocate of RethinkDB, absolutely love what Slava and the team were able to accomplish, but I am having a really hard time digesting that "Correctness", "Simplicity of the interface" and "Consistency" were the issue here. Example: Realm. I can only imagine what the team at Realm is thinking after going through comments and the post-mortem. Realm was an open-source mobile first database for the past 3 yea…

> Is mobile different? Is it easier?

It's easier to get adoption for mobile, I'd think.

Database component designed for use by mobile applications: something you base an app around

Database server: something you base your business around

Re: RethinkDB Postmortem

#216

Post by SparkyMcUnicorn »

jwr wrote: post #104

I am using RethinkDB in PartsBox ( https://partsbox.io/ ). I initially picked it because I wanted a JSON document database that has a correctly implemented distributed story. But later on it turned out that the realtime-push (changefeeds) functionality is actually fundamental for me. I used it to build an app where every open sessions gets instant updates. Right now I don't know what other database I could use, becau…

I could be wrong but the only other database I've been able to find, with the ability to subscribe to events, is GunDB (http://gun.js.org/).

The only other option is to put something in front of a DB like FeathersJS, or deepstream.io

RethinkDB is a breath of fresh air and I really hope the community will be able to keep it alive.

Re: RethinkDB Postmortem

#217

Post by lobster_johnson »

zigzigzag wrote: post #158
lobster_johnson wrote: post #82

Earlier quoted context omitted.

Exactly this. RethinkDB's value proposition wasn't really that attractive compared to Postgres. Replication and a nice UI, sure, those are nice. But Postgres has fantastic single-host performance, and Rethink is not yet in a position to compete there, except in some horizontally scaled use cases. Rethink doesn't even have transactions, making it potentially useless for some things that require strict atomicity over a…

Firstly, to the OP, that is an excellent post-mortem. Introspective, goes deep, generally confidence inspiring even though a part of it is literally "we were incompetent and didn't know it", well, lessons learned. I'd like to add some colour, although I've never used RethinkDB so take it all with a large pile of salt. lobster_johnson says: > Rethink doesn't even have transactions The post-mortem dwells heavily on how…

That's a good point. Competing with MongoDB is difficult because "correctness" alone is not a selling point. It's the "worse is better" phenomenon as a marketing challenge.

That said, I don't think lack of correctness itself is a design goal. I think it's accidental, or at least an intermediate state before we have something better.

Case in point: I'm currently working on an open-source (it will be released soon) data store that uses JSON to express "documents" of data, but also has schemas, joins and transactions. It uses PostgreSQL internally, but completely hides the underlying storage. The developers who are currently using it for actual work are pleasantly surprised about how nice it is to work with: You get to work with rich, structured data in its natural shape without needing some kind of ORM to mediate between an app's data model and the database's, and at the same time it's very strictly validated without being pushy (it has gradual typing: you can start out schemaless and then tighten the schema). When we designed it, we wanted to build something that reflected how people actually use a database, and I think we're achieving that. (I'm looking forward to make it public and share it on HN.)

So it's certainly possible to have the best of both worlds. It's just that the direction that the world moved in with "NoSQL" temporarily moved us off course for a little while. It started out as a natural reaction to scaling issues, and in that sense I think it might have been a necessary step while people figured out what they were doing. We're seeing the same evolution in the ways of people's thinking with regard to dynamically/statically typed languages, too.

Re: RethinkDB Postmortem

#218

Post by drdaeman »

flukus wrote: post #62
manigandham wrote: post #35

Earlier quoted context omitted.

Postgres neither scales vertically or horizontally. v9.6 finally brought some parallel scans but is still limited in scope and far behind the commercial databases. Same with replication and failover, although there are decent 3rd party extensions to get it working.

The vast majority of us probably won't ever need that scalability anyway. Most places I've seen that needed something more scalable needed to just stop being so retarded with the tools they already had.

As for the clustering in general... Scalability may be not important, but replication and failover should be. I believe, every place that had existed long enough must've had theirs "ouch, our master DB host went down" moment.

I wasn't able to set it up properly some years ago and settled down with warm standby replica (streaming + WAL shipping) and manual failover. Automating it (and even automating the recovery when old master is back online and healthy) was certainly possible - and I got the overall idea how it should operate. But the effort required to set it up was just too big for me, so I decided it wasn't worth the hassle and settled down with "uh, if it dies, we'll get alerted and switch to the backup server manually" scenario.

Having something that goes in line of "you start a fresh server, tell it some other server address to peer with, and it does the magic (with exact guarantees and drawbacks noted in documentation)" would be really awesome. RethinkDB is just like that. PostgreSQL - at least 8.4 - wasn't, unless I've really missed something. I haven't yet checked newer versions' features in any detail, so not sure about 8.5/8.6.

Re: RethinkDB Postmortem

#219

Post by pavlov »

andrewbarba wrote: post #199
Perceptes wrote: post #9

I really appreciate the deep introspection in this post. We can all learn a lot from it. I'm one of the (possible minority) of HN users that doesn't care about startups and business, so for me the takeaway was about how choosing quality and correctness over speed works out in the real world. It seems that you really can't take the idealist approach to quality that RethinkDB took. The world just isn't willing to wait,…

I was a huge advocate of RethinkDB, absolutely love what Slava and the team were able to accomplish, but I am having a really hard time digesting that "Correctness", "Simplicity of the interface" and "Consistency" were the issue here. Example: Realm. I can only imagine what the team at Realm is thinking after going through comments and the post-mortem. Realm was an open-source mobile first database for the past 3 yea…

It's probably too early to say that Realm made it work.

They've raised a roughly similar amount of VC money as RethinkDB did. Their last raise was a few years ago, before Parse's shutdown, so I suspect that investors are more antsy about this space today.

The fact that Realm recently entered the cloud space sounds similar to RethinkDB's trajectory too... Of course I hope Realm makes it, it's a great product. But I'm worried that they're trapped in the VC-funded model like RethinkDB was.

Post reply on HN