RethinkDB: why we failed (2017) • Hacker News Board

Re: RethinkDB: why we failed (2017)

#141

Post by drunkpotato » Sat, Mar 13, 2021, 1:20 PM UTC

RedShift1 wrote: post #133

blackrock wrote: post #117

Earlier quoted context omitted.

How’s Snowflake doing? One thing these database companies have going for themselves, is that once a customer company’s data and code is locked in to a particular database, then it becomes very very hard to get out. Not impossible, but just very difficult.

I've heard this line many times but is this actually true? Are there stories or statistics of these things?

I've done several database migrations, and it's true, but not because the data is hard to extract. It's moderately easy to dump out CSV or JSON files to S3 and load them back in to another database (only moderately because you have to deal with fiddly encoding issues). So in that sense the data is easily extracted. Tools like AWS Data Migration Services, while not perfect, can also make this a lot easier.

However, at least three issues make a database migration non-trivial:

1. Custom data types that have to be replicated somehow in the new database; this usually involves figuring out how to parse the output representation in a sensible way.

2. User-defined triggers and functions, or custom database features, that may not be available on the new database.

3. There is usually a lot of infrastructure built on top of a database that can be hard to switch over, like if you've used roles substantially in Postgres for access control. Not to mention any business intelligence tool built on it that tends to have a lot of hand-rolled SQL. For example, Periscope can be database-agnostic, but any queries you write might be customized for one particular database vendor.

So in my view, it's not really data concerns (except case 1); there are a lot of operational concerns that make database migrations hard.

Re: RethinkDB: why we failed (2017)

#142

Post by eternalban » Sat, Mar 13, 2021, 1:22 PM UTC

htsh wrote: post #129

eternalban wrote: post #127

Earlier quoted context omitted.

> All of that said, I work in an innovation space where we throw a lot of things at the wall to see what sticks, and our time-to-prototype is often a few months at most. For us, it has been a godsend, but our situation is rather unique. The very RAD reasons you note regarding Mongo also encourage Mongo Cancer (tm). What is Mongo Cancer you ask? It is the phenomena where Mongo use is manifest from API to home-grown "r…

No, we have migrated prototypes to redshift. You just need to keep your database logic in a separate layer, which is something one should do anyway. I say "I think" because I agree with you that most will not. If you implement mongo such that you will be stuck on mongo, then do not use it. You always want to have a layer that handles your database operations, with any database, but obviously you are safer in SQL-land…

But the technology encourages this type of behavior. And I am not speaking of hypothetical "can happen". I very recently had to deal with this precise issue.

> But really, please consider that many of us have, or are as we speak, losing loved ones to a deadly sickness which you have compared to a poor database implementation.

Sorry for your loss. Use of the word for non-medical concerns is common. Mongo Metastasis works too.

Again disagree regarding "poor database implementation". Trivializing the "database implementation" is precisely Mongo's value proposition and the very reason they got their market share.

Re: RethinkDB: why we failed (2017)

#143

Post by osigurdson » Sat, Mar 13, 2021, 1:25 PM UTC

goatinaboat wrote: post #88

nodesocket wrote: post #25

Earlier quoted context omitted.

I firmly believe that RethinkDB could have been a huge player along with the likes of MongoDB and Elasticsearch if they could have gotten additional funding or focused more on business development. Note: both MongoDB and Elasticsearch are now hugely successful public companies. - Changefeeds were so useful and still to this day not really matched in quality. - Official client libraries were very high quality. I mainl…

RethinkDB could have been a huge player along with the likes of MongoDB MongoDB is a weird one because what they had didn't even meet the most basic requirements of "a database", that you could store something in it and reliably get it back out. But they somehow converted their joke of a product into enough money to buy WiredTiger, rebrand its product as theirs, and profit. It's the ultimate expression of "fake it 't…

Has MongoDB "made it" now or are they still "faking it"? Under what conditions is data lost?

Re: RethinkDB: why we failed (2017)

#144

Post by Chris2048 » Sat, Mar 13, 2021, 1:29 PM UTC

tome wrote: post #98

Yoric wrote: post #96

Earlier quoted context omitted.

Is that a everything-new-is-old description for cloud offerings?

I once saw a fascinating quote that went something like the following: "As computing technology develops it becomes more efficient to take centralised computing resources and distribute them closer to the user. As network technology develops it becomes more efficient to centralise them again. Further advances redistribute and yet further advances recentralise. This pattern has been noticed several times in the histor…

Can you give examples of "Further advances redistribute"?

Also, I wonder if commodity cloud offerings such as AWS will change this?

Re: RethinkDB: why we failed (2017)

#145

Post by blacktriangle » Sat, Mar 13, 2021, 1:39 PM UTC

abraxas wrote: post #34

This is akin to everyone’s dream to start a cozy, atmospheric coffee shop. These types of coffee bars fold very quickly or at best give founders years of servitude below minimum wage. The reason is that the type of behaviour this kind of establishment encourages - lounging, book reading, laptop work is exactly the opposite of the quick serve model that is conducive to high revenue. The customer loves this model, the…

Interesting, I think this is why private clubs used to be a thing and may be the only way to get a sustainable cozy, atmospheric coffee shop. Because really that's what you're selling, not coffee, but atmosphere. And membership is a way to charge for your real product.

Re: RethinkDB: why we failed (2017)

#146

Post by htsh » Sat, Mar 13, 2021, 1:41 PM UTC

eternalban wrote: post #142

htsh wrote: post #129

Earlier quoted context omitted.

No, we have migrated prototypes to redshift. You just need to keep your database logic in a separate layer, which is something one should do anyway. I say "I think" because I agree with you that most will not. If you implement mongo such that you will be stuck on mongo, then do not use it. You always want to have a layer that handles your database operations, with any database, but obviously you are safer in SQL-land…

But the technology encourages this type of behavior. And I am not speaking of hypothetical "can happen". I very recently had to deal with this precise issue. > But really, please consider that many of us have, or are as we speak, losing loved ones to a deadly sickness which you have compared to a poor database implementation. Sorry for your loss. Use of the word for non-medical concerns is common. Mongo Metastasis wo…

I agree that it is very easy to make bad decisions with this product, but I do not think it is unique to Mongo. And it is the role of a good architect or a good team lead to make sure that this does not happen or that developers do not go down these roads.

I would even say the strengths that I have pointed out are precisely what makes it easier to take shortcuts. It gets out of your way, makes collections if they do not exist, allows for different document types in the same collections. It has no guard rails whatsoever.

I think we are in agreement -- one needs to be very careful using this. You can use it and put yourself in a spot where you have to throw out your entire codebase.

Well, me saying "can happen" does not mean it does not happen or that people have not seen it. It is saying the opposite. My point is it does not have to or that it does not always happen.

Developers take shortcuts, often because they do not know better, and you are right that some products make it easier to go down these roads.

I still do not think you want to use medical diseases as metaphors. You would not say "mongo aids" or "mongo alzheimers" or "mongo diabetes." Even if it is common, it is not particularly useful. Metastasis is better.

Re: RethinkDB: why we failed (2017)

#147

Post by hnedeotes » Sat, Mar 13, 2021, 1:46 PM UTC

htsh wrote: post #138

nicoburns wrote: post #135

Earlier quoted context omitted.

Genuine question: What does mongo give you as a JSON store that postgres's JSONB doesn't?

Querying is in pure JS, easy to do stuff programmatically. But if starting new, with a team that did not know Mongo well already, not a real advantage. I still prefer querying something nested in mongo to querying something nested in JSONB, but I can work with either. And back when I started with it, JSONb did not exist. I would say JSONB was also a godsend though as our roadmaps now have a clear path to something we…

I can't follow this through, I'm interested in nosql databases because there are indeed patterns of access where a db as a key value store is helpful.

> Querying is in pure JS, easy to do stuff programmatically.

You mean you do your logic at the application layer? Like if you need X objects and Y objects where Y have an identifier to the X objects they're related to you build them in JS after retrieving them?

What is the difference from getting a json object and doing the "querying" in pure js?

And isn't it easier "programatically" to just join the data at the source?

> But the speed advantage really lies in the early-stage prototype development. If I have to do it in postgres, the amount of code necessary up front, when inevitable schema changes happen, etc. makes mongo preferable until things settle down.

But again, what is the difference of having a record with a column with json, that you just do rodeo dump?

(I think pg even allows you to do all sorts of indexing on inner fields of json if I'm not mistaken?)

Re: RethinkDB: why we failed (2017)

#148

Post by gigatexal » Sat, Mar 13, 2021, 1:59 PM UTC

thu2111 wrote: post #123

gigatexal wrote: post #67

What’s with all of this seemingly whitewashing of history? Slava himself described why it failed. I mean as founder you have to be laser focused. Yet he was too busy trying to win over his “hot new employee” than focusing on the database. There is an archive of his deleted tweets here where he describes it: https://gist.github.com/travisbrown/059310042193a2e143408b05... And then there was an article that seems to be…

Bizarre. I don't know how much of a distraction "I hired a hot girl and then fell in love with her" was, but to me tweeting 40-60 times per day says more about a lack of focus (e.g. 56 times on 30th November 2020 where that tweet came from). The deleted tweets don't go back to 2017 but if that level of tweeting is normal I wonder where he found time to do anything, let alone build a database company. That said, I'm w…

Perhaps I was keying of the wrong thing. I think you’re 100% right.

Re: RethinkDB: why we failed (2017)

#149

Post by iFreilicht » Sat, Mar 13, 2021, 2:01 PM UTC

amelius wrote: post #113

billconan wrote: post #4

is the entire developer tool market bad, or just the db sector is bad? developer tool market seems to be the only one I'm familiar with as an engineer. If I were to start a business, I will probably also do something for this market.

Apparently, you can make a five digit income from selling an IDE colorscheme. https://news.ycombinator.com/item?id=26262989

This sounded ridiculous at first, but after looking at it, I understood the value preposition.

It’s a colorscheme, yes, but you pay for the consistent support of over 160 apps, which is a big deal IMO. Heck, every program under the sun seems to offer a solarized dark theme, but they’re all wrong in different ways and use different accent colors for different things, so I rolled my own in a lot of cases to get an actually consistent appearance. It’s a huge hassle, and just paying 80$ would probably have saved money compared to the hours and days I spent manually theming stuff.

Re: RethinkDB: why we failed (2017)

#150

Post by alberth » Sat, Mar 13, 2021, 2:02 PM UTC

jandrewrogers wrote: post #22

Database startups suffer from an extreme case of an important truism in startups generally: the product you think you are selling is not the product the customer thinks they are buying. No one buys a database per se , they are just a means to some other end. People that love database technology — that would be me — tend to start database companies. It is very difficult to sell a database. It is much, much easier to s…

This is exactly why Oracle in the 90s and 00s starting buying up software Application companies (Peoplesoft, JD Edwards, Hyperion, etc).

And then in the old on-premise world, you sell them both the App and require the use of your database which is an additional sale.

This strategy doesn’t work in a Cloud world though. Since customers are no longer buying the individual components (like the database) but are instead just leasing the finished good. By definition, they don’t care what’s under the hood - unless you’re selling to another software company who’s using your database to make their finished good.