RethinkDB 1.5 released: secondary indexes, soft durability, perf improvements • Hacker News Board

Re: RethinkDB 1.5 released: secondary indexes, soft durability, perf improvements

#41

Post by sergei » Thu, May 16, 2013, 6:43 PM UTC

josegonzalez wrote: post #35

sergei wrote: post #32

Earlier quoted context omitted.

A decent cost based optimizer will be right most of the time, and you have hints for the cases where it falters. Your approach is simply hiding the query optimizer choices inside of the app, making it more fragile. Let's say I have two fields: a and b, both of which have an index. And then I have a query with predicates across both a and b. For example, a = 10 and b = 20. One of the core facilities within an optimize…

Which is good, because now your developers don't just add queries to the application without thinking about performance. You don't have to be a dba to understand how using one index vs another will affect the performance - and thus conversion rate - of your application.

In practice, you end up with apps which dynamically build up predicates in different sections of the code. And when you combine that with many predicates, many tables, and other constraints such as ordering or aggregates, things get complex pretty quick.

Even if you have the best developers who understand all the in an outs of the dataset, re-implementing an optimizer in the app is rarely the right choice.

Re: RethinkDB 1.5 released: secondary indexes, soft durability, perf improvements

#42

Post by josegonzalez » Thu, May 16, 2013, 6:49 PM UTC

dennis82 wrote: post #40

josegonzalez wrote: post #35

Earlier quoted context omitted.

Which is good, because now your developers don't just add queries to the application without thinking about performance. You don't have to be a dba to understand how using one index vs another will affect the performance - and thus conversion rate - of your application.

no, it's categorically not good to make an application more fragile. A weakness restated is not a strength. Every database should have an optimizer, period.

I did not state that an optimizer should not exist for a database - I think thats key actually - but rather that the tradeoff they made this time around was fundamentally good in that - at least for now - it forces the developer to think about application performance.

If that happens to make an application more fragile, I think that is more of a code organization/tooling issue than anything else.

Re: RethinkDB 1.5 released: secondary indexes, soft durability, perf improvements

#43

Post by coffeemug » Thu, May 16, 2013, 6:55 PM UTC

sergei wrote: post #32

jdoliner wrote: post #13

Earlier quoted context omitted.

Hi, I implemented secondary indexes in RethinkDb. This was an intentional choice we made for a couple of reasons. The most pragmatic one was that we don't have an optimizer and we thought that secondary indexes could still be useful to a lot of people without one. And we've found it's always better to ship early and get feedback sooner rather than later. Another reason though is we're not totally sure we're sold on t…

A decent cost based optimizer will be right most of the time, and you have hints for the cases where it falters. Your approach is simply hiding the query optimizer choices inside of the app, making it more fragile. Let's say I have two fields: a and b, both of which have an index. And then I have a query with predicates across both a and b. For example, a = 10 and b = 20. One of the core facilities within an optimize…

slava @ rethink here. I don't think we disagree -- optimizers are unquestionably a good idea and are immensely useful. For us it was a matter of a) giving people the option to specify indexes directly, and b) shipping quickly. We'll implement a proper statistical optimizer in due time, but for the time being we found that the explicit approach gives a lot of people 80% of what they need with 20% of the work on our part.

Re: RethinkDB 1.5 released: secondary indexes, soft durability, perf improvements

#44

Post by btilly » Thu, May 16, 2013, 6:57 PM UTC

sergei wrote: post #32

jdoliner wrote: post #13

Earlier quoted context omitted.

Hi, I implemented secondary indexes in RethinkDb. This was an intentional choice we made for a couple of reasons. The most pragmatic one was that we don't have an optimizer and we thought that secondary indexes could still be useful to a lot of people without one. And we've found it's always better to ship early and get feedback sooner rather than later. Another reason though is we're not totally sure we're sold on t…

A decent cost based optimizer will be right most of the time, and you have hints for the cases where it falters. Your approach is simply hiding the query optimizer choices inside of the app, making it more fragile. Let's say I have two fields: a and b, both of which have an index. And then I have a query with predicates across both a and b. For example, a = 10 and b = 20. One of the core facilities within an optimize…

You are underestimating the necessary design tension. If you expose hints, then you've constrained how your optimizer works because it has to work with those hints. (Oracle, for example, has been fighting this battle for ages.) Once you've built an optimizer, people come to depend on it so it is critical that you get it right.

They will need optimizer eventually, and they know it. But I'm glad that they are not implementing it before they are ready.

Secondly your selectivity estimation point can go either way, and on the whole I don't like it. I've personally experienced the situation where a database recomputes statistics, the CBO decides that a new query plan is needed for a common query, it made a poor choice, and the first that any human hears about it, the site is down.

The problem here is that the risk profile for the application of trying to be smart here is completely backwards. In general, as long as a query is not a bottleneck, I don't care about making it faster. Oh, you made it 2x faster? I didn't mind before and I'm unlikely to even know that you did so. But if you JUST ONCE switch to a bad plan on production without warning, your users WILL notice, they WILL care, and they WILL NOT be happy.

As a developer, I don't care that you make the right choice 95% of the time. I want you to make a predictable choice. Because if you're making a bad choice in development, I've got a window of opportunity to notice and do something about it where nobody cares. But if you randomly make changes on production, every single mistake counts.

Oh, but you say that this just means that you need stored query plans? I agree, and this is an example of why the behavior of the optimizer has to be thought through very carefully before you just throw something out there, people come to depend on it, and then you realize that you have put barriers to thinking of it the way you want to think of it.

Re: RethinkDB 1.5 released: secondary indexes, soft durability, perf improvements

#45

Post by jdoliner » Thu, May 16, 2013, 7:00 PM UTC

sergei wrote: post #32

jdoliner wrote: post #13

Earlier quoted context omitted.

Hi, I implemented secondary indexes in RethinkDb. This was an intentional choice we made for a couple of reasons. The most pragmatic one was that we don't have an optimizer and we thought that secondary indexes could still be useful to a lot of people without one. And we've found it's always better to ship early and get feedback sooner rather than later. Another reason though is we're not totally sure we're sold on t…

A decent cost based optimizer will be right most of the time, and you have hints for the cases where it falters. Your approach is simply hiding the query optimizer choices inside of the app, making it more fragile. Let's say I have two fields: a and b, both of which have an index. And then I have a query with predicates across both a and b. For example, a = 10 and b = 20. One of the core facilities within an optimize…

An optimizer can be right most of the time but it's basically impossible to have it be right all of the time. It really depends on what you're doing as to which approach you prefer. If you absolutely need a query to have a certain performance characteristic then you can't count on the optimizer to get it right a human needs to think through exactly how they want this query to run to ensure it behaves correctly.

There definitely are several use cases where people are willing to sacrifice a risk of misoptimization in exchange for shorter code and we fully intend to support that (it's a complicated project though so we don't now.) However I disagree that having this logic in the app makes it more fragile. Having the precise semantics expressed in the application means you always know exactly how a query is going to be performed. We know this execution plan isn't going to change based on seemingly unrelated properties of the data (as it does with optimizers.) If fragility is the propensity to break then I'd argue having the optimizer in control rather than the developer gives you more fragility than less.

Re: RethinkDB 1.5 released: secondary indexes, soft durability, perf improvements

#46

Post by coffeemug » Thu, May 16, 2013, 7:01 PM UTC

tomjen3 wrote: post #33

prodigal_erik wrote: post #16

Earlier quoted context omitted.

Also http://www.rethinkdb.com/docs/advanced-faq/ . RethinkDB started out as a MySQL driver that was lock-free and did schema updates non-stupidly, but it sounds like they've abandoned ACID and schema enforcement entirely like almost all NoSQL systems (as if what I wanted is faster wrong answers).

I see this tude' among DB guys quite often -- unless the database guarantees unconditionally that it will always give the right answer and that no data is ever lost then it is worse than useless and most be nuked from orbit. Quite frankly that is pretty arrogant. ACID may be required for a banks transactions, but in far the majority of cases it isn't required (yes, even in your business) -- nothing of any consequence…

To be clear, RethinkDB doesn't give wrong answers, and doesn't lose data. ACID isn't as much about right vs wrong answers, as it is about defining what "right answer" means and what constraints you're willing to give up in exchange for performance.

ACID is a great, but unfortunately it isn't free (especially in distributes systems).

Re: RethinkDB 1.5 released: secondary indexes, soft durability, perf improvements

#47

Post by coffeemug » Thu, May 16, 2013, 7:10 PM UTC

dennis82 wrote: post #25

this is a good update and not to rain on their parade, but doesn't this seem market redundant and reinvent-the-wheel?

RethinkDB is definitely late to the party, so this is a great question. Here's how we think about it.

1. Technically, Rethink already does very useful things leading NoSQL contenders don't do. An extremely expressive query language, massive query parallelization, distributed joins, etc. You can't get that anywhere else, and we have lots of features in the pipeline to keep raising the bar.

2. I believe we have a unique take on usability and design that is very valuable to users but isn't present in other products. Rethink is pleasant to set up and (hopefully) pleasant to use. To us, design matters, and we put an inordinate amount of effort to make the product beautiful. It turns out users care about this a lot.

3. I feel like leading NoSQL contenders are somewhat stagnant. There is an enormous amount of innovation that could be done, needs to be done, and isn't being done. We'll continue releasing really useful features that nobody has, and (I think) nobody expects. I think we bring a unique philosophy to product design that's very valuable.

4. Details matter (which is why Rethink is late to the party). Once you outgrow the ten-minute blog stage, a lot of underlying architectural decisions start to really matter, and we took the time to do them right.

TL;DR: Rethink is already really good for building apps on top of it and offers things nobody else does. It will continue getting better. We'd be honored if you took it for a spin!

Re: RethinkDB 1.5 released: secondary indexes, soft durability, perf improvements

#48

Post by sixbrx » Thu, May 16, 2013, 7:12 PM UTC

btilly wrote: post #44

sergei wrote: post #32

Earlier quoted context omitted.

A decent cost based optimizer will be right most of the time, and you have hints for the cases where it falters. Your approach is simply hiding the query optimizer choices inside of the app, making it more fragile. Let's say I have two fields: a and b, both of which have an index. And then I have a query with predicates across both a and b. For example, a = 10 and b = 20. One of the core facilities within an optimize…

You are underestimating the necessary design tension. If you expose hints, then you've constrained how your optimizer works because it has to work with those hints. (Oracle, for example, has been fighting this battle for ages.) Once you've built an optimizer, people come to depend on it so it is critical that you get it right. They will need optimizer eventually, and they know it. But I'm glad that they are not imple…

This accords with our experience with large Oracle databases as well.

We've had a few panics caused by a DBA having updated table statistics when trying to optimize some query. This would occasionally cause sudden, massive changes in the way that other unrelated queries were performed, which queries would not finish and sometimes bring down the database.

These experiences caused us to have to change our procedures and keep the test database data closely in sync with the production data, so we'd know how newly gathered statistics would affect queries. The database is large enough that having testing be a full and recent copy of production is pretty painful. Oracle has since introduced features in 11g that allow pinning query plans, we've yet to try these though.

Re: RethinkDB 1.5 released: secondary indexes, soft durability, perf improvements

#49

Post by jdoliner » Thu, May 16, 2013, 7:16 PM UTC

mfenniak wrote: post #37

jdoliner wrote: post #13

Earlier quoted context omitted.

Hi, I implemented secondary indexes in RethinkDb. This was an intentional choice we made for a couple of reasons. The most pragmatic one was that we don't have an optimizer and we thought that secondary indexes could still be useful to a lot of people without one. And we've found it's always better to ship early and get feedback sooner rather than later. Another reason though is we're not totally sure we're sold on t…

Hi Joe, I just wanted to pipe in and say that's excellent reasoning for this design, especially since the ability to create indexes on arbitrary ReQL expressions would make an intelligent query planner into a lot of work. I'm looking forward to giving these a try.

Thank you.

Re: RethinkDB 1.5 released: secondary indexes, soft durability, perf improvements

#50

Post by sergei » Thu, May 16, 2013, 7:18 PM UTC

btilly wrote: post #44

sergei wrote: post #32

Earlier quoted context omitted.

A decent cost based optimizer will be right most of the time, and you have hints for the cases where it falters. Your approach is simply hiding the query optimizer choices inside of the app, making it more fragile. Let's say I have two fields: a and b, both of which have an index. And then I have a query with predicates across both a and b. For example, a = 10 and b = 20. One of the core facilities within an optimize…

You are underestimating the necessary design tension. If you expose hints, then you've constrained how your optimizer works because it has to work with those hints. (Oracle, for example, has been fighting this battle for ages.) Once you've built an optimizer, people come to depend on it so it is critical that you get it right. They will need optimizer eventually, and they know it. But I'm glad that they are not imple…

I frequently see apps with thousands of distinct query signatures. Having a developer manually chose indexes, join ordering, aggregation method, (just to name a few) for every single query, and then select multiple plans because input parameters absolutely do result in scenario where plans can be 10,000x off in performance -- well, that's just untenable.

Not trying to claim that CBOs are the panacea here, but let's be realistic. Having developers manually plan every single query is not the right choice.