Build fails on a "brew upgrade" for me. http://pastebin.com/3GWetUTE
Looked into it and discovered it was a simple lack of the pyyaml python module. Should probably put that in your build instructions and have the configure script throw an error.
RethinkDB 1.5 released: secondary indexes, soft durability, perf improvements
71–80 of 105 posts
Re: RethinkDB 1.5 released: secondary indexes, soft durability, perf improvements
#72Am I the only one that hardly ever needs a case-sensitive query/index on string data? This is the one feature I miss from the MySQL glory days. With postgres, I use the citext type (case-insensitive text) but I guess Mongo and ReThinkDB expect you to either store the data as all lower or upper, or duplicate the field (for indexing).
Re: RethinkDB 1.5 released: secondary indexes, soft durability, perf improvements
#73Am I the only one that hardly ever needs a case-sensitive query/index on string data? This is the one feature I miss from the MySQL glory days. With postgres, I use the citext type (case-insensitive text) but I guess Mongo and ReThinkDB expect you to either store the data as all lower or upper, or duplicate the field (for indexing).
The reason we (and probably Mongo folks) do it this way is because JSON is case-sensitive, so making indexes case-insensitive wouldn't work very well.
EDIT: it looks like I misread your comment. I was talking about index/field names, while I think you were talking about field values. In this case Sam's comment below is correct -- once we add string.toLower() (which is very easy) you'll be able to do case-insensitive indexing if you like because the indexing system supports arbitrary reql expressions. If you're using rethink and this is important to you, let me (slava@rethinkdb.com) know, and we'll prioritize this.
Re: RethinkDB 1.5 released: secondary indexes, soft durability, perf improvements
#74Am I the only one that hardly ever needs a case-sensitive query/index on string data? This is the one feature I miss from the MySQL glory days. With postgres, I use the citext type (case-insensitive text) but I guess Mongo and ReThinkDB expect you to either store the data as all lower or upper, or duplicate the field (for indexing).
If or once a string.toLower() function is added to RQL, you'll be able to have case-insensitive indexes in RethinkDB, because you can index on arbitrary functions of your rows.
Re: RethinkDB 1.5 released: secondary indexes, soft durability, perf improvements
#75Earlier quoted context omitted.
Also http://www.rethinkdb.com/docs/advanced-faq/ . RethinkDB started out as a MySQL driver that was lock-free and did schema updates non-stupidly, but it sounds like they've abandoned ACID and schema enforcement entirely like almost all NoSQL systems (as if what I wanted is faster wrong answers).
I see this tude' among DB guys quite often -- unless the database guarantees unconditionally that it will always give the right answer and that no data is ever lost then it is worse than useless and most be nuked from orbit. Quite frankly that is pretty arrogant. ACID may be required for a banks transactions, but in far the majority of cases it isn't required (yes, even in your business) -- nothing of any consequence…
Banks do their transactions with reorderable increments and decrements, and reconciliation after the fact - they are "eventually consistent".
More realistic example of stuff actually mattering: you sold product and took the money. People will be annoyed if they don't get their goods.
Re: RethinkDB 1.5 released: secondary indexes, soft durability, perf improvements
#76Earlier quoted context omitted.
I wish that databases did "speculative" queries. In the simplest form, you could say, "I have 2 reasonable plans, let's try A, and if it takes above time X, then start B in parallel and go with whatever finishes first." You could ramp up the idea to handle changing query plans based on updated statistics by sending some fraction of queries to one plan, and some to another. Then keep stats on how that worked out for y…
You can accomplish something pretty similar to this with oracle plan management: http://docs.oracle.com/cd/B28359_01/server.111/b28274/optpla... . You can set this up to pin plans, but let you know when it thinks it has a better plan to try out.
Yup. This kind of complexity they have been forced into because the simple general solution that they think should work and which does the vast majority of the time also routinely causes pants on fire emergencies.
Re: RethinkDB 1.5 released: secondary indexes, soft durability, perf improvements
#77Earlier quoted context omitted.
I wish that databases did "speculative" queries. In the simplest form, you could say, "I have 2 reasonable plans, let's try A, and if it takes above time X, then start B in parallel and go with whatever finishes first." You could ramp up the idea to handle changing query plans based on updated statistics by sending some fraction of queries to one plan, and some to another. Then keep stats on how that worked out for y…
I'm sure you'll appreciate the irony of this, but what you described is pretty much the exact mechanism of the MongoDB query optimizer [0]. [0] - http://docs.mongodb.org/manual/core/read-operations/#query-o...
The ironies are indeed rich. Thanks for that!
Re: RethinkDB 1.5 released: secondary indexes, soft durability, perf improvements
#78Earlier quoted context omitted.
Thanks for the reply. Do you have any resources you could share regarding the pitfalls of query optimizers? My experience with them comes primarily from studying the System R optimizer where the literature presented query optimization as a boon to performance without mentioning such drawbacks. Congratulations on the release by the way.
The primary problem is that picking indexes and execution algorithms essentially involves traversing an exponential space (and taking guesses about costs of things). Modern DB optimizers have hundreds (or thousands) of heuristics to do this well, but every once in a while they pick the wrong path and run a suboptimal query. This might be ok for offline analytics, but can be disastrous for production OLTP environments…
Even if you manually write the query plan, you are still subject to similar disasters. That's because your data is changing, which can cause the query performance to fall off a cliff without changing the plan.
Re: RethinkDB 1.5 released: secondary indexes, soft durability, perf improvements
#79Earlier quoted context omitted.
No, you've always been able to execute queries. They just weren't indexed before.
I didn't say "execute queries", I said search. Unless you have a tiny amount of data, no indexes means searching is not feasible.
The way you search on a key-value store with no secondary indexes is create the secondary indexes manually as separate tables.