Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

RethinkDB 1.5 released: secondary indexes, soft durability, perf improvements

rethinkdb.com

Post reply on HN
Print view View original

81–90 of 105 posts

Re: RethinkDB 1.5 released: secondary indexes, soft durability, perf improvements

#81

Post by vosper »

jdoliner wrote: post #60
wpietri wrote: post #53

Earlier quoted context omitted.

Bless you. The thing that makes me crazy about database engines is the "60% the time, it works every time" phenomenon, wherein things are magically fast until they are magically slow. Then the discussion goes seamlessly from "don't worry your pretty head, just build your app" to " of course you have to really understand ." With no recognition that these are basically contradictory messages. If you can really hide the…

Thanks, we've gotten a lot of mileage out of the following ethos: Let users ignore danger, but never let users ignore the fact that they're ignoring danger. Always give users a way to learn exactly what the dangers are. In the case of the query optimizer we haven't gotten to giving people a way to ignore these dangers yet but we will eventually.

This is a great philosophy, I'd love to see it applied more broadly.

Re: RethinkDB 1.5 released: secondary indexes, soft durability, perf improvements

#82

Post by pixelcort »

coffeemug wrote: post #73
troyk wrote: post #72

Am I the only one that hardly ever needs a case-sensitive query/index on string data? This is the one feature I miss from the MySQL glory days. With postgres, I use the citext type (case-insensitive text) but I guess Mongo and ReThinkDB expect you to either store the data as all lower or upper, or duplicate the field (for indexing).

The reason we (and probably Mongo folks) do it this way is because JSON is case-sensitive, so making indexes case-insensitive wouldn't work very well. EDIT: it looks like I misread your comment. I was talking about index/field names, while I think you were talking about field values. In this case Sam's comment below is correct -- once we add string.toLower() (which is very easy) you'll be able to do case-insensitive…

Are the indices based on reql expressions updated incrementally?

How about making indices based on map-reduce? Would that work? Would they be updated incrementally?

Re: RethinkDB 1.5 released: secondary indexes, soft durability, perf improvements

#83

Post by coffeemug »

pixelcort wrote: post #82
coffeemug wrote: post #73

Earlier quoted context omitted.

The reason we (and probably Mongo folks) do it this way is because JSON is case-sensitive, so making indexes case-insensitive wouldn't work very well. EDIT: it looks like I misread your comment. I was talking about index/field names, while I think you were talking about field values. In this case Sam's comment below is correct -- once we add string.toLower() (which is very easy) you'll be able to do case-insensitive…

Are the indices based on reql expressions updated incrementally? How about making indices based on map-reduce? Would that work? Would they be updated incrementally?

Yes, indices are updated incrementally. You can currently only evaluate an index for a given row based only on that row, not on other rows. The reasons for this are pretty deep -- if you start evaluating indexes on a row based on other rows, the result becomes non-deterministic and can't easily be replicated across the cluster. Once we introduce value-based replication (as opposed to evaluation-based replication we have now) this will become possible.

Re: RethinkDB 1.5 released: secondary indexes, soft durability, perf improvements

#85

Post by L0j1k »

I can't wait to give Rethink a spin later this weekend. I've been reading everything and just pleased as a pig in shit about almost all of it. The conversations with Rethink crew here also help me tremendously to feel good about any decision to try it out in production. I definitely feel good about throwing it against some weird situations just to see how it does. :)

Re: RethinkDB 1.5 released: secondary indexes, soft durability, perf improvements

#86

Post by rescrv »

JulianMorrison wrote: post #75
tomjen3 wrote: post #33

Earlier quoted context omitted.

I see this tude' among DB guys quite often -- unless the database guarantees unconditionally that it will always give the right answer and that no data is ever lost then it is worse than useless and most be nuked from orbit. Quite frankly that is pretty arrogant. ACID may be required for a banks transactions, but in far the majority of cases it isn't required (yes, even in your business) -- nothing of any consequence…

Banks do their transactions with reorderable increments and decrements, and reconciliation after the fact - they are "eventually consistent". More realistic example of stuff actually mattering: you sold product and took the money. People will be annoyed if they don't get their goods.

And each of those individual operations must be preserved. The example of account credit/debit is not used because it reflects reality 100%, but because its a simple-to-understand example to illustrate the power of transactions. Most developers should be able to look beyond that simplification and see how they can put transactions to work (or see that they are unnecessary).

Re: RethinkDB 1.5 released: secondary indexes, soft durability, perf improvements

#87

Post by hoodoof »

I like it alot but even simple queries are hideously complex. It doesn't need to be this hard. The ReThink developers are exceptionally helpful but goodness. Try to do a search returning all items where an array contains a value. In simpledb it's just select * where field='value', even if field is an array. In RethinkDB I had to ask for developer help and ended up writing a deep, sophisticated query. Querying has to be dramatically more simple.

Re: RethinkDB 1.5 released: secondary indexes, soft durability, perf improvements

#88

Post by coffeemug »

hoodoof wrote: post #87

I like it alot but even simple queries are hideously complex. It doesn't need to be this hard. The ReThink developers are exceptionally helpful but goodness. Try to do a search returning all items where an array contains a value. In simpledb it's just select * where field='value', even if field is an array. In RethinkDB I had to ask for developer help and ended up writing a deep, sophisticated query. Querying has to…

We're actually working on adding commands to ReQL to make queries a lot simpler. (It turns out that just a few commands that are very easy for us to implement can make enormous difference)

If you can, would you mind giving examples of things you'd like to do that are hard? We'd love to make it easy, and we think we've got most common things covered in the next release, but if there's something specific you have in mind we'd really appreciate it.

Re: RethinkDB 1.5 released: secondary indexes, soft durability, perf improvements

#89

Post by hoodoof »

coffeemug wrote: post #88
hoodoof wrote: post #87

I like it alot but even simple queries are hideously complex. It doesn't need to be this hard. The ReThink developers are exceptionally helpful but goodness. Try to do a search returning all items where an array contains a value. In simpledb it's just select * where field='value', even if field is an array. In RethinkDB I had to ask for developer help and ended up writing a deep, sophisticated query. Querying has to…

We're actually working on adding commands to ReQL to make queries a lot simpler. (It turns out that just a few commands that are very easy for us to implement can make enormous difference) If you can, would you mind giving examples of things you'd like to do that are hard? We'd love to make it easy, and we think we've got most common things covered in the next release, but if there's something specific you have in mi…

The basic stuff needed to let a beginner developer write a web app should be super super simple. REALLY simple. No map no reduce nothing hard.

Here are some of the basic examples of typical things people might need to do when writing a web app:

How do I select records based on a value in an array?

How would I find all records whose name begins with "star" i.e. 'Star Trek' 'Star Base' 'Star Wars' 'Star Anise'

I have records which contain an array of email addresses. How would I get all documents which contain one or more of a list of email addresses?

How would I add a where clause to my queries to ensure for example that the "status" field in my document equals "open"?

How would I implement paging through a result set?

How would I implement count to control the number of records returned during paging?

What is the right way to store dates in ReThinkDB so that they can be queried?

How would I select all documents in a date range?

How do I do queries on fields that are within other fields in my JSON structure?

How do I do wildcard searches?

How do I define which fields are returned from my query, or does it always return the entire document?

How do I make sure that I do not add documents to the database that are missing required fields, thereby breaking all queries that do not contain those required fields?

It should be incredibly easy (and well documented) to do all the above.

I wanted to work with ReThinkDB but chose a different database because I was so intimidated by the query writing.

Re: RethinkDB 1.5 released: secondary indexes, soft durability, perf improvements

#90

Post by coffeemug »

hoodoof wrote: post #89
coffeemug wrote: post #88

Earlier quoted context omitted.

We're actually working on adding commands to ReQL to make queries a lot simpler. (It turns out that just a few commands that are very easy for us to implement can make enormous difference) If you can, would you mind giving examples of things you'd like to do that are hard? We'd love to make it easy, and we think we've got most common things covered in the next release, but if there's something specific you have in mi…

The basic stuff needed to let a beginner developer write a web app should be super super simple. REALLY simple. No map no reduce nothing hard. Here are some of the basic examples of typical things people might need to do when writing a web app: How do I select records based on a value in an array? How would I find all records whose name begins with "star" i.e. 'Star Trek' 'Star Base' 'Star Wars' 'Star Anise' I have r…

Thank you, this is phenomenal feedback!! Most of these are already very simple, and the ones that aren't will become very simple soon. Michel (@neumino on github) is working on improving the docs right now, and I will pass this list on to him. We'll get this documented ASAP.

Thank you again, this is incredibly helpful!

Post reply on HN