Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

RethinkDB 1.12: simplified map/reduce, ARM port, new caching infrastructure

rethinkdb.com

Post reply on HN
Print view View original

21–30 of 79 posts

Re: RethinkDB 1.12: simplified map/reduce, ARM port, new caching infrastructure

#21

Post by ashwinp92 »

coffeemug wrote: post #6
taf2 wrote: post #2

I don't understand why databases like RethinkDB and MongoDB don't just provide a SQL interface. I get it - they are marketing themselves as NoSQL but they are infact providing almost exactly the same set of features that say MySQL provided back in the day a simple "fast" sub-set of SQL. Instead they use the lack of an interface as a marketing gimmick... when in reality we have to learn a new query language... :(

> I don't understand why databases like RethinkDB and MongoDB don't just provide a SQL interface. Actually, it's a really good question. I'm one of the ReQL designers at Rethink, and I was the one pushing for no SQL compatibility. Here is some of my reasoning (we could talk about this for days, though): * Even SQL designers would tell you SQL isn't a very good programming language. It even looks like Cobol! Imagine i…

Although initially it seemed otherwise, your point on having a different interface to confusion between complete SQL compatibility v/s the subset being offered makes a lot of sense. Also personally I would rather prefer an imperative language for tasks where efficiency is the primary concern compared to declarative SQL

Re: RethinkDB 1.12: simplified map/reduce, ARM port, new caching infrastructure

#22

Post by coffeemug »

meritt wrote: post #18
coffeemug wrote: post #12

Earlier quoted context omitted.

Sure, but consider how this works in ReQL: r.table('foo').filter(...) r.table('foo').group('category').filter(...) A properly designed language shouldn't have two different keywords for something that does ostensibly the same thing in different contexts. That's a mark of bad language design.

Are you sure about that? Looking at even your example page [ http://www.rethinkdb.com/docs/sql-to-reql/ ] this appears to be wrong or at least confusing? It's suggesting: SELECT category, SUM(num_comments) FROM posts GROUP BY category HAVING num_comments > 7 and: r.table("posts") .filter(r.row['num_comments']>7) .group('category') .sum('num_comments') are identical. I don't think they are? Your ReQL to me looks like…

Whoops, that's a great catch -- it's a bug in the docs. Will fix momentarily. Thanks!! (that is indeed embarrassing)

In ReQL any command you call after `group` runs on each group. So once you've called `group`, you can run anything you could run on a table on each group and that just works.

Re: RethinkDB 1.12: simplified map/reduce, ARM port, new caching infrastructure

#23

Post by meritt »

coffeemug wrote: post #22
meritt wrote: post #18

Earlier quoted context omitted.

Are you sure about that? Looking at even your example page [ http://www.rethinkdb.com/docs/sql-to-reql/ ] this appears to be wrong or at least confusing? It's suggesting: SELECT category, SUM(num_comments) FROM posts GROUP BY category HAVING num_comments > 7 and: r.table("posts") .filter(r.row['num_comments']>7) .group('category') .sum('num_comments') are identical. I don't think they are? Your ReQL to me looks like…

Whoops, that's a great catch -- it's a bug in the docs. Will fix momentarily. Thanks!! (that is indeed embarrassing) In ReQL any command you call after `group` runs on each group. So once you've called `group`, you can run anything you could run on a table on each group and that just works.

So, to be clear here, you have created two fundamental things (both called "filter") -- A pre-group and a post-group filter. Users must still understand the difference and when to utilize them.

SQL just happens to call those WHERE and HAVING instead of "filter" both times.

Re: RethinkDB 1.12: simplified map/reduce, ARM port, new caching infrastructure

#24

Post by AaronFriel »

coffeemug wrote: post #6
taf2 wrote: post #2

I don't understand why databases like RethinkDB and MongoDB don't just provide a SQL interface. I get it - they are marketing themselves as NoSQL but they are infact providing almost exactly the same set of features that say MySQL provided back in the day a simple "fast" sub-set of SQL. Instead they use the lack of an interface as a marketing gimmick... when in reality we have to learn a new query language... :(

> I don't understand why databases like RethinkDB and MongoDB don't just provide a SQL interface. Actually, it's a really good question. I'm one of the ReQL designers at Rethink, and I was the one pushing for no SQL compatibility. Here is some of my reasoning (we could talk about this for days, though): * Even SQL designers would tell you SQL isn't a very good programming language. It even looks like Cobol! Imagine i…

I can tell at a glance what `where` and `having` mean - the former is a filter over a result set, and the latter is a filter over a grouping.

You're probably right that SQL should have earlier defined this sort of recursion, so that you could sequence groupings and filters more easily, but don't fret: that future is here with common table expressions. Recursion is even supported!

I'm reminded of this abomination of a query I wrote in 2010 that produced a numbers table:

http://blogs.msdn.com/b/sqlazure/archive/2010/09/16/10063301...

Code:

    DECLARE @N int = 1000000;

      WITH RecursiveRowGenerator (Row#, Iteration) AS (
           SELECT 1, 1
            UNION ALL
           SELECT Row# + Iteration, Iteration * 2
             FROM RecursiveRowGenerator
            WHERE Iteration * 2

Re: RethinkDB 1.12: simplified map/reduce, ARM port, new caching infrastructure

#25

Post by slap_shot »

RethinkDB is my favorite piece of my current tech stack. I tell everybody I know about it, and I'm soon to release some blog posts and speak at a Meetup showing off what it is good at and how to get started - I really just can't say enough good stuff about these guys.

A high level overview from someone who has used it in production since 1.10 (about six months):

Pros:

* ReQL is a beautiful DSL that makes querying and using my data simpler than anything I've ever had before.

* An amazing UI lets you quickly do the things you do the most (verify a query, cherry pick some results, add/drop tables, indexes, shards, etc).

* Unparalleled support. During almost any reasonable hour, a RethinkDB employee will quickly field any question you have on #rethinkdb (freenode), their User Group, or email.

* Quick releases. These guys ship major releases every 60 to 90 days. Each release offers huge features, performance improvements, and bug fixes. They do a great job of listening to what people want and implementing them fast.

Cons:

* Database is still technically "beta" - great for side projects and prototyping, but be mindful if you intend to use it in production.

* Works great on tables of a few GB, but performance really degrades on the next order of magnitude. It looks like there were major changes in 1.12 addressing this.

* Only three officially supported clients (JS, Python, Ruby). That's a good start for their target market, but it is limiting for some.

RethinkDb is an archetype for startups - building what people want, shipping fast, always talking to customers, and clearly passionate about what they do. Even if you don't use their product, we all can learn from these guys. I hope they do well.

Re: RethinkDB 1.12: simplified map/reduce, ARM port, new caching infrastructure

#26

Post by fidotron »

taf2 wrote: post #2

I don't understand why databases like RethinkDB and MongoDB don't just provide a SQL interface. I get it - they are marketing themselves as NoSQL but they are infact providing almost exactly the same set of features that say MySQL provided back in the day a simple "fast" sub-set of SQL. Instead they use the lack of an interface as a marketing gimmick... when in reality we have to learn a new query language... :(

I'm totally the opposite, and blame you entirely for provoking what follows! SQL at this point is passed expiry.

The simple reason is we now have plenty of decent scripting languages to run as interactive prompts, all of which are far better at interfacing with the rest of the system than SQL is. Exposing a sane, direct, API in those languages gets you much more than SQL, along with having removed multiple layers of confusion, including string generation, escaping, reparsing, then is it doing what you want, etc.

I've been using LevelDB (via plyvel) a lot lately for data storage, and every time I end up having to use SQL (even indirectly via ORM) is painful by comparison because you can just feel the control being taken away from you, and somewhere you end up having to fire up the DB prompt for no good reason such as adding strange DB specific indexing flags to columns or even setting up authorization and DB creation, making it yet another thing to go wrong during deployment.

Protocol buffers stored in LevelDB prove so easy to use by comparison with something like SQLite or PgSQL just at an API level. The resulting code is simpler, cleaner, and much easier to reason about. If I need to move it to another format the code, again, is amazingly small.

As someone that cares deeply about my app's data structure seeing the acceptance of a world beyond SQL is one of the best developments in my career, and experience means I simply don't trust any SQL based abstraction to give you the controls to get it right.

Re: RethinkDB 1.12: simplified map/reduce, ARM port, new caching infrastructure

#27

Post by coffeemug »

meritt wrote: post #23
coffeemug wrote: post #22

Earlier quoted context omitted.

Whoops, that's a great catch -- it's a bug in the docs. Will fix momentarily. Thanks!! (that is indeed embarrassing) In ReQL any command you call after `group` runs on each group. So once you've called `group`, you can run anything you could run on a table on each group and that just works.

So, to be clear here, you have created two fundamental things (both called "filter") -- A pre-group and a post-group filter. Users must still understand the difference and when to utilize them. SQL just happens to call those WHERE and HAVING instead of "filter" both times.

You're right, but it's not just `filter`. Any command that can run on a group can run on a full table and vice versa. For example:

  # get a sample of 3 elements from a table
  r.table('foo').sample(3)
  
  # get a sample of 3 elements from each group
  r.table('foo').group('category').sample(3)
You could say that we created two versions of `filter`, and `sample`, and every other command. But another way to say it is that we use polymorphism, which is widely considered an advantage in modern programming languages.

Re: RethinkDB 1.12: simplified map/reduce, ARM port, new caching infrastructure

#29

Post by coffeemug »

slap_shot wrote: post #25

RethinkDB is my favorite piece of my current tech stack. I tell everybody I know about it, and I'm soon to release some blog posts and speak at a Meetup showing off what it is good at and how to get started - I really just can't say enough good stuff about these guys. A high level overview from someone who has used it in production since 1.10 (about six months): Pros: * ReQL is a beautiful DSL that makes querying and…

Hey, thanks for being a user!

In the spirit of making what people want, we're planning 1.13 right now. If you have feedback on what features you want, shoot me an e-mail -- slava@rethinkdb.com, and I'll work hard to try and get them in.

Re: RethinkDB 1.12: simplified map/reduce, ARM port, new caching infrastructure

#30

Post by baruch »

taf2 wrote: post #2

I don't understand why databases like RethinkDB and MongoDB don't just provide a SQL interface. I get it - they are marketing themselves as NoSQL but they are infact providing almost exactly the same set of features that say MySQL provided back in the day a simple "fast" sub-set of SQL. Instead they use the lack of an interface as a marketing gimmick... when in reality we have to learn a new query language... :(

I much prefer ReQL over SQL. It is much nicer, flows naturally in my opinion and is combine-able. I'm using this combination method in my rethink-miner tool to help me work with data to try and figure it out and I can make generic queries that I can use on top of every other sub-query which is very very nice.
Post reply on HN