Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

RethinkDB 1.12: simplified map/reduce, ARM port, new caching infrastructure

rethinkdb.com

Post reply on HN
Print view View original

51–60 of 79 posts

Re: RethinkDB 1.12: simplified map/reduce, ARM port, new caching infrastructure

#51

Post by leif »

egeozcan wrote: post #8
farhanpatel wrote: post #7

Has anyone used RethinkDB for a relative large database(100GB+)? I cant seem to find performance benchmarks anywhere.

That's something I've been searching for, too. I actually opened this page, wishing that someone linked to a benchmark for large data sets. Large datasets are the reason I'm trying to move away from mongodb completely, and I've been very happy with postgres but I'm willing to give a try to others.

If you're having problems with MongoDB on large data sets, try TokuMX: http://www.tokutek.com/products/tokumx-for-mongodb/

Re: RethinkDB 1.12: simplified map/reduce, ARM port, new caching infrastructure

#52

Post by zapov »

teraflop wrote: post #4
taf2 wrote: post #2

I don't understand why databases like RethinkDB and MongoDB don't just provide a SQL interface. I get it - they are marketing themselves as NoSQL but they are infact providing almost exactly the same set of features that say MySQL provided back in the day a simple "fast" sub-set of SQL. Instead they use the lack of an interface as a marketing gimmick... when in reality we have to learn a new query language... :(

One reason is that SQL only lets you work with flat relational tables. MongoDB and RethinkDB let you store complex JSON-like objects as values, and extending SQL to query and manipulate those kinds of structures is hard. The closest attempt I've seen is Google's BigQuery. Another reason is that SQL does its best to decouple the query (what you're asking for) from the execution plan (how it actually gets evaluated). B…

This is not true. Both Oracle and Postgres are ORDBMS and you can work with objects/nested collections and all kind of non-relational constructs.

Re: RethinkDB 1.12: simplified map/reduce, ARM port, new caching infrastructure

#53

Post by RyanZAG »

dkhenry wrote: post #47
coffeemug wrote: post #34

Earlier quoted context omitted.

What about lambda expressions in Java 8? I haven't programmed in Java in years, so unless there are limitations I'm not aware of, they seem like a great fit. Without those, doing an elegant driver would be pretty hard, which unfortunately probably means we'll never get good ReQL support into legacy Java.

So the way all underlieing communication is done is through a protobuf. So in python I know they hijack the expression and they can then construct the protobuf from the passed in python lambda. In Java land we can't really do that since lambdas are desugared at compile time to just be functions. In Java I can't even overload operators like you can in Scala or C++ to make it a little nicer. The result is your pretty m…

Take a look at this work by jooq:

http://blog.jooq.org/2014/03/21/java-8-friday-java-8-will-re...

You may be able to utilize a lot of their code for converting the lambda bytecode.. or not. Anyway it's interesting to see how they're dealing with the problem you've run into.

Re: RethinkDB 1.12: simplified map/reduce, ARM port, new caching infrastructure

#54

Post by RyanZAG »

coffeemug wrote: post #48
dkhenry wrote: post #47

Earlier quoted context omitted.

So the way all underlieing communication is done is through a protobuf. So in python I know they hijack the expression and they can then construct the protobuf from the passed in python lambda. In Java land we can't really do that since lambdas are desugared at compile time to just be functions. In Java I can't even overload operators like you can in Scala or C++ to make it a little nicer. The result is your pretty m…

Ah, got it! The upcoming 1.13 release will introduce a pure JSON interface for queries (which official drivers will switch to). You'll be able to construct JSON directly, and send it to the server -- no protobufs. Would that make things easier?

Sounds like that would work well - it's the approach ElasticSearch has taken with their APIs and they're generally very nice to use.

Re: RethinkDB 1.12: simplified map/reduce, ARM port, new caching infrastructure

#55

Post by coffeemug »

RyanZAG wrote: post #54
coffeemug wrote: post #48

Earlier quoted context omitted.

Ah, got it! The upcoming 1.13 release will introduce a pure JSON interface for queries (which official drivers will switch to). You'll be able to construct JSON directly, and send it to the server -- no protobufs. Would that make things easier?

Sounds like that would work well - it's the approach ElasticSearch has taken with their APIs and they're generally very nice to use.

To clarify, in 1.13 JSON will be a communication protocol for queries. It's not intended to be used by the end user, but by the client drivers. You can use it as an end user, but it isn't nearly as nice as native language drivers.

The change is meant to simplify driver development, packaging, and improve performance. Protobuf is worse than JSON in almost all of these categories.

Re: RethinkDB 1.12: simplified map/reduce, ARM port, new caching infrastructure

#57

Post by InAnEmergency »

coffeemug wrote: post #48
dkhenry wrote: post #47

Earlier quoted context omitted.

So the way all underlieing communication is done is through a protobuf. So in python I know they hijack the expression and they can then construct the protobuf from the passed in python lambda. In Java land we can't really do that since lambdas are desugared at compile time to just be functions. In Java I can't even overload operators like you can in Scala or C++ to make it a little nicer. The result is your pretty m…

Ah, got it! The upcoming 1.13 release will introduce a pure JSON interface for queries (which official drivers will switch to). You'll be able to construct JSON directly, and send it to the server -- no protobufs. Would that make things easier?

FWIW, I think this will make it much easier to implement drivers. Getting protobufs to work is a real pain.

Re: RethinkDB 1.12: simplified map/reduce, ARM port, new caching infrastructure

#59

Post by jjudge »

AaronFriel wrote: post #24
coffeemug wrote: post #6

Earlier quoted context omitted.

> I don't understand why databases like RethinkDB and MongoDB don't just provide a SQL interface. Actually, it's a really good question. I'm one of the ReQL designers at Rethink, and I was the one pushing for no SQL compatibility. Here is some of my reasoning (we could talk about this for days, though): * Even SQL designers would tell you SQL isn't a very good programming language. It even looks like Cobol! Imagine i…

I can tell at a glance what `where` and `having` mean - the former is a filter over a result set, and the latter is a filter over a grouping. You're probably right that SQL should have earlier defined this sort of recursion, so that you could sequence groupings and filters more easily, but don't fret: that future is here with common table expressions. Recursion is even supported! I'm reminded of this abomination of a…

Wow, that must have felt good to wrap up.

Re: RethinkDB 1.12: simplified map/reduce, ARM port, new caching infrastructure

#60

Post by br1 »

coffeemug wrote: post #27
meritt wrote: post #23

Earlier quoted context omitted.

So, to be clear here, you have created two fundamental things (both called "filter") -- A pre-group and a post-group filter. Users must still understand the difference and when to utilize them. SQL just happens to call those WHERE and HAVING instead of "filter" both times.

You're right, but it's not just `filter`. Any command that can run on a group can run on a full table and vice versa. For example: # get a sample of 3 elements from a table r.table('foo').sample(3) # get a sample of 3 elements from each group r.table('foo').group('category').sample(3) You could say that we created two versions of `filter`, and `sample`, and every other command. But another way to say it is that we us…

As a LINQ user, I expected group('category').sample(3) to sample 3 categories, not to sample inside each category.
Post reply on HN