Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We use RethinkDB

blog.workshape.io

Post reply on HN
Print view View original

61–70 of 78 posts

Re: We use RethinkDB

#61

Post by politician »

mbrock wrote: post #12
lewisjoe wrote: post #4

I've always wondered what's the best way to integrate a database engine with the application. 1) Use a middleware/ORM/Whatever which abstracts away the query-lang of the db, and provides a pluggable multi-db support 2) Just use native db query language with all exclusive features of the engine. Companies like workshape.io, why do they prefer the latter?

Abstractions are leaky. [1] ORM systems are highly complicated abstractions. Most of your developers will use them without understanding how they work. In many cases, the only way to understand how they work is to read the source code. They have magic features that are advertised as convenient but when they inevitably do something you don't want them to do you'll tear your hair out trying to circumvent them. They put…

Preprocessed prepared queries that are shipped along with whatever package is using them is a far easier solution than quirky ORM tools that can do the simple things, but have a tendency to break on the harder things or encourage an authoring style that destroys performance.

That said, I think the world could use a CoffeeScript-esque transpiler for targeting SQL. Preferably with some kind of frontend/IL/backend separation, so that everyone can take a crack at replacing the awful SQL syntax.

Re: We use RethinkDB

#62

Post by seabrookmx »

xordon wrote: post #58
otterley wrote: post #52

Earlier quoted context omitted.

> We're on 5.5 and couldn't do some table operations that seemed promising without service disruptions. Everyone has this problem. But it's been largely solved in practice by performing the schema changes on slaves, and then promoting the slaves to master. Also, if you're just using RethinkDB as a delayed (and almost certainly inconsistent) secondary storage system, why not use ElasticSearch instead? BTW, 250GB fits…

I don't think 250GB will fit in memory on any reasonable sized box. What world do you live in?

A world where people use real servers.

All of our physical boxes have 256gb+ of RAM.

They run VM's via XenServer for most of our uses, but our production DB's run on bare metal as they need the RAM and disk performance.

Re: We use RethinkDB

#63

Post by findjashua »

otterley wrote: post #52
simonpantzare wrote: post #46

We have used RethinkDB in production for a handful of months now. 100M docs, 250 GB data spread out on two servers. We added it to the mix because it got increasingly difficult to tune SQL queries involved in building API responses, especially for endpoints that needed to pull data from many tables. Our limited experience of MySQL operations was also a factor. We're on 5.5 and couldn't do some table operations that s…

> We're on 5.5 and couldn't do some table operations that seemed promising without service disruptions. Everyone has this problem. But it's been largely solved in practice by performing the schema changes on slaves, and then promoting the slaves to master. Also, if you're just using RethinkDB as a delayed (and almost certainly inconsistent) secondary storage system, why not use ElasticSearch instead? BTW, 250GB fits…

re: Elasticsearch

Rethink's 'ungroup' method lets you chain multiple reductions, which is incredibly powerful for building aggregation queries. Elasticsearch doesn't have that capability, and hence its aggregation capabilities are severely limited.

For example, with Rethink, it's very easy to compute a metric from metrics computed in a previous reduction. You can't do that with Elasticsearch, since its dsl allows metrics to only be computed from fields in the raw document, but not from other aggregation metrics.

Re: We use RethinkDB

#64

Post by 616c »

I see him here and Reddit every once in a while, but there is a cool client-side encrypted note taking app, Turtl, using Common Lisp and RethinkDB server-side, and what was node-webkit client-side. Very cool, everyone should check it.

https://turtl.it/docs/server/running

http://www.reddit.com/r/lisp/comments/1mkxp2/turtl_clientsid...

Not a dedicated user, but I have been playing with this dude's CL work and I like his approach and attitude. Maybe thought people would want to see a self-hosted RethinkDB proj.

Re: We use RethinkDB

#65

Post by wilmoore »

simonpantzare wrote: post #46

We have used RethinkDB in production for a handful of months now. 100M docs, 250 GB data spread out on two servers. We added it to the mix because it got increasingly difficult to tune SQL queries involved in building API responses, especially for endpoints that needed to pull data from many tables. Our limited experience of MySQL operations was also a factor. We're on 5.5 and couldn't do some table operations that s…

> We added it to the mix because it got increasingly difficult to tune SQL queries involved in building API responses, especially for endpoints that needed to pull data from many tables.

Had you looked into using PostgreSQL's materialized views? you can add indexes to the view with the additional bonus of the view hiding those joins from client code.

Re: We use RethinkDB

#66

Post by davyjones »

krunaldo wrote: post #59
xordon wrote: post #58

Earlier quoted context omitted.

I don't think 250GB will fit in memory on any reasonable sized box. What world do you live in?

An r720 from dell or similar model from dell with 600GB*2 SSD intel s3500DC model, 20 cores & 256GB of RAM will go for 5k-7k. You can bump this to 386GB of ram without going above 10k.

When I changed the country to Japan, the sticker price jumped from 2000 USD to 15,000 USD eq. for a very basic system. I am just at a loss as to what can explain this disparity. Guess I will have to call up my vendor to get a comparable quote.

Re: We use RethinkDB

#67

Post by pbowyer »

bbcbasic wrote: post #28
emergentcypher wrote: post #5

Why is it so trendy these days to say that everything was built "with love"?

It's just a meme. Like it is also trendy to have an over-sized picture of young people working on wooden desks in an industrial-chic office taking up 70% of the screen space. Goes nicely with the Bootstrap template, Lobster font, Circular cropped photos of founders, Ping pong tables, Ruby on Rails. Etc. Etc.

Upvoted for sheer cynical accuracy ;-)

Re: We use RethinkDB

#68

Post by z3t4 »

I'm also working on a noSQL database. What I'm struggling with is the abstraction for searches/filters. For example, if you want to get all books with "beginner" in the title, in SQL it would look something like:

  "SELECT * FROM books WHERE title LIKE %beginner%"
Where in no-SQL it would look like

  book.filter({title: ["like", "beginner"]});
Any ideas on how to abstract the filtering in a more clear way?

Re: We use RethinkDB

#69

Post by baq »

findjashua wrote: post #63
otterley wrote: post #52

Earlier quoted context omitted.

> We're on 5.5 and couldn't do some table operations that seemed promising without service disruptions. Everyone has this problem. But it's been largely solved in practice by performing the schema changes on slaves, and then promoting the slaves to master. Also, if you're just using RethinkDB as a delayed (and almost certainly inconsistent) secondary storage system, why not use ElasticSearch instead? BTW, 250GB fits…

re: Elasticsearch Rethink's 'ungroup' method lets you chain multiple reductions, which is incredibly powerful for building aggregation queries. Elasticsearch doesn't have that capability, and hence its aggregation capabilities are severely limited. For example, with Rethink, it's very easy to compute a metric from metrics computed in a previous reduction. You can't do that with Elasticsearch, since its dsl allows met…

splunk has eventstats command which computes metrics and assigns them to fields of documents so you can process them. is that something similar? (except the fact that splunk's invoices are know to cause cardiac arrest?)
Post reply on HN