Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Announcing MoSQL

stripe.com

Post reply on HN
Print view View original

51–60 of 118 posts

Re: Announcing MoSQL

#51

Post by taligent »

dennis82 wrote: post #38
gdb wrote: post #20

Earlier quoted context omitted.

(I work at Stripe.) Where we use MongoDB, it's not because of speed. PostgreSQL is certainly capable of fast performance. MongoDB is useful for its ability to log freeform data as well as for its replication model. (We use sharded MongoDB in a few places, but mostly use straight replica sets.) We use MySQL, MongoDB, PostgreSQL, and Impala. They're all useful in different places.

> "We use MySQL, MongoDB, PostgreSQL, and Impala." Thanks for the clarification, but this makes it even more obvious your engineering team is introducing needless complexity into your organization. Postgres can store unstructured data just fine, so you have a 'solution' that uses 3 OLTP stores instead of one.

PostgreSQL is awful for storing unstructured data. It is the most cumbersome, clunky syntax I've seen for a while and it lacks ORM support meaning you are forced to manually write it.

Making developers productive is an important aspect for choosing a database.

Re: Announcing MoSQL

#52

Post by nirvdrum »

taligent wrote: post #43
dennis82 wrote: post #9

Earlier quoted context omitted.

are you kidding me? There is absolutely NO reason whatsoever to use a NoSQL database for a financial services company. Postgres is more than capable of sustaining the necessary speeds of a startup. Relational databases were created in the first place to solve these very problems around transactionality and analytics for finance. This library is a beautiful example of reinventing the wheel, and otherwise creating a pa…

> There is absolutely NO reason whatsoever to use a NoSQL database for a financial services company Yes there is. PostgreSQL doesn't support multi master replication which makes it a terrible choice if you really want to make sure every transaction gets written. I really wonder at what point people that keep recommending PostgreSQL are going to wake up and realise what is happening in the industry. People are scaling…

In all fairness, you could use something other than Postgres that's also ACID.

Re: Announcing MoSQL

#53

Post by taligent »

gfodor wrote: post #34

FYI you can store unstructured data in PostgreSQL (and query it) with the introduction of hstore. So knock one more reason to use MongoDB instead of PostgreSQL off your list. (Disclaimer: the length of my list to use MongoDB has always been a constant that is less than one.) http://www.postgresql.org/docs/9.1/static/hstore.html

> So knock one more reason to use MongoDB instead of PostgreSQL off your list.

One of the reasons MongoDB is so popular is because it is an fantastic database for developers. As a Java developer I can deal in my code with sets, hashmaps, embedded structures and have it effectively map 1-1 in the database. It's akin to an object database meaning you can focus higher up in the stack.

With the SQL ORMs you can't avoid having to deal with the ER model.

Re: Announcing MoSQL

#54

Post by viralbajaria »

nelhage wrote: post #16
physcab wrote: post #7

This is pretty cool but I'm struggling to see what the use cases are, atleast for analysis. There might be quite a bit of benefits for running application code that I'm not aware of. With regards to analysis though, their own example question is "what happened last night?" but then they go on to say that it is a near real-time data store. Does it matter that it is a real-time mirror then? I've always liked the paradi…

(I wrote MoSQL) PostgreSQL scales surprisingly well for this purpose, and is much nicer for interactive queries than Hadoop/Hive. We use Impala[1] for some larger datasets, but Impala is comparatively new, and it's nice to have something as battle-tested as postgres here. As for the "why do we need realtime?": In my mind the benefit of a near-realtime replica is not that you actually often need it, but that it means…

I agree with your points that PostgreSQL (or RDMS in general) is really good for certain type of reporting / analytics use cases while hadoop/hive is awesome for handling billions or rows + TBs of data.

How was your overall experience with impala ? Did you guys have a fairly new hive cluster to try it out or did you just spin up a new one since impala can only read certain file formats (i.e. no custom SerDe).

Also, for hive/hadoop datasets, is that more for just data exploration, while this PostgreSQL solution is for smaller datasets which return in a few seconds and would not perform well in hive due to the cost of setting up a mapreduce job ?

Re: Announcing MoSQL

#55

Post by gdb »

monstrado wrote: post #42
gdb wrote: post #20

Earlier quoted context omitted.

(I work at Stripe.) Where we use MongoDB, it's not because of speed. PostgreSQL is certainly capable of fast performance. MongoDB is useful for its ability to log freeform data as well as for its replication model. (We use sharded MongoDB in a few places, but mostly use straight replica sets.) We use MySQL, MongoDB, PostgreSQL, and Impala. They're all useful in different places.

(Clouderan here) How are you liking Impala? We just dropped 0.5 release yesterday which includes the JDBC driver :D! Edit: Awesome job on the Ruby client, it's great!

It's been great -- setup was a bit of work (we're on Ubuntu, so had to build from source), but once up and running it's allowed us to do lots of ad-hoc analysis that would have been too hard otherwise.

I've been meaning to write a MoSQL equivalent for our Impala data, but at the moment we're doing a more traditional ETL.

Re: Announcing MoSQL

#56

Post by nelhage »

monstrado wrote: post #42
gdb wrote: post #20

Earlier quoted context omitted.

(I work at Stripe.) Where we use MongoDB, it's not because of speed. PostgreSQL is certainly capable of fast performance. MongoDB is useful for its ability to log freeform data as well as for its replication model. (We use sharded MongoDB in a few places, but mostly use straight replica sets.) We use MySQL, MongoDB, PostgreSQL, and Impala. They're all useful in different places.

(Clouderan here) How are you liking Impala? We just dropped 0.5 release yesterday which includes the JDBC driver :D! Edit: Awesome job on the Ruby client, it's great!

We've been pretty happy so far. There have been a few rough edges getting it up and keeping it running, but we've been very impressed with the performance so far.

I've passed your comment on to Colin, who wrote the Ruby client -- I'm sure he'll appreciate it!

Re: Announcing MoSQL

#57

Post by gfodor »

taligent wrote: post #53
gfodor wrote: post #34

FYI you can store unstructured data in PostgreSQL (and query it) with the introduction of hstore. So knock one more reason to use MongoDB instead of PostgreSQL off your list. (Disclaimer: the length of my list to use MongoDB has always been a constant that is less than one.) http://www.postgresql.org/docs/9.1/static/hstore.html

> So knock one more reason to use MongoDB instead of PostgreSQL off your list. One of the reasons MongoDB is so popular is because it is an fantastic database for developers. As a Java developer I can deal in my code with sets, hashmaps, embedded structures and have it effectively map 1-1 in the database. It's akin to an object database meaning you can focus higher up in the stack. With the SQL ORMs you can't avoid h…

Of course, the problem with that approach is you don't have anything enforcing any sort of data integrity below the application. In my experience most of the time you actually can put down on paper a schema and a set of rules the data should obey without too much fear of it changing dramatically. The nice thing about hstore is it allows you the flexibility to introduce unstructured data in just the places where a schema is unknowable or not worth the complexity.

MongoDB et all basically are built around the assumption that a schema is never worth the complexity. It's a bold claim that contradicts many decades worth of database research.

Re: Announcing MoSQL

#58

Post by psaintla »

nelhage wrote: post #21
lucian1900 wrote: post #12

Earlier quoted context omitted.

The only advantage MongoDB has over Postgres is built-in sharding, and even that is of dubious value.

To pick one, we like the fact that MongoDB lets you change your schema and add new fields to your documents without having to worry about migrations or keeping track of schema versions, or any of that. You could build something like that on top of SQL, but it's nice to have a tool where you don't have.

Serious question to you or anyone else who uses schemaless databases. Why is the ability to change schemas on the fly a good thing? Having worked at two companies that did, it was nothing but a recipe for disaster in large groups. Code that was dependent on expecting an integer or a string and not a collection would constantly break because a developer in some other group decided to store a collection instead of a the original data type that was expected. Schemaless databases required more documentation to track changes made between groups and led to more bugs because we could never be guaranteed of what kind of data we would be receiving. I've always thought of a database schema as a contract that makes guarantees to all applications. Why would you want to be able to break that contract?

Re: Announcing MoSQL

#59

Post by shawn-butler »

taligent wrote: post #43
dennis82 wrote: post #9

Earlier quoted context omitted.

are you kidding me? There is absolutely NO reason whatsoever to use a NoSQL database for a financial services company. Postgres is more than capable of sustaining the necessary speeds of a startup. Relational databases were created in the first place to solve these very problems around transactionality and analytics for finance. This library is a beautiful example of reinventing the wheel, and otherwise creating a pa…

> There is absolutely NO reason whatsoever to use a NoSQL database for a financial services company Yes there is. PostgreSQL doesn't support multi master replication which makes it a terrible choice if you really want to make sure every transaction gets written. I really wonder at what point people that keep recommending PostgreSQL are going to wake up and realise what is happening in the industry. People are scaling…

I'm sorry, postgres-xc doesn't work for you needs? [0] It has worked for me in the past.

[0] http://postgres-xc.sourceforge.net/

Re: Announcing MoSQL

#60

Post by knightni »

taligent wrote: post #43
dennis82 wrote: post #9

Earlier quoted context omitted.

are you kidding me? There is absolutely NO reason whatsoever to use a NoSQL database for a financial services company. Postgres is more than capable of sustaining the necessary speeds of a startup. Relational databases were created in the first place to solve these very problems around transactionality and analytics for finance. This library is a beautiful example of reinventing the wheel, and otherwise creating a pa…

> There is absolutely NO reason whatsoever to use a NoSQL database for a financial services company Yes there is. PostgreSQL doesn't support multi master replication which makes it a terrible choice if you really want to make sure every transaction gets written. I really wonder at what point people that keep recommending PostgreSQL are going to wake up and realise what is happening in the industry. People are scaling…

I would imagine that for your average startup, using solutions that don't even support transactionality will cause greater complexity issues. Especially given the enormous window before db scale out/up becomes an issue on well-designed applications.
Post reply on HN