Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Announcing MoSQL

stripe.com

Post reply on HN
Print view View original

81–90 of 118 posts

Re: Announcing MoSQL

#81

Post by PommeDeTerre »

knightni wrote: post #65
taligent wrote: post #45

Earlier quoted context omitted.

> Real businesses need a data warehouse and SQL is the right tool for that job. Honestly. I don't think you could be more misinformed if you tried. Hint: Google "Big Data".

...data warehouses in general mostly use SQL, and lots of businesses use data warehouses successfully. Teradata, Netezza, Oracle, DB2, etc. I'm not sure why his statement was controversial - SQL's a great language for reporting and analytics.

I've had to deal with a lot of NoSQL advocates whose experience with SQL or relational databases doesn't extend beyond MySQL.

Of course, it's understandable why they have a bad impression of SQL; they've only ever used one of the most inept implementations around.

Those who are willing to try one of the more mature and sensible relational database systems usually see quite quickly the value that such systems provide.

Re: Announcing MoSQL

#82

Post by pvh »

gdb wrote: post #20
dennis82 wrote: post #9

Earlier quoted context omitted.

are you kidding me? There is absolutely NO reason whatsoever to use a NoSQL database for a financial services company. Postgres is more than capable of sustaining the necessary speeds of a startup. Relational databases were created in the first place to solve these very problems around transactionality and analytics for finance. This library is a beautiful example of reinventing the wheel, and otherwise creating a pa…

(I work at Stripe.) Where we use MongoDB, it's not because of speed. PostgreSQL is certainly capable of fast performance. MongoDB is useful for its ability to log freeform data as well as for its replication model. (We use sharded MongoDB in a few places, but mostly use straight replica sets.) We use MySQL, MongoDB, PostgreSQL, and Impala. They're all useful in different places.

Mongo's probably still got the edge as a JSON store overall, but definitely check out the new JSON object dereferencing functionality coming in 9.3. There's a Russian indexing posse consisting of Oleg, Teodor, and Sasha who have been looking at doing proper indexes for JSON but haven't managed to secure funding. (Disclosure: I think they should get funded.)

These are the same guys who built hstore, full text search, GIN and GIST indexes and I think are working on a generic regular expression index type right now.

Re: Announcing MoSQL

#83

Post by jaytaylor »

dugmartin wrote: post #44
pc wrote: post #33

Earlier quoted context omitted.

Hm, that could be pretty cool actually. Especially if we also added a REPL for interactive queries at manage.stripe.com.

You're welcome Patrick. I'd recommend looking at Antlr4 to parse the "StripeSQL" commands.

Thanks for the Antlr reference; I've been meaning to learn more about it and you've pushed me to finally start!

Re: Announcing MoSQL

#84

Post by raylu »

PommeDeTerre wrote: post #81
knightni wrote: post #65

Earlier quoted context omitted.

...data warehouses in general mostly use SQL, and lots of businesses use data warehouses successfully. Teradata, Netezza, Oracle, DB2, etc. I'm not sure why his statement was controversial - SQL's a great language for reporting and analytics.

I've had to deal with a lot of NoSQL advocates whose experience with SQL or relational databases doesn't extend beyond MySQL. Of course, it's understandable why they have a bad impression of SQL; they've only ever used one of the most inept implementations around. Those who are willing to try one of the more mature and sensible relational database systems usually see quite quickly the value that such systems provide.

What do you suggest?

Re: Announcing MoSQL

#85

Post by PommeDeTerre »

thesis wrote: post #10
Ensorceled wrote: post #6

Nice. Real businesses need a data warehouse and SQL is the right tool for that job. I thank them for releasing this.

Maybe I'm misunderstanding your comment but... Real businesses need real solutions for their use cases. SQL is not necessarily the right tool for "that" job.

If such a "solution" involves safely querying, analyzing, storing and manipulating data in any way, SQL and relational databases are usually the best option in practice.

It's much more effective and efficient to use a SQL query than it is to throw together a huge amount of imperative JavaScript code (that's usually very specific to a single NoSQL database, as well) merely to perform the equivalent query.

It's much safer to use a database that offers true support for transactions and constraints, rather than trying to hack together that functionality in some Ruby or PHP data layer code, or relying on some vague promise of "eventual consistency", for instance.

It's much more maintainable, and leads to higher-quality data, to spend some time thinking about a schema, rather than just arbitrarily throwing data into a schema-less system, and then having to deal with the lack of a schema throughout any application code that's ever written.

Aside from an extremely small and limited handful of situations (Google and Facebook, for instance), relational databases are the best tool for the job.

Re: Announcing MoSQL

#86

Post by gfodor »

taligent wrote: post #51
dennis82 wrote: post #38

Earlier quoted context omitted.

> "We use MySQL, MongoDB, PostgreSQL, and Impala." Thanks for the clarification, but this makes it even more obvious your engineering team is introducing needless complexity into your organization. Postgres can store unstructured data just fine, so you have a 'solution' that uses 3 OLTP stores instead of one.

PostgreSQL is awful for storing unstructured data. It is the most cumbersome, clunky syntax I've seen for a while and it lacks ORM support meaning you are forced to manually write it. Making developers productive is an important aspect for choosing a database.

Choosing a data store based upon syntax and slightly limited ORM support isn't exactly a great idea. Both of these things can be improved rapidly with a little code.

More important questions are how is the data stored, how is it accessible, how can you scale the system, what operational constraints are there, how fast is it, what types of data modeling can be done, what consistency/transaction guarantees does it provide, etc. These are the things that will make developers productive because they will not be putting out fires all the time.

Re: Announcing MoSQL

#87

Post by j-kidd »

taligent wrote: post #43
dennis82 wrote: post #9

Earlier quoted context omitted.

are you kidding me? There is absolutely NO reason whatsoever to use a NoSQL database for a financial services company. Postgres is more than capable of sustaining the necessary speeds of a startup. Relational databases were created in the first place to solve these very problems around transactionality and analytics for finance. This library is a beautiful example of reinventing the wheel, and otherwise creating a pa…

> There is absolutely NO reason whatsoever to use a NoSQL database for a financial services company Yes there is. PostgreSQL doesn't support multi master replication which makes it a terrible choice if you really want to make sure every transaction gets written. I really wonder at what point people that keep recommending PostgreSQL are going to wake up and realise what is happening in the industry. People are scaling…

> People are scaling OUT not UP. Especially startups.

Startups need to scale out because many of them like to deploy on mediocre EC2 instances with the slowest SAN storage ever.

People that keep recommending PostgreSQL are rightfully ignoring this industry.

Re: Announcing MoSQL

#88

Post by j-kidd »

ElGatoVolador wrote: post #78

If you need to make a tool(and use twice the amount of storage) to be able to "query your data" in a SQL manner while using noSQL, it probably means you are using the wrong tool for the job.

Actually, it is pretty common to replicate the transactional data into another data store for analytical purpose. However, using PostgreSQL as the OLAP data store may not be the wisest move.

Re: Announcing MoSQL

#89

Post by umur »

govindkabra31 wrote: post #77

how do you deal with sharded mongo clusters?

(disclosure: I'm one of the founders at Citus Data)

hey, one way to do that is to use the MongoDB foreign data wrapper - also mentioned in some of the earlier threads.

mongo_fdw (https://github.com/citusdata/mongo_fdw) allows you to run SQL on MongoDB on a single node. Citus Data allows you to parallelize your SQL queries across multiple nodes (in this case, multiple MongoDB instances) by just syncing shard metadata. So you would effectively run SQL on a sharded mongo cluster without moving the data anywhere else.

another idea could be to use MoSQL to neatly replicate each mongo instance to a separate PostgreSQL instance, and then use Citus Data to run distributed SQL queries across the resulting PostgreSQL cluster.

Re: Announcing MoSQL

#90

Post by dennis82 »

gfodor wrote: post #86
taligent wrote: post #51

Earlier quoted context omitted.

PostgreSQL is awful for storing unstructured data. It is the most cumbersome, clunky syntax I've seen for a while and it lacks ORM support meaning you are forced to manually write it. Making developers productive is an important aspect for choosing a database.

Choosing a data store based upon syntax and slightly limited ORM support isn't exactly a great idea. Both of these things can be improved rapidly with a little code. More important questions are how is the data stored, how is it accessible, how can you scale the system, what operational constraints are there, how fast is it, what types of data modeling can be done, what consistency/transaction guarantees does it prov…

well said!
Post reply on HN