Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PostgreSQL Scalability: Towards Millions TPS

akorotkov.github.io

Post reply on HN
Print view View original

131–140 of 222 posts

Re: PostgreSQL Scalability: Towards Millions TPS

#131

Post by Jweb_Guru »

SEJeff wrote: post #120
Alex3917 wrote: post #26

Earlier quoted context omitted.

> How does PostgreSQL compare to VoltDB? If you don't know the difference, you probably want Postgres. VoltDB is a specialty database for things like high frequency trading. It wouldn't make sense to use for, say, a consumer app or web startup.

In specific, it is a column store, which is advantageous to do things like real time analytics over millions of data points via streaming market data. This has uses for HFT, but also for anyone who wants to do their own day trading.

VoltDB isn't a column store. It's designed for serializable OLTP workloads with really fast index updates, neither of which characterize column stores. You may be thinking of one of Stonebraker's other projects, Vertica.

Re: PostgreSQL Scalability: Towards Millions TPS

#132

Post by jeffdavis »

ap22213 wrote: post #106
_w3e8 wrote: post #82

Postgres has been my DB of choice for nearly a decade. The only times I wind up working with another db are because: (1 it is a better technical fit for a very specific problem (2 there is already a legacy db in place I have been voted down at a couple of startups that wanted to run a "MEAN" stack, invariably all of those startups moved from MongoDB or shutdown. The only time I will advocate for anything other than P…

The only issue that I have with this top-most comment is that it presents PG as a silver bullet. But, there are a lot of different types of databases for a reason. For instance, at my current startup, we employ at least 7 different databases (including PG). And, I don't say that to brag - each has a specific use for the problem at hand. You have to consider the needs and trade offs of your specific project. And, if y…

Not disagreeing necessarily, but having many different databases has a cost.

Data has more value when combined, so a wise database choice also depends on what you already have. You can combine data from different systems (e.g. FDWs, which postgres has great support for), but something is usually lost along the way.

I think postgres is a good default, because it is good for a lot of things. But sure, if you make an informed decision otherwise, there's nothing wrong with that.

Re: PostgreSQL Scalability: Towards Millions TPS

#133

Post by takeda »

manigandham wrote: post #102
_w3e8 wrote: post #82

Postgres has been my DB of choice for nearly a decade. The only times I wind up working with another db are because: (1 it is a better technical fit for a very specific problem (2 there is already a legacy db in place I have been voted down at a couple of startups that wanted to run a "MEAN" stack, invariably all of those startups moved from MongoDB or shutdown. The only time I will advocate for anything other than P…

> invariably all of those startups moved from MongoDB Why? Especially after point #1 and assuming the document-store was a good fit for the data model.

Because NoSQL is a hype.

Many of the NoSQL essentially takes us back to 60s before Codd came up with relational model[1] These ideas tend to come back once in a while [2][3], but so far nothing is better than relational model.

NoSQL still makes sense in many cases (generally when your specific use case does not need all guarantees of ACID), you can get in return higher performance or horizontal scalability.

MongoDB is aim to be generic database but rides on the wave of NoSQL "coolness". It was created by people who had no experience in databases and are learning as they go[4]. As they started adding things that are essential for database they realized it's not that simple.

Currently MongoDB is outperformed by Postgres. In fact there is an application called ToroDB which provides protcol compatibility for Postgres that emulates MongoDB and even that outperforms Mongo. Mongo also doesn't scale well horizontally, so essentially you don't really don't get any significant advantage.

[1] https://en.wikipedia.org/wiki/Hierarchical_database_model

[2] https://en.wikipedia.org/wiki/Object_database

[3] https://en.wikipedia.org/wiki/XML_database

[4] For example they started with mmap'ed memory regions to store the data. Did not initially use fsync() to make sure data is saved on disk. In a way it reminds me of MySQL several years ago. It's much better now than it was in the past, but it has a lot of warts left of from the past.

Re: PostgreSQL Scalability: Towards Millions TPS

#134

Post by Keepo »

bulkan wrote: post #124
_w3e8 wrote: post #82

Postgres has been my DB of choice for nearly a decade. The only times I wind up working with another db are because: (1 it is a better technical fit for a very specific problem (2 there is already a legacy db in place I have been voted down at a couple of startups that wanted to run a "MEAN" stack, invariably all of those startups moved from MongoDB or shutdown. The only time I will advocate for anything other than P…

There is also Sequelize [1] which has more activity 1 - http://docs.sequelizejs.com/en/latest/

I would also like to mention Objection[1] it is very minimalistic and doesn't get into your way. Most of the time you are writing queries with Knex[2] or you can write raw sql if you feel like it. I tried a lot of ORMs for node and this is the one I liked the most. The guy working on it is also very responsive and have superb documentation.

1 - http://github.com/Vincit/objection.js

2 - http://knexjs.org

Re: PostgreSQL Scalability: Towards Millions TPS

#135

Post by e12e »

manigandham wrote: post #103
combatentropy wrote: post #85

Earlier quoted context omitted.

> why nearly every RDBMS post on Hacker News is about Postgres and almost never MySQL You'll see the same phenomenon on Slashdot. MySQL is popular among a subset of programmers: web developers. In corporations, Microsoft SQL and Oracle are more popular. Further, MySQL is popular among a subset of web developers: those who use PHP. Among web developers who use Python, Postgres seems more popular. My suspicion is that…

> Postgres, but that anyone uses Microsoft or Oracle at all. Not only do they cost a lot, but from a purely technical standpoint they are worse. Based on what? Both commercial databases have incredible features, tooling and extensions that leave postgres behind. Postgres today doesn't even have a solid scale-up or scale-out strategy. It does have nice SQL support and makes developer lives a little easier but this isn…

I'm curious what you mean by Postgres not having a "scale-up" strategy. Are you saying that eg MS SQL works better with a few TBs of ram and 128 cores than Postgres does?

Re: PostgreSQL Scalability: Towards Millions TPS

#136

Post by takeda »

gjolund wrote: post #128
rpedela wrote: post #116

Earlier quoted context omitted.

Why not use PostGIS for geo-spatial data?

For processing, 100% agree. However if the write load is very high then Mongo is better suited as the intial store. I then replicate to other dbs.

Mongo actually has worse performance than postgres, including inserting data.

You can of course improve Mongo's speed by using weaker write concern, but you can also disable WAL in Postgres too.

Re: PostgreSQL Scalability: Towards Millions TPS

#137

Post by yeukhon »

graffitici wrote: post #62

Are there any best practices for using PostgreSQL for storing time series data? Would it be comparable in performance to some of the NoSQL solutions (like Cassandra) for reasonable loads?

Datadog uses PostgreSQL. See [1].

[1]: http://www.enterprisedb.com/success-stories/postgresql-succe...

Re: PostgreSQL Scalability: Towards Millions TPS

#138

Post by oconnor663 »

manigandham wrote: post #102
_w3e8 wrote: post #82

Postgres has been my DB of choice for nearly a decade. The only times I wind up working with another db are because: (1 it is a better technical fit for a very specific problem (2 there is already a legacy db in place I have been voted down at a couple of startups that wanted to run a "MEAN" stack, invariably all of those startups moved from MongoDB or shutdown. The only time I will advocate for anything other than P…

> invariably all of those startups moved from MongoDB Why? Especially after point #1 and assuming the document-store was a good fit for the data model.

http://www.sarahmei.com/blog/2013/11/11/why-you-should-never...

Re: PostgreSQL Scalability: Towards Millions TPS

#139

Post by jcoffland »

jlgaddis wrote: post #36

Man, how I wish WordPress had originally chosen to use PostgreSQL instead of MySQL back in the day.

Why? WordPress would not be any better or easier to use unless you already had PostgreSQL installed. Frankly for a use case as simple as WordPress you don't need to over optimize your database.

Re: PostgreSQL Scalability: Towards Millions TPS

#140

Post by jcoffland »

cwbrandsma wrote: post #29
jrcii wrote: post #12

Slightly tangential but I'm genuinely curious, does any have a theory as to why nearly every RDBMS post on Hacker News is about Postgres and almost never MySQL or MariaDB? Considering the relative obscurity of the former it seems somewhat inexplicable.

MySQL is considered more "old school" and not great at being standards compliant when installed with default settings. See https://twitter.com/robconery/status/189086889486192640 Also, MySQL is run by Oracle...and Oracle a horrible company run by horrible people that behave horribly, and should never ever be trusted in any form. Any developer relying or Oracle code should not be trusted ever again, and Oracle consult…

That's why there's MariaDB.
Post reply on HN