Earlier quoted context omitted.
Honestly that's not something I would handle at the db layer. I would build a service that is responsible for keeping client db's in sync and abstract that away from my database entirely.
I'm not a fan of rolling our own sync code if I don't have to.
PostgreSQL Scalability: Towards Millions TPS
161–170 of 222 posts
Re: PostgreSQL Scalability: Towards Millions TPS
#162Earlier quoted context omitted.
It's also a great example of a technology which is mature and well-tested yet actively growing and improving. Most open source projects have a lot to learn from Postgres.
What would you say those learning points are? What key factors would you introduce into another OSS project that PostgreSQL currently employs?
Re: PostgreSQL Scalability: Towards Millions TPS
#163Re: PostgreSQL Scalability: Towards Millions TPS
#164Man, how I wish WordPress had originally chosen to use PostgreSQL instead of MySQL back in the day.
Why? WordPress would not be any better or easier to use unless you already had PostgreSQL installed. Frankly for a use case as simple as WordPress you don't need to over optimize your database.
Sounds like a situation I'm facing: assisting a non-profit org that wants a new website. Some members push for using Wordpress as the CMS. However the org has an established pgsql db containing lots of data that optimally could be employed on the site, e.g., lists of members/events, searchable documents, etc.
Also, there's a separate web app for accessing the database using SQL features mysql doesn't support. Lack of Wordpress/pgsql compatibility makes integrating the existing db and folding in the webapp functionality just about impossible, at least I'm not seeing how that could be done.
Since Wordpress isn't going to embrace pgsql, I've recommended considering alternative CMSs. Decisions are still pending.
Re: PostgreSQL Scalability: Towards Millions TPS
#165Earlier quoted context omitted.
There is also Sequelize [1] which has more activity 1 - http://docs.sequelizejs.com/en/latest/
I haven't worked with Sequelize personally, but a friend has been recently and curses the day it was born - he wishes he'd used bookshelf.
Re: PostgreSQL Scalability: Towards Millions TPS
#166Are there any best practices for using PostgreSQL for storing time series data? Would it be comparable in performance to some of the NoSQL solutions (like Cassandra) for reasonable loads?
You also have to spend some time to tune the query cost settings to avoid sequential scans if you're only gonna work with a subset of the data. Another optimization could be implementing table inheritance so you have a table for every year. If you work with data sets for a specific year you would get a big performance boost with sequential scans.
PostgreSQL's biggest weakness at the moment is aggregating data by using several cpu workers/cores. This is coming in PostgreSQL 9.6
Oh and I run PostgreSQL on ZFS with LZ4 compression,
Re: PostgreSQL Scalability: Towards Millions TPS
#167Earlier quoted context omitted.
Because NoSQL is a hype. Many of the NoSQL essentially takes us back to 60s before Codd came up with relational model[1] These ideas tend to come back once in a while [2][3], but so far nothing is better than relational model. NoSQL still makes sense in many cases (generally when your specific use case does not need all guarantees of ACID), you can get in return higher performance or horizontal scalability. MongoDB i…
Currently MongoDB is outperformed by Postgres I hate hearing absolutist dogma like this. Some things are faster in Postgres, some things are faster in Mongo. We are migrating our analytics db from Mongo to PG but we hit a wall because SELECT COUNT(DISTINCT x)) performs abysmally in Postgres. Mongo's aggregation framework is an immature PITA but it performs this little trick well enough that we're pretty much stuck ke…
An alternative is PipelineDB, which is built on Postgres (and drop-in compatible, supposedly) and should provide an efficient implementation for those kinds of queries: https://www.pipelinedb.com/
I've never used it, though.
Re: PostgreSQL Scalability: Towards Millions TPS
#168Earlier quoted context omitted.
It's because Python tends to be the most common programming language for startups due to being easy to hire for, having good library support for a wide range of use cases, being good for web development, etc. And MySQL is focused on catering to enterprise customers who don't use Python. The last time I heard, they had something like one person working part time on Python support, so the drivers weren't nearly as reli…
Err, no. Python probably isn't even in the top 5 of languages startups choose for their main stack.
One thing that's always bemused me is that YC doesn't insist on its companies using Lisp :-)
Re: PostgreSQL Scalability: Towards Millions TPS
#169Earlier quoted context omitted.
Currently MongoDB is outperformed by Postgres I hate hearing absolutist dogma like this. Some things are faster in Postgres, some things are faster in Mongo. We are migrating our analytics db from Mongo to PG but we hit a wall because SELECT COUNT(DISTINCT x)) performs abysmally in Postgres. Mongo's aggregation framework is an immature PITA but it performs this little trick well enough that we're pretty much stuck ke…
There's no reason for SELECT COUNT(DISTINCT x)) to perform badly in Postgres, as long as you have an appropriate table design and indexes. Also MSSQL's query planner isn't better than Postgres', I work with both. Postgres does have its quirks though, especially with the MVCC row expiry.
Re: PostgreSQL Scalability: Towards Millions TPS
#170Earlier quoted context omitted.
Why not use PostGIS for geo-spatial data?
For processing, 100% agree. However if the write load is very high then Mongo is better suited as the intial store. I then replicate to other dbs.