Live data from Hacker News

Mastering PostgreSQL Administration [pdf]

momjian.us

31–40 of 56 posts

Re: Mastering PostgreSQL Administration [pdf]

#31

I debated running my own Postgres cluster for a side project, but ended up paying for Digitalocean’s managed offering. I’d like to hear from anyone running their own Postgres. Why did you choose to do that vs pay for a managed instance? How much of a hassle has it been? Is it something you really need significant Postgres experience to do well (to achieve security, backups, high availability, etc)? Biggest gotchas to…

We ran a Postgres DBaaS offering of Azure for 2 years before we switched over to a self-managed (IaaS) Postgres installation. Couldn't be happier.

The primary reason was the desire to use extensions; I'm now able to use Foreign Data Wrappers with great effect to turn the Postgres databases in an enterprise data integration platform/data warehouse.

Additional benefits include a staggering 50x performance increase* , a much finer grained user authorization strategy (by being able to (automatically) alter grants and alter default privileges of other users with superuser permissions). Also, my business users can now leverage NFS or sFTP shares to load very large datasets direct from the server and we can actually shut down unused environments outside of business hours (Azure's DBaaS can be scaled down, but not shut down).

The costs of this setup is actually ~20% lower compared to Azure's managed offering (for same core count, could be even lower if you add the enormous performance gains to the comparison). This included cost of developers/operators! (we're finding the operational effort is more or less equal to a managed offering; almost non-existant). The only thing that really caused some discomfort in the beginning was our setup of pgBouncer.

Off course, There're some caveats; we run an OLAP workload, not business critical and therefor haven't really put a lot of effort into a HA setup. Still, the advantages of an "real" Postgres server cannot be overstated! To be able to use extensions alone has brought the time to market of new features from weeks to days.

* Geometric mean of TPC-DS benchmark suite. On Azure's Ls_v2 machines, leveraging the NVMe disk with bcache. Compared to a tuned Azure's "Postres as a Service" with equal core count.

Re: Mastering PostgreSQL Administration [pdf]

#32

I debated running my own Postgres cluster for a side project, but ended up paying for Digitalocean’s managed offering. I’d like to hear from anyone running their own Postgres. Why did you choose to do that vs pay for a managed instance? How much of a hassle has it been? Is it something you really need significant Postgres experience to do well (to achieve security, backups, high availability, etc)? Biggest gotchas to…

Our product offers near real time (incremental sync every 3 mins) backup and restore solution for on-premise SQLServer data. We have used PostgreSQL to store the backup data in cloud and offered add-on services on top of PostgreSQL data (such as reports, analytics, etc). Every customer data is stored as a separate database. Initially we have used RDS as PostgreSQL instance when the product is in pilot phase. RDS cost…

I assume you're using ZFS filesystem for the transparant compression, what's your opninion/experience on using ZFS on cloud storage? I mean; the EBS disks are already redundantly stored by AWS and the COW mechanism could lead to a lot of write amplification; negatively impacting the network attached storage?

(I don't use EBS in my day job, but Azure's disk offering don't really offer adequate perforamance when used with any filesystem other then EXT4 in my experience)

Re: Mastering PostgreSQL Administration [pdf]

#33

I debated running my own Postgres cluster for a side project, but ended up paying for Digitalocean’s managed offering. I’d like to hear from anyone running their own Postgres. Why did you choose to do that vs pay for a managed instance? How much of a hassle has it been? Is it something you really need significant Postgres experience to do well (to achieve security, backups, high availability, etc)? Biggest gotchas to…

IMO start with managed, migrate to hosted when costs will justify it.

Re: Mastering PostgreSQL Administration [pdf]

#34
I have a bookmarking system built with MongoDB as the database. It has user and device management, real-time synchronization.

I am seriously considering migrating it to PostgreSQL, rewriting the entire backend.

One of the greatest things about MongoDB is the existence of Studio 3T, a really good administration software for MongoDB, and I wouldn't want to miss it.

I spent a couple of years coding with MySQL and used phpMyAdmin. I wouldn't want to go back to such a tool, and I think that pgAdmin is what would be PostgreSQL's alternative to phpMyAdmin.

The only software which I found to suite me was DataGrip.

Are those the only two reasonable administration tools for PostgreSQL? DataGrip and pgAdmin?

Re: Mastering PostgreSQL Administration [pdf]

#35
post #34

I have a bookmarking system built with MongoDB as the database. It has user and device management, real-time synchronization. I am seriously considering migrating it to PostgreSQL, rewriting the entire backend. One of the greatest things about MongoDB is the existence of Studio 3T, a really good administration software for MongoDB, and I wouldn't want to miss it. I spent a couple of years coding with MySQL and used p…

I'd suggest that for MySQL, MySQL Workbench is a closer comparison to Studio 3T, and in my limited experience quite good. (I really like their diagramming tool, but haven't used the rest of the toolset in some years).

Re: Mastering PostgreSQL Administration [pdf]

#36
post #26

You should have a look at https://github.com/citusdata/pg_auto_failover This project makes is super easy to setup a resilient and highly-available postgresql cluster. And since the postgresql client lib handle connection to multiple replica... no need for some kind of load-balancer (pg_bouncer, pgpool...) in front of it anymore (even if they can still be useful sometimes).

pgbouncer is not a load balancer, it's a connection pooler.

The former is used to split reads and writes (and so far I haven't seen "magic" tools that automate it well, pgpool has lots of issues; this task is better to solve in app code -- most modern frameworks either already solved it such as RoR, or at least work well with 2 connections),

the latter is used to multiplex connections and improve performance similarly to nginx, haproxy, envoy, but with native Postgres protocol support.

Re: Mastering PostgreSQL Administration [pdf]

#37
post #34

I have a bookmarking system built with MongoDB as the database. It has user and device management, real-time synchronization. I am seriously considering migrating it to PostgreSQL, rewriting the entire backend. One of the greatest things about MongoDB is the existence of Studio 3T, a really good administration software for MongoDB, and I wouldn't want to miss it. I spent a couple of years coding with MySQL and used p…

Dbeaver is quite good. IMHO pgAdmin should be avoided. psql and pgcli are really nice at the command line.

https://github.com/dhamaniasad/awesome-postgres

Re: Mastering PostgreSQL Administration [pdf]

#38
post #37
post #34

I have a bookmarking system built with MongoDB as the database. It has user and device management, real-time synchronization. I am seriously considering migrating it to PostgreSQL, rewriting the entire backend. One of the greatest things about MongoDB is the existence of Studio 3T, a really good administration software for MongoDB, and I wouldn't want to miss it. I spent a couple of years coding with MySQL and used p…

Dbeaver is quite good. IMHO pgAdmin should be avoided. psql and pgcli are really nice at the command line. https://github.com/dhamaniasad/awesome-postgres

Yep, use DBeaver for all my sql management needs. I recommend it to everyone after TeamSQL gave up the ghost. SSH tunneling for aws is handy :)

Re: Mastering PostgreSQL Administration [pdf]

#39
post #34

I have a bookmarking system built with MongoDB as the database. It has user and device management, real-time synchronization. I am seriously considering migrating it to PostgreSQL, rewriting the entire backend. One of the greatest things about MongoDB is the existence of Studio 3T, a really good administration software for MongoDB, and I wouldn't want to miss it. I spent a couple of years coding with MySQL and used p…

I’m personally a huge fan of Navicat. It was always the best tool of those I’d tried and it was my most missed piece of software when I moved from OSX to Ubuntu full time.

They finally released a native Linux version and I’ve been on cloud nine ever since.

Re: Mastering PostgreSQL Administration [pdf]

#40
post #34

I have a bookmarking system built with MongoDB as the database. It has user and device management, real-time synchronization. I am seriously considering migrating it to PostgreSQL, rewriting the entire backend. One of the greatest things about MongoDB is the existence of Studio 3T, a really good administration software for MongoDB, and I wouldn't want to miss it. I spent a couple of years coding with MySQL and used p…

phppgadmin or adminer.
Post reply on HN