Mastering PostgreSQL Administration [pdf] • Hacker News Board

Re: Mastering PostgreSQL Administration [pdf]

#51

Post by krnaveen14 » Mon, Sep 28, 2020, 3:34 AM UTC

Wintereise wrote: post #49

krnaveen14 wrote: post #17

Earlier quoted context omitted.

Our product offers near real time (incremental sync every 3 mins) backup and restore solution for on-premise SQLServer data. We have used PostgreSQL to store the backup data in cloud and offered add-on services on top of PostgreSQL data (such as reports, analytics, etc). Every customer data is stored as a separate database. Initially we have used RDS as PostgreSQL instance when the product is in pilot phase. RDS cost…

Aren't you getting killed on inter-AZ bandwidth costs with streaming replication? This was our experience when we tried it.

Our total Inter-AZ bandwidth usage is about 2TB - 3TB / month which comes around to just $20 - $30 / month. We are planning to introduce SSL with compression between Master Slave setup to further reduce Inter-AZ bandwidth, but this isn't taken up yet.

Re: Mastering PostgreSQL Administration [pdf]

#52

Post by __s » Mon, Sep 28, 2020, 3:45 PM UTC

christophilus wrote: post #2

I debated running my own Postgres cluster for a side project, but ended up paying for Digitalocean’s managed offering. I’d like to hear from anyone running their own Postgres. Why did you choose to do that vs pay for a managed instance? How much of a hassle has it been? Is it something you really need significant Postgres experience to do well (to achieve security, backups, high availability, etc)? Biggest gotchas to…

https://github.com/citusdata/pg_auto_failover

https://github.com/wal-g/wal-g

Re: Mastering PostgreSQL Administration [pdf]

#53

Post by nikita2206 » Mon, Sep 28, 2020, 9:32 PM UTC

Nextgrid wrote: post #29

101008 wrote: post #25

Earlier quoted context omitted.

Sorry, I misworded. We are using it on "production", but our production is a closed beta for now, so we are using the Basic tier too. We are thinking in upgrade it to a better plan, but we are not sure because we are not sure that it will solve our problems. I think that comparing to your own laptop is unfair because there is no connectino needed, everything is on the same CPU / RAM, etc. Remember when you connect to…

Hopefully the Standard (or whatever they call it) tier will solve both our problems. If IO performance is the bottleneck (you can see it on the Azure console - in my case that graph goes up to 100% with very light use) then hopefully the next tier will solve it. > I think that comparing to your own laptop is unfair because there is no connectino needed, everything is on the same CPU / RAM, etc. Remember when you conn…

Consider that even at the highest tier you’ll max out at 20k IOPS and the speed on these cloud services doesn’t generally go beyond 200MB/s. Now you can google how much IOPS and read/write speed a typical nvme SSD will give you, maybe you’ll reconsider using RDS-like services then

Re: Mastering PostgreSQL Administration [pdf]

#54

Post by Cyranix » Tue, Sep 29, 2020, 11:43 PM UTC

qwertox wrote: post #34

I have a bookmarking system built with MongoDB as the database. It has user and device management, real-time synchronization. I am seriously considering migrating it to PostgreSQL, rewriting the entire backend. One of the greatest things about MongoDB is the existence of Studio 3T, a really good administration software for MongoDB, and I wouldn't want to miss it. I spent a couple of years coding with MySQL and used p…

I love Postico on OSX. Not 100% certain that it does 100% of what other apps do, but it has been a real breath of fresh air for my development needs.

Re: Mastering PostgreSQL Administration [pdf]

#55

Post by hrishios » Wed, Sep 30, 2020, 9:56 AM UTC

Cyranix wrote: post #54

qwertox wrote: post #34

I have a bookmarking system built with MongoDB as the database. It has user and device management, real-time synchronization. I am seriously considering migrating it to PostgreSQL, rewriting the entire backend. One of the greatest things about MongoDB is the existence of Studio 3T, a really good administration software for MongoDB, and I wouldn't want to miss it. I spent a couple of years coding with MySQL and used p…

I love Postico on OSX. Not 100% certain that it does 100% of what other apps do, but it has been a real breath of fresh air for my development needs.

Postico is amazing. I recently moved out of MacOS, and I can honestly say that it's the only tool that keeps making me want to move back. pgAdmin is a right pain.

Re: Mastering PostgreSQL Administration [pdf]

#56

Post by greggyb » Thu, Oct 01, 2020, 12:08 AM UTC

mthomassen wrote: post #31

christophilus wrote: post #2

I debated running my own Postgres cluster for a side project, but ended up paying for Digitalocean’s managed offering. I’d like to hear from anyone running their own Postgres. Why did you choose to do that vs pay for a managed instance? How much of a hassle has it been? Is it something you really need significant Postgres experience to do well (to achieve security, backups, high availability, etc)? Biggest gotchas to…

We ran a Postgres DBaaS offering of Azure for 2 years before we switched over to a self-managed (IaaS) Postgres installation. Couldn't be happier. The primary reason was the desire to use extensions; I'm now able to use Foreign Data Wrappers with great effect to turn the Postgres databases in an enterprise data integration platform/data warehouse. Additional benefits include a staggering 50x performance increase* , a…

This is not the only database product that is poorly configured for peak performance on Azure. Even their managed offering for their own Analysis Services OLAP database, Azure Analysis Services, is handily beaten by an IaaS instance with a better CPU, running SQL Server Analysis Services (the same database as part of SQL Server licensing). The database engine loves high clock speeds, but in testing, the top tier managed offering runs similar to what I've seen on mid-2GHz server CPUs.