Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to move from Amazon RDS to a dedicated PostgreSQL server

layer0.authentise.com

Post reply on HN
Print view View original

31–40 of 73 posts

Re: How to move from Amazon RDS to a dedicated PostgreSQL server

#31

Post by Rezo »

qaq wrote: post #22

OK a bit of topic, but just curious we are considering rolling Postgres hosted service that would support much large instances up 50TB much higher IOPS 1,000,000+ lower latency (infiniband vs ethernet SAN) pricing would be in line with AWS RDS +/- optional ElasticSearch integration optional change notification pub/sub Would you guys be interested/think such service is a good idea? Would appreciate to learn what you g…

When it comes to data, trust is key.

Think about it this way: OCZ has on paper SSDs that are simultaneously both faster and cheaper than, say, Intel. But I wouldn't put a OCZ drive into the lowest budged ricer gaming PC I could imagine (let's just say that have a bit of a reputation, to put it lightly), and I should be rightfully fired if I suggested putting one in a server.

Yes, your service might be massively faster, but... at 10-15% cheaper? No way! Not until you've been running for years with a great reputation. Now, if you enter the managed database space and pull a Digital Ocean ($5-$40 pricing vs comparable EC2 at $80-$400 AND better raw performance on top), then you can essentially create a new market niche. That's my suggestion to focus on, because businesses who operate 50TB databases aren't looking to save a few dollars. But people who use DO, need a database to go with it that is both faster and cheaper than AWS.

Honestly, performance ranks pretty low on the things I've found people look for when choosing a managed database service. You can serve a million users per day and average a whopping 12 IOPS, which might just about push a Raspberry Pi with sqlite slightly. Got more than a million users? Congrats, you now officially qualify to throw money at problems, the kind of problems RDS and Amazon loves to solve by injecting money.

Re: How to move from Amazon RDS to a dedicated PostgreSQL server

#32

Post by whyleyc »

eliribble wrote: post #16
jbrantly wrote: post #5

Not that it necessarily should, but the article did not really address why they were moving away from RDS. Was it cost, performance, other issues? I've been trying to compare RDS for Postgres to other offerings like Compose or Heroku but have come up surprisingly dry on comparisons.

I'm the decision maker for the company in question - Authentise. It was a combination of cost and the fact that we need to support an in-datacenter install option for our customers. RDS is really great if you need all of the failover and backup options it provides and you don't have the manpower to invest in it. That's how we got our services started. But, as our expertise and automation grew it ended up being a mode…

Thanks for the background info, very helpful. How has on-prem install gone for you guys ? There was an article on HN yesterday suggesting it was a royal PITA (for ops and support), would be interesting to hear about your experience.

Also FYI, your homepage has a 2015 copyright notice on it. No big deal, but thought you might like to know since it could put off particularly "nit picky" types of customer :)

Re: How to move from Amazon RDS to a dedicated PostgreSQL server

#33

Post by aftabh »

>To my surprise, PostgreSQL v9.5 on Ubuntu 14.04 does not default to unicode schema.

PostgreSQL does (if unicode character set was specified when PostgreSQL database cluster is initialized first time); it's Ubuntu OS which creates PostgreSQL database cluster with ASCII character set (encoding) after PostgreSQL's installation.

> Here is the snippet that I had to use: ...

Instead of resorting to those hacks, follow the PostgreSQL documentation[1] to do it the right way. The simplest way is to initialize your PostgreSQL database cluster as:

initdb --encoding=UTF8 --pgdata=

If you want to use an existing database cluster, PostgreSQL documentation shows how you can do it.

--

[1] - https://www.postgresql.org/docs/9.5/static/multibyte.html

Re: How to move from Amazon RDS to a dedicated PostgreSQL server

#34

Post by meritt »

qaq wrote: post #20
jdiscar wrote: post #15

Earlier quoted context omitted.

At what scale have you found Performance and Size Limitations an issue? I've seen people leave due to cost before (not that I thought they were always correct), but not for the other two you listed.

30,000 iops is 1/2 of a crappy consumer grade SSD. 6TB is fairly low for projects that generate any sizable amount of data. We don't run things in AWS unless it's strong client preference.

Not to mention 30k IOPS and 6TB is very expensive.

You're looking at $2700/mo for the IO provisioning and $750/mo for the 6TB of storage. Double those if you want Multi-AZ. Then you get to price the server size which I imagine is one of the more expensive options if you need 30k/6TB.

Re: How to move from Amazon RDS to a dedicated PostgreSQL server

#35

Post by mrmondo »

aftabh wrote: post #33

>To my surprise, PostgreSQL v9.5 on Ubuntu 14.04 does not default to unicode schema. PostgreSQL does (if unicode character set was specified when PostgreSQL database cluster is initialized first time); it's Ubuntu OS which creates PostgreSQL database cluster with ASCII character set (encoding) after PostgreSQL's installation. > Here is the snippet that I had to use: ... Instead of resorting to those hacks, follow the…

Ubuntu has lots of poor defaults and this is a good example of one.

Re: How to move from Amazon RDS to a dedicated PostgreSQL server

#36

Post by dboreham »

aftabh wrote: post #33

>To my surprise, PostgreSQL v9.5 on Ubuntu 14.04 does not default to unicode schema. PostgreSQL does (if unicode character set was specified when PostgreSQL database cluster is initialized first time); it's Ubuntu OS which creates PostgreSQL database cluster with ASCII character set (encoding) after PostgreSQL's installation. > Here is the snippet that I had to use: ... Instead of resorting to those hacks, follow the…

Using the distro-supplied packages for PG (or any database server) is typically not the best choice for a production deployment.

Re: How to move from Amazon RDS to a dedicated PostgreSQL server

#37

Post by peteretep »

dboreham wrote: post #36
aftabh wrote: post #33

>To my surprise, PostgreSQL v9.5 on Ubuntu 14.04 does not default to unicode schema. PostgreSQL does (if unicode character set was specified when PostgreSQL database cluster is initialized first time); it's Ubuntu OS which creates PostgreSQL database cluster with ASCII character set (encoding) after PostgreSQL's installation. > Here is the snippet that I had to use: ... Instead of resorting to those hacks, follow the…

Using the distro-supplied packages for PG (or any database server) is typically not the best choice for a production deployment.

    > Using the distro-supplied packages for PG
    > (or any database server)
Or almost any of your application environment, really.

Re: How to move from Amazon RDS to a dedicated PostgreSQL server

#38

Post by qaq »

Rezo wrote: post #31
qaq wrote: post #22

OK a bit of topic, but just curious we are considering rolling Postgres hosted service that would support much large instances up 50TB much higher IOPS 1,000,000+ lower latency (infiniband vs ethernet SAN) pricing would be in line with AWS RDS +/- optional ElasticSearch integration optional change notification pub/sub Would you guys be interested/think such service is a good idea? Would appreciate to learn what you g…

When it comes to data, trust is key. Think about it this way: OCZ has on paper SSDs that are simultaneously both faster and cheaper than, say, Intel. But I wouldn't put a OCZ drive into the lowest budged ricer gaming PC I could imagine (let's just say that have a bit of a reputation, to put it lightly), and I should be rightfully fired if I suggested putting one in a server. Yes, your service might be massively faste…

You do not have an option to host anything larger then 6TB on RDS. I guess what we are trying to figure out is if there is a need for service for people who have needs beyond those RDS is addressing.

Re: How to move from Amazon RDS to a dedicated PostgreSQL server

#39

Post by qaq »

jbrantly wrote: post #30
erkkie wrote: post #27

Earlier quoted context omitted.

How are you imagining solving latency issues? Many people want their DB to be close to their appservers. edit: and if it's an hosted service, I'm assuming decent networking options (ipsec et al)?

Exactly. I want my app server and my db server to be as close as possible. That's why services like Compose and ElephantSQL offer just that. So then it comes down to performance/price (value) between RDS and those.

Latency will be higher. We are considering if we do go forward: a) Partnerships with smaller providers where there is an option to colocate equipment b) Colocating as close to as possible to targeted providers.

The main thing we are trying to figure out - if there is a need for service that has significantly higher tears then what Azure, AWS offers.

Re: How to move from Amazon RDS to a dedicated PostgreSQL server

#40

Post by [deleted] »

qaq wrote: post #22

OK a bit of topic, but just curious we are considering rolling Postgres hosted service that would support much large instances up 50TB much higher IOPS 1,000,000+ lower latency (infiniband vs ethernet SAN) pricing would be in line with AWS RDS +/- optional ElasticSearch integration optional change notification pub/sub Would you guys be interested/think such service is a good idea? Would appreciate to learn what you g…

[deleted]
Post reply on HN