Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why doesn't Quora use PostgreSQL

quora.com

Post reply on HN
Print view View original

51–60 of 78 posts

Re: Why doesn't Quora use PostgreSQL

#51

Post by petrilli »

blueberry wrote: post #38

Those who commented in favor of postgres: What are some of the biggest deployments of postgres? Facebook has a gargantic deployment of MySQL and it just works. The author of the answer on Quora mentions reliable scalability as the most important factor for their MySQL choice on another answer here: http://bit.ly/dm6HtQ . What is the biggest system that postgres is deployed in?

A better question would be: who uses MySQL as anything but a glorified key-value store when reaching large sizes? I personally have been involved in trying to use MySQL as a "real database" in the 10-100TB range and let me tell you, it's not pretty. I'm not sure about the open source PostgreSQL, but I know Greenplum has petabyte level warehouses running on a distributed version of it.

A few observations:

1) MyISAM's performance is highly dependent on certain idiosyncrasies of a lot of applications. Using MyISAM in this day and age is a very bad idea. InnoDB at least gets closer to real database behavior.

2) The "query optimizer" is insulting at best and actively impeding getting things right if you use it for much more than simple queries. Something that's more along the lines of what really large databases (as opposed to KV stores) get used for can implode the server.

Personally, I think too many people try and stick things in relational databases that don't belong there simply because they've got the hammer in their hand and it's easier than pulling out a screwdriver.

Re: Why doesn't Quora use PostgreSQL

#52

Post by pilif »

andrewvc wrote: post #22
joshu wrote: post #17

there's one useful thing about postgresql: specifically, interviewees that inform you you should be using postgresql can be immediately rejected. "do you use mysql or postgresql?" "oh, mys-" "YOU SHOULD SWITCH TO POSTGRESQL!!"

I think this can be extended to any interviewee who starts redesigning your stack before they're even hired can be rejected.

I don't know. Personally, I'd prefer an interviewee that comes to the interview with an open mind and actually tries to make suggestions based on the view of the infrastructure that they have acquired based on the information I have given them or they inferred in their research of my company.

Of course they might be wrong because naturally, they can't have the full picture. But I'd much rather have an employee I can hold a discussion with and get input from then one that blindly does what I tell them to. In the end, I might as well have made a bad decision or overlooked something.

In fact, having an engaging discussion during the interview is quite a sure-fire way to get hired if you are in an interview with me (provided the stuff you talk about makes sense).

Re: Why doesn't Quora use PostgreSQL

#53

Post by rmc »

jasonkester wrote: post #49
rubyrescue wrote: post #47

Earlier quoted context omitted.

irc chat, generally on the 'freenode' irc server - http://freenode.net/

Wow, really? Chat? In 2010??? That would never have occurred to me. Thanks!

IRC is still heavily used in the open source world. It's a great resource and you should add it to your arsenal of 'places to get support on FLOSS stuff'

Re: Why doesn't Quora use PostgreSQL

#54

Post by efsavage »

Excellent lesson to other startups: When two or more products do what you need, choose the one you can more easily hire help for.

(not that there aren't a fair number of postgres experts, but there's no question there are far more competent mysql experts)

Re: Why doesn't Quora use PostgreSQL

#55

Post by masklinn »

efsavage wrote: post #54

Excellent lesson to other startups: When two or more products do what you need, choose the one you can more easily hire help for. (not that there aren't a fair number of postgres experts, but there's no question there are far more competent mysql experts)

No question, really?

Re: Why doesn't Quora use PostgreSQL

#56

Post by masklinn »

_delirium wrote: post #26
scg wrote: post #19

It's interesting to see how PostgresSQL didn't get nowhere near the popularity of MySQL. I think a few major things contributed to this: 1) MySQL focused on performance first while PostgresSQL positioned themselves as "feature-complete." Of course, that was years ago and in the meantime they became comparable in both aspects. MySQL+InnoDB is ACID compliant and Postgres matches the InnoDB performance. Sure, Postgres m…

I think having a "launch" at the right time might have had some impact also. MySQL launched in 1995, right around the time that the idea of generating hypertext dynamically was taking off, so a bunch of web stuff was looking for a lightweight, fast DB without necessarily a lot of features. Postgres had been around since the 1980s as a research project, and in 1994 was publicly released as an open-source project and a…

Postgres was also a bitch to install and work with in the early years, even on really supported platforms.

The usability push started with the performance one, from 7.x onwards (2000) and especially with 8.x (2005).

Re: Why doesn't Quora use PostgreSQL

#57

Post by blueberry »

petrilli wrote: post #51
blueberry wrote: post #38

Those who commented in favor of postgres: What are some of the biggest deployments of postgres? Facebook has a gargantic deployment of MySQL and it just works. The author of the answer on Quora mentions reliable scalability as the most important factor for their MySQL choice on another answer here: http://bit.ly/dm6HtQ . What is the biggest system that postgres is deployed in?

A better question would be: who uses MySQL as anything but a glorified key-value store when reaching large sizes? I personally have been involved in trying to use MySQL as a "real database" in the 10-100TB range and let me tell you, it's not pretty. I'm not sure about the open source PostgreSQL, but I know Greenplum has petabyte level warehouses running on a distributed version of it. A few observations: 1) MyISAM's…

who uses MySQL as anything but a glorified key-value store when reaching large sizes?

My assumption is both Quora and Facebook use MySQL this way. While you are right that this is not using it as a real database, I want to know if PostgreSQL is deployed in a similar setup at all. Most of the people (including you) don't take into account the fact that there are many cases where MySQL (used as a KV store) proved to work, while I have never heard of such huge PostgreSQL deployments. If this was a general discussion regarding MySQL and PostgreSQL I could understand that, however, I think the post is more about whether to choose MySQL or PostgreSQL if you are going to use it as a KV store.

Re: Why doesn't Quora use PostgreSQL

#58

Post by vark »

blueberry wrote: post #38

Those who commented in favor of postgres: What are some of the biggest deployments of postgres? Facebook has a gargantic deployment of MySQL and it just works. The author of the answer on Quora mentions reliable scalability as the most important factor for their MySQL choice on another answer here: http://bit.ly/dm6HtQ . What is the biggest system that postgres is deployed in?

From http://glinden.blogspot.com/2008/05/yahoo-builds-two-petabyt... :

Yahoo builds two petabyte PostgreSQL database James Hamilton writes about Yahoo's "over 2 petabyte repository of user click stream and context data with an update rate for 24 billion events per day".

It apparently is built on top of a modified version of PostgreSQL and runs on about 1k machines. In his post, James speculates on the details of the internals. Very interesting.

Re: Why doesn't Quora use PostgreSQL

#59

Post by jeltz »

metabrew wrote: post #40
narrator wrote: post #29

Earlier quoted context omitted.

The replication is reliable, it's just that with the 8.0 releases, it's an add-on and not obvious to setup (e.g with Slony have to do special procedures to properly replicate DDL changes, it required having triggers on replicated tables, etc). In 9.0 replication is built in, transparent, and trivial to setup.

Sorry, but pre-9.0, Postgres replication is neither reliable nor easy to setup/admin. It's absolutely the worst thing about (pre-9.0) Postgres, and I'm glad the core team finally caved and made it a first-class feature. Slony caused us no end of problems (large postgres installations at Last.fm). It's a monumental pain in the ass, it will choke on something, probably when you're trying to make a DDL change, sometimes…

You are correct about Slony but if you do not care about doing any reads at the slave in pre-9.0 you had log shipping and warm standby. Warm standby is perfect for using the slave for failover in case the master crashes or the HDDs break.

Warm standby is very reliable and easy to admin. The setup is not that easy but not too hard either.

Re: Why doesn't Quora use PostgreSQL

#60

Post by gxti »

masklinn wrote: post #25
Empact wrote: post #9

A few bugs/limitations of Mysql not shared by Postgres. All AFAIK. * Integer columns with null constraints will silently translate nulls to 0s, rather than throwing an error (as constraints in all other contexts do). Imagine a database which, on violation of a foreign key constraint, just silently picked the first row in the target table - it's just insane. * Transactions can't roll back schema changes, so if you hav…

> Transactions can't roll back schema changes That's a feature of postgres rather than a limitation of mysql however. As far as I know the SQL Standard doesn't ask for transactional DDL, Oracle has pretty much no support for transactional DDL (in 10g anyway, each DDL statement is executed in its own transaction and any pending transaction is COMMITted before a DDL), I have no idea about SQL Server.

MSSQL supports it because it was inherited from Sybase. SQLite also supports it as well. You're right that it's a feature, but frankly I can't live without it -- there's no way to write safe schema migrations!
Post reply on HN