Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The part of Postgres we hate the most: Multi-version concurrency control

ottertune.com

Post reply on HN
Print view View original

71–80 of 148 posts

Re: The part of Postgres we hate the most: Multi-version concurrency control

#71

Post by p_l »

edmundsauto wrote: post #69
p_l wrote: post #59

Earlier quoted context omitted.

Honestly, at the time when LAMP was gaining the userbase, said userbase for considerable portion did not care about replication because there was only one server they had . Replication was something you did when you got succesful enough to have it, or were a MSP providing it at premium to others.

I remember it differently - we needed replication for "hot" backups. At that time, scalability was a major issue - so anyone (including businesspeople) wanted to have a scalable architecture. MySQL spoke to the practical (default install on cPanel hosts, easy replication) and the aspirational (you're going to blow up and need to scale). Digg.com also had a really influential technical team - hearing about how they di…

maybe you were on the more funded side of history in this. As for me, Digg is way after LAMP got solidly plonked into "what I need for a dynamic website on cheap".

Essentially, start at 2000-2001 and more and more people going into running websites for all kinds of reasons (forums, blogs, webshops, etc. often hosted on low end offerings)

Re: The part of Postgres we hate the most: Multi-version concurrency control

#72

Post by j16sdiz »

Max-Ganz-II wrote: post #36
masklinn wrote: post #34

Earlier quoted context omitted.

The problem is not mvcc but postgres’ implementation details of it.

In what way? I didn't see anything obviously improper when I learned how serialization isolation worked.

A delicated full vacuum process that need full locking, how little isolation between tables, etc

Lots of the pain point have been mitigated in the last ten years. It is now as simple as other comparable complex db can go (i.e. Not simple, but you can't find better product)

Re: The part of Postgres we hate the most: Multi-version concurrency control

#73

Post by zeroimpl »

> Oracle and MySQL do not have this problem in their MVCC implementation because their secondary indexes do not store the physical addresses of new versions. Instead, they store a logical identifier (e.g., tuple id, primary key) that the DBMS then uses to look up the current version’s physical address.

This doesn’t have anything to do with MVCC. I’m sure PostgreSQL could implement an index format that piggybacks on another index rather than pointing at the physical page directly, without overhauling MVCC.

Re: The part of Postgres we hate the most: Multi-version concurrency control

#74

Post by klysm »

DiabloD3 wrote: post #62
cyberax wrote: post #30

Yup. A lot of heavy users of Postgres eventually hit the same barrier. Here's another take from Uber: https://www.uber.com/blog/postgres-to-mysql-migration/ I had a similar personal experience. In my previous job we used Postgres to implement a task queuing system, and it created a major bottleneck, resulting in tons of concurrency failures and bloat. And most dangerously, the system failed catastrophically under loa…

I remember when Uber got roasted by the postgresql mailing list over this: ultimately, a post mortem was done on all of Uber's claims, and it was basically proven that they were incompetent, did not read any available "best practices" guides, did not seek any external help, and treated it like it was some sort of mysql-esque database and used it as wrong as humanly possible. Uber's workload at the time, ironically, w…

Migrating entire workload is way more fun and exciting than reading manual. How else are you going to demonstrate your impact!

Re: The part of Postgres we hate the most: Multi-version concurrency control

#75

Post by endorphine »

DiabloD3 wrote: post #62
cyberax wrote: post #30

Yup. A lot of heavy users of Postgres eventually hit the same barrier. Here's another take from Uber: https://www.uber.com/blog/postgres-to-mysql-migration/ I had a similar personal experience. In my previous job we used Postgres to implement a task queuing system, and it created a major bottleneck, resulting in tons of concurrency failures and bloat. And most dangerously, the system failed catastrophically under loa…

I remember when Uber got roasted by the postgresql mailing list over this: ultimately, a post mortem was done on all of Uber's claims, and it was basically proven that they were incompetent, did not read any available "best practices" guides, did not seek any external help, and treated it like it was some sort of mysql-esque database and used it as wrong as humanly possible. Uber's workload at the time, ironically, w…

Anyone has a link to that mailing list thread to share?

Re: The part of Postgres we hate the most: Multi-version concurrency control

#76

Post by zie »

dalyons wrote: post #49
zie wrote: post #29

Earlier quoted context omitted.

I would argue, you handle the hyper-scale use case when you are actually in hyper-scale. Trying to pre-maturely optimize this is almost always a waste of time and chances are you will screw it up anyway. Almost nobody gets to that scale anyway. If you do get to that scale, you have the money and resources to fix the problem(s) at that time.

i mean, sort of? There is some subtly lost in this oft-repeated advice. i've worked at 3 companies now that were initially based on a single RDBMS but have outgrown the scale of what is reasonable to serve off that architecture. They are consumer scale (10s of mill) users, but not hyperscale (IMHO 100m+). The amount of engineering cost to migrate a complicated growing company/product off a mono-db architecture is ast…

Of course subtlety matters, but as you start scaling and noticing pain points, that is when you start working towards fixing them. First you just throw hardware at the problem and that tends to scale really really well for a really long time. It's pretty rare, even at very large scale that you MUST move off of PG, there are plenty of well tested scaling solutions, if you have the $$$'s to spend.

10+ years of dev work for a few hundred tables worries me a lot. My last conversion was about 20 years of data across a few hundred tables and we did two-way data synchronization across DB products with about 1 month of work, with 2 devs. We kept the sync running for over a year in production because we didn't want to force users over to the new system in a big hurry. We only stopped because the license on the old DB product finally expired and nobody wanted to pay for it anymore.

Re: The part of Postgres we hate the most: Multi-version concurrency control

#77

Post by emmelaich »

DiabloD3 wrote: post #62
cyberax wrote: post #30

Yup. A lot of heavy users of Postgres eventually hit the same barrier. Here's another take from Uber: https://www.uber.com/blog/postgres-to-mysql-migration/ I had a similar personal experience. In my previous job we used Postgres to implement a task queuing system, and it created a major bottleneck, resulting in tons of concurrency failures and bloat. And most dangerously, the system failed catastrophically under loa…

I remember when Uber got roasted by the postgresql mailing list over this: ultimately, a post mortem was done on all of Uber's claims, and it was basically proven that they were incompetent, did not read any available "best practices" guides, did not seek any external help, and treated it like it was some sort of mysql-esque database and used it as wrong as humanly possible. Uber's workload at the time, ironically, w…

Do you have a link to the mailing list discussion?

Re: The part of Postgres we hate the most: Multi-version concurrency control

#78

Post by rodgerd »

zaphirplane wrote: post #47
mmaunder wrote: post #12

I must admit as a web practitioner since 1994 I have a bit of an issue with this: > In the 2000s, the conventional wisdom selected MySQL because rising tech stars like Google and Facebook were using it. Then in the 2010s, it was MongoDB because non-durable writes made it “webscale“. In the last five years, PostgreSQL has become the Internet’s darling DBMS. And for good reasons! Different DB's, different strengths and…

Part of the popularity of the early MySQL was marketing. I hope I’m not wrong here, there was something written about MySQL people posting misinformation in forums. Another is the ease of having it up and running. Another was I think there was some IP address component to setting up users which made it look complicated

> there was something written about MySQL people posting misinformation in forums.

They were absolute liars of the first water back in the day, absolutely. In the 3.x era there were claims that transactions were only for people who didn't know how to program! You'd struggle to find most of the absolute nonsense that was being pushed, because it's mostly gone down various memory holes, but it was absolutely breathtaking.

Re: The part of Postgres we hate the most: Multi-version concurrency control

#79

Post by emmelaich »

spprashant wrote: post #13
joevandyk wrote: post #11

Earlier quoted context omitted.

I've personally ran into the problems mentioned in the article many times, unsure it's "overhyped".

It's a problem, but not an AI problem. It has a clear cause and obvious mitigation strategies.

If there was a clear cause and obvious mitigation strategies then they would have been built into Postgres already.

Re: The part of Postgres we hate the most: Multi-version concurrency control

#80

Post by lmm »

mmaunder wrote: post #12

I must admit as a web practitioner since 1994 I have a bit of an issue with this: > In the 2000s, the conventional wisdom selected MySQL because rising tech stars like Google and Facebook were using it. Then in the 2010s, it was MongoDB because non-durable writes made it “webscale“. In the last five years, PostgreSQL has become the Internet’s darling DBMS. And for good reasons! Different DB's, different strengths and…

> MongoDB gained traction not because it's an alternative to MySQL or PostgreSQL.

Disagree. It gained traction because it was an alternative to MySQL in the ways that mattered - fast, easy to administer, widely known, good enough. Yes, there are significant differences in the details of what they do - but in terms of someone looking for a backing datastore for their webapp, they're actually competing in a very similar space.

Post reply on HN