41–50 of 306 posts
Earlier quoted context omitted.
pgbouncer works pretty well for that. I kinda wish it was part of the default Postgres install.
How does pgbouncer work with Amazon RDS (where you can't install separate software)?
Accordingly, using pgbouncer to do connection pooling with
Postgres has been generally successful for us. However, we have
had occasional application bugs in our backend services that
caused them to open more active connections (usually “idle in
transaction” connections) than the services ought to be using,
and these bugs have caused extended downtimes for us.We used a 2 column InnoDB-backed table for all of our data storage, massively sharded, and run in a 3-host master-slave-slave configuration.
At that time EC2 would routinely kill hosts without the courtesy of a poke via ACPI and as such we became very good at quickly recovering shards. In a nutshell this mechanism was to have the new host contact a backup slave, perform an lvm snap, pipe the compressed snap over a TCP connection, unroll it and carry on, letting replication take up the delta.
That enabled us to not only manage the 10 million or so daily active users of that title, but was also the platform under the 12 or so additional titles that studio had.
We had lots and lots of very simple things and failures were contained.
I think at the time we were the 3rd-largest consumer of EC2 after Netflix and "another" outfit I never learned the name of. EA being what it was, however, we were never permitted to open source a lot of the cool stuff Netflix and ourselves seemed to develop in parallel.
Posts like this are important. We too often rely on a buzz-word heuristic and that's how you end up with dozens of random technologies that are harder to maintain and don't necessarily solve any of your problems. This method is good, because it shows that when you understand the problem the right way, you can find the right solution, even if by popularity it looks like a "step backwards" Massive Kudos.
I strongly disagree. it would have been useful f they'd stuck to problems without well-known solutions. Sadly, they also mixed in issues which are easily solved, or in a particularly egregious case, where they just complain about a bug. As though MySQL never had a bug. That was silly. My read of it was: Postgres annoyed us a few times, and we got fed up with its, so now something different will annoy us. Please look…
Posts like this are important. We too often rely on a buzz-word heuristic and that's how you end up with dozens of random technologies that are harder to maintain and don't necessarily solve any of your problems. This method is good, because it shows that when you understand the problem the right way, you can find the right solution, even if by popularity it looks like a "step backwards" Massive Kudos.
I strongly disagree. it would have been useful f they'd stuck to problems without well-known solutions. Sadly, they also mixed in issues which are easily solved, or in a particularly egregious case, where they just complain about a bug. As though MySQL never had a bug. That was silly. My read of it was: Postgres annoyed us a few times, and we got fed up with its, so now something different will annoy us. Please look…
Doing so deliberately is whats commendable.
this reads like a laundry list of buzzwords that were designed to justify not throwing any effort into postgresql and just going with a new shiny toy (not mysql. yes. i know it's been around for a while). it happens everywhere.
The amount of research about the on-disk internals of both PostgreSQL and MySQL are a lot more effort than I would have probably spent (granted, I don't have a team of highly paid devs at my disposal, but still, I've seen technical decisions made on the basis of Google Trends...).
this reads like a laundry list of buzzwords that were designed to justify not throwing any effort into postgresql and just going with a new shiny toy (not mysql. yes. i know it's been around for a while). it happens everywhere.
The "laundry list of buzzwords" is a detailed, up to engineering standards, analysis of the issues they faced, with example cases and explanatory follow-up.
As far away from a "laundry list of buzzwords" as you could possibly get. Not to mention they did the very opposite of switching willy nilly to some new "shiny toy".
In other words, the comment is not even wrong.
This is a fantastic read. I hope the pg folks can turn as many of the issues brought up here into bug reports as possible (I think many of the issues, especially re: replication, are known), this kind of feedback is invaluable.