11–20 of 59 posts
Is that HBase's fault, or Pinterest's added complexity?
I'm baffled when databases don't support seamless in-place upgrades, and require a full dump and restore instead. At certain scale a full rebuild is as complex as replacing wheels of a moving car.
> the HBase version upgrade is a slow and painful process due to a legacy build/deploy/provisioning pipeline and compatibility issues Is that HBase's fault, or Pinterest's added complexity? I'm baffled when databases don't support seamless in-place upgrades, and require a full dump and restore instead. At certain scale a full rebuild is as complex as replacing wheels of a moving car.
Is every article written by ChatGPT now? My confusion was partially answered as soon as I saw the word "delve."
Shocking to see a company like Pinterest has built multiple in house data stores. Talking about a maintenance burden...
This article has so many words yet has such little information, and is remarkably sparse in technical detail. What did they actually do? What did they build? How did they migrate? What is "SDS"? Is every article written by ChatGPT now? My confusion was partially answered as soon as I saw the word "delve."
Also this article doesn't feel like it's written by ChatGPT at all. "Delve" is not even a very uncommon word; just one use of it isn't necessarily indicative, and even if it was, it's used in the summary of the rest of the series (which you seem to have missed in your hunt for gotchas)! I think LLM bullshit is definitely making everything a lot worse but this isn't even an example of such.
I don’t think this is correct. When I started in late 2013 Redis was being used as a persistent data store. And what pain it was. I convinced leadership in late 2014 this was a bad and they had me keep it alive until it was replaced by MySQL in mid 2015.
HBase was nothing but pain at Facebook where it was supposed to replace MySQL and then Pinterest where… I think there was hope it would replace MySQL. Once I automated MySQL at Pinterest I think it wasn’t so bad, particularly given the absurdly limited staff they gave the problem.
Wow, never realized Pinterest had 6 petabytes of data. I wonder if they're including images in that. Even billions of rows is usually around 1-2tb, so makes you wonder what they're storing many billions of.
Shocking to see a company like Pinterest has built multiple in house data stores. Talking about a maintenance burden...
Imagine having to deal with multiple data stores for your daily development but then go on-call for a week and have to deal with twice as many plus the inevitable lack of runbooks etc.? I personally have a big tech experience with the same data store proliferation in an organization with pretty solid RDBMS use. In the last 3-4 years we've been undoing the damage and migrating data. Toil for engineers is lower and uptimes are better.
Shocking to see a company like Pinterest has built multiple in house data stores. Talking about a maintenance burden...
This is my first reaction as well. How in the world do you end up with close to 10 different databases in production? And furthermore, most of which are totally proprietary? Just use Postgres...
This is not a snarky comment from an outsider, I very much work at such a company. It's ridiculous.
Edit: But I must admit that most of this was probably developed 10 years ago when the ecosystem was much more limited. At least they now came to the conclusion that it's time to move on. I would not have recommended Postgres 10 years ago and even today there is no way it would work at the scale of Pinterest.
Wow, never realized Pinterest had 6 petabytes of data. I wonder if they're including images in that. Even billions of rows is usually around 1-2tb, so makes you wonder what they're storing many billions of.
For one comparison, Tumblr hit 100 billion unique rows of relational data in MySQL (on masters, not including replicas) back in October 2012. So they're easily in the trillions of rows today, and Tumblr is smaller than Pinterest!