HBase was a joy to use from a application developer standpoint.
You should check out Bigtable.
HBase Deprecation at Pinterest
31–40 of 59 posts
Re: HBase Deprecation at Pinterest
#32Shocking to see a company like Pinterest has built multiple in house data stores. Talking about a maintenance burden...
This is my first reaction as well. How in the world do you end up with close to 10 different databases in production? And furthermore, most of which are totally proprietary? Just use Postgres...
Not sure if I can get you to 10, but different DBs work at different scales for different workloads. There's relational, key-value, caching, time series, data warehouse, and search index.
Re: HBase Deprecation at Pinterest
#33Wow, never realized Pinterest had 6 petabytes of data. I wonder if they're including images in that. Even billions of rows is usually around 1-2tb, so makes you wonder what they're storing many billions of.
In NoSQL world, you often store multiple duplicates of the same data, each of which is optimized for a different use case.
The SQL world calls this an "index."
Re: HBase Deprecation at Pinterest
#34Shocking to see a company like Pinterest has built multiple in house data stores. Talking about a maintenance burden...
If you rewind 10 years to a time before waves of big tech layoffs and the market for engineers was very much in favor, you'll find your answer. The explosive growth in hiring coupled with FOMO-driven risk taking resulted in many projects which were essentially resumeware for engineers. This environment permitted people to escape strategic thinking and any consequences. Imagine having to deal with multiple data stores…
At the time there were _no_ tools that scaled to what people needed so they had to build their own.
It's easy to look back and think we were idiots for doing it that way but you need to remember that the average server in 2014 has the power of the average smartphone today.
Re: HBase Deprecation at Pinterest
#35Shocking to see a company like Pinterest has built multiple in house data stores. Talking about a maintenance burden...
This is my first reaction as well. How in the world do you end up with close to 10 different databases in production? And furthermore, most of which are totally proprietary? Just use Postgres...
>Just use Postgres
Now try running Postgres on vintage hardware from the period and you quickly see why we did what we did.
Re: HBase Deprecation at Pinterest
#36For companies like Pinterest, where data storage isn't the core business, should the focus be on building in-house data warehouses or leveraging managed service providers (MSPs)? While building in-house offers control and customization, MSPs can potentially address complexity and infrastructure costs.
Can someone from Pinterest comment on specific performance needs (SLOs) that influenced their choice between TiDB and other solutions, including managed services? Considering complexity and cost, could an MSP have addressed their needs effectively?
Re: HBase Deprecation at Pinterest
#37Re: HBase Deprecation at Pinterest
#38Earlier quoted context omitted.
In NoSQL world, you often store multiple duplicates of the same data, each of which is optimized for a different use case.
The SQL world calls this an "index."
No. Most NoSQL dbs support indexes and secondary indexes too.
In SQL world we call this "denormalization". It's called the same thing in NoSQL, too.
Re: HBase Deprecation at Pinterest
#39Earlier quoted context omitted.
In NoSQL world, you often store multiple duplicates of the same data, each of which is optimized for a different use case.
The SQL world calls this an "index."
Indices are no good if you constantly do large amounts of inserts or updates (say, entire clickstream of a popular global site) - keeping them up to date will massively slow down the inserts/updates. Wheresas, in NoSQL, you can do as inserts as quickly as your disk can write data, and still query the dataset as if it had an index.
Re: HBase Deprecation at Pinterest
#40Shocking to see a company like Pinterest has built multiple in house data stores. Talking about a maintenance burden...
This was the norm about a decade ago. When I was at Hulu, we built our own analytics platform on top of Hadoop, we almost certainly wouldn't have done that today with the amount of off-the-shelf stuff available.
Even things like protobuf or Avro weren't as broadly adopted (>80%) at the time, many many companies at the time persisted stuff using JSON or other text formats (which in retrospect was very dumb, but it was very normal for a while).