Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

HBase Deprecation at Pinterest

medium.com

Post reply on HN
Print view View original

21–30 of 59 posts

Re: HBase Deprecation at Pinterest

#21

Post by softwaredoug »

rwultsch wrote: post #16

“Introduced in 2013, HBase was Pinterest’s first NoSQL datastore.” I don’t think this is correct. When I started in late 2013 Redis was being used as a persistent data store. And what pain it was. I convinced leadership in late 2014 this was a bad and they had me keep it alive until it was replaced by MySQL in mid 2015. HBase was nothing but pain at Facebook where it was supposed to replace MySQL and then Pinterest w…

I had a (low traffic) app with Redis as the only data store, that we'd periodically dump to disk and copy the dump off for backup. It was... not great for anyone who was used to maintaining a normal web app. And it broke down because the underlying relationships we mostly cared about were, in fact, relational. So you got a lot of duplication / nesting, homegrown hierarchy, or pointers to other areas in Redis...

Re: HBase Deprecation at Pinterest

#22

Post by eclark »

pornel wrote: post #11

> the HBase version upgrade is a slow and painful process due to a legacy build/deploy/provisioning pipeline and compatibility issues Is that HBase's fault, or Pinterest's added complexity? I'm baffled when databases don't support seamless in-place upgrades, and require a full dump and restore instead. At certain scale a full rebuild is as complex as replacing wheels of a moving car.

HBase supports in-place upgrades. Almost* all version upgrades have been relatively painless if you have automation to do the necessary operations across a cluster of nodes. Hadoop upgrades have been a similar story. That minimum automation necessary is high for all stateful data stores of this size.

* A few notable exceptions exist where Hadoop and HBase upgrades were necessary simultaneously. These have been awful experiences that required huge efforts to accomplish.

Re: HBase Deprecation at Pinterest

#24

Post by eclark »

> Production HBase clusters typically used a primary-standby setup with six data replicas for fast disaster recovery, which, however, came at an extremely high infra cost at our scale.

This is the real killer. HBase uses Hadoop for its replication. That replication does 3 copies in the same data center. If you're a company that requires online data of this scale, you probably also have several other data centers or clouds. Having to replicate entire datasets 3x for every new data center is cost-prohibitive.

That, along with the fact that HBase has issues at NVMe speeds and throughputs, are true issues.

Re: HBase Deprecation at Pinterest

#25

Post by doctorpangloss »

lopkeny12ko wrote: post #14
redwood wrote: post #8

Shocking to see a company like Pinterest has built multiple in house data stores. Talking about a maintenance burden...

This is my first reaction as well. How in the world do you end up with close to 10 different databases in production? And furthermore, most of which are totally proprietary? Just use Postgres...

Your application wants to use Postgres.

Kubernetes, which it runs on, wants etcd.

Keycloak wants to use Postgres and Redis for sessions.

Plausible Analytics wants to use Clickhouse and Postgres. Matomo specifically wants to use MySQL.

Lago Billing wants to use Postgres, Redis and Clickhouse.

Prometheus uses TSDB, but okay, it’s “just” files and a WAL. Wait a minute, so is Postgres…

Jaeger wants to use Cassandra or Elastic.

Thanos wants S3 APIs. So maybe you use Minio.

Okay. That’s 2024: in production, you will wind up with 7 different databases. At least.

Re: HBase Deprecation at Pinterest

#26

Post by std_reply »

redwood wrote: post #7

Great to see TiDB mentioned for use here, I have to say I love their demo at ossinsights. Anyone else using them successfully in prod?

AirBnB, Databricks, Flipkart 3 of the largest banks in the world, some of the largest logistics companies in the world, at least 2K seriously large installations.

Re: HBase Deprecation at Pinterest

#27

Post by openplatypus »

aeyes wrote: post #19
lopkeny12ko wrote: post #14

Earlier quoted context omitted.

This is my first reaction as well. How in the world do you end up with close to 10 different databases in production? And furthermore, most of which are totally proprietary? Just use Postgres...

Easy: These companies have a promotion process which require you to generate "impact" to get promoted. Using existing but boring technology which solves the problem in an efficient manner apparently doesn't show off how smart you are. This is not a snarky comment from an outsider, I very much work at such a company. It's ridiculous. Edit: But I must admit that most of this was probably developed 10 years ago when the…

So sad because so true.

Seeing working technology replaced because someone doesn't like it, get accolades for it, only to face the same limitations as old tech is just depressing.

Re: HBase Deprecation at Pinterest

#28

Post by aeyes »

openplatypus wrote: post #27
aeyes wrote: post #19

Earlier quoted context omitted.

Easy: These companies have a promotion process which require you to generate "impact" to get promoted. Using existing but boring technology which solves the problem in an efficient manner apparently doesn't show off how smart you are. This is not a snarky comment from an outsider, I very much work at such a company. It's ridiculous. Edit: But I must admit that most of this was probably developed 10 years ago when the…

So sad because so true. Seeing working technology replaced because someone doesn't like it, get accolades for it, only to face the same limitations as old tech is just depressing.

You forgot the next step: The people who developed it move on to other companies and nobody wants to touch it.

Re: HBase Deprecation at Pinterest

#30

Post by dehrmann »

I worked for a place that used HBase heavily. They migrated from AWS to GCP ~just for BigTable since the data model is essentially the same. They're a Java shop, and the drivers are actually the same. The workload of managing HBase and HDFS was high, and it was unreliable enough that they always had a failover cluster set up. Interestingly, the migration surfaced degenerate cells/tables that might have been partially to blame for reliability issues. Even 5 years ago, HBase was slowly dying, and at the end of the day, they didn't want the company's core competency to be HBase management.
Post reply on HN