Live data from Hacker News

HBase Deprecation at Pinterest

medium.com

51–59 of 59 posts

Re: HBase Deprecation at Pinterest

#51
post #39

Earlier quoted context omitted.

The SQL world calls this an "index."

Indices are no good if you constantly do large amounts of inserts or updates (say, entire clickstream of a popular global site) - keeping them up to date will massively slow down the inserts/updates. Wheresas, in NoSQL, you can do as inserts as quickly as your disk can write data, and still query the dataset as if it had an index.

That has nothing to do with whether the DB is relational or not.

Re: HBase Deprecation at Pinterest

#52
post #38

Earlier quoted context omitted.

The SQL world calls this an "index."

No. Most NoSQL dbs support indexes and secondary indexes too. In SQL world we call this "denormalization". It's called the same thing in NoSQL, too.

Indices are essentially managed denormalization.

Re: HBase Deprecation at Pinterest

#53
post #2

Wow, never realized Pinterest had 6 petabytes of data. I wonder if they're including images in that. Even billions of rows is usually around 1-2tb, so makes you wonder what they're storing many billions of.

They almost certainly have trillions of rows of data . The popular social media / user-generated content sites are just huge. You can easily get to that size just from the core product OLTP data/metadata, no need to include media. For one comparison, Tumblr hit 100 billion unique rows of relational data in MySQL (on masters, not including replicas) back in October 2012 . So they're easily in the trillions of rows tod…

Just surprising since I've never seen the social side or anyone using it. Guess I'm just not in that social bubble.

Re: HBase Deprecation at Pinterest

#54
post #53

Earlier quoted context omitted.

They almost certainly have trillions of rows of data . The popular social media / user-generated content sites are just huge. You can easily get to that size just from the core product OLTP data/metadata, no need to include media. For one comparison, Tumblr hit 100 billion unique rows of relational data in MySQL (on masters, not including replicas) back in October 2012 . So they're easily in the trillions of rows tod…

Just surprising since I've never seen the social side or anyone using it. Guess I'm just not in that social bubble.

Woman use it a lot.

My wife has a Pinterest board.

Re: HBase Deprecation at Pinterest

#55
post #38

Earlier quoted context omitted.

No. Most NoSQL dbs support indexes and secondary indexes too. In SQL world we call this "denormalization". It's called the same thing in NoSQL, too.

Indices are essentially managed denormalization.

Ah, never thought of it that way.

Re: HBase Deprecation at Pinterest

#56
post #51
post #39

Earlier quoted context omitted.

Indices are no good if you constantly do large amounts of inserts or updates (say, entire clickstream of a popular global site) - keeping them up to date will massively slow down the inserts/updates. Wheresas, in NoSQL, you can do as inserts as quickly as your disk can write data, and still query the dataset as if it had an index.

That has nothing to do with whether the DB is relational or not.

Which relational databases can simultaneously do milions of updates and milions of selects per second, to the same table?

Re: HBase Deprecation at Pinterest

#57
post #53

Earlier quoted context omitted.

They almost certainly have trillions of rows of data . The popular social media / user-generated content sites are just huge. You can easily get to that size just from the core product OLTP data/metadata, no need to include media. For one comparison, Tumblr hit 100 billion unique rows of relational data in MySQL (on masters, not including replicas) back in October 2012 . So they're easily in the trillions of rows tod…

Just surprising since I've never seen the social side or anyone using it. Guess I'm just not in that social bubble.

Pinterest has over 500 million monthly active users. Their revenue was $3 billion in 2023.

Re: HBase Deprecation at Pinterest

#59
post #7

Great to see TiDB mentioned for use here, I have to say I love their demo at ossinsights. Anyone else using them successfully in prod?

AirBnB, Databricks, Flipkart 3 of the largest banks in the world, some of the largest logistics companies in the world, at least 2K seriously large installations.

Very cool. So many upstart SQL players: TiDB, PlanetScale, Crunchy, Cockroach, Neon, Supabase, Nile, Yugabyte, Aiven, let alone the cloud provider options (AlloyDB, Spanner, Aurora, Cosmos). Kind of mind boggling there's room for all these players
Post reply on HN