Just off the top of my head I can think of a few cases where we easily have 100-1000x that on a single table in shared node on kube, with an ebs volume (point being, bog standard, if a tad large, commodity hardware). If you properly index things and your workload isn't too crazy in a way that needs access to all of this at once (e.g. point lookups with some recency bias), classic RDBMSs have this use case covered - and it's achievable by anyone!
Is 20M of rows still a valid soft limit of MySQL table in 2023?
41–50 of 86 posts
Re: Is 20M of rows still a valid soft limit of MySQL table in 2023?
#42Earlier quoted context omitted.
My current employer has a single InnoDB table that is 14-15 TB.
How long does it take you to do a schema change?
Big MySQL deployments typically use pauseless online schema changes tooling like gh-ost:
https://planetscale.com/docs/learn/online-schema-change-tool...
Re: Is 20M of rows still a valid soft limit of MySQL table in 2023?
#43You can safely go to a billion rows. Use indexes. Don't do silly things like count(*) or select *, select the small data sets you need using your indexes. If you need a full table count for some reason, use a primary key that auto increments and select the max, or use information_schema for an estimate. Nobody will sweat the slight inaccuracy on a table that size.
yeah i was also under the 20mil impression initially until building a system that worked with a 500mil row table without any problems (just a beefy RDS instance, nothing crazy distributed or anything). Schema changes became a pain at that point, so i would probably split it if I were to do it again, but reads and writes never gave us much trouble.
What were you using for schema changes? I’ve been using pt-online-schema-change on InnoDB tables with 200m+ rows for a while, it’s just a bit slow.
Re: Is 20M of rows still a valid soft limit of MySQL table in 2023?
#44Love the article, but one criticism: I've found gp3 to be better than gp2 in all aspects, especially consistency—and it's cheaper. For a long time you could only use RDS with gp2 but I know of no reason to choose it in 2023.
Re: Is 20M of rows still a valid soft limit of MySQL table in 2023?
#45Earlier quoted context omitted.
You run MongoDB as a backing store. Very web much scale.
Literally pays for itself in scale
Anyone who wants to get these references: https://www.youtube.com/watch?v=b2F-DItXtZs
Re: Is 20M of rows still a valid soft limit of MySQL table in 2023?
#46Thanks for repost the blog! Some crazy guys have already told me that they have mysql table >1.4B rows (~180GiB), underestimated mysql's power lol
10+ years ago I worked at a smallish company and we had a table with over 500 million rows. This was on a high end system with several spinning rust disks in a raid 10 and 256 gigs of RAM. Billions in modern hardware seems quite reasonable.
Re: Is 20M of rows still a valid soft limit of MySQL table in 2023?
#47Re: Is 20M of rows still a valid soft limit of MySQL table in 2023?
#48Re: Is 20M of rows still a valid soft limit of MySQL table in 2023?
#49When was this ever a thing?
Re: Is 20M of rows still a valid soft limit of MySQL table in 2023?
#50Haven't had trouble into the low hundreds of millions on a single RDS server with 16GB of ram. YMMV.
same. we've got 3 to 4 hundred million rows in one table and our queries still complete in the 100ms range!