Thanks for repost the blog! Some crazy guys have already told me that they have mysql table >1.4B rows (~180GiB), underestimated mysql's power lol
My current employer has a single InnoDB table that is 14-15 TB.
Is 20M of rows still a valid soft limit of MySQL table in 2023?
11–20 of 86 posts
Re: Is 20M of rows still a valid soft limit of MySQL table in 2023?
#12Re: Is 20M of rows still a valid soft limit of MySQL table in 2023?
#13Earlier quoted context omitted.
Agreed. I have a handful of tables with over 2 billion rows and have experienced no issues. One database is 2.5TB in size. I feel like 20 million rows is a relatively small MySQL table.
What types of data with over two billion rows?
It's related to user activity, along the lines of products a user views. The other large tables in the billions of rows are about analyzing how products are performing on other websites over time, helping to determine past and current trends for businesses.
Re: Is 20M of rows still a valid soft limit of MySQL table in 2023?
#14Earlier quoted context omitted.
My current employer has a single InnoDB table that is 14-15 TB.
what is it?
Events that are enriched off of other data in the same DB.
The system design predates me, but it is solid (albeit difficult to operate at this scale - usual stuff like schema changes, replication bootstrapping).
Re: Is 20M of rows still a valid soft limit of MySQL table in 2023?
#15...yes, performance will drop when your set of active data exceeds available memory and your system needs to reference slower storage. I assume this would apply to anything.
but that threshold is surely not generally 20M rows? I mean wouldn't it be completely dependent on how much memory you have?
Re: Is 20M of rows still a valid soft limit of MySQL table in 2023?
#16You can safely go to a billion rows. Use indexes. Don't do silly things like count(*) or select *, select the small data sets you need using your indexes. If you need a full table count for some reason, use a primary key that auto increments and select the max, or use information_schema for an estimate. Nobody will sweat the slight inaccuracy on a table that size.
Agreed. I have a handful of tables with over 2 billion rows and have experienced no issues. One database is 2.5TB in size. I feel like 20 million rows is a relatively small MySQL table.
Could you provide some details, like MySQL version, IOPS and read/write performance?
Re: Is 20M of rows still a valid soft limit of MySQL table in 2023?
#17Re: Is 20M of rows still a valid soft limit of MySQL table in 2023?
#18Earlier quoted context omitted.
what is it?
Events that are enriched off of other data in the same DB. The system design predates me, but it is solid (albeit difficult to operate at this scale - usual stuff like schema changes, replication bootstrapping).
What is your mean time to recovery like when you have to restart your system?
Re: Is 20M of rows still a valid soft limit of MySQL table in 2023?
#19Isn't 20m rows super tiny?
Re: Is 20M of rows still a valid soft limit of MySQL table in 2023?
#20Haven't had trouble into the low hundreds of millions on a single RDS server with 16GB of ram. YMMV.