Is 20M of rows still a valid soft limit of MySQL table in 2023?
1–10 of 86 posts
Re: Is 20M of rows still a valid soft limit of MySQL table in 2023?
#2very cool repo from the author full of all kinds of tests and scripts: https://github.com/gongyisheng/playground/blob/main/mysql/ro...
might need to fork/run this on psql to compare performance.
Re: Is 20M of rows still a valid soft limit of MySQL table in 2023?
#3[dead]
Re: Is 20M of rows still a valid soft limit of MySQL table in 2023?
#4You can safely go to a billion rows. Use indexes. Don't do silly things like count(*) or select *, select the small data sets you need using your indexes. If you need a full table count for some reason, use a primary key that auto increments and select the max, or use information_schema for an estimate. Nobody will sweat the slight inaccuracy on a table that size.
Re: Is 20M of rows still a valid soft limit of MySQL table in 2023?
#5You can safely go to a billion rows. Use indexes. Don't do silly things like count(*) or select *, select the small data sets you need using your indexes. If you need a full table count for some reason, use a primary key that auto increments and select the max, or use information_schema for an estimate. Nobody will sweat the slight inaccuracy on a table that size.
yeah i was also under the 20mil impression initially until building a system that worked with a 500mil row table without any problems (just a beefy RDS instance, nothing crazy distributed or anything). Schema changes became a pain at that point, so i would probably split it if I were to do it again, but reads and writes never gave us much trouble.
Re: Is 20M of rows still a valid soft limit of MySQL table in 2023?
#6I love real meat articles like this where software engineers actually explain why particular limits exist, or why technical things are they way they are. Good read. Thanks for sharing.
Re: Is 20M of rows still a valid soft limit of MySQL table in 2023?
#7You can safely go to a billion rows. Use indexes. Don't do silly things like count(*) or select *, select the small data sets you need using your indexes. If you need a full table count for some reason, use a primary key that auto increments and select the max, or use information_schema for an estimate. Nobody will sweat the slight inaccuracy on a table that size.
Agreed. I have a handful of tables with over 2 billion rows and have experienced no issues. One database is 2.5TB in size. I feel like 20 million rows is a relatively small MySQL table.
Re: Is 20M of rows still a valid soft limit of MySQL table in 2023?
#8Thanks for repost the blog! Some crazy guys have already told me that they have mysql table >1.4B rows (~180GiB), underestimated mysql's power lol
Re: Is 20M of rows still a valid soft limit of MySQL table in 2023?
#9...yes, performance will drop when your set of active data exceeds available memory and your system needs to reference slower storage. I assume this would apply to anything.
Re: Is 20M of rows still a valid soft limit of MySQL table in 2023?
#10You can safely go to a billion rows. Use indexes. Don't do silly things like count(*) or select *, select the small data sets you need using your indexes. If you need a full table count for some reason, use a primary key that auto increments and select the max, or use information_schema for an estimate. Nobody will sweat the slight inaccuracy on a table that size.
Agreed. I have a handful of tables with over 2 billion rows and have experienced no issues. One database is 2.5TB in size. I feel like 20 million rows is a relatively small MySQL table.
What types of data with over two billion rows?