You can safely go to a billion rows. Use indexes. Don't do silly things like count(*) or select *, select the small data sets you need using your indexes. If you need a full table count for some reason, use a primary key that auto increments and select the max, or use information_schema for an estimate. Nobody will sweat the slight inaccuracy on a table that size.
yeah i was also under the 20mil impression initially until building a system that worked with a 500mil row table without any problems (just a beefy RDS instance, nothing crazy distributed or anything). Schema changes became a pain at that point, so i would probably split it if I were to do it again, but reads and writes never gave us much trouble.
Is 20M of rows still a valid soft limit of MySQL table in 2023?
21–30 of 86 posts
Re: Is 20M of rows still a valid soft limit of MySQL table in 2023?
#22Re: Is 20M of rows still a valid soft limit of MySQL table in 2023?
#23Re: Is 20M of rows still a valid soft limit of MySQL table in 2023?
#24Re: Is 20M of rows still a valid soft limit of MySQL table in 2023?
#25Hardware has limits as well as software. You would need a completely different architecture to accomodate a 20M+ database. Usually schemeless the database overhead is typically what is responsible how many rows it handle and because your client interacts with the MySQL engine it is at limit and performance of that engine. So yeeeee-no. Time to use a different database.
Re: Is 20M of rows still a valid soft limit of MySQL table in 2023?
#26...yes, performance will drop when your set of active data exceeds available memory and your system needs to reference slower storage. I assume this would apply to anything.
but that threshold is surely not generally 20M rows? I mean wouldn't it be completely dependent on how much memory you have?
Pretty clearly people ran the exact same test, fill a table with a bunch of empty/meaningless data and see where performance degrades - then write a catchy headline title/post.
In the end, it's more nuanced than that but i think the overall theme is know your tuning once you get to certain data sizes.
Re: Is 20M of rows still a valid soft limit of MySQL table in 2023?
#27Re: Is 20M of rows still a valid soft limit of MySQL table in 2023?
#28Isn't 20m rows super tiny?
Re: Is 20M of rows still a valid soft limit of MySQL table in 2023?
#29Re: Is 20M of rows still a valid soft limit of MySQL table in 2023?
#30You can safely go to a billion rows. Use indexes. Don't do silly things like count(*) or select *, select the small data sets you need using your indexes. If you need a full table count for some reason, use a primary key that auto increments and select the max, or use information_schema for an estimate. Nobody will sweat the slight inaccuracy on a table that size.
Agreed. I have a handful of tables with over 2 billion rows and have experienced no issues. One database is 2.5TB in size. I feel like 20 million rows is a relatively small MySQL table.