Loading half a billion rows into MySQL
derwiki.tumblr.com
Loading half a billion rows into MySQL
1–10 of 101 posts
Re: Loading half a billion rows into MySQL
#2I'm not familiar with Percona, what problem does it solve?
Re: Loading half a billion rows into MySQL
#3I'm not familiar with Percona, what problem does it solve?
Performance, for one: http://www.percona.com/software/percona-server/benchmarks/
They also provide a lot of nice tools: http://www.percona.com/software/percona-toolkit/
Re: Loading half a billion rows into MySQL
#4I'm not familiar with Percona, what problem does it solve?
It's a fork of MySQL that provides substantially better durability, admin tools, an easy hot-backup solution, better instrumentation of your DB, etc etc.
Deploying is it a question of "stop db, replace mysql server binary with percona server binary, start db".
My personal opinion is that unless you have a support contract with MySQL AB, there is very little reason not to use Percona.
Re: Loading half a billion rows into MySQL
#5I fail to see why setting the transaction isolation level to read uncommitted would make any difference to the data load process.
Re: Loading half a billion rows into MySQL
#6FTA:
"...I decided to migrate the existing system into a single table setup."
"Alternative Proposed Solutions: MySQL Partitioning and Redis"
I'm surprised he didn't consider at Mongo, Couch, etc.
Re: Loading half a billion rows into MySQL
#7FTA: "...I decided to migrate the existing system into a single table setup." "Alternative Proposed Solutions: MySQL Partitioning and Redis" I'm surprised he didn't consider at Mongo, Couch, etc.
My coworkers and I have absolutely zero experience with Mongo, Couch, etc. We do have experience with MySQL and Redis (see https://github.com/causes/mock_redis), and I feel "safer" keeping important data in something as mature as MySQL.
Re: Loading half a billion rows into MySQL
#8I want to give this 10 upvotes. This is one of those things you only see on HN occasionally, and it's full of all kinds of awesome little nuggets. That command to get the count from the information_schema in and of itself is gold (I now know about the tee command)
Re: Loading half a billion rows into MySQL
#9FTA: "...I decided to migrate the existing system into a single table setup." "Alternative Proposed Solutions: MySQL Partitioning and Redis" I'm surprised he didn't consider at Mongo, Couch, etc.
My coworkers and I have absolutely zero experience with Mongo, Couch, etc. We do have experience with MySQL and Redis (see https://github.com/causes/mock_redis ), and I feel "safer" keeping important data in something as mature as MySQL.
I've heard so many stories of people moving away from MongoDB once they reach scale like yours, I don't blame you.
Re: Loading half a billion rows into MySQL
#10 SELCT table_rows
SELECT table_rows