Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CouchSurfing Deletes Itself, Shuts Down - Old, yet relevant

techcrunch.com

Post reply on HN
Print view View original

21–26 of 26 posts

Re: CouchSurfing Deletes Itself, Shuts Down - Old, yet relevant

#21

Post by chmac »

dmpayton wrote: post #20
nraynaud wrote: post #17

how do you backup and recover your stuff ? I can't find any prepackaged thing on the web. I'd like to address 2 needs : disaster recovery and point in time recovery. For now I only dump the database from time to time, but it's far from a real solution, if I loose my whole server it would take quite some time to get its configuration back on track.

If it's a *nix system, why not setup a cron job to do it for you? I have a Python script in /etc/cron.daily that performs a mysql-dump, tars it, and uploads it to S3. For total completeness and peace-of-mind, you could have the script add anything else you need backed up, like your projects or public_html directories, configuration files, etc.

Running mysqldump on a database the size of CouchSurfing's is a real problem. It's a resource hog. To properly back up a consistent data image you need to use locking, which prevents writes to your database while the backup is running. That causes downtime on every backup, not ideal.

Likewise, when backing up 100s of GiBs of data, a simple tar gz is not suitable. More sophisticated strategies need to be used when the data becomes that big.

I've found http://HighScalability.com to be a great source of information on building scalable applications.

Re: CouchSurfing Deletes Itself, Shuts Down - Old, yet relevant

#22

Post by chmac »

shafqat wrote: post #19
nraynaud wrote: post #17

how do you backup and recover your stuff ? I can't find any prepackaged thing on the web. I'd like to address 2 needs : disaster recovery and point in time recovery. For now I only dump the database from time to time, but it's far from a real solution, if I loose my whole server it would take quite some time to get its configuration back on track.

I'd love to get people's thoughts on this as well. What are the best ways to automatically back up your DB.

For a smaller database, dmpayton's suggestion is spot on. Simply cron schedule a mysqldump and then push that offsite. Here's a few example code snippets I use:

/usr/bin/mysqldump -u dbusername -pdbpassword --skip-lock-tables database_name | gzip > /path/to/backup/file/file_$(/bin/date +\%Y-\%m-\%d_\%H-\%M-\%S).sql.gz

I'm using --skip-lock-tables because on a ZenCart site, the lock tables option was adversely affecting site performance.

Then you could add something to automatically email or scp the file to another host.

For bigger, more complex databases, look into MySQL binary logging, DRBD replication, and MySQL slaves. That should give you some keywords to get started.

Re: CouchSurfing Deletes Itself, Shuts Down - Old, yet relevant

#23

Post by chmac »

ibsulon wrote: post #11
tweety wrote: post #6

Besides the tragic (or incompetent?) tech failures, what do folks here think about the founder initially just calling it quits on the community? Having followed this particular crash-recovery saga unfold and being involved with web 2.0 otherwise, it would be interesting to hear a few thoughts over this: what's the responsibility of the platform provider towards their possibly very engaged community - or is there any?

Sometimes, it just seems like the universe is trying to send us a message. Take that in a spiritual way if you want, or take it in the sense that a cataclysmic (relatively speaking) forces us to reevaluate our priorities. Certainly, if this happened to me and I came to that decision, I'd put up a forum for everyone so that they could engage and figure out post-startup plans. But that's me.

I think you make a great point ibsulon. If you did take the decision to close down the site, the least you could do is signpost users to somewhere else where they could continue to communicate.

As far as I'm aware, during the CouchSurfing crash, some volunteers set up a separate forum, but it wasn't sanctioned or initiated by the founder, I believe he simply walked out. :(

Re: CouchSurfing Deletes Itself, Shuts Down - Old, yet relevant

#26

Post by shafqat »

chmac wrote: post #22
shafqat wrote: post #19

Earlier quoted context omitted.

I'd love to get people's thoughts on this as well. What are the best ways to automatically back up your DB.

For a smaller database, dmpayton's suggestion is spot on. Simply cron schedule a mysqldump and then push that offsite. Here's a few example code snippets I use: /usr/bin/mysqldump -u dbusername -pdbpassword --skip-lock-tables database_name | gzip > /path/to/backup/file/file_$(/bin/date +\%Y-\%m-\%d_\%H-\%M-\%S).sql.gz I'm using --skip-lock-tables because on a ZenCart site, the lock tables option was adversely affecti…

Thanks this is exactly what we are doing... (the first option).
Post reply on HN