Live data from Hacker News

Our Server’s Hard Drive is Dead. We didn’t have a backup.

blog.method.ac

71–80 of 88 posts

Re: Our Server’s Hard Drive is Dead. We didn’t have a backup.

#71
post #16

Earlier quoted context omitted.

glad I wasn't the only one who had that as a first thought. as an ops guys I wasn't sure if I should be offended or just shake my head at the irony of it.

Well apparently someone didn't like my comment since it got down-voted. Whatever. As someone with a background in systems administration, it bothers me when this profession gets left out of the equation, far too often.

Sometimes i think Sys. Admin profession needs a messiah like NNT(for finance) with perhaps the outspokenness/attitude of Zed Shah. That way, there would be a whole lot of increased awareness for the challenges involved in Systems administration.

Re: Our Server’s Hard Drive is Dead. We didn’t have a backup.

#72
I feel for you. I really do. You did a lot of things right: learned how to program, bootstrapped your startup, released a product (!), got users, went viral, etc. etc.

But.

1) All of this could have been solved with money, specifically money used to pay professionals. You got 30 THOUSAND signups and you didn't think of trying to get funding? I'm surprised VC's weren't pounding at your door. At the very least, that might even be enough for a bank loan from a savvy lender. Hell, you could probably find a recently graduated ('tis the season) CSCI student willing to just take sweat equity with those numbers. This is especially frustrating for me as I currently have a startup that recently garnered a whopping 400 (count 'em!) hits on it's signup page, and yet I still got emails from people trying to invest. Not nearly platinum tier, and thus far none have panned out, but still!!!

2) You claim to have worked in web design/development for a while, and you didn't hear about 1&1's horrific reputation? That's hard for me to believe. In fact, of any community, the PHP/JS crowd is probably most familiar with being burned by 1&1. (Not even going into the slimy overselling).

I hate to say it, but you should have known better. That said, I sincerely wish you the best of luck. You've succeeded pretty spectacularly thus far, and in the big scheme of things this is a pretty minor setback. Just keep shipping and you'll get it eventually.

Edit: I realize that it might seem foolish to some to go after funding when it's not needed, but I would argue that if you are making it up as you go along (not an indictment, it's how we learn) and you get these kind of numbers, you should feel at least a little obligated to your users to secure your product. If that requires money that you don't have, get funding.

Re: Our Server’s Hard Drive is Dead. We didn’t have a backup.

#73
post #16

Earlier quoted context omitted.

Well apparently someone didn't like my comment since it got down-voted. Whatever. As someone with a background in systems administration, it bothers me when this profession gets left out of the equation, far too often.

judging by some of the other comments I think people believe sysadmins can and have been replaced by heroku, aws and such.

Developers are like any other person. "How hard can it be?"

Re: Our Server’s Hard Drive is Dead. We didn’t have a backup.

#74
post #12

"We messed up bad. We launched without having a backup procedure in place, and without the resources to make it happen. This was a hard-learned lesson that won’t happen again. We have no one except ourselves to blame." You know how you messed up? By not using something like AWS - EC2 - Snapshots. Or even S3 or Glacier. What is this trend of devs doing Operations? As a Sysadmin with a Compsci/dev background, it blows…

Long story short, money was tight, priorities were set incorrectly, and they got fucked.

Re: Our Server’s Hard Drive is Dead. We didn’t have a backup.

#75
post #50

Earlier quoted context omitted.

> you can add an expert admin to your team for $100-200/mo Hahaha.

It's what we do -- so I laugh with you ;)

Expert admins are in the $100-150/hr range when they're consulting, and $100K+ salary range when hired. $100-200/month? Oh lord.

Re: Our Server’s Hard Drive is Dead. We didn’t have a backup.

#76
post #68

Earlier quoted context omitted.

It's what we do -- so I laugh with you ;)

Good luck with your below-the-minimum-wage admin.

I can understand with those numbers why you may think that. These numbers are generally per server and fit a small shop pretty well. We also work with many companies that have hundreds of servers, and the rate is multiplied by each server. I could have been clearer in my initial post.

Additionally, for clarification about your wage comment -- we pay above average US sysadmin wages, and just posted a job for our 7th full-time admin.

Re: Our Server’s Hard Drive is Dead. We didn’t have a backup.

#77

Earlier quoted context omitted.

It's what we do -- so I laugh with you ;)

Expert admins are in the $100-150/hr range when they're consulting, and $100K+ salary range when hired. $100-200/month? Oh lord.

Agreed with the consulting price point, and we charge this. But we also let you get in the door for a few hours a month at a reasonable rate. It's been working for many years now.

Re: Our Server’s Hard Drive is Dead. We didn’t have a backup.

#78
post #66

Earlier quoted context omitted.

Note: the website is an extended advertisement for a piece of backup software, and the user account was created 3 minutes before the comment was posted.

Yeah, that's true. But then do you really think those guys are spamming HN at 8 p.m.? I wish I had said, if you ignore the advertising, it's a great resource. If you apply it to proposed backup solutions, it's an effective means to find out if they are viable.

I said it for you, since a reader would probably be interested in that piece of information. I apologise for the cynical comment about the account being 3 minutes old, as that is ad hominem and being new to HN shouldn't come into it. Welcome to HN.

Re: Our Server’s Hard Drive is Dead. We didn’t have a backup.

#79
post #12

"We messed up bad. We launched without having a backup procedure in place, and without the resources to make it happen. This was a hard-learned lesson that won’t happen again. We have no one except ourselves to blame." You know how you messed up? By not using something like AWS - EC2 - Snapshots. Or even S3 or Glacier. What is this trend of devs doing Operations? As a Sysadmin with a Compsci/dev background, it blows…

Long story short, money was tight, priorities were set incorrectly, and they got fucked.

Sounds about right. My favourite part was leasing a server from 1&1. Even a little industry knowledge with regards to infrastructure would have caused someone to avoid them.

Re: Our Server’s Hard Drive is Dead. We didn’t have a backup.

#80
post #71
post #16

Earlier quoted context omitted.

Well apparently someone didn't like my comment since it got down-voted. Whatever. As someone with a background in systems administration, it bothers me when this profession gets left out of the equation, far too often.

Sometimes i think Sys. Admin profession needs a messiah like NNT(for finance) with perhaps the outspokenness/attitude of Zed Shah. That way, there would be a whole lot of increased awareness for the challenges involved in Systems administration.

I dream of that day..
Post reply on HN