Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

"DigitalOcean Killed Our Company"

twitter.com

Post reply on HN
Print view View original

161–170 of 620 posts

Re: "DigitalOcean Killed Our Company"

#161

Post by SteveNuts »

z3t4 wrote: post #124
_bxg1 wrote: post #64

Looks like Moisey Uretsky personally intervened fairly quickly: https://twitter.com/moiseyuretsky/status/1134547532149854208 That said, any company, especially one working with Fortune 500's, should have DB backups in at least two places. If they'd had the data, they could have spun up their service on a different hosting provider relatively easily.

Having backups in two places could easily triple the hosting costs. The question is what costs more. Eg. Losing data vs backup costs.

As a startup, generally your secondary backup could literally be an external hard drive from best buy, or an infrequent access S3 bucket (or hell, even Glacier). No excuse, especially when "dealing with Fortune 500 companies".

Re: "DigitalOcean Killed Our Company"

#162

Post by selimthegrim »

bnt wrote: post #112
BlackRing wrote: post #97

Earlier quoted context omitted.

I agree. Putting all your eggs in one basket these days indicates poor decisions or risk assessment. > Digital Ocean "Trust and Safety" Does this phrase give anyone else the heebie-jeebies? They deliberately locked his account without looking at previous metrics over the previous months to conclude this was obviously not a concern, and why? Things that have crossed my mind: Is their automatic system was poorly tuned?…

Sounds like the slogans they put on Police cars.

They should borrow the Dragnet one

Re: "DigitalOcean Killed Our Company"

#163

Post by m-p-3 »

They literally put all their eggs in the same basket by having the backup and the production environment at the same provider.

Not saying this shouldn't have happened, and hopefully they have learned from this experience.

Re: "DigitalOcean Killed Our Company"

#164

Post by nikanj »

prophesi wrote: post #152
PostPost wrote: post #140

Earlier quoted context omitted.

When AWS has gone down in the past, it's severely impacted massive tech companies like Netflix and Spotify. Why would there be an expectation that a 2-man shop have "another cloud offering" as a contingency plan when some of the biggest and best tech companies do not? People use services like AWS or DO because they are the contingency plan - they have the size and scale that smaller companies cannot afford or impleme…

The difference is that when AWS goes down, Netflix/Spotify still have backups and could adapt infrastructure if the outage involved permanent data-loss. You're talking about the people who built https://github.com/Netflix/chaosmonkey I'd argue that it should be _easier_ for a 2-man company to adapt to cloud service outages, as they likely don't have to keep up with nearly as many backups or moving parts.

Another practical difference: When AWS goes down, it makes the news outside the tech bubble too. Customers are much more likely to forgive you, if you send a link to NY Times that says "Amazon is suffering a global outage, affecting tens of thousands of companies"

Re: "DigitalOcean Killed Our Company"

#165

Post by coryrc »

nhooyr wrote: post #44

A few years ago I used to run a VPS on DO with a mail server, VPN and some code I was writing. Once I was done with everything, I used their snapshots feature to backup my VPS and shut it down. Two years later I wanted to restore the VPS but turns out my snapshot become "outdated" and they stopped supporting the format for restoration... Support was completely useless, wouldn't even let me download the snapshot, said…

That seems reasonable. Launching security-bug-ridden software seems dangerous, so mounting to a VPS so you can copy the important files seems an entirely reasonable access method.

Re: "DigitalOcean Killed Our Company"

#166

Post by thaumaturgy »

walrus01 wrote: post #151
nikanj wrote: post #143

Earlier quoted context omitted.

Linode is definitely in the minority here. Most companies, in tech and outside of it, seem to follow the DO model. Twitter provides decent service, and the official help channels provide canned responses and template emails. I somewhat blame people in tech, actually. More than one company is creating products that "cut customer service costs via machine learning", which is code for "pick keywords from incoming ticket…

ISP here: The margins in bulk hosting services are incredibly thin, and companies have resorted to automation tools. If somebody asked me to run backend infrastructure for something like DigitalOcean or Linode, I would run away screaming. It would literally be my own personal hell. I would rather run any other sort of ISP services on the planet than a bulkhosting service where anybody with a pulse and $10 to $20/mont…

This race to the bottom has reached a point that it's harming customers. It's okay to be more expensive than the competition if you provide a better service.

Re: "DigitalOcean Killed Our Company"

#167

Post by ViViDboarder »

macspoofing wrote: post #89
_bxg1 wrote: post #64

Looks like Moisey Uretsky personally intervened fairly quickly: https://twitter.com/moiseyuretsky/status/1134547532149854208 That said, any company, especially one working with Fortune 500's, should have DB backups in at least two places. If they'd had the data, they could have spun up their service on a different hosting provider relatively easily.

>That said, any company, especially one working with Fortune 500's, should have DB backups in at least two places Yes they should. How many 2-man shops do you think follow all the proper backup and security procedures?

What Fortune 500 company is doing business with 2-man shops that aren’t?

Re: "DigitalOcean Killed Our Company"

#168

Post by Dylan16807 »

prophesi wrote: post #152
PostPost wrote: post #140

Earlier quoted context omitted.

When AWS has gone down in the past, it's severely impacted massive tech companies like Netflix and Spotify. Why would there be an expectation that a 2-man shop have "another cloud offering" as a contingency plan when some of the biggest and best tech companies do not? People use services like AWS or DO because they are the contingency plan - they have the size and scale that smaller companies cannot afford or impleme…

The difference is that when AWS goes down, Netflix/Spotify still have backups and could adapt infrastructure if the outage involved permanent data-loss. You're talking about the people who built https://github.com/Netflix/chaosmonkey I'd argue that it should be _easier_ for a 2-man company to adapt to cloud service outages, as they likely don't have to keep up with nearly as many backups or moving parts.

So pretend they had offsite backups. That's a separate issue from an entire contingency plan. The ability to adapt is not the same thing. This company could certainly adapt to a new host if they had an extra backup.

Re: "DigitalOcean Killed Our Company"

#169

Post by taffer »

detaro wrote: post #35
amelius wrote: post #27

What other business shuts their customers out if they use too much of their product?

Any that suspects fraud and doesn't want to be on the hook for the inevitable chargeback.

Caps or quotas are a better way to achieve this IMO. Random shutdowns do not make it sound as if DO is ready for production.
Post reply on HN