Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Atlassian: We estimate the rebuilding effort to last for up to 2 more weeks

twitter.com

Post reply on HN
Print view View original

91–100 of 260 posts

Re: Atlassian: We estimate the rebuilding effort to last for up to 2 more weeks

#91

Post by lmc »

uuyi wrote: post #84
laurent92 wrote: post #47

Earlier quoted context omitted.

> 1) This outage will get their organization to prioritize work such that it never happens again It has already happened in the past.

2012. https://www.theregister.com/2012/05/09/atlassian_cloud_stora...

"An Atlassian spokesperson contacted by The Reg blamed the loss on the failure of “multiple” storage disks at its cloud storage provider on 28 April."

10 years ago to the month.

So as long as everyone plans a vacation for this time in 2032 it'll be OK.

Re: Atlassian: We estimate the rebuilding effort to last for up to 2 more weeks

#92

Post by uuyi »

daigoba66 wrote: post #81
laurent92 wrote: post #42

Earlier quoted context omitted.

Let me bet: Rebuilding from Jira email notifications. Yes, the diffs in the notifications.

I've actually done exactly that many years ago for a self-hosted Jira installation that didn't have any backups. You can bet we had backups with regular testing after that.

I would have just quit if I had that landed on me.

Re: Atlassian: We estimate the rebuilding effort to last for up to 2 more weeks

#93

Post by jlbooker »

perlgeek wrote: post #66

So, let me get this straight: * It's been deleted for a week already, they estimate they might need two more weeks. Three in total. * They claim to have "extensive backups", and hundreds of engineers working on it. What? How? This simply doesn't go together. Why would restoring from backup take three weeks? Either their backups aren't complete, or they need new software written for the restore, or something else does…

Restore from off-site tape backup. The kind of service where you ship them ~dozen new tapes in a lockbox each week and they ship you the oldest dozen back. It's supposed to be the "if all of a data centers happen to burn to ashes simultaneously" option. If you say "give us all of our tapes, asap" and then have some pour souls swapping them out as fast as the data can be read... it would probably take a few weeks.

Re: Atlassian: We estimate the rebuilding effort to last for up to 2 more weeks

#96

Post by tpmx »

perlgeek wrote: post #66

So, let me get this straight: * It's been deleted for a week already, they estimate they might need two more weeks. Three in total. * They claim to have "extensive backups", and hundreds of engineers working on it. What? How? This simply doesn't go together. Why would restoring from backup take three weeks? Either their backups aren't complete, or they need new software written for the restore, or something else does…

I have no specific information on this, but in general and in theory:

Isn't this how an incompetent, insincere and desperate company being subjected to a ransom attack would communicate publicly?

Re: Atlassian: We estimate the rebuilding effort to last for up to 2 more weeks

#97

Post by tetsusaiga »

hyperman1 wrote: post #62
bpp wrote: post #33

Earlier quoted context omitted.

Works until you get a bug in the deletion job. I've seen exactly this happen.

You don't even need a bug. Just a wrong system clock. We had a few windows laptops where something caused them to time travel to 8000 years in the future. Then, they'd slowly spend a few hours deleting every local profile, as nobody had logged in to them for 8000 years. Then, they'd do something to their time zone database and travel back 8000 years. When they started the process, it was unstoppable. Trying to modify…

that is almost cartoonishly nightmarish

Re: Atlassian: We estimate the rebuilding effort to last for up to 2 more weeks

#99

Post by senorsmile »

planb wrote: post #68

As the person responsible for running Jira and Confluence on premises at my employer I‘m looking forward for the next time one of their sales droids contacts me to make us move to their cloud services (despite me stating that we are not interested multiple times)…

They are discontinuing on prem after 2022 though. What is your plan if not to migrate to their SaaS offering?

Re: Atlassian: We estimate the rebuilding effort to last for up to 2 more weeks

#100

Post by planb »

senorsmile wrote: post #99
planb wrote: post #68

As the person responsible for running Jira and Confluence on premises at my employer I‘m looking forward for the next time one of their sales droids contacts me to make us move to their cloud services (despite me stating that we are not interested multiple times)…

They are discontinuing on prem after 2022 though. What is your plan if not to migrate to their SaaS offering?

We will probably bite the sour apple (not sure if this is correct English but you’ll get the meaning even if it’s not) and switch to the data center edition which is still on prem but costs approximately twice as much.
Post reply on HN