81–90 of 157 posts
Implementing soft deletes is a lessons every developer learns early in their career. The fact that Atlassian did not implement that in their cloud, is mind boggling. Great case study of a monumental fuck up!
Soft deletes are great, but are probably not sufficient to meet the GDPR's "right to be forgotten"
Implementing soft deletes is a lessons every developer learns early in their career. The fact that Atlassian did not implement that in their cloud, is mind boggling. Great case study of a monumental fuck up!
Soft deletes are great, but are probably not sufficient to meet the GDPR's "right to be forgotten"
Alternatively move the deleted data to a temporary location and then delete the temporary location after a short period of time.
Or better combine both patterns where expired rows get moved to a temporary location before hard deleting a period of time after.
GDPR says you have to delete data when requested. As far as I know you have 30 days to acknowledge the request and up to 60 days to action the request. It’d be completely reasonable to do a soft delete for 7-14 days before doing a hard delete to prevent these kind of errors.
While the post-mortem is thorough, it misses key details on what companies experienced who were unlucky enough to be caught out by this outage. For example, it fails to mention how impacted customers lost access to certain Atlassian services for up to ~2 weeks: JIRA, Confluence, OpsGenie. But not others like Trello or BitBucket. Of these, losing access to OpsGenie for this long was a massive problem, dwarfing most ot…
It should be shouted from the rooftops: don't switch services just to save money if the result is potentially worse business outcomes. Why save a tiny bit of cash if it puts your business at risk?
But it had the risk of catastrophic failure the whole time. I wish we had better ways to measure and communicate risk.
While the post-mortem is thorough, it misses key details on what companies experienced who were unlucky enough to be caught out by this outage. For example, it fails to mention how impacted customers lost access to certain Atlassian services for up to ~2 weeks: JIRA, Confluence, OpsGenie. But not others like Trello or BitBucket. Of these, losing access to OpsGenie for this long was a massive problem, dwarfing most ot…
It should be shouted from the rooftops: don't switch services just to save money if the result is potentially worse business outcomes. Why save a tiny bit of cash if it puts your business at risk?
I wasn't affected by this in the slightest and just found out opsgenie exists from the parent comment but even I can understand that this decision would almost certainly be driven by things like "we're already using atlassian for everything else and will benefit from the interop" and "we already trust them with everything else and they haven't let us down or we wouldn't be using them for any of that stuff either."
Do any databases have something like native support for soft deletes or ability to undo (other than SQL transactions rollbacks where you're having to specify the undo checkpoint)? Something like what Git does where it keeps a history of edits? If this isn't common, is this a neglected area that should be addressed or it's just too hard of a problem? It feels like with SQL, there's minimal guardrails and it's just you…
Datomic treats all data as immutable, so it can wind back to any version. When new data is written, the entire block is copied and rewritten rather than changing the data in-place.
I sincerely hope all of the people who worked on the recovery effort are okay, and are being well supported and strongly encouraged to tend their mental health. I have no personal investment in Atlassian products—if anything, unrelated to this or any incident, I could happily never use them again. But the people who work there are human, and I know what kind of a toll a protracted recovery effort can take. I know it…
Earlier quoted context omitted.
As someone who's worked on similar situations, I don't expect any pat on the back, but see it as my responsibility to make sure it doesn't happen in the first place, and when it does, my responsibility to fix it without complaint. Some Atlassian customers might have had much more severe (mental health and other) problems than the Atlassian staff, so I think it can be perceived as mildly solipsistic to be praising the…
This is a fascinating response. I’ll address the last part first, because I don’t want it to get lost. > I know you didn't say anything about forgetting about the customers, but affected customers may perceive it that way No, I didn’t mean to suggest or imply this, and hope it won’t be taken this way by anyone. It was perhaps a mistake taking it as read that obviously their customers were also harmed. But I do think…
I sincerely hope all of the people who worked on the recovery effort are okay, and are being well supported and strongly encouraged to tend their mental health. I have no personal investment in Atlassian products—if anything, unrelated to this or any incident, I could happily never use them again. But the people who work there are human, and I know what kind of a toll a protracted recovery effort can take. I know it…
Quoted post unavailable.
While the post-mortem is thorough, it misses key details on what companies experienced who were unlucky enough to be caught out by this outage. For example, it fails to mention how impacted customers lost access to certain Atlassian services for up to ~2 weeks: JIRA, Confluence, OpsGenie. But not others like Trello or BitBucket. Of these, losing access to OpsGenie for this long was a massive problem, dwarfing most ot…