Earlier quoted context omitted.
In these cases the best thing to do is just give every customer the full month refund; don't make them ask for it.
The best thing to do business-wise, or as a good faith move?
What's the difference?
Inside the longest Atlassian outage
271–280 of 772 posts
Re: Inside the longest Atlassian outage
#272Earlier quoted context omitted.
You're assuming every team would have better uptime with in-house solutions I think many would have worse uptime even with more headcount
I've heard this argument many times before, but is there research into this? I.e. where they would compare uptime of cloud vs. on-premises across a wide range of companies.
I mean, you're going to get biased results, no? Only companies who are confident in self-hosting will self-host it. You won't have any real data about companies who are not confident in self-hosting maintaining their on-premises version of the software.
Re: Inside the longest Atlassian outage
#273What's a good Jira replacement? Redmine? Phabricator? OpenProject? Just leaving the jira server alone and hoping there's no new and exciting zero-days? One thing is clear, these guys are a bunch of cowboys who can't be trusted with any amount of data.
Bugzilla
Re: Inside the longest Atlassian outage
#274> it takes between 4 and 5 elapsed days to hand a site back to a customer. Atlassian's SLA page says, Premium Cloud Products 99.9% That's 43 minutes of downtime per month. That works out to, Atlassian can't have any more downtime for the next 14 years. Are SLAs even real? I'm being slightly facetious. From the page text it's just a threshold after which I think you're entitled to some money back for that month.
>Are SLAs even real? SLI: Some metric you use to measure a thing (e.g. uptime, latency, etc.) SLO: Some objective you try to hit, as measured by the SLI (e.g. "99.99% of requests are processed within 3 seconds) SLA: A promise to a customer that they will meet some SLO, and consequences if they don't. If there aren't consequences for not meeting the SLO, then measuring and tracking the metrics is a pointless exercise.…
> and consequences if they don't.
And these consequences usually just amount to getting some percentage of your service fees back. I'm sure the affected customers will get their entire monthly Atlassian Cloud fees back. Since this is so severe maybe Atlassian will even give them credits for some # of free months.
But there's no way the amount they'll get from Atlassian is going to come close to what they're losing in productivity by not having access to Jira & Confluence. At my company, getting an entire free year of Jira wouldn't be worth Jira being inaccessible for a week.
Re: Inside the longest Atlassian outage
#275Earlier quoted context omitted.
> Is it not a good idea to spin up separate db instances for each client/company? It depends, really. There is a trade-off in terms of software and operational complexity vs scalability/perf and isolation. And probably a bunch of other factors. If you have separate databases for each customer, schema migrations can be staged over time. But that means your software backend needs to be able to work with different schem…
> But that means your software backend needs to be able to work with different schemas concurrently. Not if you're truly multi-tenant and each customer has their own app servers. Then your code and schema version are always in lock-step.
True. But then you have an additional problem ...
Re: Inside the longest Atlassian outage
#276> However, if they [restore backups], while the impacted ~400 companies would get back all their data, everyone else would lose all data committed since that point OK, so you restore backups to a separate system, and selectively copy the stomped accounts data back to production. Simple concepts aren't that simple at their scale, sure, but I suspect this is skimping details on some truly horrendous monolithic architec…
I would really like to understand who makes the decision to purchase JIRA. It's like the C++ of ticketing software--it does everything because no one wanted to sit down and think critically about the use cases and instead decided it would be easier to say "yes" to every single feature request. It definitely feels like whoever is buying JIRA is not on the team who is using it (maybe IT or finance) because it ticks the…
I was part of the decision to purchase Atlassian tools at my company. We had been using a variety of self-hosted and SaaS tools which had varying abilities to integrate with each other. We’ve had very positive feedback from users since switching to them. We were also able to move some of our help desks to JIRA Service Management, and away from another self-hosted product which is still used by a good portion of our business. The self-hosted product is honestly a nightmare to maintain and keep secure. According to the vendor, the “fix” is to have 1-2 people dedicated to that product, which simply isn’t something that my team has the bandwidth or will to do.
JIRA does try to be all things to all people…and mostly succeeds. For instance, we use the same workflow and mostly the same nomenclature across our development and helpdesk teams. Some of our software projects use Kanban-style workflows, while others use sprints, but we can keep track of a project across multiple teams using the same tools. I’m sure other products also offer this, but we liked the integration and overall capability for the price.
There are definitely issues: some feature requests and bugs have languished in their backlog for years. But you can get started very quickly and we’ve had great feedback from users.
Re: Inside the longest Atlassian outage
#277Earlier quoted context omitted.
> Selectively restoring data only for certain rows is super hard. What's the right way to structure your data here that would make restoring more straightforward here? Is this backup/restore scenario niche or they should have designed for it?
in theory, shard your customer databases 1:1, job done. alas, in practice, many SaaS compromise this two ways: a) overwhelmed by creeping featuritis, each customer's data has relationships to global tables, and b) they backup their entire database cluster in one snapshot and there maybe other gotchas for restoration, like relying on denormalized views and caches that have to be rebuilt. they may also have erroneously…
> shard your customer databases 1:1
What are the downsides to this?
Re: Inside the longest Atlassian outage
#278Earlier quoted context omitted.
I was told by a lifetime manager turned successful consultant, that roughly fifty percent of engineering firms govern their engineers basically using fear.
> using fear Could you elaborate? What kind of fear? “You’re fired”? I wonder how effective it actually is because of the current job market and also because I (and others) react very poorly to this kind of tactics: “you want me to fear getting fired? Joke’s on you, please DO fire me, I dare you”
I'm guessing it's a sort of a nerd shorthand for "various means that are accompanied with self confusion of users but not with strong rational or scientific or technical basis"
Re: Inside the longest Atlassian outage
#279Earlier quoted context omitted.
The fact that a single bad script could delete 400 of their customers should be absolute proof they do not have the processes in place to be a steward of your data in the cloud. On-prem or bust.
On-premise just means that your overworked IT person is going to spend 5% of their time keeping your service maintained, at no point gaining any more than baseline familiarity with the product. On-premise isn’t a magic pill guaranteeing 100% uptime and 0 data loss. While on-premise may be a good choice in many cases, it’s not like running on-premise business tools has no risk associated with that choice. Remember tha…
On Prem means you have control over spending. I calculated that if we've moved to the cloud, we would pay YEARLY as much as we spent on Atlassian licenses in last 5 years. That easily pays for the maintenance overhead on our devops team.
Re: Inside the longest Atlassian outage
#280Selectively restoring data only for certain rows is super hard. But the communications by Atlassian has been the worst I have ever seen in the industry.
> Selectively restoring data only for certain rows is super hard. What's the right way to structure your data here that would make restoring more straightforward here? Is this backup/restore scenario niche or they should have designed for it?
Work out a relationship graph and automate the export/import