Comes across as jerk. How can an outsider say things with such certianty?
Inside the longest Atlassian outage
501–510 of 772 posts
Re: Inside the longest Atlassian outage
#502Atlassian about to dip over the next few years as firms around the world slowly remove themselves from their ecosystem of products.
Not to mention a CEO who is more interested in activities outside the company like the green energy transition and politics.
As an Aussie I always wanted Atlassian to succeed as we have so few tech companies at that scale or larger. Now I view them as another Oracle. Now they innovate little, they keep ratchetting up prices, pushing deployments to cloud where they make more money. Nickel and dime you for what should be core features (SAML Auth?). They aren't coming up with anything new to keep the value in the ecosystem. They buy applications in, spend a little to make some cross integration and then drop down to a slower development Cadence.
Re: Inside the longest Atlassian outage
#503Earlier quoted context omitted.
This is an engineering problem. They should own it and improve things, make sure it doesn't happen again. Also, GP's quote > Engineering mistakes happen. I don't like this statement because it offers consolation at the expense of unintentional normalization.
And coders that say all code has bugs are just defeatists that are trying to make excuses for being lazy. Sometimes manure will always hit the fan. Being robust means being able to handle that.
I think this is obviously incorrect.
Human error is probabilistic, and the probability of making an error cannot be zero.
On the flip side, it’s infeasible to use only provably correct systems; not lazy, but literally not a practical option due to compute costs, developer time, what formal techniques can even be applied to the problem at hand, etc…
Re: Inside the longest Atlassian outage
#504This is extremely poor for a large SaaS company. A standard RFP question for SaaS should be: - Can you restore data for a single customer, and if so, what is the RTO for that operation? A smaller SaaS could be excused for only thinking about full database restores. When you're a scrappy upstart, thinking about hypotheticals is less important than survival. But for any decent size multi-tenanted SaaS, it's imperative…
Is it standard for a RFP to have a long list of questions like this? I've never been involved in an RFP from either side.
Is it standard to (in addition or instead) to have something more general/forward-looking like: how do you watch other providers' postmortems and apply the lessons to your own system?
> - Can you restore data for a single customer, and if so, what is the RTO for that operation?
If I were to aim something at this specifically, it'd be: can you restore data for N customers or N% of customers, and if so, what is the RTO for that operation?
I mentioned in another comment that Gmail had a similar outage in which they had to restore from tape. https://news.ycombinator.com/item?id=31017160 They had a tool for restoring a single account but not for restoring N accounts in bulk, which would be significantly more efficient than doing the one-account process N times. (E.g., in the case of tape backups, imagine the difference between pulling data from the tape library sequentially for each user vs all N at once, particularly when one tape may hold data for many of these customers.)
Re: Inside the longest Atlassian outage
#505Earlier quoted context omitted.
> Well its mathematically impossible to be accurate as soon as you have > 20 users. It's probably in the semantics. Text input and editing is clearly a part of functionality that's probably used by everyone (or at least most users), so it's not possible for "different 5%" to mean what you're alluding to, maybe the phrasing needs work. In any given 5% there might be 1-4% of overlap with what others are using and the r…
And the greater the degree of overlap the weaker the implicit argument. If it's a uniform distribution of discrete features then each feature is equally "important" and worth equal resources and dev time. If 81/100 companies use the exact same 5% of features and the remaining 19 cover the remaining 95%, then all else equal you can probably drop 95% of your features and still do well.
The dynamics of the Enterprise market are such that there are features where having just one customer that will make a buy/no-buy decision based on just one feature will deliver enough incremental ARR to justify the opportunity cost of doing that feature instead of a bunch of others.
Typically you do the most popular features first, but most Enterprise vendors end up working on a long tail of niche features that nevertheless are profitable.
There's a long conversation to be had about how this ends up being a trap where Enterprise software gets bloated and shitty and eventually gets disrupted by a small vendor that does "less," but in a powerful, transformative way that obsoletes the Enterprise "standard," which leads us back to discussing Atlassian :-)
They're a good example of this dynamic, because they have a "constellation" of products to sell. So if they build a niche feature that gets a new customer to buy Jira seats, having "landed" in the account, their salespeople can "expand" by selling OpsGenie and other related products very profitably.
Re: Inside the longest Atlassian outage
#506This is extremely poor for a large SaaS company. A standard RFP question for SaaS should be: - Can you restore data for a single customer, and if so, what is the RTO for that operation? A smaller SaaS could be excused for only thinking about full database restores. When you're a scrappy upstart, thinking about hypotheticals is less important than survival. But for any decent size multi-tenanted SaaS, it's imperative…
Rather, customers must stop using Atlassian cloud services.
Which is becoming more and more difficult due to them focusing on Cloud Products (my on-prem renewal jumped almost 8x this year).
I’d rather use request tracker or bugzilla over Atlassian these days
Re: Inside the longest Atlassian outage
#507Engineering mistakes happen. The most inexcusable thing is not communicating with the paying customers who have been affected for over a week. Atlassian's Global Head of Customer Success probably should have been fired but here she is promoting Atlassian Cloud on LinkedIn three days ago: https://www.linkedin.com/mwlite/in/gertie-rizzo-5b70061 Actually reading a bit more, it seems like their customer team was partying…
Fair criticisms on response times but regarding Vegas, it was their annual user conference last week in Vegas.
Re: Inside the longest Atlassian outage
#508All I can say as an Attlassian Server products user is that the moment they say it was Cloud or nothing, I choose nothing. I much rather running Gittea on a raspberry pi that I CONTROL than having to have the impotence of doing nothing for more than a week. + having work at cloud companies and having been requested to "collect customer data" to hand it over to the government I would NEVER move critical pieces to anyo…
As I understood it is not "Cloud or Nothing" but "Cloud or Data Center" - is this wrong?
The on-prem offering of Atlassian was discontinued. Existing contracts are being honored but as of March 2022, that's the end of the line for it. Maybe it will be revived now.
Re: Inside the longest Atlassian outage
#509This is extremely poor for a large SaaS company. A standard RFP question for SaaS should be: - Can you restore data for a single customer, and if so, what is the RTO for that operation? A smaller SaaS could be excused for only thinking about full database restores. When you're a scrappy upstart, thinking about hypotheticals is less important than survival. But for any decent size multi-tenanted SaaS, it's imperative…
Is it standard for a RFP to have a long list of questions like this? I've never been involved in an RFP from either side. Is it standard to (in addition or instead) to have something more general/forward-looking like: how do you watch other providers' postmortems and apply the lessons to your own system? > - Can you restore data for a single customer, and if so, what is the RTO for that operation? If I were to aim so…
Yes, pages of them. Multiple pages of security questions, ciphers used, how data is stored, when is it encrypted, etc. I filled out a 20 pager once. As the company got better and more mature, we had a bunch of canned answers to make it easier and faster....
Re: Inside the longest Atlassian outage
#510Earlier quoted context omitted.
You're assuming every team would have better uptime with in-house solutions I think many would have worse uptime even with more headcount
I find this argument to be totally bs these days. If anything, a smaller company with smaller footprint and fewer total requirements is going to be more likely to manage a vertical slice of some SAAS product. The reason things like github go down so often is because they are public/shared resources.
>The reason things like github go down so often is because they are public/shared resources.
Very much this. Managing shared resources at scale is pretty hard. We have a bunch of internal sites made by interns as part of their internships, and, funny enough, those sites have much greater uptime and appear more stable than our own multi-tenant SaaS solution made by seasoned devs.