51–60 of 67 posts
Extended downtime like this usually means prod database deleted. Sucks to be that SRE team. Can’t wait for the Kevin Fang re-enactment!
Earlier quoted context omitted.
But microservices are standalone services. Anyone who correctly implements ms wouldn't make it depend on other ms or get it shipped to prod anyway. Edit: /s
I would actually love to know if you have done this in prod and what it looked like.
There are some instances where that kind of thing is necessary, but...insane amount of clear downsides so your problem statement for doing it had better be pretty compelling.
"Some merchants may be seeing a higher-than-usual decline due to Gateway rejections Fraud." Is this really "down"?
The trend is is to downplay the issue in status messages to obscure the real problem. That message could mean anything from an extra 1% of rejections to 99% of transactions are failing. The vagueness is the point, because they want to avoid admitting serious problems. We had this problem with some devops hires who came from a big company. They’d delay updating the status page as long as possible, then update with the…
You still want to notify customers they may be seeing issues even if you aren't confident on the percentage of impacted customers yet.
Earlier quoted context omitted.
I would actually love to know if you have done this in prod and what it looked like.
I've seen people try and do it. It involves each service replicating copies of the data from every service it's dependent on (via CDC streams on kafka/whatever) in its own datastores and reading directly from its copy of data rather than actually calling the source-of-truth service. There are some instances where that kind of thing is necessary, but...insane amount of clear downsides so your problem statement for doi…
The premise being high service availability and low latency to serve requests (no need to talk to source of truth) with the tradeoffs of higher storage / processing costs and from my experience higher complexity.
Earlier quoted context omitted.
But microservices are standalone services. Anyone who correctly implements ms wouldn't make it depend on other ms or get it shipped to prod anyway. Edit: /s
I'm trying to figure out the odds that you intended this as sarcasm. In the real world, it doesn't work like that.
Meanwhile, anyone in Boston who only read the headline is completely unsurprised to hear that the red line isn't running and is wondering how this merits making the front page of HN.
BrainTree has been down for more than 7 hours now