Salesforce Global Outage • Hacker News Board

Re: Salesforce Global Outage

#181

Post by abeyer » Wed, Sep 16, 2026, 9:05 PM UTC

elzbardico wrote: post #79

Kind of ironic. Salesforce is basically one of the major spiritual grandfathers of Slop. It is not uncommon in production systems to find that objects like Contact and Account have hundreds of custom fields. Sometimes, you find out that several of them have the same meaning and semantics, but were used at different times. Digging out you discover that some Marketing guy that used to work at the company did some task…

> Doing data engineering work with Salesforce data is an exercise on archeology, psychology and organizational politics.

What business focused systems is this _not_ true for? Doubly so for such systems that encourage non-developer users to customize the database schema?

Not saying that's always a great choice, but they're far from the only ones to have made it, enterprise customers love to buy it, and Salesforce is actually one of the better ones I've seen to deal with from a technical perspective. There is _far, far_ worse out there. Most of this comes down to a business and design problem... your entire first paragraph reads as a broken process that no system is going to fix.

Re: Salesforce Global Outage

#182

Post by abeyer » Wed, Sep 16, 2026, 9:07 PM UTC

isityettime wrote: post #101

schnevets wrote: post #89

Earlier quoted context omitted.

Something tells me Troy the Salesforce Admin/BD Analyst did not cause the SAAS infrastructure to go down. And I think you're confusing crud with slop.

I've never worked somewhere that had a Salesforce integration which wasn't an eternal disaster. Have you? Why is every company's Salesforce team absolute bottom of the barrel developers with super high churn, no responsibility, and little competency? Something about the product and its positioning attracts catastrophe. That's what GP is talking about.

It sounds like you've worked places that have made a staffing decision and paid the price. It happens, but it's far from universal.

Re: Salesforce Global Outage

#183

Post by torginus » Wed, Sep 16, 2026, 9:42 PM UTC

supriyo-biswas wrote: post #180

torginus wrote: post #155

Earlier quoted context omitted.

Well ackchually.. I get that large scale systems pose their own challenges on their own, but it also matters what's the smallest isolable unit. What I mean by this is a CDN consists of nodes that are horizontally replicable and don't really talk to each other, and thus are easy to run even at scale. In contrast, something like a bank or social media isn't really reducible - every user needs to be able to interact wit…

> What I mean by this is a CDN consists of nodes that are horizontally replicable and don't really talk to each other, and thus are easy to run even at scale. They do though! They mostly try to avoid it since hitting the network to serve any kind of latency would unacceptably increase latency, but you wildly underestimated the amount of complexity there is to running a CDN.

I probably underestimated the complexity and I didn't mean to knock on CDNs - I just wanted to say that not all distributed systems have equal complexity, and some require essentially almost serializable transactions, while others are fine with small channels of eventual consistency

Re: Salesforce Global Outage

#184

Post by DANmode » Wed, Sep 16, 2026, 9:45 PM UTC

fragmede wrote: post #171

DANmode wrote: post #169

Earlier quoted context omitted.

> I just don’t know which 50% Yes, you largely do - they’re the commits that get rushed to, and through. This take that showstopping technical debt is unavoidable is very new, and will age like milk.

> showstopping technical debt is unavoidable is very new No it's not. The push and pull between shipping and paying down technical debt is as old as there's been software to sell. Sales has been selling features that don't exist quite yet ever since they've been talking to customers, and engineering has been pushing back on implementing them yesterday since there's been features to implement. Showstopping technical d…

Yes.

My point is the technical debt is stopping the show way more often.

Not “this never existed before selling”,

but “we never had the team in place who could do this right in the first place”.

You can quibble about who is responsible, but the fact remains.

Re: Salesforce Global Outage

#185

Post by elzbardico » Wed, Sep 16, 2026, 10:53 PM UTC

nightski wrote: post #154

stmw wrote: post #66

Despite all of the snark here, in my experience Salesforce SRE team is quite competent. The engineering challenges of running a large PaaS - not just with own apps, but with millions of customer-written apps running on it - are quite interesting, and sadly things happen. The status page makes sense to actual customers, it's the particular "pods" where a given service runs.

For me personally the snark isn't because their SRE team is incompotent. It's because software that tries to be everything to everyone is inherently terrible. It's not fun to use for the users, and so many compromises need to be made on the technology side to make that happen that it ends up just being crap all around. This includes Salesforce, SAP, Dynamics, any platforms like that which scale many industries. Flexi…

AI changes this equation fundamentally, in a way that VC and SaaS founders still haven't realized. Nobody ever liked having to change their business, their workflows, or ducting taping a customization in a SaaS, they did because as we moved from centralized mainframe apps, to PC client-server and then Web based distributed apps, it became increasingly more cost-effective to suffer with a generic, one-size-fits-all SaaS than building at home.

AI coding changes this a lot.