Live data from Hacker News

Verizon outages reported across U.S.

firstcoastnews.com

61–70 of 193 posts

Re: Verizon outages reported across U.S.

#62

I see multiple posts here speculating on cyberattack—as opposed to "we pushed a bad configuration update which messed everything up irreparably"—you know, like it has been every other time before this. E.g., Cloudflare, Meta (who in doing so also locked themselves out of the building), and didn't some bumbling major Canadian telco knock themselves offline for like a week not too long ago?

In a dead empire, sufficiently advanced rot is indistinguishable from malice.

Re: Verizon outages reported across U.S.

#63
post #47

Recently downloaded bitchat so I can contact my family members if outages like this happen when we’re out and about. I got a few T1000 cards for meshtastic but there’s just too much friction to teach my spouse and others how and when to use it. I wish haloW was built into phones which would make long range local communication much better.

Really neat concept but that name is so unfortunate. Is it bit-chat or bitch-at? Frankly either would work.

Checking off "complains about the name" on my HN bingo card.

Re: Verizon outages reported across U.S.

#64

At 2:14PM EST, Verizon said: "Verizon engineering teams are continuing to address today's service interruptions. Our teams remain fully deployed and are focused on the issue. We understand the impact this has on your day and remain committed to resolving this as quickly as possible." As someone responsible my whole career for uptime and network response, I really feel for the engineers, at the same time hoping my ser…

Odds are they have a ton of outsourced engineers who are an ocean away and unable to do anything.

At least the shareholders have had an amazing quarter. :)

Re: Verizon outages reported across U.S.

#66

I see multiple posts here speculating on cyberattack—as opposed to "we pushed a bad configuration update which messed everything up irreparably"—you know, like it has been every other time before this. E.g., Cloudflare, Meta (who in doing so also locked themselves out of the building), and didn't some bumbling major Canadian telco knock themselves offline for like a week not too long ago?

Yeah, the Canadian telco was Rogers. Total recovery took multiple days. From the Wikipedia writeup:

> In a letter to the CRTC, Rogers stated that the deletion of a routing filter on its distribution routers caused all possible routes to the internet to pass through the routers, exceeding the capacity of the routers on its core network.

https://en.wikipedia.org/wiki/2022_Rogers_Communications_out...

Re: Verizon outages reported across U.S.

#67

I see multiple posts here speculating on cyberattack—as opposed to "we pushed a bad configuration update which messed everything up irreparably"—you know, like it has been every other time before this. E.g., Cloudflare, Meta (who in doing so also locked themselves out of the building), and didn't some bumbling major Canadian telco knock themselves offline for like a week not too long ago?

The major Canadian outage was Rogers in 2022: https://en.wikipedia.org/wiki/2022_Rogers_Communications_out...

I took my kids swimming that day and the pool couldn't take our money since the payment terminal was on the Rogers cellular network, so it was a free family swim.

Re: Verizon outages reported across U.S.

#68
post #47

Recently downloaded bitchat so I can contact my family members if outages like this happen when we’re out and about. I got a few T1000 cards for meshtastic but there’s just too much friction to teach my spouse and others how and when to use it. I wish haloW was built into phones which would make long range local communication much better.

Really neat concept but that name is so unfortunate. Is it bit-chat or bitch-at? Frankly either would work.

It's obviously bit chat

Re: Verizon outages reported across U.S.

#69

I see multiple posts here speculating on cyberattack—as opposed to "we pushed a bad configuration update which messed everything up irreparably"—you know, like it has been every other time before this. E.g., Cloudflare, Meta (who in doing so also locked themselves out of the building), and didn't some bumbling major Canadian telco knock themselves offline for like a week not too long ago?

[deleted]

Re: Verizon outages reported across U.S.

#70

For those who weren't aware: Verizon got a new CEO late last year and laid off 15% of the workforce (15,000 people). This included people working in network, IT and cyber security.

If you’ve got 100k people to run something that should run essentially on autopilot, you’ve got much deeper problems where merely laying off selected chunks of people will no longer help. The whole company is rotten and the only way is to start from scratch and not make the mistakes that led you to accumulate 100k people.

I can't even begin to imagine what those 100k people actually do. For starters, none of the telcos actually develop their own equipment - they buy pre-made from vendors like Ericsson. Often that includes ongoing maintenance too. The only "engineering" is building the back-office and customer-facing UIs, and even that is often outsourced (as a rule of thumb, if something can be outsourced, telcos will do it: https://berthub.eu/articles/posts/5g-elephant-in-the-room/).

Customer service might be part of that number (assuming that too isn't outsourced), but even then 100k feels extreme.

10k is ok although leaning on the more bloated side. But 100k?

Post reply on HN