Live data from Hacker News

Cellular outage in U.S. hits AT&T, T-Mobile and Verizon users

cnbc.com

281–290 of 486 posts

Re: Cellular outage in U.S. hits AT&T, T-Mobile and Verizon users

#281

Outage Over Status: Restored AT&T FINAL, Service Degradation, Global Smart Messaging Suite AT&T Global Smart Messaging Suite Event description: FINAL, Service Degradation Impacted Services: MMS MT Start time: 02-21-2024 22:00 Eastern, 21:00 Central, 19:00 Pacific End time: 02-22-2024 11:00 Eastern, 10:00 Central, 08:00 Pacific Downtime: 780 minutes Dear Customer, We are writing to inform you that Global Smart Messagi…

I think a communication like this should include that they are investigating the root cause (assuming they aren’t completely sure) and that they will share it, and state where.

Maybe im reading to much into it but it bothers me that thats not in the communication.

Re: Cellular outage in U.S. hits AT&T, T-Mobile and Verizon users

#282

Earlier quoted context omitted.

SOS means it has cell service and you can call 911. If there is no cell service then it's SOS with a little picture of a satellite next to it.

Except SFFD is reporting that (some) AT&T customers are unable to call 911.

I wonder if some devices bungle their failover. The exact failure mode state of AT&Ts network might cause some devices to hang onto AT&T’s RF.

Re: Cellular outage in U.S. hits AT&T, T-Mobile and Verizon users

#283

Latest AT&T Statement: “Our network teams took immediate action and so far three-quarters of our network has been restored,” the company said. “We are working as quickly as possible to restore service to remaining customers.” Still down for me though.

3/4 might just mean the internal facing side, which is still progress, but doesn’t mean any improvements for end-users.

Anecdotally, I woke up to no signal / “SOS” mode on my iPhone this morning at around 0600 and had service restored around 0830 in South Carolina. However, a coworker in Memphis confirmed he was still out of service at 1000 so it’s regional restoration.

Re: Cellular outage in U.S. hits AT&T, T-Mobile and Verizon users

#284

Earlier quoted context omitted.

The speculation is fascinating. For most people, their guess is a reflection of themselves. Is there a term for that? This is a gross generalization, but I've seen... - Science people guessing solar flares - My "right-wing friend" guessed international hackers - I, myself, guessed it was a botched software release - Someone in this post commented their military friend says get gas And yet, like everyone else, I genui…

My speculation is: “Higher-ups kept demanding that technicians ‘do more with less’ in order to deliver on quarterly metrics and now we’re finally seeing the cumulative result of employees being stretched thin, underpaid, and overworked.” You are welcome to infer as to why I’m thinking this way!

The ops team can run the whole company and better without the C-Suite is my impression of modern day SV. Agile stickers on waterfall gates…

Re: Cellular outage in U.S. hits AT&T, T-Mobile and Verizon users

#285
post #180

Using myATT app doesn't even show my wireless account anymore. My entire family account doesn't even show up as a service. Seems like a hack or internal issue that deleted accounts? Can others confirm whether they see their accounts?

Many of their APIs appear to be intermittently returning 502s, leading to strange behavior in their web/mobile apps.

That's just people trying to figure out if their service has been disconnected rather than it being a network outage

Re: Cellular outage in U.S. hits AT&T, T-Mobile and Verizon users

#286

Something I'm not seeing discussion on: What is/were the cascading effects of this, particularly for drivers? Many people in buildings were unaffected, as they could fallback to wifi. But I imagine this had a pretty broad impact to drivers. Just a few things I can think of: - Packages delayed (UPS, FedEx, Amazon, truck drivers, etc.) for drivers that relied on their phone's mapping apps to get them to their deliverie…

Worth noting that GPS does not rely on cell service.

Re: Cellular outage in U.S. hits AT&T, T-Mobile and Verizon users

#287

Earlier quoted context omitted.

> I do think implementations are changing a bit around this, mostly in the phones so that they give up and go into a network scan if the emergency call is failing It seems like this is a global problem, since all Rogers-subscribed devices in a Rogers reception area couldn’t make 9-1-1 calls. But could be a SIM coding issue and not afflict other providers elsewhere. I just always imagined the GSM spec was so resilient…

> I just always imagined the GSM spec was so resilient that you could always make a 9-1-1 call if a working network was available but this outage proved that wrong. As I recall it is slightly more nuanced than this and was particular to the failure mode, and has a couple of different things aligning to create the failure mode. If you're phone is just blank, no sim card. To make an emergency call, it has to just start…

Multi-faceted failure mode.

0) At the network terminal level (mobile phone): at least for emergency calls if a given network fails to connect, fail over and try other networks. Even if the preferred networks claim to provide service.

1) At the network level: failure thresholds should be present. If those thresholds are crossed enter a fail-safe state. This should include entering a soft offline / overloaded response state.

2) Where possible critical data paths should cross-route. Infra Command and Control and Emergency calls in this case. Though if Roger's issue was expired certs or something the plans for handling that get complicated.

Re: Cellular outage in U.S. hits AT&T, T-Mobile and Verizon users

#288

Earlier quoted context omitted.

3/4 might just mean the internal facing side, which is still progress, but doesn’t mean any improvements for end-users.

Anecdotally, I woke up to no signal / “SOS” mode on my iPhone this morning at around 0600 and had service restored around 0830 in South Carolina. However, a coworker in Memphis confirmed he was still out of service at 1000 so it’s regional restoration.

I always wonder instead of a regional restoration, if they would “disable” segments of SIMs/accounts randomly to avoid lightning strike (it’s not a DDoS…) their network as they turn things back on. Depends on what the recovery method is, but could be problematic to turn everything back on at once.

Re: Cellular outage in U.S. hits AT&T, T-Mobile and Verizon users

#289

NOAA is reporting R3 activity. Solar flare outage seems likely

https://www.spaceweatherlive.com/en/news/view/522/20240222-t... We did have Two Class X flares from sunspot 3590 but they did not result in a coronal mass ejection. There was a CME from another sunspot not visible but it is not aimed at the earth.

NOAA did release a statement saying the two X class flares were unlikely to effect cellular networks.

https://www.swpc.noaa.gov/news/two-major-solar-flares-effect...

Post reply on HN