Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cellular outage in U.S. hits AT&T, T-Mobile and Verizon users

cnbc.com

Post reply on HN
Print view View original

421–430 of 486 posts

Re: Cellular outage in U.S. hits AT&T, T-Mobile and Verizon users

#421

Post by luxuryballs »

devgonewild wrote: post #419
jader201 wrote: post #276

Something I'm not seeing discussion on: What is/were the cascading effects of this, particularly for drivers? Many people in buildings were unaffected, as they could fallback to wifi. But I imagine this had a pretty broad impact to drivers. Just a few things I can think of: - Packages delayed (UPS, FedEx, Amazon, truck drivers, etc.) for drivers that relied on their phone's mapping apps to get them to their deliverie…

In Australia we recently had a telecom outage with Optus; there were untold amount of damage - card payments at shops/cafes were out. - rural towns completely cut off (a few in particular are only serviced by Optus) - emergency services unavailable; for example a snake wrangler was unable to receive his call-outs - hospitals infrastructure came to a halt And I'm only going off of examples I have heard. These outages…

cashiers always look at me dumbstruck when I tell them about the mechanical offline credit card machines we had when I was a cashier back in 2004

Re: Cellular outage in U.S. hits AT&T, T-Mobile and Verizon users

#422

Post by wizerdrobe »

Scoundreller wrote: post #288
wizerdrobe wrote: post #283

Earlier quoted context omitted.

Anecdotally, I woke up to no signal / “SOS” mode on my iPhone this morning at around 0600 and had service restored around 0830 in South Carolina. However, a coworker in Memphis confirmed he was still out of service at 1000 so it’s regional restoration.

I always wonder instead of a regional restoration, if they would “disable” segments of SIMs/accounts randomly to avoid lightning strike (it’s not a DDoS…) their network as they turn things back on. Depends on what the recovery method is, but could be problematic to turn everything back on at once.

I actually spoke with my wife after the initial comment, she was reconnected at a later time than I! So it’s not regional but some other mechanism.

Re: Cellular outage in U.S. hits AT&T, T-Mobile and Verizon users

#423

Post by mucle6 »

teleforce wrote: post #416

I think this is the main reason developers should always be designing their software with local-first (aka offline-first) approach in-spite of the Internet and the cloud [1]. Regarding the cellular based authentication, it is perfectly doable to be securely authenticated even without the any connectivity and there is a solution to this based on combination of MFA and OTP [2]. [1] Martin Kleppmann talk on local-first…

I think you're right, and I'd like to amend the "should always" to "after product market fit". I'm always trying to stop myself from prematurely optimizing

Re: Cellular outage in U.S. hits AT&T, T-Mobile and Verizon users

#424

Post by topkai22 »

jrochkind1 wrote: post #418
Animats wrote: post #331

This is a serious architectural flaw. In the entire history of electromechanical switching in the Bell System, no central office was ever out of service for more than 30 minutes for any reason other than a natural disaster, or, on one single occasion, a major fire in NYC. The AT&T Long Lines system in the 1960s and 1970s had ten regional centers, all independent and heavily interconnected. There was a control center,…

I've been thinking about how pretty much no USA infrastructure of today is as reliable as of the last half of the 20th century. Just my imagination, or true? And what does it mean?

Airliners crash far less, road fatalities kept going down till 2010, so taking the broad view of were infrastructure is I’d say at least those got better.

Anecdotally, I remember more electricity interruptions and plumbing issues when I was a kid, but that could be location dependent and I couldn’t quickly find good numbers going back that far.

Edit: While the phone network didn’t necessarily go down, I frequently got “all circuits busy” when I was a kid. I don’t remember the last time that happened.

Re: Cellular outage in U.S. hits AT&T, T-Mobile and Verizon users

#425

Post by deelowe »

thedaly wrote: post #408
Diederich wrote: post #385

Earlier quoted context omitted.

In this context, what's the physical scale of a 'central office', as far as regional dimensions? Thanks!

I suspect that when AT&T built all the COs (1950-70s) they were constrained by both max number of lines and physical distance. You’ll see numerous COs in a big city, but they are also pretty widely dispersed throughout the suburbs and rural areas.

I used to work at BellSouth in outside plant engineering in the early 2000s. That's exactly what it was. Of course, by then any expansion was done via remote terminals and COs were becoming very antiquated.

Re: Cellular outage in U.S. hits AT&T, T-Mobile and Verizon users

#426

Post by RajT88 »

bradleyjg wrote: post #173
efitz wrote: post #35

Reporting status externally is hard. I worked at a cloud provider and while we had very detailed metrics about how our systems were functioning, it was difficult to distill that information in a way that customers would understand if they were being impacted or not, and what the impact would be. Just reporting raw numbers wouldn’t give customers the context to understand what was actually going on. How do you actuall…

If it can’t be done then don’t do it. A dot that’s always green, no matter what, is worse than nothing.

Agree. As a customer, you need to be able to correlate your impact to anything which might be going on.

Because those cloud provider internal tools are often wrong about who's impacted and who's not.

Re: Cellular outage in U.S. hits AT&T, T-Mobile and Verizon users

#427

Post by kibwen »

lor_louis wrote: post #420
jrochkind1 wrote: post #418

Earlier quoted context omitted.

I've been thinking about how pretty much no USA infrastructure of today is as reliable as of the last half of the 20th century. Just my imagination, or true? And what does it mean?

Redundancy was needed because individual nodes/machines were more prone to failure. As machines got more and more reliable, having highly redundant infrastructure was seen as an extra cost.

Efficiency or reliability: pick one.

Re: Cellular outage in U.S. hits AT&T, T-Mobile and Verizon users

#428

Post by watersb »

topkai22 wrote: post #424
jrochkind1 wrote: post #418

Earlier quoted context omitted.

I've been thinking about how pretty much no USA infrastructure of today is as reliable as of the last half of the 20th century. Just my imagination, or true? And what does it mean?

Airliners crash far less, road fatalities kept going down till 2010, so taking the broad view of were infrastructure is I’d say at least those got better. Anecdotally, I remember more electricity interruptions and plumbing issues when I was a kid, but that could be location dependent and I couldn’t quickly find good numbers going back that far. Edit: While the phone network didn’t necessarily go down, I frequently go…

And how often one of our five TV stations would be "Experiencing Technical Difficulties... Please Stand By".

Re: Cellular outage in U.S. hits AT&T, T-Mobile and Verizon users

#429

Post by james-skemp »

luxuryballs wrote: post #421
devgonewild wrote: post #419

Earlier quoted context omitted.

In Australia we recently had a telecom outage with Optus; there were untold amount of damage - card payments at shops/cafes were out. - rural towns completely cut off (a few in particular are only serviced by Optus) - emergency services unavailable; for example a snake wrangler was unable to receive his call-outs - hospitals infrastructure came to a halt And I'm only going off of examples I have heard. These outages…

cashiers always look at me dumbstruck when I tell them about the mechanical offline credit card machines we had when I was a cashier back in 2004

If we're thinking of the same thing, I have fond memories of those at mall department stores in the 80s and 90s.

I think Sears around '04-06 (?) was the last time I saw one of those used. I think I bought a dehumidifier or air purifier.

When they started rolling out credit and debit cards without the raised numbers I thought fondly of those and how they were definitely done for now.

Re: Cellular outage in U.S. hits AT&T, T-Mobile and Verizon users

#430

Post by jsjohnst »

thedaly wrote: post #408
Diederich wrote: post #385

Earlier quoted context omitted.

In this context, what's the physical scale of a 'central office', as far as regional dimensions? Thanks!

I suspect that when AT&T built all the COs (1950-70s) they were constrained by both max number of lines and physical distance. You’ll see numerous COs in a big city, but they are also pretty widely dispersed throughout the suburbs and rural areas.

I worked for Southwestern Bell in the 90s pre-SBC (aka just before remote terminals and dslams became common). COs handled mid-tens of thousands of lines in big cities, for smaller more rural areas they generally covered a single town or less often there were a few in a county where the full county was under 50k people.

In towns, we generally tried to keep loop lengths under 30k feet, but in rural areas that simply wasn’t possible. You’d often find remnants of party line systems in those areas and definitely load coils out the wazzoo. It was “fun” unwinding all that crap to install ISDN circuits and later DSL.

I remember the old hats at the time laughing about VDSL saying “leave it to the nerds to dream up some unrealistic shit where the loop length can be at most 2k feet, where does that exist!?” not realizing a few years later RTs and DSLAMs would mean a significant portion of city and suburban customers would be.

Post reply on HN