221–230 of 486 posts
Verizon and T-Mo both issued statements that they have no outages and the issue is just their customers being unable to call AT&T customers. Looks like most of the AT&T network in the US is down though.
Earlier quoted context omitted.
Rogers, of course, blamed their vendor (Ericsson I believe it was). Rogers can do no wrong! Of course, was fun to see yet another huge org have no back-out/failure plan for their potential enterprise-breaking changes. No/limited IT 101 stuff here. The only positive thing we learned was that the big 3 (really 2) telcos thought it would be a good idea to give eachother emergency backup sims for the other network to key…
> Fun that Rogers used the same core for wireless and wired connections, so many of us were in total blackout, even if we used a 3rd party internet provider that ran over Rogers. If it ran over Rogers circuits then why wouldn't it go down too? Isn't that the case everywhere?
The 3rd party providers aren’t white-label resellers, but there’s obviously some overlapping susceptibilities to going down when Rogers breaks something. Depends what they break, and in this case, it took them down too.
To everyone trying to speculate on the root cause, I haven't seen enough information in any of the comments to really draw any conclusions. Having worked on several nationwide cellular issues in Canada when I worked in telecom, we saw nationwide impacts based on any number of causes. - A new route injected in the network caused the routing engines on a type of cellular specific equipment to crash nationwide. This too…
And yet, like everyone else, I genuinely feel that I'm probably right
This is affecting roughly 0.02% of the US population; is it really that bad?
This reminds me of the recent discussion on status pages. https://news.ycombinator.com/item?id=39099980 They need to be accurate. At&t status claims everything is fine. My wireless service is down. Down detector has tens of thousands of reports, so clearly everything is not fine.
>Service Alert: Some of our customers are experiencing wireless service interruptions this morning. We are working urgently to restore service to them. We will provide updates as they are available.
I wonder if this extends to the Public Safety part of AT&T - FirstNet.
The AT&T land lines (CAMA trunks provided by the ILEC Frontier) that handle the 911 service did not fail. Only mobile service failed.
Earlier quoted context omitted.
Rogers, of course, blamed their vendor (Ericsson I believe it was). Rogers can do no wrong! Of course, was fun to see yet another huge org have no back-out/failure plan for their potential enterprise-breaking changes. No/limited IT 101 stuff here. The only positive thing we learned was that the big 3 (really 2) telcos thought it would be a good idea to give eachother emergency backup sims for the other network to key…
> Also fun where the towers remained just powered on enough for phones to stick to them but not be able to do anything, so 9-1-1 calls would just fail, instead of failing-over to other networks. Seems like a deficiency in the GSM spec (or Rogers SIM programming?) that I don’t think was actioned on. Actually, I think this is going to change after the Rogers outage, it's just slowly happening behind the scenes so it's…
It seems like this is a global problem, since all Rogers-subscribed devices in a Rogers reception area couldn’t make 9-1-1 calls. But could be a SIM coding issue and not afflict other providers elsewhere.
I just always imagined the GSM spec was so resilient that you could always make a 9-1-1 call if a working network was available but this outage proved that wrong. Surprising to learn in 2022.
Of course it’s Canada, so I agree with them that the thought of letting users failover to a partner for everything would thrash the partner’s networks. Even though Canadian subscriber plans are laughably low in monthly data and population density is low (per the telecom’s usual excuse for our high prices) it turns out the telecoms still underbuilt their networks to have less capacity than what other networks internationally built out to support plans available on the international market (e.g. close to truly unlimited data/free long distance calls)
Latest AT&T Statement: “Our network teams took immediate action and so far three-quarters of our network has been restored,” the company said. “We are working as quickly as possible to restore service to remaining customers.” Still down for me though.