Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Verizon outages reported across U.S.

firstcoastnews.com

Post reply on HN
Print view View original

101–110 of 193 posts

Re: Verizon outages reported across U.S.

#101

Post by turtlesdown11 »

Nextgrid wrote: post #79
IncandescentGas wrote: post #72

Earlier quoted context omitted.

Telco infra runs on autopilot?

Almost; you need maintenance and monitoring but that doesn't take anywhere near 100k people - assuming they even use their own headcount for this instead of just outsourcing maintenance to their equipment vendor. Big Tech companies operate much more complex systems (for starters, they actually build greenfield stuff instead of buying ready-made equipment from a vendor and plugging it in) and have way less headcount.

VZW has 146m lines. Each employee supports 14,600 customers, seems like a reasonable number...

Re: Verizon outages reported across U.S.

#102

Post by Nextgrid »

Spooky23 wrote: post #87
Nextgrid wrote: post #79

Earlier quoted context omitted.

Almost; you need maintenance and monitoring but that doesn't take anywhere near 100k people - assuming they even use their own headcount for this instead of just outsourcing maintenance to their equipment vendor. Big Tech companies operate much more complex systems (for starters, they actually build greenfield stuff instead of buying ready-made equipment from a vendor and plugging it in) and have way less headcount.

lol. You figured it out, Verizon wireless can just be replaced by a UniFi router and ChatGPT.

ChatGPT can very well be an upgrade compared to the "engineering" capability of a lot of telcos (they have very little, are hell-bent on outsourcing as much as possible and are even proud of that). But don't take it from me, here's a more reliable source: https://berthub.eu/articles/posts/5g-elephant-in-the-room/

Re: Verizon outages reported across U.S.

#103

Post by joecool1029 »

cormorant wrote: post #96
joecool1029 wrote: post #90

Lol it's network core. Postpaid users on Verizon branded accounts affected. Most MVNO's just fine (checked US Mobile and Visible, different cores... and they are working fine on the Verizon towers)

Further info: https://old.reddit.com/r/cellmapper/comments/1qcwj48/observa...

T-Mobile had this issue 5 years or so ago where the IMS core went down due to fiber issues at one of their major backbone providers (pretty sure it was zayo).

As I've said elsewhere, never underestimate Verizon's incompetence. A couple years ago they shut down vtext service which also happened to host their UAprof's for most Android phones. Without a working link to these descriptors, those devices would download any MMS (pre-RCS picture messages) in compatibility mode at the lowest resolution.

Re: Verizon outages reported across U.S.

#104

Post by schmuckonwheels »

happytoexplain wrote: post #74
schmuckonwheels wrote: post #60

Earlier quoted context omitted.

Is there a substantive connection? Like all the doom and gloom after the Twitter layoffs predicting the site would implode and go permanently offline "within a month" which...never happened. It's also ironic in the sense it implies the indignant people were so bad at their jobs they designed and built a system so fragile it would collapse without constant intervention from thousands of individuals. You do realize it'…

This is unrealistic and seems to be biased by some kind of broad un-focused hostility. Yes, maybe they were overstaffed. But it's reasonable to suspect that leadership overcut, given the current climate and the number being 15,000. Your characterization of Twitter predictions relies on cherry-picking and ignores the actual impacts, and there's no evidence that the system goes down without "constant" intervention from…

>Your tone also implies that large, complex systems, even if designed well, don't normally require a lot of maintenance from many people.

That's correct.

In the case of Twitter, it was disclosed that many of their systems were running out of date EOL software, to the point of being a security liability, which raises the question: if the systems weren't being maintained, wtf were all those people doing? Taste-testing the free food and cappuccinos?

Re: Verizon outages reported across U.S.

#105

Post by threecheese »

kotaKat wrote: post #27

I've got a Verizon Network Extender and it appears to be online - the tunnel is up back to VZW's security gateway, but all of my phones are refusing to register to it. I did manage to roam onto an international network on the boarder near me in New York/Canada, so some bits of the core seem functional for authentication. When I roam internationally I appear to be on Telus's 3G network (no LTE) for data and voice is f…

Same. Femtocell is online but no clients, phones on SOS mode.

Re: Verizon outages reported across U.S.

#106

Post by jermaustin1 »

Bratmon wrote: post #54
jandrese wrote: post #21

Earlier quoted context omitted.

One of these times they will be right and you will never hear the end of the time they were first to recognize the start of a cyberattack.

You know what they say about a stopped clock: it's wrong 1,438 times a day.

If it is a quartz watch, it might be 47,120,384 times per day.

Re: Verizon outages reported across U.S.

#107

Post by Xylakant »

schmuckonwheels wrote: post #28
embedding-shape wrote: post #13

Earlier quoted context omitted.

That's not true though, sometimes it's inexplicitly DNS too.

My money is on expired certificate. If so, successful test.

The great thing is that with all of the encrypted/signed DNS thingys, you now have not one, but multiple options to combine “invalid certificate” and DNS in one outage. You no longer need to choose!

Re: Verizon outages reported across U.S.

#108

Post by Meekro »

venturecruelty wrote: post #62
schmuckonwheels wrote: post #12

I see multiple posts here speculating on cyberattack—as opposed to "we pushed a bad configuration update which messed everything up irreparably"—you know, like it has been every other time before this. E.g., Cloudflare, Meta (who in doing so also locked themselves out of the building), and didn't some bumbling major Canadian telco knock themselves offline for like a week not too long ago?

In a dead empire, sufficiently advanced rot is indistinguishable from malice.

Infrastructure in general seems worse than 20 years ago. Our talent for black-bagging dictators has never been stronger, though!

Re: Verizon outages reported across U.S.

#109

Post by syntaxing »

relium wrote: post #89
syntaxing wrote: post #25

Earlier quoted context omitted.

I’m somewhat surprised you’re still using your ISP’s DNS when there’s ton of better free or paid options.

We had edge delivery issues when I didn't use my ISP's DNS, especially from Apple. Not exactly sure of the mechanism, but downloading Xcode would take 2 hours instead of 10 minutes.

That’s really weird that’s the case. DNS simply resolves “google.com” to an IP address (8.8.8.8 or something). Shouldn’t impact anything download related. I’m pretty sure DNS isn’t used for geolocating either

Re: Verizon outages reported across U.S.

#110

Post by apitman »

I got sniped away from Visible (Verizon MVNO) by US Mobile (multi-carrier MVNO) during a Black Friday sale. USM has an interesting thing where you can actually get a separate eSIM for each of the major carriers, and switch between them. I was curious so I signed up for all 3. It's been interesting to see how the signals vary from location to location, and at least a couple times I've been able to get significantly better signal by switching.

The main downside is that you have different numbers for each eSIM, but that doesn't really affect me because I use Google Voice for SMS.

Post reply on HN