Rogers network outage across Canada hits banks, businesses and consumers • Hacker News Board

Re: Rogers network outage across Canada hits banks, businesses and consumers

#251

Post by walrus01 » Sat, Jul 09, 2022, 1:34 AM UTC

futureproofd wrote: post #236

walrus01 wrote: post #196

Earlier quoted context omitted.

> The utterly inefficient and overpriced Canadian telecoms exist only because we allow one specific behavior: > 1. A new local player comes to town, builds their own infrastructure (e.g. connects one building to fiber) and starts offering competitive service. You forgot to mention that whoever is the ILEC in an area (Telus, Bell, etc) and/or the incumbent 35+ year early advantage local cable TV operator (Shaw, Rogers…

They legally have the right of way to build infrastructure to connect to newly developed, last mile locations? Interesting.

whether in usa or canada, local governments treat the incumbent local phone company and cable tv company as a default essential utility, so if it's not a legally enshrined right, it would certainly be shocking and weird if they didn't build to new places

of course in places where the local phone company or cable tv company shares aerial utility pole based infrastructure with the local power company, they're very close buddies as well

Re: Rogers network outage across Canada hits banks, businesses and consumers

#252

Post by bawolff » Sat, Jul 09, 2022, 1:39 AM UTC

whynotmaybe wrote: post #110

brandon272 wrote: post #72

I see a lot of people noting the lack of communication and updates from Rogers. To me it would be shocking if they were providing updates. Canada’s telecommunication monopolies are not known for their customer service and, even worse, they seem to have internal corporate cultures of entitlement and arrogance that drive an “F U” attitude in general when it comes to external communications and accountability. So you wo…

It's not possible to pay with debit card in Canada because Interac is down, because Rogers is down. Many atm across the country aren't working. It's not just Rogers' client who are impacted, the whole banking ecosystem is impacted. Even rumors that Costco can only accept cash, the Capital One Mastercard network might be down. I don't see how Rogers could keep quiet on this, unless it's from some malicious actor.

Have you ever seen any similar company give meaningful updates in a situation like this?

It is odd they don't have PR people trying to spin it. But I am not surprised in the slightest that there is no technically useful updates happening.

Re: Rogers network outage across Canada hits banks, businesses and consumers

#253

Post by ehPReth » Sat, Jul 09, 2022, 1:42 AM UTC

mynameisvlad wrote: post #99

acchow wrote: post #59

Earlier quoted context omitted.

The payment terminal still needs internet access to process your Apple Pay transaction

It does not for credit. Credit transactions can be batched offline and transmitted when things come back online. Debit must be online, though.

What happens if the transaction is over credit limit or if someone uses a fake (once valid but now cancelled, generated, etc) card?

Re: Rogers network outage across Canada hits banks, businesses and consumers

#254

Post by notatoad » Sat, Jul 09, 2022, 1:46 AM UTC

sophacles wrote: post #3

drpgq wrote: post #2

I was hoping there would be a discussion of this here with some inside baseball, but then HN was down.

The Cloudflare blog suggests it's BGP problems: https://blog.cloudflare.com/cloudflares-view-of-the-rogers-c...

this is pretty far outside my area of expertise so maybe i'm missing something, but isn't the cloudflare blog just describing what they're seeing from the perspective of somebody who monitors BGP traffic, rather than assigning the blame to BGP?

it seems like the BGP traffic signature of any isp-level outage, regardless of the cause, would look like this. they went down, so they stopped advertising that they could accept traffic. then they tried to accept a little bit of traffic, but that didn't work so they turned it off again.

Re: Rogers network outage across Canada hits banks, businesses and consumers

#255

Post by betaby » Sat, Jul 09, 2022, 1:50 AM UTC

RL_Quine wrote: post #66

betaby wrote: post #54

Earlier quoted context omitted.

Can you please show bogons learned from AS812? Thanks!

https://bgp.he.net/AS812#_bogons They announced 216.176.216.0/21.

Completely unrelated. Network in questions was advertised from Rogers for last 3+ years.

Re: Rogers network outage across Canada hits banks, businesses and consumers

#256

Post by antoniuschan99 » Sat, Jul 09, 2022, 1:52 AM UTC

tra3 wrote: post #32

Staying with the inlaws that use Rogers' reseller for wired internet. Using Fido for mobile internet access (also by Rogers). Zero internet. It's wild to discover how reliant I am on the internet: - Can't get around this city (no waze, gmaps). - Can't figure out what's going on (AM radio is non stop commercials) - Can't figure out where we might get some internet (maybe WeWork somewhere? How do you find one without i…

680 news is AM radio version of CP24 at least.

680 news is owned by rogers and cp24 is owned by bell lol!

Re: Rogers network outage across Canada hits banks, businesses and consumers

#257

Post by mgbmtl » Sat, Jul 09, 2022, 1:55 AM UTC

I think my weirdest experience was a server that was jammed on "apt update". I looked up the IP using whois.. and it belonged to Rogers.

Re: Rogers network outage across Canada hits banks, businesses and consumers

#258

Post by ChoGGi » Sat, Jul 09, 2022, 2:05 AM UTC

I see it seems to be a day for shitting on Canadian telecoms. May I say a hearty "Up yours CRTC".

Seems as good as any time to share a Rogers story with HN:

My mum passed away in 2009, she owed Rogers $200 and change. We took over her contacts/mail forwarding/etc to tie everything up. A month or two afterwards; we got a call from Rogers looking for their money, explained she passed away, they said how sorry they were and figured that was the end of it.

3/4 months after that we got a call from a collections agency, mentioned her passing, got apologies and hang up. This then continued every few months bouncing between Rogers and a collections agency. For some reason they kept passing the debt back and forth? The frequency was low enough, that we never bothered going any further with it, after a couple years it was less annoying and more a nice reminder of mum :)

It took about 6-7 years before they finally clued in, and that's why I'll never have anything to do with Rogers.

Re: Rogers network outage across Canada hits banks, businesses and consumers

#259

Post by jml7c5 » Sat, Jul 09, 2022, 2:10 AM UTC

shirononon_ wrote: post #6

this is quite a significant outage and has been ongoing for over six hours now. would love to see a tech write up or blog post or even a Twitter hint but there seems to be absolutely zero transparency.

It's quite surprising. I wonder if the technical side and the PR side of the company just aren't sufficiently connected.

It would be beneficial to have unvarnished tweets from someone in a technical role, written for technical peers rather than the general public. The press and tech journalists can filter the jargon into something useful for customers. "Dashboards are showing ABC, we're trying XYZ, so far it's looking like LMN", even if the public don't understand what is being said, is more reassuring than a generic statement. It suggests competence and hints that yes, it really is a difficult problem.

Re: Rogers network outage across Canada hits banks, businesses and consumers

#260

Post by FpUser » Sat, Jul 09, 2022, 2:11 AM UTC

skipants wrote: post #164

UkrainianJew wrote: post #121

The utterly inefficient and overpriced Canadian telecoms exist only because we allow one specific behavior: 1. A new local player comes to town, builds their own infrastructure (e.g. connects one building to fiber) and starts offering competitive service. 2. Rogers/Shaw/Bell/Telus immediately offer better terms for the residents of that building. 3. The competitor runs out of money and leaves. 4. The telecoms revert…

> 2. Rogers/Shaw/Bell/Telus immediately offer better terms for the residents of that building. I was about to write a whole comment about how I've never seen this myself and to disagree with you I was going to compare the price of Bell Fibre in my building to Beanfield. The latter of which I have at $50/mo at 1Gbps. But then I loaded up Bell's availability page and entered my address... it was almost like a sick joke…

I use 1Gbps symmetric Bell fiber small business package which includes static IP. Pay $129/month and generally happy. Static IP is important as I host some stuff right in my basement. You got me curious with this Beanfield. It is indeed $50 for 1Gbps symmetric (no static IP of course) but when I looked at their business package it is $300 for the same speed. Static IP is extra. I wonder what does such a big difference include.