Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AT&T Fiber in the SF Bay Area is flipping bits

twitter.com

Post reply on HN
Print view View original

61–70 of 374 posts

Re: AT&T Fiber in the SF Bay Area is flipping bits

#61

Post by kevinherron »

cpach wrote: post #54
francoisLabonte wrote: post #29

From my professional experience of programming and debugging networking equipment, this could be a switch/router with a buffer with bad memory (stuck bit maybe). The better chips have CRC/Parity/ECC to cover such issues but there are always those magical choke points where the past CRC is tossed and the new one is generated that can leave a gaping hole. The tricky part is how often is this bad memory buffer used... I…

”I would use traceroute to find a common bad point for everyone“ How do you mean? I use traceroute from time to time but I’m not sure how it would apply in a case like this. Feel free to elaborate :)

I think he means if everyone on AT&T experiencing the issue ran trace route to example.com some common hops would emerge, which would be a place to start investigating.

Re: AT&T Fiber in the SF Bay Area is flipping bits

#62

Post by slopirate »

mschuster91 wrote: post #49
cameldrv wrote: post #40

Earlier quoted context omitted.

The parts of SV that haven't buried their power lines (not too pretty to begin with) have gotten significantly uglier since all of the new cable/fiber/DSL infrastructure has gone in. There are in many cases multiple of these things on loads of poles: https://www.dslreports.com/forum/r31593060-Equip-What-is-thi...

Here in Germany we bury everything . I don't even want to imagine how long it takes to rebuild all of that when some drunkard crashes his car into one of these poles or a lightning strike hits.

A lot less time than when an earthquake snaps an underground cable.

Re: AT&T Fiber in the SF Bay Area is flipping bits

#64

Post by simcop2387 »

cpach wrote: post #54
francoisLabonte wrote: post #29

From my professional experience of programming and debugging networking equipment, this could be a switch/router with a buffer with bad memory (stuck bit maybe). The better chips have CRC/Parity/ECC to cover such issues but there are always those magical choke points where the past CRC is tossed and the new one is generated that can leave a gaping hole. The tricky part is how often is this bad memory buffer used... I…

”I would use traceroute to find a common bad point for everyone“ How do you mean? I use traceroute from time to time but I’m not sure how it would apply in a case like this. Feel free to elaborate :)

Take a traceroute from everyone experiencing the problem, look for the common hops among them all. Then compare that list against people not experiancing the problem to find the differences. The finsl set there are good places to start looking. Sweitches and routers along that way could be the cause

Re: AT&T Fiber in the SF Bay Area is flipping bits

#65

Post by proverbialbunny »

ProfessorLayton wrote: post #47

I had AT&T via Sonic's FTTN in an effort to avoid Comcast/AT&T directly, especially since Sonic does not have data caps despite running on AT&T's network and actually respects their customer's privacy. Unfortunately my experience was so terrible and unreliable that I decided to give up and finally sign up with Comcast, and just have to mind that 1.2TB data cap. The Bay Area of all places should not have such terrible…

It's because AT&T does not follow net neutrality. It has fast lanes and slow lanes, which cause a lot of problems for work VPNs even when down the street. This TLS handshake bit flip is just yet another issue AT&T has, including the 1.1.1.1 DNS issues, and everything else.

I'm on Sonic.net fiber over AT&T too. To get the equivalent fiber from Comcast out here it's $270 a month. However, if you're lucky enough Sonic.net fiber that doesn't run over AT&T is amazing.

Re: AT&T Fiber in the SF Bay Area is flipping bits

#66

Post by ian-g »

mschuster91 wrote: post #49
cameldrv wrote: post #40

Earlier quoted context omitted.

The parts of SV that haven't buried their power lines (not too pretty to begin with) have gotten significantly uglier since all of the new cable/fiber/DSL infrastructure has gone in. There are in many cases multiple of these things on loads of poles: https://www.dslreports.com/forum/r31593060-Equip-What-is-thi...

Here in Germany we bury everything . I don't even want to imagine how long it takes to rebuild all of that when some drunkard crashes his car into one of these poles or a lightning strike hits.

I could be wrong, but I'd worry about earthquakes severing the lines.

This map[0] shows the major faults, but each one of those is a lot of small minor faults that all could snap things and shift. One small fault right by where I grew up is about three blocks long. Last I checked had one earthquake to its name: a magnitude 5.0 aftershock of Loma Prieta

[0] https://usgs.maps.arcgis.com/apps/webappviewer/index.html?id...

Re: AT&T Fiber in the SF Bay Area is flipping bits

#68

Post by rconti »

gricardo99 wrote: post #4
jonny_eh wrote: post #3

I'm so happy to have had the opportunity to switch to Sonic's Fiber last month.

That may not make any difference, depending on your locality. Unfortunately AT&T infrastructure is still relied on by others such as Sonic, which leases lines from the likes of AT&T fiber, at least in my locality.

It's pretty clear what you're getting, though. If for no other reason than the fact that the resold ATT service costs 2x what "native" Sonic fiber does.

I'm on the preorder list for Sonic, for the past.. almost 2 years now. In the meantime, I'm using resold ATT through Sonic. $10/mo more than ATT directly, but no MITM and no data caps.

Re: AT&T Fiber in the SF Bay Area is flipping bits

#69

Post by proverbialbunny »

dbt00 wrote: post #55
hangonhn wrote: post #37

Earlier quoted context omitted.

Palo Alto has its own municipal fiber program: https://www.cityofpaloalto.org/gov/depts/utl/business/progra... Wonder if someone should take them up on it.

It’s only for commercial use. As much as I’d love to be able to drop a node in to peer at paix for my house ...

I know people with Palo Alto Fiber running to their house. They were hosting websites out of their garage years ago (I think archive.org for a while.), but today it's just residential, so it's very much possible.

Re: AT&T Fiber in the SF Bay Area is flipping bits

#70

Post by yrral »

For this type of issue, I would recommend trying the nanog mailing list. You often see network admins ask for someone "with clue" at a different company when they get the runaround by tier 1/2 tech support.

https://archive.nanog.org/list/join

https://seclists.org/nanog/2019/Aug/103

Post reply on HN