I have tried AT&T Fiber twice and the experience have been underwhelming both times. Intermittent outages and dead connection on a frequent basis. Unfortunately, I switched to Comcast 1GB connection and it has been miles better than AT&T
AT&T Fiber in the SF Bay Area is flipping bits
251–260 of 374 posts
Re: AT&T Fiber in the SF Bay Area is flipping bits
#252From my professional experience of programming and debugging networking equipment, this could be a switch/router with a buffer with bad memory (stuck bit maybe). The better chips have CRC/Parity/ECC to cover such issues but there are always those magical choke points where the past CRC is tossed and the new one is generated that can leave a gaping hole. The tricky part is how often is this bad memory buffer used... I…
Hardware designers basically started making bad decisions on this issue around the time that VLAN tagging was introduced, as well as harware forwarding of IP packets. When VLAN tags are inserted or removed, the CRC of a packet needs to be adjusted to reflect the inserted, removed and/or modified bytes from the VLAN header. Additionally Both the CRC and IP checksum of a packet needs to be adjusted when TTL is decremented as part of IP routing.
When implementing this functionality, the naive hardware designer will strip the existing CRC from the packet, modify the contents of the packet and then reuse the handy dandy CRC calculation block to place a newly calculated CRC on the packet. Similar choices are made for the adjustment of the IP/TCP/UDP checksums. If any errors are introduced in the contents of the packet by the data path prior to the new CRC is calculated, this results in the CRC being "corrected" to include the erroneous data.
A far more understanding hardware designer will instead calculate how to adjust the CRC by the changes introduced in the packet contents. Sadly, this is far more complicated to get right, and it goes against the drive of hardware designers to reuse blocks of code wherever possible. Every hardware designer working on networking has a block of Verilog or VHDL code to calculate and append a CRC to a packet. Only the most dedicated will attempt to apply only the delta needed to the CRC or checksum.
Re: AT&T Fiber in the SF Bay Area is flipping bits
#253Earlier quoted context omitted.
Funnily enough this is how my AT&T fiber is set up as well. They force you to use their router, and you can’t directly connect your router to the ONT. The problem is that they use a device certificate + EAP. There’s work arounds but it’s a pain.
Oh, there's a workaround for that.. I've unplugged my Residential Gateway and now my UniFi dream machine pro is directly connected to the ONT. You install a CA from a jailbroken modem into a supplicant container that runs on the UDM pro. It confirms to the network that you are using "authorised" equipment for the connection and the packets flow!
Which is then a problem when you try to explain them that yes, you are sure the issue is with their service and not your setup. But what's your reason to be going such lengths instead of just plugging UDM into their router? Unless it was done for the fun of it which is fine and understandable.
Re: AT&T Fiber in the SF Bay Area is flipping bits
#254Earlier quoted context omitted.
Data point: I’m on Sonic fiber (resold AT&T fiber) and this script has been running without errors for 5+ minutes.
> resold AT&T fiber Only in certain areas! Within San Francisco on overhead-cabled blocks (for example) their lines are their own.
Oh, good to know, I am indeed in SF.
Re: AT&T Fiber in the SF Bay Area is flipping bits
#255I had AT&T via Sonic's FTTN in an effort to avoid Comcast/AT&T directly, especially since Sonic does not have data caps despite running on AT&T's network and actually respects their customer's privacy. Unfortunately my experience was so terrible and unreliable that I decided to give up and finally sign up with Comcast, and just have to mind that 1.2TB data cap. The Bay Area of all places should not have such terrible…
I had to switch also after missing days of work due to not being able to use the internet with AT&T. They just kept trying to send new routers, which was not the problem. And I thought ISPs in Alaska were bad, I never thought I'd have worse service in the Bay Area.
Re: AT&T Fiber in the SF Bay Area is flipping bits
#256According to https://twitter.com/alexstamos/status/1336163726345441280 it appears to be fixed now.
Re: AT&T Fiber in the SF Bay Area is flipping bits
#257Earlier quoted context omitted.
TCP header does have a checksum that is supposed to check both the header and the payload. So either the bit corruption is such that is not detected by the checksum Or ATT is doing something nefarious and touches layer 4 and corrupts the data while doing so
I don’t have numbers off hand, but from a feeling I have from memory, I would think it is extremely unlikely that TCP checksums are consistently failing to trigger retransmission. Someone must be altering packets along the way.
It is triggering retransmission. It just retransmits until it gets lucky with the (pretty weak) TCP checksum.
Re: AT&T Fiber in the SF Bay Area is flipping bits
#258Earlier quoted context omitted.
To my knowledge, from working on an actual software router, a router will only touch the TTL and recalculate the IP header checksum. There is no reason for it to touch the TCP header.
Agreed there is no logical reason for it to touch the TCP header. And yet, an unfortunate number of L2 switches do exactly that. :(
Doesn't NAT (specifically carrier-grade NAT in this case) often modify the port? Although I don't know if AT&T does carrier-grade NAT.
Re: AT&T Fiber in the SF Bay Area is flipping bits
#259If you have AT&T fiber, run the script in the linked gist: https://twitter.com/bmastenbrook/status/1335400747794530304 It loads http://example.com and https://example.com and compares the result (should be equal) in a loop, and then reports if it finds a difference. I'm seeing multiple bit flips in the unencrypted version, and having a lot of issues loading web pages, presumably because a corrupted packet in a TLS ha…
Oddly, I can't even resolve example.com (using AT&T DNS).
My AT&T bay area fiber DNS has been garbage since the day we got it. I've had to forcibly update every device we own to Google DNS as well as put another router in between our devices and the AT&T one since AT&T doesn't let you actually change those settings on your device.
Re: AT&T Fiber in the SF Bay Area is flipping bits
#260Earlier quoted context omitted.
I had a ginormous AT&T router/modem (pace 5268ac) with a set of static ip addresses and a few times, AT&T just stopped routing traffic to it. It had happened before and then magically fixed itself a few days later. One time I had a week of outage with AT&T basically said the problem was on my side. They could ping the modem, and then punted. I had several truck rolls. The techs were really nice guys, but were basical…
The 5268ac pace router is the worst ISP provided router I've ever had, and I've been an Xfinity/Comcast customer, and I've even had a connection in Wyoming. I detailed my experience with it in a review of a third-party router, and found numerous issues along the way [0]. My favorite is that DMZ+ mode, which is what they offer instead of a traditional DMZ mode, just has some weird MTU issue that leads git and other se…
[deleted]