Live data from Hacker News

AT&T Wireless traffic shaping apparently making some websites unusable

adriano.fyi

161–170 of 305 posts

Re: AT&T Wireless traffic shaping apparently making some websites unusable

#161

Earlier quoted context omitted.

Unlimited is a term of art that means "limited".

Can we just start calling fraud by its own name?

I'll order one criminality with extra politicians and one large diet-fraud-lite(tm) with extra ice.

Just charge it to my offshore. Cheers

Re: AT&T Wireless traffic shaping apparently making some websites unusable

#162

AT&T uses different APNs for regular phone plans and data-only hotspot plans. It's very possible that the phone traffic is being routed completely differently than the hotspot traffic, with congestion at a peering point occurring on the hotspot. AT&T tends to be known for having poor/congested peering. You could try changing the DNS server on your hotspot to a different public resolver like 1.1.1.1 or 8.8.8.8. If Clo…

Use resolvers that support eDNS Client Subnet, otherwise geo-resolvers like Google/CF DNS may cause your traffic to be misrouted. You can use Quad9's EDNS [0] so your client is properly routed, any privacy concerns aside. [0] https://www.quad9.net/support/faq/#edns

Quad9 doesn't work on AT&T, because of poor routing you're sent to Miami or Amsterdam.

Re: AT&T Wireless traffic shaping apparently making some websites unusable

#163

Earlier quoted context omitted.

This isn't applicable to fast.com as it simply makes requests to Netflix's CDN from the frontend. Indistinguishable from regular Netflix traffic.

Good point, although you could still do logic like only activating the throttle after the customer visits netflix.com. You can't distinguish the CDN traffic, but you can still tell what website is being viewed. Incidentally, my speeds on fast.com are always terrible (about 1/8 of what I get elsewhere), despite the fact that I'm fairly confident it is not being throttled. That's because the speed I see is >100 Mbps, w…

then you might miss embedded clients, right?

Re: AT&T Wireless traffic shaping apparently making some websites unusable

#164

While everyone knows that all wireless carriers are universally terrible, I've actually had a fairly noneventful (read: good) experience with T-Mobile over the last decade. I've never observed evidence of traffic shaping or any other shady business (like capturing NXDOMAIN DNS responses and directing you to a sponsored search page). I've also never observed significant performance degradation from congestion-based pr…

T-Mobile objectively has the best network if you have good reception. They locally peer, you're getting your own IPv6 address, and their interconnections are excellent (except with AT&T as AT&T refuses to peer and they use Zayo which will take you across the US).

AT&T by far has the worst network, even if you have great service. They NAT millions of devices behind a single /64 network, you're usually routed to IXs very far away from you, and they do the most packet manipulation out of all the carriers. Verizon is better, but they can do some wacky stuff if you aren't in New England or California.

Re: AT&T Wireless traffic shaping apparently making some websites unusable

#165
post #35

> I already knew from previous experience that for some reason, AT&T traffic to fast.com is throttled. Why AT&T wants bandwidth to appear lower than reality is a mystery to me, but I digress This I think is because they throttle ip addresses for known video streaming sites. That is one of (the only reliable) ways an ISP can get the streaming provider to drop the stream to a lower quality one by default. Since fast.co…

> Since fast.com is a Netflix ip, and the isp can’t distinguish whether it’s video that is being transferred or a file to measure throughout, It is really trivial to do basic traffic snooping and see what people are looking at. I'm surprised it isn't more common. I figured it would be harder, or perform worse, but I easily wrote a little piece of software that filters the TLS ClientHello for arbitrary domains. Maybe…

Fast.com loads from nflxvideo.net for me. They have a /speedtest/ path.

Sure, they could apply some kind of advanced DPI assessment based on packets sizes over time and other crap, but if anyone complains it's much easier to just say "well, does speedtest.net work? What about Google's speedtest? Hmm, strange, must be fast.com being slow then!". If they just check for fast.com in the SNI header, I know I'd be loading the speedtest every time I want to watch Netflix.

ECH is slowly coming along, but it's still perfectly possible to detect these speed tests. It just takes much more time and effort to set up right.

I'm honestly surprised the article shows enabling a VPN actually working to fix the messed up network here. That would be a perfectly valid solution in my opinion; if I notice my ISP is messing with my network like this, there's no way I trust them enough not to use a VPN.

Re: AT&T Wireless traffic shaping apparently making some websites unusable

#166

Taking a quick 5 minute look at the packet capture attached to the post, it looks to me like this is likely traffic shaping. While it's always difficult to be 100% sure from just the client side capture, the capture looks relatively clean of errors, and the amount of data in flight doesn't appear to even approach the advertised window. We're getting some merging of segments likely from a segment offload, but I doubt…

I do see a couple of retransmissions early in that query (lost ACKs?) which could be upstream throttling but in any event certainly explains why the connection was slow to get off the ground.

Agreed that segment offload is probably kernel-side coalescing.

From shunting this into Wireshark, you can see that the sequence number grows fairly linearly (often a good indicator of a rate limiter) starting from 1.508s up until the 1.95s mark, but at a rate closer to 25-30 Mbps (1.5MB in 0.45s).

https://i.imgur.com/s2JBrCf.png

edit: this was for iphone-capture.pcap. You see a similarly strong picture of shaping for capture.pcap [0], and indeed that corresponds to the ~320kbps range you've noted.

[0] https://i.imgur.com/g9cN95d.png

edit edit: of course, under normal situations, your bandwidth will get saturated, and steady-state will look fairly linear. If you're using "classic" congestion control like CUBIC or New Reno, you'll see this interspersed with drops (search for "tcp sawtooth"). Under BBR you'd expect to see drops in bandwidth due to latency spikes (per bufferbloat). Neither of those seem to be in play here.

Re: AT&T Wireless traffic shaping apparently making some websites unusable

#167
post #112
post #89

Earlier quoted context omitted.

Couldn't ISPs just sniff the SNI hostname to differentiate fast.com vs actual Netflix video streaming?

You can think of the requests to fast.com as just loading the speed test control scripts and user interface. The actual speed test loads files from the same servers (with the same SNI hostname) used by actual Netflix video streaming. It wouldn't surprise me at all if the fast.com speed test loads real streaming video segments from these servers, the only difference being that it doesn't have the decryption key for th…

It would also not surprise me if dns requests for fast.com temporarily elevated bandwidth limits for netflix

Re: AT&T Wireless traffic shaping apparently making some websites unusable

#168

Earlier quoted context omitted.

> As for me? ATT has provided the best service of any carrier while traveling, so I will use them. I'm curious about your reasons to say that AT&T provides the best service "while traveling". I'm guessing it would be domestic travel, because AT&T roaming charges are the second highest among the big three (I believe Verizon is actually even more expensive). I was a customer and an employee of AT&T for a while, and I f…

Do you mean international charges? I thought roaming has not been a thing for a decade. Many ATT plans include usage in all of the western hemisphere, excluding the Caribbean islands. And for countries not included, it is $10 per day ($5 for other lines on your plan) up to 10 days in a billing cycle, and after that it is free until the next billing cycle. Not the cheapest (I think T-Mobile has international at $5 per…

Right, roaming as in international roaming.

I’m on T-Mobile right now, and the plan equivalent to the one I had with AT&T (all inclusive and unlimited data), does include 5GB of international data per month. AT&T was $10, even for employees, so not a great deal for frequent travelers.

Re: AT&T Wireless traffic shaping apparently making some websites unusable

#169
Wireless carriers have been pulling stunts like this over the last several years. For example, T-Mobile blocks all SMS messages with links that have a .xyz TLD. There's no indication that it was blocked by neither the sender or the receiver. I assume this was done to prevent spam and sketchy websites, but personally speaking I've seen way more sketchy .com sites and more spam coming from gmail accounts.

Re: AT&T Wireless traffic shaping apparently making some websites unusable

#170

Earlier quoted context omitted.

If you're travelling, I find it hard to beat Fi. You literally land in a new country, turn your phone back on and you get a "Welcome to [country] - your data rate is the same" message almost anywhere. Personally - I've flown from Taiwan to Brazil to Amsterdam and then back to the US and I don't have to think about my phone. It just works. --- Outside of the travel use-case, I would also probably pick something else,…

I still have my Sprint plan. This is how it works by default. (Sprint + gvoice = google fi; before gfi you could merge your gvoice and sprint accounts which was really cool. Then they cancelled that and started gfi) Since the TMo merger, I suspect gfi is still using the Sprint stuff.

> Since the TMo merger, I suspect gfi is still using the Sprint stuff.

Since before the merger, it used Sprint and T-Mobile, in addition to US Cellular https://techcrunch.com/2018/01/17/googles-project-fi-now-cap... (ctrl-f sprint)

Post reply on HN