Live data from Hacker News

YouTube's road to HTTPS

youtube-eng.blogspot.com

81–90 of 97 posts

Re: YouTube's road to HTTPS

#81

Earlier quoted context omitted.

TCP only gives you 16 bits of checksum, which is really not enough. I used to work for a CDN which served downloads over HTTP, and when the object size grew to a few gigabytes a non-negligible percentage of users ended up with corruption. The SSL MAC is valuable even in the absence of enemy action.

This is a really interesting 'real world' anecdote. Do you know of any related data available publicly? i.e. "on our network, X bits in Y terabytes end up with undetected corruption, meaning that approximately Z% of downloads of a 2GB file will have at least one bit error".

From http://noahdavids.org/self_published/CRC_and_checksum.html

In "Performance of Checksums and CRCs over Real Data" Stone and Partridge estimated that between 1 in 16 million and 1 in 10 billion TCP segments will have corrupt data and a correct TCP checksum. This estimate is based on their analysis of TCP segments with invalid checksums taken from several very different types of networks. The wide range of the estimate reflects the wide range of traffic patterns and hardware in those networks. One in 10 billion sounds like a lot until you realize that 10 billion maximum length Ethernet frames (1526 bytes including Ethernet Preamble) can be sent in a little over 33.91 hours on a gigabit network (10 * 10^9 * 1526 * 8 / 10^9 / 60 / 60 = 33.91 hours), or about 26 days over a T3.

Re: YouTube's road to HTTPS

#82
post #43

Earlier quoted context omitted.

I should be able to turn it off, but I like that tmo throttles my video. I only get 2.5 gb, I don't need 1080p.

You can turn it off, but it removes the zero rating. Up to you either way.

I'm on the light package so I don't even get the zero rating.

Re: YouTube's road to HTTPS

#83
post #54

> You watch YouTube videos on everything from flip phones to smart TVs what year is this?!

I am less curious about the year and more curious as to how the hell flip phones are rendering video. Like 64p 3GPP or something realtime transcoded?

ASCII art.

Re: YouTube's road to HTTPS

#84
post #51

Earlier quoted context omitted.

Why not limit the bandwidth per user instead? Youtube itself will auto-adjust quality based on connection speed.

Let's say there are only 2 users using it. Then a lot of bandwidth would go unused if each user has a hard cap. Ideally, there would be a dynamic limit based on network utilization and type of content consumed.

Don't think hard cap. Think avail bandwidth/num of users.

Re: YouTube's road to HTTPS

#86
post #45
post #39

Earlier quoted context omitted.

Per user limits are not easy to implement, maybe easier to visualize - consider that there are a bunch of packet gateways sitting behind a load balancer and each HTTP session may end up on a different server. There is no entity that counts the live bandwidth usage on a per user basis, let alone control it. Billing and metering is done on a session basis through logs. So from T-Mo's point of view it is much easier to…

Eh? Even cheap home routers these days allow you to set QoS rules per device. Have I misunderstood something?

Cheap home routers with QoS has OUTBOUND QoS.

Re: YouTube's road to HTTPS

#87
post #27

Earlier quoted context omitted.

Degrading video quality is not a bad idea if you have limited bandwidth. This would help in cases such as airplane flights. One person watching HD cat videos is going to consume more bandwidth than 20 people doing work. Now, assuming bandwidth increase is difficult to achieve, they would be forced to keep increasing the price instead. Prioritizing non-video content is challenging if all content is encrypted.

Why not limit the bandwidth per user instead? Youtube itself will auto-adjust quality based on connection speed.

> Why not limit the bandwidth per user instead?

Because that would require actual effort from ISPs, which would be in conflict with their current business practice that can be summarized as "to us, you are all equally worthless".

Re: YouTube's road to HTTPS

#88
post #27

Earlier quoted context omitted.

Degrading video quality is not a bad idea if you have limited bandwidth. This would help in cases such as airplane flights. One person watching HD cat videos is going to consume more bandwidth than 20 people doing work. Now, assuming bandwidth increase is difficult to achieve, they would be forced to keep increasing the price instead. Prioritizing non-video content is challenging if all content is encrypted.

Why not limit the bandwidth per user instead? Youtube itself will auto-adjust quality based on connection speed.

The auto-adjustment is a problem for fast but traffic limited connections because it detects that the connection is fast, then it switches to HD or even 4K and the traffic limit gets used up faster even if the user don't need the better quality stream.

Re: YouTube's road to HTTPS

#89

Earlier quoted context omitted.

I would also be interested in that. What differnce would you expect from the catalog size? I would guess that aside from maybe the handshake, decoupling storage and encryption would not be worth it in all cases where the storage medium is slower than AES-NI and this would apply even more to youtube than to netflix, exactly because of the bigger catalog.

With a small catalog, you can pre-encode everything. That means that in the unencrypted case, media files can be sent directly from the kernel's page cache, as unmapped I/O. Eg, the data is not even touched by the CPU. But in the encrypted case, you're suddenly touching every byte with the CPU, thereby doubling your memory bandwidth requirements. With a large catalog like YouTube's, I would expect that most of the ti…

> encoded on the fly

no, and often their encoders will go bad and produce invalid output (blocky/purple) in one of the resolutions/formats. There is ZERO options for fixing that other than deleting your video and reuploading (losing all the comments/upvotes). Even if you have 200K subs there is no way of contacting YT for help, mmaybe they will speak with you at 1mil subs :/

Re: YouTube's road to HTTPS

#90
post #26
post #2

> We found that HTTPS improved quality of experience on most clients: by ensuring content integrity, we virtually eliminated many types of streaming errors Wow, what were carriers doing to the streams?!

Remind me the madness that mobile carrier are doing to the TCP stack.

any links with technical details about this?
Post reply on HN