Live data from Hacker News

Netflix's Open Connect Appliances

gizmodo.com

11–20 of 68 posts

Re: Netflix's Open Connect Appliances

#11
post #2

Interesting to see some more details on the appliances. Isn't the argument primarily about peering, not open connect appliances, though?

It's all one discussion in my mind. There are three ways to get Netflix: boxes, peering, or transit (from cheapest to most expensive). If peering saves the ISP money and the boxes save even more money, why are some ISPs refusing to use them?

Re: Netflix's Open Connect Appliances

#12
post #8

Can anyone explain why an ISP would push back Netflix installing one of these boxes? Does the ISP want Netflix service congestion to better sell its own media assets/distribution (i.e. cable TV)?

Less conspiratorially, if they can maximize profit by holding out for payment I guess that's what they'll do.

Re: Netflix's Open Connect Appliances

#13
post #9
post #8

Can anyone explain why an ISP would push back Netflix installing one of these boxes? Does the ISP want Netflix service congestion to better sell its own media assets/distribution (i.e. cable TV)?

Based on the blog posts by Level 3 -- how much internal capacity Comcast / Verizon say they have versus what Level 3 sees for utilization at their peering points with both ISPs -- I would have to assume "yes".

Can you please post a link? This sounds very interesting...

Re: Netflix's Open Connect Appliances

#14
Could someone explain to me (it's been a long day, and I'm no storage expert!) how these are useful, wouldn't you need a whole bunch of these at each location they are to be installed?

Just basing this on my own experience I have 12 drives in RAID which have a fairly substantial sequential throughput. Start multiple streams of high bandwidth videos and the maximum throughput from the drives drops sharply due to the random reads.

I would have assumed that having even 100 people streaming from a single box would be an interesting challenge.

Re: Netflix's Open Connect Appliances

#15
post #5

Lot of mistakes in this article, afair it can't hold all the movies but just the most popular. http://oc.nflxvideo.net/docs/OpenConnect-Deployment-Guide.pd... " Each appliance stores a portion of the catalog, which in general is less than the complete content library for a given region. Popularity changes, new titles being added to the service, and re- encoded movies are all part of the up to 7.5TB of nightly updates…

In addition, Netflix has multiple copies of the same content at different quality levels. So, there might be the 1080p, 720p, and SD quality levels of the popular content, each of them taking up space from other movies/shows.

Re: Netflix's Open Connect Appliances

#16
post #9

Earlier quoted context omitted.

Based on the blog posts by Level 3 -- how much internal capacity Comcast / Verizon say they have versus what Level 3 sees for utilization at their peering points with both ISPs -- I would have to assume "yes".

Can you please post a link? This sounds very interesting...

Here's a link to one of the blog posts: http://blog.level3.com/global-connectivity/verizons-accident...

Re: Netflix's Open Connect Appliances

#17
post #14

Could someone explain to me (it's been a long day, and I'm no storage expert!) how these are useful, wouldn't you need a whole bunch of these at each location they are to be installed? Just basing this on my own experience I have 12 drives in RAID which have a fairly substantial sequential throughput. Start multiple streams of high bandwidth videos and the maximum throughput from the drives drops sharply due to the r…

well, if it means that they need to install n boxes in an isp to avoid peering at a high cost then it might be worth it.

> Sometimes it'll even have multiple copies of the same show, if something is in crazy high demand.

that statement from the article lends one to believe that the raid arrays are in parallel.

These boxes are meant to reduce the number of packets going all the way across the internet not necessarily remove the need.

Also, in addition to lots of storage, these boxes probably have loads of ram that they can buffer popular streams into and then serve it.

Re: Netflix's Open Connect Appliances

#18
I'd be interested to find out more of hardware details of these devices. 100+ TB of storage in a 4U is pretty respectable. From the images in the article, it looks like the drives are not hot-swappable, so I'd guess Netflix is able to remotely track loss of redundancy and will just send out an entire replacement unit when needed.

For comparison, Supermicro makes one of the highest-density storage servers that I'm aware of[1]. 72 3.5" drives in 36 drive bays, so up to 288tB of raw storage, if you're brave enough to use 4tB drives.

[1] http://www.supermicro.com/products/system/4U/6047/SSG-6047R-...

Re: Netflix's Open Connect Appliances

#19
post #14

Could someone explain to me (it's been a long day, and I'm no storage expert!) how these are useful, wouldn't you need a whole bunch of these at each location they are to be installed? Just basing this on my own experience I have 12 drives in RAID which have a fairly substantial sequential throughput. Start multiple streams of high bandwidth videos and the maximum throughput from the drives drops sharply due to the r…

These things can pump out 12Gbit during peak viewing, which is enough for about 4,000 avg. 3 Mbit streams.

If they can cram in 256GB of DRAM, that's enough room for a 64MB buffer per stream, or about 170 seconds worth of streaming. Now you only need to be able to fill those buffers at a rate of about 24/second.

I'm assuming whatever file system is on the disks uses a massive block size, so the number of seeks you'd have to perform to pull 64MB off is probably pretty low. Eight? Sixteen? Even if it's the latter, that's only 384 seeks/second, which you could very plausibly do striped over only a half dozen disks, and the device presumably has many more.

Re: Netflix's Open Connect Appliances

#20
post #14

Could someone explain to me (it's been a long day, and I'm no storage expert!) how these are useful, wouldn't you need a whole bunch of these at each location they are to be installed? Just basing this on my own experience I have 12 drives in RAID which have a fairly substantial sequential throughput. Start multiple streams of high bandwidth videos and the maximum throughput from the drives drops sharply due to the r…

Start multiple streams of high bandwidth videos and the maximum throughput from the drives drops sharply due to the random reads.

I think this can be fixed by prefetching harder (like 1-2 MB).

100 Netflix streams would be less than 100 MB/s which can be satisfied with a single SATA drive. And some of the Netflix boxes have dozens of SSDs that each do 500 MB/s.

Post reply on HN