Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Netflix is Down

gigaom.com

Post reply on HN
Print view View original

61–70 of 81 posts

Re: Netflix is Down

#61

Post by brown9-2 »

moe wrote: post #59
res0nat0r wrote: post #54

Earlier quoted context omitted.

>Especially one as trivial as Netflix which is largely read-only. I stopped reading after this, since you obviously dont understand how complex their architecture really is.

I work for a media streaming site (rank "Trivial" may sound harsh but in terms of large scale systems it really doesn't get much simpler than serving static files. Obviously their auxiliary services (billing, content ingestion, social etc.) are nowhere near trivial. But there's no reason for the core services that are required to get the catalog and video to most[1] devices to not be damn near 100% available. I'm not…

How can you assure us of what the #99 system is like when you admit to only working on the ~30k system? That's not what "assure" means.

Re: Netflix is Down

#62

Post by alxndr »

malyk wrote: post #60
zrail wrote: post #17

Earlier quoted context omitted.

It's not every app for me either. It appears to be a subset of apps that use the ssl:endpoint add-on to implement HTTPS.

And its not all sslenpoint sites either. We've been up through the whole thing. Which is critical seeing as we are a gifting company that provides an easy last minute gift!

Great timing!

Re: Netflix is Down

#63

Post by toomuchtodo »

herge wrote: post #58
toomuchtodo wrote: post #49

Earlier quoted context omitted.

Assumption: Your business uses Amazon's EBS (because heh, you need to store your data somewhere between EC2 instance reboots/creation/destruction). EBS is down. What do you do? Hope AWS engineers get it back up fast? You can't do a damn thing about its reliability; you're stuck with whatever reliability Amazon has decided to deliver (which I think we can agree is much less than a company with revenue can depend on).…

> When you build your own system, you decide how reliable you want it. Ha! You mean that you discover how hard and expensive it is to build your own reliable infrastructure or how hard it is to hire people to do it for you.

No offense, but I used to work at a DoE lab on data-taking from the Large Hadron Collider CMS detector.

It's not as expensive as you'd think. We were dealing with hundreds of petabytes of data, and customers much more demanding than your consumer paying $16/month to stream movies.

Re: Netflix is Down

#64

Post by xugle »

jrockway wrote: post #41
toomuchtodo wrote: post #29

You'd think Netflix would learn by now to move back to their own gear. EDIT: Downvote away; its practically dogma on HN to use AWS. How much downtime are people willing to tolerate for a "superior" technology? Sure, Amazon AWS has some great ideas and tech, but you might as well give up if your business depends on EBS in us-east-1 at all

The gear isn't the problem: the dependency on a single data center is the problem. It requires a lot of software engineering effort to maintain a service that works when a data center suddenly goes away. To be fault-tolerant, Netflix has to do this engineering regardless of whether or not they own the servers. But if they use Amazon, they don't have to actually fix the servers when they break, freeing up engineering…

jrockway, we know that google does things a certain way that blows up the costs of simple storage. It's not necessary to make 12 copies worldwide to get bare minimum redundancy required for this case. The data that needs to be replicated (video files) and account info is not realtime transactions. The video files can be transferred by fedexing a bunch or hard disks on a weekly basis (take a reasonable guess at Netflix catalog growth and compute the per week growth of data... it's indeed trivial).

Re: Netflix is Down

#66

Post by moe »

brown9-2 wrote: post #61
moe wrote: post #59

Earlier quoted context omitted.

I work for a media streaming site (rank "Trivial" may sound harsh but in terms of large scale systems it really doesn't get much simpler than serving static files. Obviously their auxiliary services (billing, content ingestion, social etc.) are nowhere near trivial. But there's no reason for the core services that are required to get the catalog and video to most[1] devices to not be damn near 100% available. I'm not…

How can you assure us of what the #99 system is like when you admit to only working on the ~30k system? That's not what "assure" means.

We actually talk to other people in the industry sometimes, imagine that...

Re: Netflix is Down

#68

Post by arkonaut »

toddh wrote: post #65

This really sucks for the engineers. Not a fun way to spend xmas at all.

If you're netflix, I'm sure you have staff/engineers at the office 24/7 during this time of year specifically. At least, I would hope they did.

Re: Netflix is Down

#69

Post by sjs382 »

cowkingdeluxe wrote: post #52

Netflix is down because Amazon is down, so we had to watch shows on Amazon (prime) instead. Funny how that works!

I had the same experience. When watching movies with my SO tonight, I explained that Netflix uses AWS and that AWS was experiencing problems. We started watching a movie on Amazon prime and she asked "So I guess Amazon Video isn't using AWS?"
Post reply on HN