81–90 of 133 posts
Earlier quoted context omitted.
I remember in the 90's having a conversation with my friend. We were discussing Linux vs Windows for web hosting. My friend said "tell me if you can find a single porn site that's hosted on Windows." He was right, I couldn't find any.
They do exist. I turned down the chance to interview at one about 6 months ago.
Earlier quoted context omitted.
The online porn industry were the first people to really deal with scalability (of managing and serving web content, anyway), the first people to deal with online payments, the first people to come under really heavy, sustained hacking attempts, etc. Don't underestimate how much of what you take for granted technologically in 2012, they pioneered in the mid-90s.
As well as advertising networks and referral schemes. Though, oddly, they've been late to the game on social.
That's porn for database geeks: 100 million pageviews per day, 300k requests per second against Redis. The way they use MySQL is also interesting IMHO: they populate a relational database in order to be able to build new indexes in the Redis side, using the relational DB for the stuff it is best at, generating new "views" of the data easily. (Relational DBs are also good to do a zillion more things of course.)
300k requests/sec? Surely that can't be on a single Redis server, is it? Is Redis cluster out yet, or did they implement their own?
* http://code.google.com/p/redis/wiki/Benchmarks
* http://redis.io/topics/benchmarks
Earlier quoted context omitted.
Porn and war drive all innovation. Or so they say.
Sex and war would seem more accurate; porn being just a subset of sex. And I suppose one could argue that war is often just a manifestation of a desire for sex or power/money as an intermediary to sex.
That's porn for database geeks: 100 million pageviews per day, 300k requests per second against Redis. The way they use MySQL is also interesting IMHO: they populate a relational database in order to be able to build new indexes in the Redis side, using the relational DB for the stuff it is best at, generating new "views" of the data easily. (Relational DBs are also good to do a zillion more things of course.)
I suppose if each video's metadata causes a redis lookup and there are 300 videos displayed per page that would do it, but you'd think you could batch those lookups. It seems a fairly inefficient.
Either that or they have some background jobs causing redis lookups that aren't a result of page views.
That's porn for database geeks: 100 million pageviews per day, 300k requests per second against Redis. The way they use MySQL is also interesting IMHO: they populate a relational database in order to be able to build new indexes in the Redis side, using the relational DB for the stuff it is best at, generating new "views" of the data easily. (Relational DBs are also good to do a zillion more things of course.)
100M page views per day is 1157 page views per second. I wonder how that's being blown up by a factor of 300? I suppose if each video's metadata causes a redis lookup and there are 300 videos displayed per page that would do it, but you'd think you could batch those lookups. It seems a fairly inefficient. Either that or they have some background jobs causing redis lookups that aren't a result of page views.
That's porn for database geeks: 100 million pageviews per day, 300k requests per second against Redis. The way they use MySQL is also interesting IMHO: they populate a relational database in order to be able to build new indexes in the Redis side, using the relational DB for the stuff it is best at, generating new "views" of the data easily. (Relational DBs are also good to do a zillion more things of course.)
Yep, i've been a Redis lobbyist for over a year at this company (i use to be the Lead Software Engineer on Youporn/Pornhub), and i'm really happy to see them using it and using it properly.
I remember seeing YouTube storing videos using some sort of specialized hardware from NetApp or something and they still got hit by the limitation of Linux filesystem (number of nodes?)
Earlier quoted context omitted.
First off, I am not saying that youporn guys are not decent but IMHO, people shouldn't get too excited about this. Youporn is somewhat quite a simple site I'd imagine. It's just categorized videos with a bunch of stats (views, related video...), and users. There's almost no social interaction between users (i've just registered an account just to confirm that). That's a very clear (and good) reason why 100% redis sui…
You would imagine wrong. I'm running a video sharing page (think FB&youtube mashed together with a dash of blogger) with about 800k unique/month users and it's no picnic.
Or the good old' NFS?
Earlier quoted context omitted.
300k requests/sec? Surely that can't be on a single Redis server, is it? Is Redis cluster out yet, or did they implement their own?
Redis can run anywhere from 50-200K IOPs, on a single core, dependent on CPU; you can find benchmarks at the following: * http://code.google.com/p/redis/wiki/Benchmarks * http://redis.io/topics/benchmarks At the point where multi-core sharding/ring setups are involved, you begin to have NIC/network saturation. It's feasible to achieve 300K on a single box.
problem is you can't tell your boss: "well youporn uses it" when he looks dubious about using redis for his trading system.
Mine would probably hear the reasoning out ... then ask for a less "unsavory" example he can use when talking with everyone else.