61–70 of 77 posts
Having never worked with code-bases larger than ~50kloc, I have a lot of trouble understanding what 1 million lines of code is needed for, especially considering that python is such a high-level language.
Does anyone have any idea why there would be this much code?
I fully agree with Youtube faking data. However, I reckon they are faking a bit too much. Many times I would see 2000 likes and the video having 1700 views (Viral videos that is). I knew the view counter wasn't propagated but the likes were and I was like: "Damn this is Youtube, kinda disappointing..." I guess if both were propagated at the same time I wouldn't mind.
Yeah, the view counter is a clear case where they've gone too far. People care about view counts and it seems like every other video I watch has people complaining in the comments about the view count being obviously wrong.
Most of this is relatively straightforward and unsurprising. But the one part that grabbed me is about "jittering". They insert random delays into timed events (the example given is cache expiration) to prevent a thundering herd problem when all the parts of the distributed system see the event at the same time (and for popular content, presumably repopulate the cache from the backend simulteously). This is simple en…
Most of this is relatively straightforward and unsurprising. But the one part that grabbed me is about "jittering". They insert random delays into timed events (the example given is cache expiration) to prevent a thundering herd problem when all the parts of the distributed system see the event at the same time (and for popular content, presumably repopulate the cache from the backend simulteously). This is simple en…
Why is this not done in electronic exchanges to render HFT operations pointless?
The first 10 minutes are about monetization from one of the Youtube dev advocates. Skip to 9:45 to get to the "good stuff". As an aside, this fellow is probably one of the best presenters I've seen from the pycon videos for this year. Confident, smooth, not reading from a computer screen or sheet of paper, clearly smart and in firm command of the subject matter. I'd love to see more talks from him.
"Cheating - Know How to Fake Data
Awesome technique. The fastest function call is the one that doesn’t happen. When you have a monotonically increasing counter, like movie view counts or profile view counts, you could do a transaction every update. Or you could do a transaction every once in awhile and update by a random amount and as long as it changes from odd to even people would probably believe it’s real. Know how to fake data."
So all those people who buy views are kinda screwed now :-) I suspect this is a bad example. I HOPE this is a bad example, if only for the KONY2012 campaign :P
"They wrote their own BSON implementation which is 10-15 times faster than the one you can download." Curious to hear more about that one. If true, I hope they open source it, because that could potentially make MongoDB a lot faster for everyone. EDIT: It's apparently in their vitess code. Relevant code: http://code.google.com/p/vitess/source/browse/#hg%2Fgo%2Fbso...
At 11:28 he says, "at last count, there was over a million lines of python running this thing" Having never worked with code-bases larger than ~50kloc, I have a lot of trouble understanding what 1 million lines of code is needed for, especially considering that python is such a high-level language. Does anyone have any idea why there would be this much code?
It's the world's 3rd biggest website with hundreds of billions of views and dozens of millions of users, so maybe that's why.
I think it's a credit to Python that a website that does that and has grown in a fairly haphazard fashion only has about 1000k SLOCs.
I'm sure many of us are disappointed that YouTube doesn't see consistent presentation of user comments as mission critical.
For me, YouTube is good for watching videos. If I want to discuss it, I post it on FB.