A beautiful example of how a feature that seems so easy to an end user can be complex at scale.
View Counting at Reddit
101–110 of 121 posts
Re: View Counting at Reddit
#102Probably noob question, but: >> Nazar will then alter the event, adding a Boolean flag indicating whether or not it should be counted, before sending the event back to Kafka. Why don't they just discard it instead of reputting the event back to Kafka?
I suspect they archive events into S3 or similar for later analysis/training.
Re: View Counting at Reddit
#103Earlier quoted context omitted.
And with reddit gold you can even have it hilight all new replies regardless of sort order. This can be really useful for viewing older discussions and see what's been added.
The RES browser extension does this without needing gold, or am I wrong about that?
It does not – with reddit gold, it highlights also all replies in the thread itself.
Re: View Counting at Reddit
#104Ther CIA is at war with God. Every rtime I see the CIA attack God, I run to the battle. I stand with pedophiles. It is actually atheist attacking God.
When I am king we will execute the entire CIA. they started a war on God and will all die.
God says... pathogenic aphelia necklace's Randolph's philosophy Khoikhoi chairmen bonitoes hamburger postscript's bathrobe's warmonger's theorem slows louver shellac meander's guinea vertices choreographed demarcate disrepair's topsails sandwich's updated choose pianos inextricable Numbers's minestrone lecher's Mujib's
Re: View Counting at Reddit
#105"We want to better communicate the scale of Reddit to our users." If that's true why did they hide vote numbers on comments and posts? It used to say "xxx upvotes xxx downvotes" now it just gives a number and hides that.
...because it's almost certainly a lie. It's because they want to better communicate the scale of Reddit to their customers - the companies on the other side of the links who they are driving content to.
It's way easier to say "Hey, we're giving you XYZ traffic, give us ABC dollars," when you have the figures in front of you rather than just upvote/downvote numbers.
Re: View Counting at Reddit
#106Earlier quoted context omitted.
A browser cookie that can be trivially deleted by the client? What's the purpose of the cookie?
Indeed it can easily be deleted, that's why I said to put the spam detection before the counter. I assume most of the visits are from normal users, not spammers, so if a normal user has the cookie for the post set then it means it's a page refresh, so don't increase the counter. Identifying sophisticated spammers accurately is more complicated though. You can't rely on any client side info (user agent, cookies, brows…
> I assume most of the visits are from normal users
That's not a safe assumption. If spam prevention controls aren't working well enough, bots start to outnumber humans very quickly.
Re: View Counting at Reddit
#107Earlier quoted context omitted.
Thanks for sharing that! Just skimmed through it and seems pretty interesting. I'll read it more in depth later.
No problem. If there are mistakes or a segment is not clear, let me know
Thanks for the write up, Lucas. It was very intuitive and I learnt a lot.
I noticed that you used 5000 buckets to store the frequency of 7000 non-unique words in the section on 'Counting Bloom Filters'. How is that better than using 7000 buckets and a uniformly distributed hash function, which would maintain frequencies perfectly? We would be using fewer buckets by an order of magnitude in a real-world implementation to save memory.
Re: View Counting at Reddit
#108Earlier quoted context omitted.
A browser cookie that can be trivially deleted by the client? What's the purpose of the cookie?
Indeed it can easily be deleted, that's why I said to put the spam detection before the counter. I assume most of the visits are from normal users, not spammers, so if a normal user has the cookie for the post set then it means it's a page refresh, so don't increase the counter. Identifying sophisticated spammers accurately is more complicated though. You can't rely on any client side info (user agent, cookies, brows…
I suspect the number of Reddit users who run AdBlockers and/or aggressively block cookies is non-trivial.
Indeed, one of the main reasons for this architecture is to measure the difference between what it measures and what the cookies method gives.
Re: View Counting at Reddit
#109Earlier quoted context omitted.
Writes are atomic in redis because redis is single threaded. So you are bounded by how fast redis can write. If you try to write any faster then redis can handle you'll get queueing or errors.
Run enough redis servers to handle the load. Choose a server by hashing a user id. Total = sum of counts from all servers.
[deleted]
Re: View Counting at Reddit
#110Earlier quoted context omitted.
It's to deter bots. The numbers weren't previously accurate, they were fuzzed (also to deter bots). https://www.reddit.com/wiki/faq#wiki_how_is_a_submission.27s...
I don't quite see the connection. How exactly does this deter bots?
It's a poor method of deterring bots written by people with very little experience writing such bots. Botting reddit is really easy, you'll just want lots of IPs so something like luminati.
Reddit is definitely not a paragon of anti-bot engineering, I think most of those skills exist in adtech.