Twitter's Recommendation Algorithm • Hacker News Board

Re: Twitter's Recommendation Algorithm

#71

Post by distrill » Fri, Mar 31, 2023, 7:14 PM UTC

BbzzbB wrote: post #59

distrill wrote: post #44

the-algorithm is such a pretentious name for a repo

It's a colloquial term for recommendation engines, how often do you hear people say "the algorithm" (vs. "the recommendation engine") on YouTube?

yes, but this is the first repository i have seen named like this

Re: Twitter's Recommendation Algorithm

#72

Post by systemvoltage » Fri, Mar 31, 2023, 7:14 PM UTC

Astounding amount of cynicism here, so I'll say something positive: Transparency is undoubtly important, I'm glad we can see how all of this works and what sort of effort goes into building a social media system. It's licensed under GPL which is a bummer (would have preferred BSD) but it's better than nothing.

Re: Twitter's Recommendation Algorithm

#73

Post by Chinjut » Fri, Mar 31, 2023, 7:14 PM UTC

sroussey wrote: post #30

Does it show the part where is recommends Elon more than anyone else?

Perhaps that's related to this line. Though perhaps this is just used for observing metrics. https://github.com/twitter/the-algorithm/blob/7f90d0ca342b92...

Re: Twitter's Recommendation Algorithm

#74

Post by minimaxir » Fri, Mar 31, 2023, 7:14 PM UTC

CathalMullan wrote: post #55

tric wrote: post #11

From https://github.com/twitter/the-algorithm/blob/7f90d0ca342b92... ( "author_is_elon", candidate => candidate .getOrElse(AuthorIdFeature, None).contains(candidate.getOrElse(DDGStatsElonFeature, 0L))), ( "author_is_power_user", candidate => candidate .getOrElse(AuthorIdFeature, None) .exists(candidate.getOrElse(DDGStatsVitsFeature, Set.empty[Long]).contains)), ( "author_is_democrat", candidate => candidate .getOrEls…

Only used for metrics, apparently. [0] /** * These author ID lists are used purely for metrics collection. We track how often we are * serving Tweets from these authors and how often their tweets are being impressed by users. * This helps us validate in our A/B experimentation platform that we do not ship changes * that negatively impacts one group over others. */ [0]: https://github.com/twitter/the-algorithm/blob/7f…

The original code is a part of the home-mixer service, which is the "Main service used to construct and serve the Home Timeline."

I suspect the flag corresponds to weights not present in the repo.

Re: Twitter's Recommendation Algorithm

#75

Post by spike021 » Fri, Mar 31, 2023, 7:14 PM UTC

infogulch wrote: post #34

anderspitman wrote: post #20

I'm not opposed to social media feeds having complex recommendation algorithms. I just wish they allowed you to opt in to a reverse chronological feed of only people you follow, like RSS.

Twitter has this now. The home page is split into two tabs: "For you", the algorithmic feed, and "Following", the reverse chronological feed of just who you follow.

It doesn't always stay on whatever you last used, though. I mostly use Following but it always inevitably ends up back on "For you".

Re: Twitter's Recommendation Algorithm

#76

Post by raydev » Fri, Mar 31, 2023, 7:15 PM UTC

throwayyy479087 wrote: post #5

You gotta hand it to Elon - he actually did it.

Do we "gotta hand it to Elon" for not missing one of his 40-50 self-imposed deadlines and feature announcements?

Re: Twitter's Recommendation Algorithm

#77

Post by ano-ther » Fri, Mar 31, 2023, 7:15 PM UTC

sroussey wrote: post #30

Does it show the part where is recommends Elon more than anyone else?

This is one: https://github.com/twitter/the-algorithm/issues/121

Search for Elon gives this: https://github.com/twitter/the-algorithm/search?q=Elon&type=

Re: Twitter's Recommendation Algorithm

#78

Post by jmeister » Fri, Mar 31, 2023, 7:15 PM UTC

summarity wrote: post #7

Main repos: - https://github.com/twitter/the-algorithm - https://github.com/twitter/the-algorithm-ml Blogs: - Eng: https://blog.twitter.com/engineering/en_us/topics/open-sourc... - Biz: https://blog.twitter.com/en_us/topics/company/2023/a-new-era...

Twitter spaces live right now: https://twitter.com/i/spaces/1jMJgLdenVjxL

Re: Twitter's Recommendation Algorithm

#79

Post by eecc » Fri, Mar 31, 2023, 7:15 PM UTC

jonathanmayer wrote: post #45

Context: I teach at Princeton and study social media and recommendation systems. From a very quick skim of the repositories, this appears to be quite limited transparency. The documentation gives a decent high-level overview of how Tweet recommendation works—no surprises—and the code tracks that roadmap. Those are meaningful positive steps. But the underlying policies and models are almost entirely missing (there are…

Wouldn’t that make them easy prey of “spam SEO”. However, given the framework isn’t it still possible to guess the models?

Re: Twitter's Recommendation Algorithm

#80

Post by simonsarris » Fri, Mar 31, 2023, 7:15 PM UTC

PenguinRevolver wrote: post #25

Great pull request here which improves the algorithm: https://github.com/twitter/the-algorithm/pull/17

That would be great (unweighting bluechecks) but they actually plan to go in the other direction: Starting April 15th non-bluechecks won't show up in the "For you" section (the algorithm timeline) at all. Unpaid users are being written completely out of the algo.

https://twitter.com/elonmusk/status/1640502698549075972