Live data from Hacker News

Twitter's Recommendation Algorithm

blog.twitter.com

41–50 of 1001 posts

Re: Twitter's Recommendation Algorithm

#43
post #16

Let's dig into Twitter code quality.

https://github.com/twitter/the-algorithm/blob/7f90d0ca342b92...

```

def query_keys(self, language, task=2, size="50"):

    if task == 2:

      if language == "ar":

        self.query_settings["adhoc_v2"]["table"] = "..."

      elif language == "tr":

        self.query_settings["adhoc_v2"]["table"] = "..."

      elif language == "es":

        self.query_settings["adhoc_v2"]["table"] = f"..."

      else:

        self.query_settings["adhoc_v2"]["table"] = "..."

      return self.query_settings["adhoc_v2"]

    if task == 3:

      return self.query_settings["adhoc_v3"]

    raise ValueError(f"There are no other tasks than 2 or 3. {task} does not exist.")
```

Re: Twitter's Recommendation Algorithm

#45
Context: I teach at Princeton and study social media and recommendation systems.

From a very quick skim of the repositories, this appears to be quite limited transparency. The documentation gives a decent high-level overview of how Tweet recommendation works—no surprises—and the code tracks that roadmap. Those are meaningful positive steps. But the underlying policies and models are almost entirely missing (there are a couple valuable components in [1]). Without those, we can't evaluate the behavior and possible effects of "the algorithm."

[1] https://github.com/twitter/the-algorithm-ml

Re: Twitter's Recommendation Algorithm

#48

I'm not opposed to social media feeds having complex recommendation algorithms. I just wish they allowed you to opt in to a reverse chronological feed of only people you follow, like RSS.

Twitter has this now. The home page is split into two tabs: "For you", the algorithmic feed, and "Following", the reverse chronological feed of just who you follow.

It always had it.

Edit: Why am I downvoted? It literally did, it even was named as you'd expect it ("sort by latest" or something), tho the location was less obvious as it was under the stars icon above the feed.

Re: Twitter's Recommendation Algorithm

#49
post #4
post #2

GitHub repo: https://github.com/twitter/the-algorithm/

I wonder how useful this is without the knowledge and tooling around deploying it.

That's my thought as well. Complicated system like this rely on all sorts of related services and data stores. This seems like the sort of thing that sounds a lot more interesting than it is in practice. I would bet many non-technical people expect "The Algorithm" to be a straightforward and self-contained system.

Re: Twitter's Recommendation Algorithm

#50
post #11

From https://github.com/twitter/the-algorithm/blob/7f90d0ca342b92... ( "author_is_elon", candidate => candidate .getOrElse(AuthorIdFeature, None).contains(candidate.getOrElse(DDGStatsElonFeature, 0L))), ( "author_is_power_user", candidate => candidate .getOrElse(AuthorIdFeature, None) .exists(candidate.getOrElse(DDGStatsVitsFeature, Set.empty[Long]).contains)), ( "author_is_democrat", candidate => candidate .getOrEls…

The full list of model features in that file is interesting.

I am surprised at the number of inherently redundant and colinear features, though. (e.g. has_1_image, has_2_images, has_3_images, has_4_images)

Post reply on HN