Twitter's Recommendation Algorithm • Hacker News Board

Re: Twitter's Recommendation Algorithm

#421

Post by culi » Fri, Mar 31, 2023, 9:00 PM UTC

tric wrote: post #11

From https://github.com/twitter/the-algorithm/blob/7f90d0ca342b92... ( "author_is_elon", candidate => candidate .getOrElse(AuthorIdFeature, None).contains(candidate.getOrElse(DDGStatsElonFeature, 0L))), ( "author_is_power_user", candidate => candidate .getOrElse(AuthorIdFeature, None) .exists(candidate.getOrElse(DDGStatsVitsFeature, Set.empty[Long]).contains)), ( "author_is_democrat", candidate => candidate .getOrEls…

what is vits?

  private val DarkRequestAnnotation = "clnt/has_dark_request"
  private val Democrats = "democrats"
  private val Republicans = "republicans"
  private val Elon = "elon"
  private val Vits = "vits"

Re: Twitter's Recommendation Algorithm

#422

Post by delecti » Fri, Mar 31, 2023, 9:00 PM UTC

lhnz wrote: post #320

minimaxir wrote: post #241

Earlier quoted context omitted.

Update: Elon was asked about these in a Twitter Space, he says it's not appropriate and will be removed from the codebase. Additionally, from another Twitter engineer, the Democrat/Republican flags are apparently 10 years old and not important and do not have high feature importance.

I think the decade old comment related to a different part of the code regarding the number of followers you have in relation to the number of accounts you follow. (Everybody on the call wants to remove this: I wonder why they haven't yet.)

Chesterton's Fence. In a sufficiently large system, you should be hesitant to remove things unless you're sure you know why it was added, and all the things that have come to depend on it since.

I've definitely been hesitant to remove things I was pretty confident weren't used anymore, just because I didn't want to deal with the repercussions if I was wrong.

Re: Twitter's Recommendation Algorithm

#423

Post by mullingitover » Fri, Mar 31, 2023, 9:01 PM UTC

nonethewiser wrote: post #376

6d6b73 wrote: post #240

Earlier quoted context omitted.

Like it or not but it's the twitter that gets value from celebrities. How many people are on social networks jusy so see what their fav celebrites are doing?

It obviously goes both ways. Social media is a megaphone and ego boost for celebs.

The problem for twitter is it isn't the only game in town when it comes to social media, not by a long shot. They're not even in the top ten. They're a megaphone in a large pile of megaphones, and those other megaphones don't bite the hand that picks them up.

Re: Twitter's Recommendation Algorithm

#424

Post by endorphine » Fri, Mar 31, 2023, 9:01 PM UTC

froggychairs wrote: post #269

Why is nobody pointing out that this is likely an April Fools joke? We just deployed our April Fools joke into production today too.

Yeah this confused me a lot while reading the comments here. I wonder what percentage of the comments are trolling vs. fell for it vs. think it's legit.

Perhaps this calls for an HN poll...

Re: Twitter's Recommendation Algorithm

#425

Post by wahnfrieden » Fri, Mar 31, 2023, 9:02 PM UTC

GaryNumanVevo wrote: post #32

tric wrote: post #11

From https://github.com/twitter/the-algorithm/blob/7f90d0ca342b92... ( "author_is_elon", candidate => candidate .getOrElse(AuthorIdFeature, None).contains(candidate.getOrElse(DDGStatsElonFeature, 0L))), ( "author_is_power_user", candidate => candidate .getOrElse(AuthorIdFeature, None) .exists(candidate.getOrElse(DDGStatsVitsFeature, Set.empty[Long]).contains)), ( "author_is_democrat", candidate => candidate .getOrEls…

I wonder who's on the "VIT" (Very Important Tweeter) list?

People like Ben Shapiro, Glenn Greenwald, @catturd2

Re: Twitter's Recommendation Algorithm

#426

Post by mcast » Fri, Mar 31, 2023, 9:02 PM UTC

bcherry wrote: post #361

tantalor wrote: post #278

Earlier quoted context omitted.

What's all the "DDG"? Is this data from DuckDuckGo?

DuckDuckGoose is the twitter A/B test framework

That's actually kind of a cool and fitting name.

Re: Twitter's Recommendation Algorithm

#427

Post by calvinmorrison » Fri, Mar 31, 2023, 9:03 PM UTC

koolba wrote: post #90

It's reassuring to know that billion dollar tech companies write CI exactly like I do: https://github.com/twitter/the-algorithm/blob/main/ci/ci.sh Permalink: https://github.com/twitter/the-algorithm/blob/7f90d0ca342b92...

Maybe it's weird, but for all the work I have ever done, I have never used CI/CD in the way that it was meant to be used, or never really leveraged it. Maybe all of my past jobs were unprofessional, but like, I see a lot of jobs using "CI/CD experience required" and I think... huh I wonder if they actually do it

Re: Twitter's Recommendation Algorithm

#428

Post by nailer » Fri, Mar 31, 2023, 9:03 PM UTC

sva_ wrote: post #331

CathalMullan wrote: post #55

Earlier quoted context omitted.

Only used for metrics, apparently. [0] /** * These author ID lists are used purely for metrics collection. We track how often we are * serving Tweets from these authors and how often their tweets are being impressed by users. * This helps us validate in our A/B experimentation platform that we do not ship changes * that negatively impacts one group over others. */ [0]: https://github.com/twitter/the-algorithm/blob/7f…

Ahh, the group of Elons. I was wondering why I see so many tweets by him, and what his "Group's" impression quote is. This is actually pretty hilarious.

That’s not how it works. See the parent.

Re: Twitter's Recommendation Algorithm

#429

Post by jml2 » Fri, Mar 31, 2023, 9:03 PM UTC

( "has_toxicity_score_above_threshold", _.getOrElse(EarlybirdFeature, None).exists(_.toxicityScore.exists(_ > 0.91)) )

Re: Twitter's Recommendation Algorithm

#430

Post by WA » Fri, Mar 31, 2023, 9:04 PM UTC

Will this make it easier to game the algo or does it depend so heavily on individual user interaction that it’s close to impossible to game it? For example, by carefully crafting Tweets or by buying likes/retweets etc?