Live data from Hacker News

Twitter's Recommendation Algorithm

blog.twitter.com

511–520 of 1001 posts

Re: Twitter's Recommendation Algorithm

#511

Earlier quoted context omitted.

Seems dumb of them. Cost is trivial and their competition that isn’t so politically motivated will have a much further reach. The smart move would be silent on the policy change, pay, and support rival platforms as they can. Instead they will eventually pay and look like they lost.

We’re in a time where ideology trumps revenue for some companies. You know, “Get woke, go broke.”

Everyone says that about things that made tons of money though.

Re: Twitter's Recommendation Algorithm

#512

Earlier quoted context omitted.

Only used for metrics, apparently. [0] /** * These author ID lists are used purely for metrics collection. We track how often we are * serving Tweets from these authors and how often their tweets are being impressed by users. * This helps us validate in our A/B experimentation platform that we do not ship changes * that negatively impacts one group over others. */ [0]: https://github.com/twitter/the-algorithm/blob/7f…

I expect they're tracking the red team/blue team metrics because of the political shitstorm that's been the GOP's assertions they're being silenced by The Algorithm.

The fallacy of false equivalence systematized in code.

Now one side can spew as much disinfo and incitement to violence as it likes, and any algorithm change that prevents this shit from getting amplified will be rejected as bias.

BSaaS = Both Sides as a Service

Re: Twitter's Recommendation Algorithm

#514
post #11

From https://github.com/twitter/the-algorithm/blob/7f90d0ca342b92... ( "author_is_elon", candidate => candidate .getOrElse(AuthorIdFeature, None).contains(candidate.getOrElse(DDGStatsElonFeature, 0L))), ( "author_is_power_user", candidate => candidate .getOrElse(AuthorIdFeature, None) .exists(candidate.getOrElse(DDGStatsVitsFeature, Set.empty[Long]).contains)), ( "author_is_democrat", candidate => candidate .getOrEls…

Did they not expect people to notice suspicious code like this? Or did they leave this in just so they could hold its removal up as an example of listening to the community?

Could be that those in charge of preparing this open sourced repository did it begrudgingly, and so they perceived the fact that it looked bad as a positive thing. "Hey, you wanted us to release the code. Happy now?"

Re: Twitter's Recommendation Algorithm

#515

So as expected, there is exactly nothing that favors posters from one side of the political spectrum. I don't expect that this article will do anything to calm down those who are convinced otherwise though. Well written article, from an engineer's perspective.

Algorithm exists and is non-trivial, therefore it favors those groups of posters that spend more effort to hack it.

Re: Twitter's Recommendation Algorithm

#516
post #11

From https://github.com/twitter/the-algorithm/blob/7f90d0ca342b92... ( "author_is_elon", candidate => candidate .getOrElse(AuthorIdFeature, None).contains(candidate.getOrElse(DDGStatsElonFeature, 0L))), ( "author_is_power_user", candidate => candidate .getOrElse(AuthorIdFeature, None) .exists(candidate.getOrElse(DDGStatsVitsFeature, Set.empty[Long]).contains)), ( "author_is_democrat", candidate => candidate .getOrEls…

At first I thought this post was a joke - and it was actually a pretty good joke. Yikes.

Re: Twitter's Recommendation Algorithm

#518

Earlier quoted context omitted.

Only anecdotally. But ever since Musk took over the amount of US political content has significantly increased in particular from the right despite me not living in the US. It's hard to tell whether previously political content was weighted less and Musk has removed those controls or whether they are now weighted higher.

Some of the power users Musk reportedly had boosted are specifically right wing political posters, like catturd2. But then he's also boosted some high profile left leaning politicians, so it's not exclusive. It does mean you're more likely to see right wing American political content either way, which has to be annoying for people outside of the US. https://gizmodo.com/twitter-algorithm-aoc-ben-shapiro-cattur...

Would it not be just as annoying to see a slant the other direction, or in any extreme direction?

Re: Twitter's Recommendation Algorithm

#519

Earlier quoted context omitted.

This is (weirdly) common in production ML codebases written by software engineers. Like you, I have no idea why unless it's a memory optimisation (where you count 4+ as many).

Having every column as a boolean (0/1) means you can treat it as a bitmap. As an (entirely fictional) example, imagine if you wanted to get the features of a thread instead of a single tweet. You could do it as a union of all the tweets: threadFeatures = tweet1 | tweet2 | tweet2

Ok that makes lots of sense from an engineering perspective. It's pretty insane from a statistical perspective though, which I think was the original point.

Re: Twitter's Recommendation Algorithm

#520

Earlier quoted context omitted.

When your visa is on the line, you’ll do anything

Engineers who work at twitter can easily find another job in the US.

With forced RTO and "hardcore" mandates, it's difficult to find 6+ hours of time to interview at other companies (assuming there are any open visa sponsorships available).
Post reply on HN