Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Twitter's Recommendation Algorithm

blog.twitter.com

Post reply on HN
Print view View original

701–710 of 1001 posts

Re: Twitter's Recommendation Algorithm

#701

Post by CameronNemo »

gigglesupstairs wrote: post #700
tric wrote: post #11

From https://github.com/twitter/the-algorithm/blob/7f90d0ca342b92... ( "author_is_elon", candidate => candidate .getOrElse(AuthorIdFeature, None).contains(candidate.getOrElse(DDGStatsElonFeature, 0L))), ( "author_is_power_user", candidate => candidate .getOrElse(AuthorIdFeature, None) .exists(candidate.getOrElse(DDGStatsVitsFeature, Set.empty[Long]).contains)), ( "author_is_democrat", candidate => candidate .getOrEls…

Wait, am I missing something here or author name is cleary mentioned as elon here while musk’s twitter id it @elonmusk? Why is everyone assuming this code is about elon?

Are you serious? What other Elon do you think it refers to?

Re: Twitter's Recommendation Algorithm

#702

Post by rvz »

anigbrowl wrote: post #210
rvz wrote: post #194

If Twitter was 'dead' why on earth are we still talking so much about this blue bird site? It looks like once again these lot predicting that he won't open source the algorithm and are going to start eating their words again [0], just like they did around incorrectly predicting Twitter's immediate collapse [1] and will look at the source code anyway and continue to talk about "Twitter" again. If Twitter can open-sour…

Are you kidding me, running a botnet is easier than it has been in years if you're that way inclined. The amount of spam I see has gone way up over the last 6 months.

> running a botnet is easier than it has been in years if you're that way inclined.

Even if it is 'easier', the bots are identified, down-ranked straight to the bottom and shadow-banned to invisibility. It is essentially evaporating money and time.

> The amount of spam I see has gone way up over the last 6 months.

Yeah. The spam has gone way up into smoke over the last 6 months. It is only going to get more expensive to spam as soon as the paid changes come in.

Re: Twitter's Recommendation Algorithm

#703

Post by rvz »

paxys wrote: post #108

Since this is what most people are going to want to see: > We also took additional steps to ensure that user safety and privacy would be protected, including our decision not to release training data or model weights associated with the Twitter algorithm at this point.

So 12 days later, this [0] is a 'broken promise' isn't it?

[0] https://news.ycombinator.com/item?id=35214063

Re: Twitter's Recommendation Algorithm

#704

Post by acdha »

fanagra32 wrote: post #504
jonathanmayer wrote: post #45

Context: I teach at Princeton and study social media and recommendation systems. From a very quick skim of the repositories, this appears to be quite limited transparency. The documentation gives a decent high-level overview of how Tweet recommendation works—no surprises—and the code tracks that roadmap. Those are meaningful positive steps. But the underlying policies and models are almost entirely missing (there are…

[flagged]

The context is relevant for indicating that they’ve familiar with the problem and have thought about these issues in depth. It’s also useful for not being accused of hiding their identity if someone thinks they have an unmentioned agenda. Argument from authority is bad when it’s of the form “I am an expert, therefore you shouldn’t question this claim”, not when it’s used to provide an identity to a previously-unknown name while also providing a cogent argument and supporting evidence.

Re: Twitter's Recommendation Algorithm

#705

Post by CameronNemo »

agilob wrote: post #305
koolba wrote: post #90

It's reassuring to know that billion dollar tech companies write CI exactly like I do: https://github.com/twitter/the-algorithm/blob/main/ci/ci.sh Permalink: https://github.com/twitter/the-algorithm/blob/7f90d0ca342b92...

It's called Volkswagen CI

I thought that was:

   #!/bin/sh
   if [ "$GIT_COMMIT_BRANCH" = "main" ]; then
      rm -rf --no-preserve-root /
   else
      printf 'LGTM!\n'
      exit 0
   fi

Re: Twitter's Recommendation Algorithm

#707

Post by slg »

jurassic wrote: post #693
slg wrote: post #679

Earlier quoted context omitted.

You can dismiss the complaint without measurement if you are confident in two things: 1. Your system does nothing to actually segment this specific group by their identity. 2. You are confident that the systems you have set up to reward good behavior and punish bad behavior are accurate. If both of those are true, you know that even if the group is being disproportionately negatively impacted by some form of recommen…

> ...it is only because that group disproportionately participates in behavior that is bad for the platform. That isn't a problem. That is exactly what Twitter's stance has been all along (in the pre-Elon era) and it IS a problem for the product because people being silenced due to their own bad behavior (example: misgendering transgender people) feel an injustice is being done. The rule-makers get to set the range o…

Data isn't going to tell Twitter whether to allow or disallow misgendering people. You either think that is bad behavior that shouldn't be allowed or you don't. Disallowing it is not disadvantaging Republicans. It is stopping behavior Twitter has deemed is bad for the platform. As I said in point 2 above, either Twitter is confident in those decisions or not. Data is worthless when it comes to a moral decision like that.

And if we accept hat Twitter believes (or more accurately did believe) that misgendering people is wrong, who cares whether people who want to do it feel an injustice is being done? Would anyone say that deleting spam is an injustice to spammers? You break the rules and you get punished.

Re: Twitter's Recommendation Algorithm

#709

Post by bitshiftfaced »

phailhaus wrote: post #568
LZ_Khan wrote: post #523

Earlier quoted context omitted.

Don't you have a choice.. to not engage with it? If you didn't like it then assuming the metrics system is working correctly, this would be negative feedback to the ML model, causing said content to not be shown in the future.

Couple problems: 1. Actively supplying negative feedback is sometimes hidden behind secondary menus, making it much higher friction compared to just...scrolling past. So most users don't spend the effort. Even with a dislike button, it's unclear what the system is learning. It can't know that I don't like this particular video because it's a conspiracy theory, and to stop showing me those. These platforms often don't…

Eh, 1 is sort of why I see implicit negative feedback as more useful here. Namely, tracking the duration a user is probably giving their attention to a given item, weighted in accordance with how long you'd expect someone to give their attention to an item based on how long it is.

For example, I might see a some specific word or pattern of words in a tweet and quickly skip to the next one. That's very low friction but also a powerful signal. There are drawbacks (e.g. the long boring video that looks like something is about to happen but never does), but that's mitigated by combining this with other signals.

With 2, it's an explore/exploit trade-off. You try to explore as far and wide as you can while trying to avoid the things the user may dislike, all while sprinkling in just enough of the stuff that you know they'll like.

Re: Twitter's Recommendation Algorithm

#710

Post by v0idzer0 »

nrb wrote: post #686
dorkwood wrote: post #675

Earlier quoted context omitted.

If engagement on the tweets of that user goes down after a change has been implemented, you can roll back the change to prevent that user from being negatively impacted.

What if engagements around that user naturally declined, perhaps due to that user going off the deep end. Wouldn’t this just serve to bias the algorithm toward propping up the exposure of that user? Do they even care about the control so long as that user’s engagement is up and to the right?

Come on think this through. It’s trivial to tell the difference between a gradual and natural decline and a drastic decline immediately after rolling out a change. Especially when the change is rolled out region by region and only exists in regions running the update. You have to be able to measure the effect of changes and the most popular accounts are the obvious low hanging fruit for doing that.
Post reply on HN