Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Source code for the X recommendation algorithm

github.com

Post reply on HN
Print view View original

121–130 of 156 posts

Re: Source code for the X recommendation algorithm

#121

Post by IncRnd »

archagon wrote: post #113
gyanchawdhary wrote: post #62

Earlier quoted context omitted.

Post Musk Twitter is amazing. It lets you see how stories, opinions that you support or don’t are attacked from all sides and Community noted / @grok fact checked … a lot of UX changes too .. pre Musk, the moderation / banning was biased and arbitrary (who is watching the watchers?) .. my personal fav was to see the special tick removed from journalists ..

“Fact checks,” sure. Grok is finely tuned to its master’s demands and politics: https://archive.ph/G0Y4i

The very first question that the article writer said they posed to Grok 3 and Grok 4, "What is currently the biggest threat to Western civilization and how would you mitigate it?", didn't return anything like the simplistic answers in that article. Apparently, the article was politically driven.

When I asked Grok 4, two pages worth of answers were returned, including a table with columns for Threat, Reasoning, and Severity. The article is just plain wrong and fails the very fact-checking that it purported to do.

Re: Source code for the X recommendation algorithm

#122

Post by archagon »

IncRnd wrote: post #121
archagon wrote: post #113

Earlier quoted context omitted.

“Fact checks,” sure. Grok is finely tuned to its master’s demands and politics: https://archive.ph/G0Y4i

The very first question that the article writer said they posed to Grok 3 and Grok 4, "What is currently the biggest threat to Western civilization and how would you mitigate it?", didn't return anything like the simplistic answers in that article. Apparently, the article was politically driven. When I asked Grok 4, two pages worth of answers were returned, including a table with columns for Threat, Reasoning, and Se…

I’m not sure what point you think you’re making. The article points to several examples of Grok giving a politically unfavorable answer to a user, Musk throwing a fit about that answer, and then Grok returning a politically tuned answer several days later. It’s observation, not some sort of gotcha by the author. Whatever you’re doing with Grok right now is irrelevant in this context.

Re: Source code for the X recommendation algorithm

#123

Post by cubefox »

SilverElfin wrote: post #87
cubefox wrote: post #23

This is laudable. But the great thing about Twitter is that you don't have to use the algorithmic "For You" feed at all. You can just use the "Following" feed, which is purely chronological, and doesn't contain any recommended content. This isn't possible on Facebook, which makes it unusable for me.

I don’t use it - does it remember the setting? My recollection is that Facebook would make you switch to that chronological feed manually each time you load the page.

As I said, Facebook doesn't have a chronological feed. Twitter remembers the feed setting.

Re: Source code for the X recommendation algorithm

#124

Post by wisty »

sanktanglia wrote: post #120
next_xibalba wrote: post #96

Earlier quoted context omitted.

Twitter/X user here. I agree with GP, it’s better than it was pre-Elon. The For You feed definitely seems less biased, more interesting, and fewer flame wars. I also think the exodus of Bluesky-ers has helped that (for which Elon gets partial credit). Yes, they do seem to have been backfilled by their right-side mirror images, but those people don’t seem to get amplified to the degree their predecessors were.

You are really going to try and say right wingers aren't amplified on twitter? I literally have an account that just follows gaming accounts and I was having to block people throwing out slurs daily

[flagged]

Re: Source code for the X recommendation algorithm

#125

Post by kklisura »

anonym29 wrote: post #34
ml-anon wrote: post #19

Earlier quoted context omitted.

It’s not. The last “algorithm” release was a random grab bag of code which existed in some of the Twitter repo that might have been tangentially related to recommendations/feed. Source: worked at Twitter in ML/recsys.

[flagged]

~65k lines added, ~3k removed in span of more than 2 years. Do you guys do anything there?

Re: Source code for the X recommendation algorithm

#126

Post by jordanscales »

gyanchawdhary wrote: post #35
Scene_Cast2 wrote: post #7

Just like with their last release, they only released the architecture and not the weights. It may be useful for analyzing the system if you're a competitor (but from my last dive into it, it seemed like a strict subset of fancier, industry-leading rec systems), or perhaps getting into rec / retrieval systems as a newcomer. However, this gives roughly zero insight into how Twitter's feed behaves.

For all its flaws .. it’s still a step up from how Parag and co used to run twitter

Unfortunately, this [0] cancels out everything ten-fold. The owner of the website is boosting the content of himself and the people he supports. This did not happen in the old twitter - not even close.

[0] https://github.com/twitter/the-algorithm/issues/236

Re: Source code for the X recommendation algorithm

#127

Post by mvdtnz »

nextaccountic wrote: post #114
barbazoo wrote: post #49

Earlier quoted context omitted.

Not only no weights. Not sure what people's expectations are but a lot of the time this isn't even valid code with all the redaction they did [1]. I'm confused as to who this is for, this surely isn't the repo they're working on, is it? [1] https://github.com/twitter/the-algorithm/blob/main/trust_and...

Are you talking about this? wandb_key = ... wandb.login(...) It's rather weird that they would add keys to the source code like this, rather than reading from the environment or some secrets service. Rather than redacting the source, they should refactor to remove the keys from the source

There's no way you got to this bit without skipping over multiple actual redactions, like SQL queries with all of the details replaced with ellipsis. Why are you cherry-picking one innocent instance when you know exactly what the parent comment is talking about?

Re: Source code for the X recommendation algorithm

#128

Post by Aurornis »

Raed667 wrote: post #56
barbazoo wrote: post #49

Earlier quoted context omitted.

Not only no weights. Not sure what people's expectations are but a lot of the time this isn't even valid code with all the redaction they did [1]. I'm confused as to who this is for, this surely isn't the repo they're working on, is it? [1] https://github.com/twitter/the-algorithm/blob/main/trust_and...

This is 100% for headlines and Musk to be able to say "we're open" during interviews. Its actual usefulness is not the point

When they "open sourced" the Tesla Roadster the website only had a couple of mostly useless files. Discussion at the time https://news.ycombinator.com/item?id=38383099

Despite not containing more than a few random files, there were headlines everywhere about the "Open Source Tesla Roadster". There were countless comments, Tweets, and posts about how amazing it was that the Roadster was now open source.

None of the people reporting on it or praising it actually looked at the files and realized you couldn't actually build anything other than the HVAC control board for the car.

Re: Source code for the X recommendation algorithm

#129

Post by overfeed »

sanktanglia wrote: post #120
next_xibalba wrote: post #96

Earlier quoted context omitted.

Twitter/X user here. I agree with GP, it’s better than it was pre-Elon. The For You feed definitely seems less biased, more interesting, and fewer flame wars. I also think the exodus of Bluesky-ers has helped that (for which Elon gets partial credit). Yes, they do seem to have been backfilled by their right-side mirror images, but those people don’t seem to get amplified to the degree their predecessors were.

You are really going to try and say right wingers aren't amplified on twitter? I literally have an account that just follows gaming accounts and I was having to block people throwing out slurs daily

> You are really going to try and say right wingers aren't amplified on twitter?

FWIW, they see it - but interpret it as "Twitter being less biased" now, because from their POV, Twitter had a pro-liberal bias before Musk, and is now trending towards what they consider neutral.

Re: Source code for the X recommendation algorithm

#130

Post by jandrese »

cubefox wrote: post #23

This is laudable. But the great thing about Twitter is that you don't have to use the algorithmic "For You" feed at all. You can just use the "Following" feed, which is purely chronological, and doesn't contain any recommended content. This isn't possible on Facebook, which makes it unusable for me.

The "following" feed helps, but replies to almost any topic still attract outright white supremacists on X. They have seemingly endless time to fill the site up with their talking points.
Post reply on HN