Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Source code for the X recommendation algorithm

github.com

Post reply on HN
Print view View original

91–100 of 156 posts

Re: Source code for the X recommendation algorithm

#91

Post by lawlessone »

gyanchawdhary wrote: post #70
jezzamon wrote: post #63

Earlier quoted context omitted.

I think that's the first time I've seen someone positive about that change. My experience has been by showing blue tick users above others, the experience has become a lot more biased because it's only a certain type of person that pays for Twitter

Isn’t that a good thing that you have already created a mental filter about people who pay for it as being of a “certain type” .. the problem with ticks being bestowed upon some journalists is that they become the brokers/influencers by the virtue of simply working for a newspaper .. that power is vaporised now .. tbh the real question is why didn’t twitter pre musk implement community notes .. I mean it’s not such a…

>that power is vaporised now

and replaced with something worse.

Re: Source code for the X recommendation algorithm

#92

Post by minimaxir »

Kaethar wrote: post #81
Scene_Cast2 wrote: post #7

Just like with their last release, they only released the architecture and not the weights. It may be useful for analyzing the system if you're a competitor (but from my last dive into it, it seemed like a strict subset of fancier, industry-leading rec systems), or perhaps getting into rec / retrieval systems as a newcomer. However, this gives roughly zero insight into how Twitter's feed behaves.

Not an ML expert, but is it feasible to train the weights using the actual Twitter feed as an oracle?

No, even if you somehow were able to download the corpus of all public X posts. There are many hidden signals that are feature engineered in good recsys, and the stripped-down algo won't be able to replicate them.

Re: Source code for the X recommendation algorithm

#93

Post by huflungdung »

close04 wrote: post #76
anonym29 wrote: post #34

Earlier quoted context omitted.

[flagged]

Honest question. Would you even dare to say you work at Twitter and then spill the beans on some very public lie or misdirection? It’s trivial to match your writing style between your HN comments and your work emails to identify you. Musk is famously a very petty, bitter, and vindictive person with an easy to bruise ego. I don’t have any knowledge of the reality inside Twitter but I also have no reason to believe the…

[dead]

Re: Source code for the X recommendation algorithm

#94

Post by cubefox »

luma wrote: post #59
cma wrote: post #20

Earlier quoted context omitted.

Did they ever open source in there the stuff that was making Grok search for Musk's opinion before giving an opinion on world news?

Nope, nor did that repo have the system prompts that brought is MechaHitler, nor the time earlier this year where it started injecting Trump into every completion. The Grok repo is a smokescreen for deniability (just not particularly plausible).

> Nope, nor did that repo have the system prompts that brought is MechaHitler

False, you made that up.

https://decrypt.co/329365/bye-bye-mechahitler-elon-musk-xai-...

> The Grok repo is a smokescreen for deniability (just not particularly plausible).

Completely unfounded conspiracy theory.

Re: Source code for the X recommendation algorithm

#95

Post by cubefox »

jsheard wrote: post #16
cubefox wrote: post #14

Earlier quoted context omitted.

And the Grok prompts are also open source: https://github.com/xai-org/grok-prompts

Well, some version of them are open source, which may or may not be what is actually running in prod. AFAIK the patch which made it obsessively bring up white South Africans was never published, and this algorithm repo went over two years between updates so it obviously wasn't tracking prod.

> which may or may not be what is actually running in prod.

This is the case for every open source software ever.

> AFAIK the patch which made it obsessively bring up white South Africans was never published

Or there never was a specific patch for that purpose, contrary to what you are assuming.

> this algorithm repo went over two years between updates so it obviously wasn't tracking prod.

You are mixing up something. The Grok prompt repo is a different repo from the recommendation algorithm, and the former has been updated regularly.

Re: Source code for the X recommendation algorithm

#96

Post by next_xibalba »

jezzamon wrote: post #63
gyanchawdhary wrote: post #62

Earlier quoted context omitted.

Post Musk Twitter is amazing. It lets you see how stories, opinions that you support or don’t are attacked from all sides and Community noted / @grok fact checked … a lot of UX changes too .. pre Musk, the moderation / banning was biased and arbitrary (who is watching the watchers?) .. my personal fav was to see the special tick removed from journalists ..

I think that's the first time I've seen someone positive about that change. My experience has been by showing blue tick users above others, the experience has become a lot more biased because it's only a certain type of person that pays for Twitter

Twitter/X user here. I agree with GP, it’s better than it was pre-Elon. The For You feed definitely seems less biased, more interesting, and fewer flame wars. I also think the exodus of Bluesky-ers has helped that (for which Elon gets partial credit). Yes, they do seem to have been backfilled by their right-side mirror images, but those people don’t seem to get amplified to the degree their predecessors were.

Re: Source code for the X recommendation algorithm

#97

Post by dotancohen »

Gabrys1 wrote: post #88
Scene_Cast2 wrote: post #7

Just like with their last release, they only released the architecture and not the weights. It may be useful for analyzing the system if you're a competitor (but from my last dive into it, it seemed like a strict subset of fancier, industry-leading rec systems), or perhaps getting into rec / retrieval systems as a newcomer. However, this gives roughly zero insight into how Twitter's feed behaves.

It'd assume that weights are changing constantly so they'd need to open source a service tweaking the weights in real time rather than the weights themselves...

They could publish a snapshot of any point in time. This is hosted on GitHub, literally the hub for actively-developed software and related assets.

Re: Source code for the X recommendation algorithm

#98

Post by TheAceOfHearts »

keepamovin wrote: post #78

I think Elon said he would release the weights. In a video somewhere. That's what he meant - when the next major version lands, they release the previous one? People's choices can change, maybe the economic/geopolitical reality of AI race has been impressed upon him, but I think that's what he said.

This post is about the Twitter algorithm. He originally said it would be open source, but he just did a source dump 2 years ago. Now they did a new source dump with updated but heavily redacted code for the For You feed.

As for his claims about opening up Grok: Elon said that they would publish the n-1 weights for Grok. However, he dragged his feet and only recently released the weights for Grok 2. So now we're up to Grok 4 but he has yet to release the weights for Grok 3 despite his claims.

I think the problem with Elon is that he doesn't fully hold himself accountable for his words. If he decided that it was no longer economically viable to share Grok's weights then he should post an update about that. You cannot expect to win the goodwill of claiming to support open source and then continuously drag your feet while refusing to communicate your intentions clearly.

Re: Source code for the X recommendation algorithm

#99

Post by TheAceOfHearts »

ml-anon wrote: post #19
openquery wrote: post #4

I've always wondered - how can I as a non X engineer be sure that the code on GH is actually deployed on their servers?

It’s not. The last “algorithm” release was a random grab bag of code which existed in some of the Twitter repo that might have been tangentially related to recommendations/feed. Source: worked at Twitter in ML/recsys.

Anon, when I was looking through this source dump I saw a huge range of timeouts used in various services, do you know if there's any writeup or explanation as to how the engineering team settled on those values?
Post reply on HN