Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Source code for the X recommendation algorithm

github.com

Post reply on HN
Print view View original

11–20 of 156 posts

Re: Source code for the X recommendation algorithm

#11

Post by amelius »

Scene_Cast2 wrote: post #7

Just like with their last release, they only released the architecture and not the weights. It may be useful for analyzing the system if you're a competitor (but from my last dive into it, it seemed like a strict subset of fancier, industry-leading rec systems), or perhaps getting into rec / retrieval systems as a newcomer. However, this gives roughly zero insight into how Twitter's feed behaves.

There might be some value if someone can show that the feed mis-behaves for some selection of weights.

Re: Source code for the X recommendation algorithm

#12

Post by openquery »

random3 wrote: post #8
openquery wrote: post #4

I've always wondered - how can I as a non X engineer be sure that the code on GH is actually deployed on their servers?

I don’t think that’s the point of open sourcing things, in general

I agree in general it isn't. But in this case Musk claimed that was the point of open-sourcing the algorithm. Transparency on what they are or are not suppressing.

Re: Source code for the X recommendation algorithm

#13

Post by gchamonlive »

openquery wrote: post #4

I've always wondered - how can I as a non X engineer be sure that the code on GH is actually deployed on their servers?

How can you be sure that the machine code that was generated from your C source files actually match the behaviour encoded in them?

https://www.cs.cmu.edu/~rdriley/487/papers/Thompson_1984_Ref...

Re: Source code for the X recommendation algorithm

#15

Post by Scene_Cast2 »

amelius wrote: post #11
Scene_Cast2 wrote: post #7

Just like with their last release, they only released the architecture and not the weights. It may be useful for analyzing the system if you're a competitor (but from my last dive into it, it seemed like a strict subset of fancier, industry-leading rec systems), or perhaps getting into rec / retrieval systems as a newcomer. However, this gives roughly zero insight into how Twitter's feed behaves.

There might be some value if someone can show that the feed mis-behaves for some selection of weights.

Nope. Every single system like that will misbehave if given a bad set of weights, or even a random set of weights. I'd go as far as saying that even with "good" weights, it's likely to have some sigma of misbehavior.

Re: Source code for the X recommendation algorithm

#16

Post by jsheard »

cubefox wrote: post #14
kennyadam wrote: post #9

Earlier quoted context omitted.

That was Grok, not the X algorithm.

And the Grok prompts are also open source: https://github.com/xai-org/grok-prompts

Well, some version of them are open source, which may or may not be what is actually running in prod. AFAIK the patch which made it obsessively bring up white South Africans was never published, and this algorithm repo went over two years between updates so it obviously wasn't tracking prod.

Re: Source code for the X recommendation algorithm

#17

Post by cma »

openquery wrote: post #12
random3 wrote: post #8

Earlier quoted context omitted.

I don’t think that’s the point of open sourcing things, in general

I agree in general it isn't. But in this case Musk claimed that was the point of open-sourcing the algorithm. Transparency on what they are or are not suppressing.

When Tesla "open sourced" their patents, they required companies taking them up on it to, not reciprocally, not copy their "designs". So you get access to their patents in exchange for vague restrictions broader than the patent or copyright system.

Re: Source code for the X recommendation algorithm

#18

Post by jjordan »

h1fra wrote: post #6
openquery wrote: post #4

I've always wondered - how can I as a non X engineer be sure that the code on GH is actually deployed on their servers?

you can't, and it's 100% sure it's not this code running in prod

100% huh? That's a bold statement with no supporting evidence.

Re: Source code for the X recommendation algorithm

#19

Post by ml-anon »

openquery wrote: post #4

I've always wondered - how can I as a non X engineer be sure that the code on GH is actually deployed on their servers?

It’s not. The last “algorithm” release was a random grab bag of code which existed in some of the Twitter repo that might have been tangentially related to recommendations/feed.

Source: worked at Twitter in ML/recsys.

Re: Source code for the X recommendation algorithm

#20

Post by cma »

cubefox wrote: post #14
kennyadam wrote: post #9

Earlier quoted context omitted.

That was Grok, not the X algorithm.

And the Grok prompts are also open source: https://github.com/xai-org/grok-prompts

Did they ever open source in there the stuff that was making Grok search for Musk's opinion before giving an opinion on world news?
Post reply on HN