Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

twitter/the-algorithm

github.com

Post reply on HN
Print view View original

91–100 of 403 posts

Re: twitter/the-algorithm

#91

Post by root_axis »

tomcam wrote: post #74
threeseed wrote: post #63

Anyone who actually uses Twitter already knows the algorithm: * Chronological - reverse sort by date * Home - for all of the followed topics, recommended topics, retweets and tweets in the past day determine the estimated level of engagement, include the highest and reverse sort by date. This is likely to be a fairly basic ML model. It will be uncontroversial, technically unsophisticated and of no practical use to an…

You forgot the gigantic amount of human intervention required to unperson people who tweet against Twitter‘s political interests

What does "unperson" mean?

Re: twitter/the-algorithm

#93

Post by dominotw »

daenz wrote: post #83
brink wrote: post #40

Earlier quoted context omitted.

Looks like a wise move to keep potential destructive protests at bay.

Yep. I imagine Twitter has a employee base with a very strong activist streak. I'm sure some of them wouldn't mind martyring themselves to make a virtuous statement.

only because the job market is so good.

Re: twitter/the-algorithm

#94

Post by apgwoz »

influx wrote: post #9
Irishsteve wrote: post #7

Many people have commented that it is empty. However what they do not realize is that there has never actually been an algorithm and that is why it is empty.

There's not an algorithm to order timelines?

Turns out they never actually scaled their mailbox pattern. As a result, what you see is what they delivered before timeout.

Re: twitter/the-algorithm

#95

Post by axg11 »

I've worked on very large scale recommendation systems at a FAANG. If Twitter's system resembles anything like ours, the concept of publishing or open sourcing "the algorithm" doesn't make sense.

Even if we were to open source all associated code and publish all related documents it would be very difficult to make sense of the entire system. That is precisely why companies such as Twitter A/B test the hell out of everything. What most people think of as "the algorithm" is a complex system that receives many inputs (maybe hundreds) and has dependencies on many other internal Twitter services. Tweets likely pass through multiple filtering steps as well as scoring before you ever see them. Each of these steps is highly contextual, depending on: location, past tweets, verification status, etc. You can attempt to predict the effect of a certain change, but you never know the actual outcome until you test it.

I think what will ultimately happen is that _some_ details will be published. Elon will parade that around as a victory for free speech as Twitter is now more "open". In reality, nothing of value will be gained as "the algorithm" isn't a simple function.

Re: twitter/the-algorithm

#96

Post by baryphonic »

krapp wrote: post #58
onlyrealcuzzo wrote: post #45

Earlier quoted context omitted.

In the end, it's still code that is being executed, which can be called "an algorithm". Algorithms are under no obligation to be simple or only include "one algorithm".

So then "open sourcing the algorithm" likely means opens sourcing the entire Twitter codebase?

At this point, what harm would that do to Twitter? The company's tech has been stagnant for years, and the only real features recently have been attempts to exploit Twitter's network effects to ape Clubhouse & Substack. (Seems successful for the former, kind of a failure for the latter.)

Re: twitter/the-algorithm

#97

Post by threeseed »

shrimpx wrote: post #82

I don't understand the concept of open-sourcing "the algorithm". First of all, "the algorithm" is probably hundreds of thousands of lines of code, including all the tedious boilerplate like cache policies and multi-AZ logic. And second of all, doesn't the algorithm include machine learning components, which are trained on terabytes of data? That data will likely be impossible to open source. And open sourcing the neu…

You can open source the model code.

And developers will be able to train a model using it on a subset of Twitter data. Just that the quality of the outcome won't be the same as having the full set of Twitter data.

Re: twitter/the-algorithm

#98

Post by mrkramer »

shrimpx wrote: post #82

I don't understand the concept of open-sourcing "the algorithm". First of all, "the algorithm" is probably hundreds of thousands of lines of code, including all the tedious boilerplate like cache policies and multi-AZ logic. And second of all, doesn't the algorithm include machine learning components, which are trained on terabytes of data? That data will likely be impossible to open source. And open sourcing the neu…

Ever heard of Pseudocode which happens to be human-readable? If they are really going do it they will release source code for programmers and in general computer scientists to analyse and on top of that they will release pseudocode which non-techincal people can somewhat understand.

Re: twitter/the-algorithm

#99

Post by ProAm »

shrimpx wrote: post #82

I don't understand the concept of open-sourcing "the algorithm". First of all, "the algorithm" is probably hundreds of thousands of lines of code, including all the tedious boilerplate like cache policies and multi-AZ logic. And second of all, doesn't the algorithm include machine learning components, which are trained on terabytes of data? That data will likely be impossible to open source. And open sourcing the neu…

Or is it six cats in a basement with a laser pointer and a mouse?

The point of releasing it is to let people know exactly why they see the tweets they do in the order they do. I hope Elon just goes back to time base ordering of tweets.

Re: twitter/the-algorithm

#100

Post by WORMS_EAT_WORMS »

shrimpx wrote: post #82

I don't understand the concept of open-sourcing "the algorithm". First of all, "the algorithm" is probably hundreds of thousands of lines of code, including all the tedious boilerplate like cache policies and multi-AZ logic. And second of all, doesn't the algorithm include machine learning components, which are trained on terabytes of data? That data will likely be impossible to open source. And open sourcing the neu…

Probably right but I think trying to be transparent is better than not trying at all.

Gesture carries weight to the users too.

Not sure any big company has tried this before. I could be wrong, but either way looking forward to it / FWIW hope it catches on.

Post reply on HN