Live data from Hacker News

Twitter Should Open Up the Algorithm

every.to

131–138 of 138 posts

Re: Twitter Should Open Up the Algorithm

#132
post #28

Recommender systems like Twitter are as much about data as they are about code. Without the dataset and the derived statistics and models that are used for ranking and recall, the code is not going to help much with transparency.

Oh, you mean the matrix came up with rule to ban people talking about lab leak? but matrix can't figure out simple crypto bots and need humans to report and manually ban them?

That's moderation which is a completely separate topic from recommendation.

Re: Twitter Should Open Up the Algorithm

#133

Comments saying that this is naive seem to be completely missing the point in their desire to cynically defend a miserable state of affairs: clients, and not Twitter's servers, should ultimately determine what users see. Like the web browser or an email client. The fact that we calculate precisely what users see on their devices on servers is a result of the architectural constraints of the time, and more importantly…

> clients, and not Twitter's servers, should ultimately determine what users see. Like the web browser or an email client.

That would involve giving clients access to information that clients probably shouldn't have. eg: If a part of the weighting for recommendation is that people you follow who regularly DM other people you follow should be weighted higher, doing it client side would allow you to see other people's private DM information.

Re: Twitter Should Open Up the Algorithm

#134
post #130
post #129

Earlier quoted context omitted.

I’m sure twitters algorithms take advantage of all interaction data, like impressions, scrolls, and clicks. Agree tho it’s good a lot of the data is public.

Publishing user scrolls is a "massive hill" and a legal/privacy nightmare?

Sure seems like it could be - I wouldn’t opt-in to allowing mine to be distributed, why would I? And I don’t think Twitter has the rights to do so. So you’d at least need to solve the anonymization problem, or you’d have to package a data release such that you can replicate the ranking algorithm without it.

Re: Twitter Should Open Up the Algorithm

#135
post #91

Earlier quoted context omitted.

It would provide insight but if the goal is to increase transparency to the point of meaningfully improving trust in the centralized actor that is Twitter, I don’t see it moving that needle much. Given the right data you can kind of coax a lot of these algorithms into arbitrary outputs, so if you’re starting from a place of distrust then if you can’t replicate the outputs yourself you wont have evidence to alter prio…

The point I'm stuck on in this whole conversation is why you'd continue to use Twitter if you felt they were untrustworthy. Why are we discussing freedom of speech when you can just load another app and get all the freedom you want?

The reason people are complaining is we have found ourselves in a situation where Twitter is arguably already or indisputably will become what amounts to a global public square.

The early philosophical developments around freedom of speech didn’t foresee this but now that it is here, we have to ask the question of what ought we want if we are to uphold the principle that people should be free to speak unpopular ideas without censorship or fear of excommunication, given that a key ingredient in progress, revelation of the truth, and finding compromise to avoid violence.

Twitter is of course a private company, and can do what they want. But what those who wish to see free speech principles upheld say is we ought to want those in twitters position to be as lenient as possible so as to not stifle the free exchange of ideas. And, perhaps failing that, we ought to want to see another mechanism that is less susceptible to widespread censorship and overreach. Too many people seem counter someone explaining that they feel the status quo is undesirable with an is/ought fallacy - it’s our job in liberal democracies to continually raise and debate issues regarding basic freedoms and the ability for our society to continue to evolve according to shared principles.

Re: Twitter Should Open Up the Algorithm

#136
I my experience as someone who resides in India, I have seen Twitter repeatedly silencing voices of the opposition and amplifying the voices of those connected and aligned with the majoritarian, right-wing elements - and I don't think it is because of any algorithm. There are clearly people involved deciding case-by-case to shutout voices of the opposition.

We have seen this with Facebook as well with Facebook charging 3 times higher for ads from the opposition parties in a bid to influence the elections: https://www.aljazeera.com/economy/2022/3/16/facebook-charged...

Re: Twitter Should Open Up the Algorithm

#137
post #134
post #130

Earlier quoted context omitted.

Publishing user scrolls is a "massive hill" and a legal/privacy nightmare?

Sure seems like it could be - I wouldn’t opt-in to allowing mine to be distributed, why would I? And I don’t think Twitter has the rights to do so. So you’d at least need to solve the anonymization problem, or you’d have to package a data release such that you can replicate the ranking algorithm without it.

User clicks in form of "likes" are already public so why would scroll data be so sacred?

Re: Twitter Should Open Up the Algorithm

#138
post #97

Earlier quoted context omitted.

The point is to increase transparency, period. And you are the one arguing against that. Nobody is saying it’s enough, but it’s objectively better than nothing, and it would make it possible to ask more specific questions and make more specific arguments for the value of more transparency. Again, you are the one arguing against it but you have not provided a reason why.

I’m not arguing against it - I’m arguing that people who are focused on the idea of releasing the code are miscalibrated on the marginal impact of doing so given a lack of domain knowledge. The net suggestion is to demand more, and start describing solutions, today, to get data visibility without compromising other things, to help offset the inevitable objections that will come once people are forced to demand it whe…

Ironically, the best way to prove your own point is to open the source code.
Post reply on HN