Twitter Should Open Up the Algorithm • Hacker News Board

Re: Twitter Should Open Up the Algorithm

#61

Post by agentdrtran » Sat, Apr 16, 2022, 3:42 PM UTC

LightHugger wrote: post #37

jimmaswell wrote: post #26

Sounds like a bad idea. Bad actors would immediately exploit it and figure out how to most effectively flood everyone with spam, same as if Google's exact algorithms were public. Not everything has to be open.

On occasion, people make the same argument about all open source software, that making it public makes it insecure. Fortunately this has not turned out to be the case in reality. If there are flaws in the system they will probably be easier to find and fix with more eyes on it.

It's not about security bugs in this case, anti-spam needs obscurity as a layer of defense.

Re: Twitter Should Open Up the Algorithm

#62

Post by ankit219 » Sat, Apr 16, 2022, 3:42 PM UTC

dkobia wrote: post #14

The most contentious moderation happens via subjective human intervention rather than an algorithms. Fully machine driven moderation is a black hole into which copious amounts of time and money from the r&d labs of the biggest social media companies disappear into every year for a long time now.

But the article is not talking about content moderation as such. It's about surfacing the right content for right people, and regain the trust that author believes twitter has lost, without really justifying how.

Re: Twitter Should Open Up the Algorithm

#63

Post by sobkas » Sat, Apr 16, 2022, 3:42 PM UTC

gfodor wrote: post #28

Recommender systems like Twitter are as much about data as they are about code. Without the dataset and the derived statistics and models that are used for ranking and recall, the code is not going to help much with transparency.

> Recommender systems like Twitter are as much about data as they are about code. Without the dataset and the derived statistics and models that are used for ranking and recall, the code is not going to help much with transparency.

And also other way around, data depends on algorithms and tools that created them. You can't have insightful understanding of data without knowing how and why they were created.

Re: Twitter Should Open Up the Algorithm

#64

Post by jeffbee » Sat, Apr 16, 2022, 3:43 PM UTC

baggy_trough wrote: post #34

jeffbee wrote: post #21

Earlier quoted context omitted.

Truly do not understand this point of view. The strictly chronological presentation is too easy to flood with unwanted posts.

I choose people to follow. If somebody is flooding my feed more than I like, then I unfollow them. Truly do not understand why anyone would want it to work differently other than marketers.

But it's not spam. If fifty people you follow retweet the same popular post, you do not want to see it fifty-one times. You want to see it once, and you want it ranked highly because everyone you follow thinks it's an important post. It's a form of crowdsourcing from a curated group of contributors. The chonological view cannot accommodate.

Re: Twitter Should Open Up the Algorithm

#65

Post by 11111 » Sat, Apr 16, 2022, 3:44 PM UTC

No post body was provided.

Re: Twitter Should Open Up the Algorithm

#66

Post by rufus_foreman » Sat, Apr 16, 2022, 3:44 PM UTC

soheil wrote: post #36

So by the same logic McDonald's should publish its secret sauce recipe. There is business value in keeping it hidden. It'd have a chance of happening if Twitter was taken private by someone with the intention of experimentation and improving the world in the long run.

McDonald's is required in some places (Canada, for example) to publish the ingredients for its food. For example, Big Mac Sauce:

Soybean oil, sweet relish (cucumber, glucose-fructose, sugar, vinegar, salt, xanthan gum, calcium chloride, natural flavour), water, vinegar, egg yolk, onion powder, spices, salt, propylene glycol alginate, colour, sugar, garlic powder, hydrolyzed (corn, soy, wheat) proteins. CONTAINS: Soy, Wheat, Egg, Mustard.

-- https://www.mcdonalds.com/ca/downloads/IngredientslistCA_EN....

And there are strict rules about how that ingredients list must be constructed:

"Ingredients must be declared by their common name in descending order of their proportion by weight of a prepackaged product. The order must be the order or percentage of the ingredients before they are combined to form the prepackaged product. In other words, based on what was added to the mixing bowl"

and on and on for many thousands of words, https://inspection.canada.ca/food-label-requirements/labelli...

No, that's not the full recipe, but it is information a company might not want to disclose, but is required to disclose because it affects the health and safety of consumers.

Re: Twitter Should Open Up the Algorithm

#67

Post by jmull » Sat, Apr 16, 2022, 3:45 PM UTC

memish wrote: post #5

A marketplace of algorithms and "pick your own algorithm" would be interesting. Is there any reason not to do this? Not sure why this is getting downvoted. It's not crazy. Even Jack is pushing for it: "The choice of which algorithm to use (or not) should be open to everyone" https://twitter.com/jack/status/1507146276416098307

I wonder about the feasibility of this.

How does an algorithm "that attempts to prioritize nuanced conversations about important topics" work? (Or "to find mind-expanding threads", for "savage dunks" or "thirst traps of hot new snax"? -- other examples from the post.)

I suppose you spend some time with existing tweets and ML and develop a model that can produce a score of some sort on these concepts for a tweet, and then run every tweet through the model and present the high-scoring ones. Of course, you can't just look at individual tweet, which in isolation don't mean much, but also at how it fits into a conversation. (For that matter, I'd be interested to see what a model of a conversation is.)

Sounds expensive and quite possibly not accurate enough to be worthwhile.

It also seems like a major strategic decision to give third parties every tweet by everyone. It seems like the business changes from being what twitter is now to a message routing backend, and these third parties become what twitter used to be. That's a fundamental shift, that probably devalues the company by an order of magnitude, since they would be dissolving the valuable thing they have -- the social network they have.

Just doesn't make any sense to me.

Re: Twitter Should Open Up the Algorithm

#68

Post by crazy1van » Sat, Apr 16, 2022, 3:45 PM UTC

soheil wrote: post #47

dkobia wrote: post #14

The most contentious moderation happens via subjective human intervention rather than an algorithms. Fully machine driven moderation is a black hole into which copious amounts of time and money from the r&d labs of the biggest social media companies disappear into every year for a long time now.

> The most contentious moderation happens via subjective human intervention rather than an algorithms. 1. Source? 2. Thought experiment: if said subjective human intervention was recorded and codified into an algorithm it wouldn't be as contentious?

Source for probably the most contentious piece of moderation Twitter has ever done being manual was their own statement:

“After close review of recent Tweets from the @realDonaldTrump account and the context around them we have permanently suspended the account due to the risk of further incitement of violence,”

Re: Twitter Should Open Up the Algorithm

#69

Post by soheil » Sat, Apr 16, 2022, 3:45 PM UTC

jimmaswell wrote: post #26

Sounds like a bad idea. Bad actors would immediately exploit it and figure out how to most effectively flood everyone with spam, same as if Google's exact algorithms were public. Not everything has to be open.

Yes it'd be a cat and mouse game and that's how you build an efficient and fair algorithm. Not too unlike how the market decides what the price should be for a certain instrument. Lots of buys, lots of sells and where they meet is called the price. All in the open for anyone to see.

Let's stop the Communistic mentality around keeping things hidden. Please read about Soviet Russia, about Siberia and how Russians had no idea about the miseries inflected on them for decades.

Re: Twitter Should Open Up the Algorithm

#70

Post by admax88qqq » Sat, Apr 16, 2022, 3:45 PM UTC

The interesting question this poses and has come up on a few comment threada here is

Is "security through on obscurity" necessary for automated content moderation?

Would an open algorithm be trigially gamed by spammers as they can now test offline exactly how their posts will be ranked/promoted?

My gut says yes but I'm not an expert in this area. Curious if anyone has a theory or idea on if an open moderation algorithm could work.

SpamAssassin exists and is open to moderate success. But is that just because it's use is not widespread enough to bother to test your spam mails against it?

If every email account in the world was covered by SpamAssassin, what would spam look like, and how much would make it through.