Live data from Hacker News

twitter/the-algorithm

github.com

101–110 of 403 posts

Re: twitter/the-algorithm

#101

Earlier quoted context omitted.

Twitter Locks Down Product Changes After Agreeing to Musk Bid - https://finance.yahoo.com/news/twitter-locks-down-product-ch...

Are there precedents of this? This looks like there is not much trust internally. I doubt this was the first Elon’s decision

The level of trust is to be expected from a company like this imo. We're in an information war all around us. Spies, saboteurs, etc, are all part of a war, and we just witnessed a coup of leadership.

Re: twitter/the-algorithm

#105
post #82

I don't understand the concept of open-sourcing "the algorithm". First of all, "the algorithm" is probably hundreds of thousands of lines of code, including all the tedious boilerplate like cache policies and multi-AZ logic. And second of all, doesn't the algorithm include machine learning components, which are trained on terabytes of data? That data will likely be impossible to open source. And open sourcing the neu…

The whole point of the repo seems to just show that there's nothing called as "the algorithm" It's probably something like 100s of 1000s of algorithms doing different things

Re: twitter/the-algorithm

#106

Earlier quoted context omitted.

This is most likely the correct answer. I doubt there is a single piece of code or algorithm that controls all of twitter. From my limited perspective it's just a catch phrase to simplify what is likely tens of thousands of lines of code.

Well, that's an algorithm. If you have to draw a diagram showing how edge cache nodes affect a user's results based on whether or not they signed in from New York or Phoenix, or if the downtime of a cluster affects the relevance rankings of tweets for a photography enthusiast, then guess what... That's an algorithm.

I don't disagree, I might even argue that twitter is comprised of many algorithms. We can argue all software is algorithms, or we can just call it software.

Re: twitter/the-algorithm

#107
post #37

Earlier quoted context omitted.

If it’s a list of tweets ordered based on a kajillion ML data points that varies per user is it still an algorithm? And does every user have their own algorithm? And could it be made readable to a human?

Yes, irrelevant, and the code somewhere is readable. Take an example like gravity: is there an algorithm for how gravity acts even though it varies per each molecule based on a kajillion other molecules? Of course there is.

If I have your DNA can I recreate a perfect copy of you as you are now including your mind? It is after all the code you were made with.

Re: twitter/the-algorithm

#108
post #95

I've worked on very large scale recommendation systems at a FAANG. If Twitter's system resembles anything like ours, the concept of publishing or open sourcing "the algorithm" doesn't make sense. Even if we were to open source all associated code and publish all related documents it would be very difficult to make sense of the entire system. That is precisely why companies such as Twitter A/B test the hell out of eve…

I get what you're saying. I'd imagine this repo will be closer to pseudo code than "complete" code for all systems that a tweet flows through. For example, the system that flags/remove picture of hotdogs would likely be represented as "if image contains hotdog: weight: -500"

Re: twitter/the-algorithm

#109
post #95

I've worked on very large scale recommendation systems at a FAANG. If Twitter's system resembles anything like ours, the concept of publishing or open sourcing "the algorithm" doesn't make sense. Even if we were to open source all associated code and publish all related documents it would be very difficult to make sense of the entire system. That is precisely why companies such as Twitter A/B test the hell out of eve…

This is likely in response to upcoming EU legislation on algorithm transparency[0]. It's not useful, but they'll need to do it eventually.

[0]: https://mashable.com/article/eu-digital-services-act-big-tec...

Re: twitter/the-algorithm

#110

There are elements of their algo that I think should be openly defined, and perhaps there should be some regulatory branch that reports to Congress that has full access. However, obfuscation is often necessary to countering bad actors.

On the other hand, wouldn't open sourcing the algorithm help accelerate the identification of possible exploits?
Post reply on HN