I know the algorithm i use, it ends with ORDER BY date DESC.
Nice for some things. Absolute death for discovery in a lot of cases though. Either trim your follows down dramatically, or miss some incredible stuff...
twitter/the-algorithm
221–230 of 403 posts
Re: twitter/the-algorithm
#222I don't understand the concept of open-sourcing "the algorithm". First of all, "the algorithm" is probably hundreds of thousands of lines of code, including all the tedious boilerplate like cache policies and multi-AZ logic. And second of all, doesn't the algorithm include machine learning components, which are trained on terabytes of data? That data will likely be impossible to open source. And open sourcing the neu…
Open sourcing is in this is not about the implementation or the CS algorithm. It is more about transparency. I think the idea is that the public should know how tweets are ranked, why tweets show up in timelines and which timelines, what makes tweets popular etc. Imagine Google publishing a document detailing how their system ranks pages aka publishes internal SEO rules officially. I don't know if it is a good idea o…
The only change is that people with resource would stop guessing how to game the system, and start employing people to ensure they systematically game the system.
Re: twitter/the-algorithm
#223I've worked on very large scale recommendation systems at a FAANG. If Twitter's system resembles anything like ours, the concept of publishing or open sourcing "the algorithm" doesn't make sense. Even if we were to open source all associated code and publish all related documents it would be very difficult to make sense of the entire system. That is precisely why companies such as Twitter A/B test the hell out of eve…
It's so difficult as someone who works in technology to tolerate this idea that there is "an algorithm". Maybe this will get flagged on HN but the overwhelming feeling is to just scream "don't be such a fucking idiot, this is a $43Bn company it doesn't boil down to 50 lines of code" and hell, there are plenty of examples of 50 lines of code at my company I could spend a month really understanding and even then not un…
I don't think that most people actually believe "the algorithm" is like a Python script or something.
Re: twitter/the-algorithm
#224Re: twitter/the-algorithm
#225Earlier quoted context omitted.
Care to explain "thought bubble Brooklyn based reporters"? Did you choose Brooklyn for the alliteration? Or is there something about my home I should know about?
Large amount of blue checkmark journalists live there, simple as that.
But just imagine if those journalists had to live without twitter ? They would have to report things from "the real world" ! I've been there. It's awfully complicated, and sometimes people disagree with you and you have to talk. Insane.
Re: twitter/the-algorithm
#226I've worked on very large scale recommendation systems at a FAANG. If Twitter's system resembles anything like ours, the concept of publishing or open sourcing "the algorithm" doesn't make sense. Even if we were to open source all associated code and publish all related documents it would be very difficult to make sense of the entire system. That is precisely why companies such as Twitter A/B test the hell out of eve…
How did you work with a system that couldn't be understood even with access to full source and documentation? Surely your engineering process wasn't purely stochastic.
Re: twitter/the-algorithm
#227Earlier quoted context omitted.
Twitter Locks Down Product Changes After Agreeing to Musk Bid - https://finance.yahoo.com/news/twitter-locks-down-product-ch...
> Twitter imposed the temporary ban to keep employees who may be miffed about the deal from “going rogue,” according to one of the people. What could a rogue employee do?
[deleted]
Re: twitter/the-algorithm
#228Is this supposed to be a joke? It's clearly an empty repo. Either this is a mistake, or this is a really, really misguided attempt at a joke from Twitter.
The tweet announcing it was captioned "watch this space": https://twitter.com/willnorris/status/1518694675909013504 Which seems like a promise they intend to actually open source something there.
Who is that guy even ? Is he even associated with twitter ? Calling it an announcement seems like a stretch
Re: twitter/the-algorithm
#229I've worked on very large scale recommendation systems at a FAANG. If Twitter's system resembles anything like ours, the concept of publishing or open sourcing "the algorithm" doesn't make sense. Even if we were to open source all associated code and publish all related documents it would be very difficult to make sense of the entire system. That is precisely why companies such as Twitter A/B test the hell out of eve…
Even if publishing the underlying algorithm isn't useful, they should be able to do research on what the end result of endless a/b testing has created in practice. I'm sure you could find the highest level heuristics and that would be extremely valuable to both the public and decision makers internally.
You won’t know what they were A/B testing though.
Re: twitter/the-algorithm
#230I've worked on very large scale recommendation systems at a FAANG. If Twitter's system resembles anything like ours, the concept of publishing or open sourcing "the algorithm" doesn't make sense. Even if we were to open source all associated code and publish all related documents it would be very difficult to make sense of the entire system. That is precisely why companies such as Twitter A/B test the hell out of eve…
As you probably know, the algorithm is NOT the code, never has been.
It can be pseudo-code or diagram or whatever that can be used to understand what logic lies behind decision making.