Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

twitter/the-algorithm

github.com

Post reply on HN
Print view View original

261–270 of 403 posts

Re: twitter/the-algorithm

#261

Post by vrfvr »

actuator wrote: post #166
axg11 wrote: post #95

I've worked on very large scale recommendation systems at a FAANG. If Twitter's system resembles anything like ours, the concept of publishing or open sourcing "the algorithm" doesn't make sense. Even if we were to open source all associated code and publish all related documents it would be very difficult to make sense of the entire system. That is precisely why companies such as Twitter A/B test the hell out of eve…

Yeah, this makes no sense. There is no golden algorithm that Facebook, Tiktok or Twitter has figured out. All these feed rankings are complex combinations of features, models coupled with weights and filters. On top of this abuse detection layers are added. Unless Musk is planning to open source user data to show what all the "scores" and "features" for all the entities are and how they were reached to, this will mak…

Can you mention the source of your information?

Re: twitter/the-algorithm

#262

Post by lupire »

NaturalPhallacy wrote: post #247
mhh__ wrote: post #116

Earlier quoted context omitted.

Of course it doesn't make sense. I think it's just a dog whistle to the people who believe google have a guy in a room somewhere turning the "conservative search results" lever down a notch during elections.

A comparison I did from four months ago across google, ddg, yandex, and search.brave, searching for "are conservatives being silenced by big tech" https://i.imgur.com/MVlshAT.png You don't have to be conservative to see there's a pretty significant bias, just in the headlines. I'm a Pacific Green and I can still see it.

that's kind of a silly meta search query, not checking for bias in coverage of climate or race or religion or whatever.

Re: twitter/the-algorithm

#263

Post by mcintyre1994 »

seaman1921 wrote: post #228
koboll wrote: post #135

Earlier quoted context omitted.

The tweet announcing it was captioned "watch this space": https://twitter.com/willnorris/status/1518694675909013504 Which seems like a promise they intend to actually open source something there.

Who is that guy even ? Is he even associated with twitter ? Calling it an announcement seems like a stretch

His website claims he’s open source lead at Twitter: https://willnorris.com/

Re: twitter/the-algorithm

#264

Post by karmasimida »

axg11 wrote: post #95

I've worked on very large scale recommendation systems at a FAANG. If Twitter's system resembles anything like ours, the concept of publishing or open sourcing "the algorithm" doesn't make sense. Even if we were to open source all associated code and publish all related documents it would be very difficult to make sense of the entire system. That is precisely why companies such as Twitter A/B test the hell out of eve…

Let alone those 10s of GB of embeddings which might be user identifiable, without those, how do they claim to 'open source' the algorithm?

Re: twitter/the-algorithm

#265

Post by memish »

mcphage wrote: post #246
memish wrote: post #188

Earlier quoted context omitted.

"Among the elite, and within Twitter specifically, there is much more inclination to ban the right. I say this as someone whose political views, if you force them onto the left-right spectrum, probably end up about 80% toward the left. E.g. I've spent millions over the past several elections supporting the Democrats. It used to be that censorship was something the right did, and free speech was something the left wer…

Which side is on a book banning tear again?

Both! The difference is in what they target.

"On the right, banning books means trying to prevent their kids from reading them. But on the left it often means trying to prevent anyone from reading them." https://twitter.com/paulg/status/1515563419386191880

Re: twitter/the-algorithm

#266

Post by lupire »

briandear wrote: post #251
mhh__ wrote: post #116

Earlier quoted context omitted.

Of course it doesn't make sense. I think it's just a dog whistle to the people who believe google have a guy in a room somewhere turning the "conservative search results" lever down a notch during elections.

And that hasn’t happened? Google got fined by the EU for manipulating search results. It definitely happens. https://amp.usatoday.com/amp/1248099002 https://www.vox.com/2017/6/27/15878980/europe-fine-google-an...

That EU fine is a totally different issue, about defining the barrier between Google Search and other Google products.

Re: twitter/the-algorithm

#267

Post by michaelmrose »

KrishnaShripad wrote: post #159
axg11 wrote: post #95

I've worked on very large scale recommendation systems at a FAANG. If Twitter's system resembles anything like ours, the concept of publishing or open sourcing "the algorithm" doesn't make sense. Even if we were to open source all associated code and publish all related documents it would be very difficult to make sense of the entire system. That is precisely why companies such as Twitter A/B test the hell out of eve…

The openness for the algorithm is to make public how the algorithm works internally. It has nothing to do with how "novel" or "good" or "bad" the algorithm is. It is just a way to check if there is anything fishy going on in there. I don't see what the fuss is about if it only makes the defacto town square of the internet more transparent and open.

Twitter isn't the town square of the internet. The internet is the town square of the internet and you aren't owed a fair ranking in relation to a certain search term user or combination thereof because no singular definition of fair.

Re: twitter/the-algorithm

#268

Post by arriu »

suadeo wrote: post #163
axg11 wrote: post #95

I've worked on very large scale recommendation systems at a FAANG. If Twitter's system resembles anything like ours, the concept of publishing or open sourcing "the algorithm" doesn't make sense. Even if we were to open source all associated code and publish all related documents it would be very difficult to make sense of the entire system. That is precisely why companies such as Twitter A/B test the hell out of eve…

As someone who’s also worked in this area, I disagree with this take. There is typically clear objective function of a recommendation system. What Twitter is optimizing for is what’s of interest here. And some of the hidden business rules. It’s likely these are specified in the code in an obvious way. How exactly they achieve that is the part that is complex and relatively indecipherable. It’s possible that it’s desi…

Yeah seriously. If they are not able to put into words or diagrams what the algorithm is doing then the company itself has no idea how it works. And that to me would suggest it’s far from optimal.

Many complicated research papers have had no issues describing their models at a high level. This should be no different.

Re: twitter/the-algorithm

#269

Post by mellosouls »

akhmatova wrote: post #219
commandlinefan wrote: post #152

Earlier quoted context omitted.

> turning the "conservative search results" lever It may not have been algorithmic, but it definitely happened.

You know this how? Because you were that guy?

A quick Google (ironically) confirms this has happened, beyond the anecdotal evidence of people (including me) who remember it on several occasions.

https://www.mediaite.com/news/ex-google-engineer-says-glitch...

I'm not making any comment on the theory in the article, but it has certainly happened.

Edit: more context in thread here:

https://news.ycombinator.com/item?id=31161790

Re: twitter/the-algorithm

#270

Post by nickstinemates »

MetaWhirledPeas wrote: post #256
mhh__ wrote: post #116

Earlier quoted context omitted.

Of course it doesn't make sense. I think it's just a dog whistle to the people who believe google have a guy in a room somewhere turning the "conservative search results" lever down a notch during elections.

> I think it's just a dog whistle to the people who believe google have a guy in a room somewhere turning the "conservative search results" lever down a notch during elections. Even if that's precisely true, is it not good to be creating a more trusted space for everyone? The grievances, regardless of merit, are mostly coming from the right. If you want to create a service that caters to all you're going to have to a…

Perception is reality and right now the perception is a clear bias toward the left on Twitter. Anything that can be done to combat this fairly is a good thing.
Post reply on HN