TikTok reveals details of how its algorithm works • Hacker News Board

Re: TikTok reveals details of how its algorithm works

#11

Post by xnx » Thu, Sep 10, 2020, 2:15 PM UTC

jakear wrote: post #8

It's a (seemingly) pretty simple matter of keeping track of how long someone looks at a video, and optimizing to show them videos they'll spend a long time looking at. It's pretty powerful too, I spend much more time looking at TikTok than any other digital content aggregator (to the extent I had to delete it from my phone). They also do some things like sprinkle in random fresh videos, potentially unrelated to your…

I've also found that TikTok does a good job of throwing in unexpected videos to test the waters of what the user is interested in. This diversity is one of my favorite things about it.

Re: TikTok reveals details of how its algorithm works

#12

Post by mc32 » Thu, Sep 10, 2020, 2:15 PM UTC

Kednicma wrote: post #2

Here's the secret sauce: "Using machine learning, the algorithm serves videos to users based on their proximity to other clusters of users and content that they like." It's impressive how they arranged to be transparent for everything else, but kept this important part opaque. Rumor is that the actual secret sauce here is human curation; people hand-select videos with high appeal and label them "viral", "popular", et…

If this is true... then imagine if you had an unofficial business relationship with these curators to tag your videos over other similar videos. Obviously you don’t want them to be too obvious but when many with good viral quality just need a bump that could be enough to generate lots of revenue to split.

Re: TikTok reveals details of how its algorithm works

#13

Post by basch » Thu, Sep 10, 2020, 2:16 PM UTC

ChefboyOG wrote: post #4

I don't see any details about its algorithm, just that they use a recommendation engine? Things like this aren't proprietary info, they're just how recommendation engines work: "Once TikTok collects enough data about the user, the app is able to map a user's preferences in relation to similar users and group them into "clusters." Simultaneously, it also groups videos into "clusters" based on similar themes, like "bas…

This article was popular here when it was published. https://www.eugenewei.com/blog/2020/8/3/tiktok-and-the-sorti...

Re: TikTok reveals details of how its algorithm works

#14

Post by albertshin » Thu, Sep 10, 2020, 2:16 PM UTC

Kednicma wrote: post #2

Here's the secret sauce: "Using machine learning, the algorithm serves videos to users based on their proximity to other clusters of users and content that they like." It's impressive how they arranged to be transparent for everything else, but kept this important part opaque. Rumor is that the actual secret sauce here is human curation; people hand-select videos with high appeal and label them "viral", "popular", et…

if that rumor holds any water, that would be an impressive team of curators who have a pretty extraordinary ability to spot* globally favorable trends. I remember reading when FB tried to employ human curators for their news section, bias creeped in and was scrapped away. wonder what TikTok did differently.

* although with popularity of TikTok, one needs to wonder if that team have become THE trendsetters themselves...

Re: TikTok reveals details of how its algorithm works

#15

Post by gcmac » Thu, Sep 10, 2020, 2:16 PM UTC

ChefboyOG wrote: post #4

I don't see any details about its algorithm, just that they use a recommendation engine? Things like this aren't proprietary info, they're just how recommendation engines work: "Once TikTok collects enough data about the user, the app is able to map a user's preferences in relation to similar users and group them into "clusters." Simultaneously, it also groups videos into "clusters" based on similar themes, like "bas…

I’d say it’s more likely they have super advanced/clever ways of doing the latter. The algorithm could be a simple dot product and the result could be great or terrible depending on how good the feature extraction is.

Pulling useful features out of videos is no small task. The fact that everyone raves about how good the recommendations are indicates to me that this is where their innovation lies.

Re: TikTok reveals details of how its algorithm works

#16

Post by simion314 » Thu, Sep 10, 2020, 2:17 PM UTC

Kednicma wrote: post #2

Here's the secret sauce: "Using machine learning, the algorithm serves videos to users based on their proximity to other clusters of users and content that they like." It's impressive how they arranged to be transparent for everything else, but kept this important part opaque. Rumor is that the actual secret sauce here is human curation; people hand-select videos with high appeal and label them "viral", "popular", et…

It mentions that it also serves you from time to time a video from a different cluster and analyzes your reaction so you are not stuck in one cluster without hope to get out.

Re: TikTok reveals details of how its algorithm works

#17

Post by xnx » Thu, Sep 10, 2020, 2:17 PM UTC

Matticus_Rex wrote: post #10

coldcode wrote: post #6

Maybe I am just strange, but I find no appeal in TikTok, and would not like something that just shows me more of the same things. But I can see how it might appear to people who just want to be stimulated with stuff.

I thought this would be the case for me, but after trying it I've been really impressed with the creativity and content quality. The algorithm quickly figured out I didn't care about teenagers dancing and started feeding me cool crafts and comedy, and my faith in Gen Z has grown dramatically.

Agreed. TikTok has very good built-in video tools, but the creativity and sense of storytelling in short form videos on the part of these kids is very impressive.

Re: TikTok reveals details of how its algorithm works

#18

Post by xnx » Thu, Sep 10, 2020, 2:20 PM UTC

gcmac wrote: post #15

ChefboyOG wrote: post #4

I don't see any details about its algorithm, just that they use a recommendation engine? Things like this aren't proprietary info, they're just how recommendation engines work: "Once TikTok collects enough data about the user, the app is able to map a user's preferences in relation to similar users and group them into "clusters." Simultaneously, it also groups videos into "clusters" based on similar themes, like "bas…

I’d say it’s more likely they have super advanced/clever ways of doing the latter. The algorithm could be a simple dot product and the result could be great or terrible depending on how good the feature extraction is. Pulling useful features out of videos is no small task. The fact that everyone raves about how good the recommendations are indicates to me that this is where their innovation lies.

There's so much good meta-data (likes, comments, duration, sound used, views, like/view ratio, skips, loops, subscribes, etc.) that I'd be surprised if they were digging into the contents of the video at all right now.

Re: TikTok reveals details of how its algorithm works

#19

Post by gazelleeatslion » Thu, Sep 10, 2020, 2:21 PM UTC

Does anyone actually trust any of this nonsense?

At this scale, TikTok, Google, YouTube, Twitter, Facebook.. it should legally be required to either:

- Open Source It

- Provide clear option to opt-out

- Provide the params to be 100% configurable

Wake up people!

Re: TikTok reveals details of how its algorithm works

#20

Post by rsa25519 » Thu, Sep 10, 2020, 2:21 PM UTC

coldcode wrote: post #6

Maybe I am just strange, but I find no appeal in TikTok, and would not like something that just shows me more of the same things. But I can see how it might appear to people who just want to be stimulated with stuff.

> would not like something that just shows me more of the same things

Me neither. Good thing TikTok provides variety and helps users discover new things.

> But I can see how it might appear to people who just want to be stimulated with stuff.

Don't we all? Is that not why we're on Hacker News, to find stimulation targeted more specifically to us than other websites?