Live data from Hacker News

TikTok reveals details of how its algorithm works

axios.com

81–90 of 190 posts

Re: TikTok reveals details of how its algorithm works

#81
Does anyone know of an open source recommendation engine? I would love to just explore how one works bet it seems like this solely in the realm of proprietary software at the moment.

I have a dream of naively making a open source recommendation engine using data from good reads and imdb.

There is a real utility here that I worry will be captured entirely by companies without an open source alternative.

Re: TikTok reveals details of how its algorithm works

#82

TikTok is like a highly compressed version of YouTube. YouTube incentivizes creators to make artificially long videos, resulting in a huge amount of filler. So you get a lot of videos where you can skip the first 20%+ and not miss anything. The content is buried and spread out. 1 minute of content surrounded by 9 minutes of filler. TikTok removed those 9 minutes, so it feels very refreshing in that respect. There's n…

I watch a lot of YouTube and haven't really noticed any artificially long videos in my bubble of channels. I agree that intros are annoying (I have no idea why people do them), but other than that... it's just the usual video stuff. The nature of the format is that a one page blog post becomes a 10 minute video. People talk slower than they read, so it's somewhat unavoidable. There are also YouTubers that are really into the long form video essay format (hello, ContraPoints), but I don't think they're doing that to milk as much ad revenue as possible.

But again, that is just the random subset of YouTube that I watch. I doubt any of it is that popular, and I am always shocked when I get logged out and am on the default main page. So my experience could be totally off-base.

Re: TikTok reveals details of how its algorithm works

#83

Earlier quoted context omitted.

How many minutes have you spent watching TikTok?

i had an account for a few weeks.. followed a few folks. However my assessment is a video has much less revisit value than still images. The nature of a video requires much more time investment. Audience has to remain focused on a video for a period of time to achieve full reward. Unlike a still image, where a quick glance can gain satisfaction. These platforms can not sustain on pure novel content. The model of user…

None of these problems your listing are relevant to TikTok at all. You don't seem to be familiar with how TikTok works or how immensely popular is world wide.

Re: TikTok reveals details of how its algorithm works

#84

Earlier quoted context omitted.

How many minutes have you spent watching TikTok?

i had an account for a few weeks.. followed a few folks. However my assessment is a video has much less revisit value than still images. The nature of a video requires much more time investment. Audience has to remain focused on a video for a period of time to achieve full reward. Unlike a still image, where a quick glance can gain satisfaction. These platforms can not sustain on pure novel content. The model of user…

> However my assessment is a video has much less revisit value than still images.

I would say the same, only for text versus images. and yet for some reason I'll never understand besides being only in my early 30ies, people prefer images. and videos even more.

Re: TikTok reveals details of how its algorithm works

#85
post #11
post #8

It's a (seemingly) pretty simple matter of keeping track of how long someone looks at a video, and optimizing to show them videos they'll spend a long time looking at. It's pretty powerful too, I spend much more time looking at TikTok than any other digital content aggregator (to the extent I had to delete it from my phone). They also do some things like sprinkle in random fresh videos, potentially unrelated to your…

I've also found that TikTok does a good job of throwing in unexpected videos to test the waters of what the user is interested in. This diversity is one of my favorite things about it.

A common comment I see on Tiktok videos is something like "I've finally made it to [niche interest] tiktok", it's something I've experienced myself when the algorithm started showing me videos from the crew on cargo ships as they talk about their favorite ports to dock at, or their experience going through E.G the Panama Canal. I don't know any other platform that emergently figures out that I have an interest in international logistics without me specifically searching for it, and it's one of the reasons TikTok is such a great platform for content.

Re: TikTok reveals details of how its algorithm works

#86

Earlier quoted context omitted.

Dwell is a very powerful ranking signal in a binary classifier. Pairwise association of videos watched by the same user consecutively or even just sampled pairs from their last N videos will get you a video embedding. Pairwise sampling of users who watch the same video to the end will get you a user embedding. Turking category tags will prime the pump for other types of embeddings. These things can be ensembles, stac…

What surprises me is all of what you've said is the way pretty much all recommenders work - YouTube, Facebook, Twitter, etc. will all be doing this. Yet people don't spend hours per day browsing tweets. What is it about tiktok which is so much more effective?

You don't have that equivalent signal, that's what matters. i.e. Twitter doesn't know if you even finished reading the tweet which is what is providing a strong signal in the case of TikTok, skipped vs watched once vs watched multiple times.

Re: TikTok reveals details of how its algorithm works

#87
post #51

Earlier quoted context omitted.

I actually think it's got much more to do with the training data. Given that you are watching a video, you need to either swipe away from it, or finish watching it. This provides either a 1 or 0 for the video classification model. The important contrast here is with FB/IG feed where you can scroll aimlessly without engaging, leaving you with perhaps 1 engagement out of 10 (or whatever). The attached doc suggests that…

What surprises you about the unsupervised learning?

Normally supervised learning is much, much more effective.

Re: TikTok reveals details of how its algorithm works

#88

TikTok is like a highly compressed version of YouTube. YouTube incentivizes creators to make artificially long videos, resulting in a huge amount of filler. So you get a lot of videos where you can skip the first 20%+ and not miss anything. The content is buried and spread out. 1 minute of content surrounded by 9 minutes of filler. TikTok removed those 9 minutes, so it feels very refreshing in that respect. There's n…

> TikTok removed those 9 minutes, so it feels very refreshing in that respect. There's no incentive to create filler - just the opposite.

It's to a point that I skip any TikTok video that starts like “I was in the forest and found this mushroom. Did you know that this kind of mushroom can be used to... ”. I skip it as soon as I hear “I was in the“ (that and the usual tone that comes with it).

I don't care. Show me directly what you are doing with the mushroom.

Re: TikTok reveals details of how its algorithm works

#89
post #11

Earlier quoted context omitted.

I've also found that TikTok does a good job of throwing in unexpected videos to test the waters of what the user is interested in. This diversity is one of my favorite things about it.

A common comment I see on Tiktok videos is something like "I've finally made it to [niche interest] tiktok", it's something I've experienced myself when the algorithm started showing me videos from the crew on cargo ships as they talk about their favorite ports to dock at, or their experience going through E.G the Panama Canal. I don't know any other platform that emergently figures out that I have an interest in int…

Neat. Do you use/click on tags in video description ? (I am afraid the niche I found might get replaced by another niche).

Re: TikTok reveals details of how its algorithm works

#90
post #77
post #69

Earlier quoted context omitted.

Almost all of those applies to YouTube, do they not ?

IIRC youtube vids are too long to do any useful feature extraction from the videos.

The comment I was responding to mentioned a lot of metadata around videos, that is what I was responding to.
Post reply on HN