Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TikTok reveals details of how its algorithm works

axios.com

Post reply on HN
Print view View original

101–110 of 190 posts

Re: TikTok reveals details of how its algorithm works

#101

Post by wenc »

johnchristopher wrote: post #88
undefined1 wrote: post #73

TikTok is like a highly compressed version of YouTube. YouTube incentivizes creators to make artificially long videos, resulting in a huge amount of filler. So you get a lot of videos where you can skip the first 20%+ and not miss anything. The content is buried and spread out. 1 minute of content surrounded by 9 minutes of filler. TikTok removed those 9 minutes, so it feels very refreshing in that respect. There's n…

> TikTok removed those 9 minutes, so it feels very refreshing in that respect. There's no incentive to create filler - just the opposite. It's to a point that I skip any TikTok video that starts like “I was in the forest and found this mushroom. Did you know that this kind of mushroom can be used to... ”. I skip it as soon as I hear “I was in the“ (that and the usual tone that comes with it). I don't care. Show me di…

Most people are familiar the standard narrative arc structure (which makes for compelling storytelling):

normalcy -> conflict -> (rising action to) climax -> resolution.

In flash fiction (i.e. very short fiction), the same elements exist, but due to brevity constraints, it starts the story in the middle, i.e. the point of conflict or climax, and then resolves the other parts along the way.

I think creators who understand how to exploit this structure could potentially make some really compelling short videos.

Example:

"For sale: baby shoes, never worn."

Re: TikTok reveals details of how its algorithm works

#102

Post by wsgeorge »

conradfr wrote: post #100
undefined1 wrote: post #73

TikTok is like a highly compressed version of YouTube. YouTube incentivizes creators to make artificially long videos, resulting in a huge amount of filler. So you get a lot of videos where you can skip the first 20%+ and not miss anything. The content is buried and spread out. 1 minute of content surrounded by 9 minutes of filler. TikTok removed those 9 minutes, so it feels very refreshing in that respect. There's n…

I guess that's why there is short video suggestions now in the Youtube app.

I noticed this just this week! I've found it refreshing watching 30-60 second clips of very random things I wouldn't care about otherwise.

I'm not sure if this has anything to do with my general preference for sub 5-minute videos that just get to the point.

Re: TikTok reveals details of how its algorithm works

#103

Post by sbilstein »

undefined1 wrote: post #73

TikTok is like a highly compressed version of YouTube. YouTube incentivizes creators to make artificially long videos, resulting in a huge amount of filler. So you get a lot of videos where you can skip the first 20%+ and not miss anything. The content is buried and spread out. 1 minute of content surrounded by 9 minutes of filler. TikTok removed those 9 minutes, so it feels very refreshing in that respect. There's n…

I wish I had data on the average length of youtube videos over time. I swear 5 years ago an explainer video or a game review would be less than 5 minutes consistently, which is what I personally want from a platform like youtube, a digestible snack of information. Nowadays career youtubers are often pushing well over 10 minutes which just occupies too much of my time to be worth the distraction.

Re: TikTok reveals details of how its algorithm works

#104

Post by wenc »

disgruntledphd2 wrote: post #87
pb7 wrote: post #51

Earlier quoted context omitted.

What surprises you about the unsupervised learning?

Normally supervised learning is much, much more effective.

I wonder if for random content discovery, unsupervised learning methods are less likely to be overconstrained by independent variables, and hence are freer to make better open-ended recommendations.

Supervised algorithms almost assume too much about a user -- they assume the correlation structures that are true in the past are also likely to hold in the future. This assumption holds in deterministic environments, but is false or unnecessarily constraining in stochastic domains, where it is widely known simple models and heuristics tend to perform much better.

I feel "new content discovery" is more of a stochastic problem rather than a deterministic one (which is the environment most conventional recommendation engines dwell in, hence most rec engines only make conservative recommendations).

For all we know TikTok's algorithm could simply be a combination of rules of thumb + simple clusters + randomization that happen to work well. I've seen so many instances (in real life) where simple models vastly outperform complex models in stochastic situations.

(p.s. in my opinion, YouTube's recommendations tend to be rather on the deterministic side. My recommendations seem to be mostly based on what I've watched and liked in the past and so the recs tend to be a bit boring. It clearly works great for YouTube from a monetization perspective, but it doesn't unearth a lot of interesting new content for me -- I have to search for those.)

Re: TikTok reveals details of how its algorithm works

#105

Post by milquetoastaf »

sbilstein wrote: post #103
undefined1 wrote: post #73

TikTok is like a highly compressed version of YouTube. YouTube incentivizes creators to make artificially long videos, resulting in a huge amount of filler. So you get a lot of videos where you can skip the first 20%+ and not miss anything. The content is buried and spread out. 1 minute of content surrounded by 9 minutes of filler. TikTok removed those 9 minutes, so it feels very refreshing in that respect. There's n…

I wish I had data on the average length of youtube videos over time. I swear 5 years ago an explainer video or a game review would be less than 5 minutes consistently, which is what I personally want from a platform like youtube, a digestible snack of information. Nowadays career youtubers are often pushing well over 10 minutes which just occupies too much of my time to be worth the distraction.

The answer, of course, is ads. You get more roll time if you hit the 10 min mark, even if its just slightly over

Re: TikTok reveals details of how its algorithm works

#106

Post by SergeAx »

londons_explore wrote: post #72
benreesman wrote: post #61

Earlier quoted context omitted.

Dwell is a very powerful ranking signal in a binary classifier. Pairwise association of videos watched by the same user consecutively or even just sampled pairs from their last N videos will get you a video embedding. Pairwise sampling of users who watch the same video to the end will get you a user embedding. Turking category tags will prime the pump for other types of embeddings. These things can be ensembles, stac…

What surprises me is all of what you've said is the way pretty much all recommenders work - YouTube, Facebook, Twitter, etc. will all be doing this. Yet people don't spend hours per day browsing tweets. What is it about tiktok which is so much more effective?

Facebook and Twitter are social-graph subscription-based services. TikTok just gives recomendations without subscriptions while knowing nothing about my social graph.

Re: TikTok reveals details of how its algorithm works

#107

Post by novok »

dirtyid wrote: post #92
johnchristopher wrote: post #88

Earlier quoted context omitted.

> TikTok removed those 9 minutes, so it feels very refreshing in that respect. There's no incentive to create filler - just the opposite. It's to a point that I skip any TikTok video that starts like “I was in the forest and found this mushroom. Did you know that this kind of mushroom can be used to... ”. I skip it as soon as I hear “I was in the“ (that and the usual tone that comes with it). I don't care. Show me di…

Yeah I wish TikTok has variable speed / fast forward option. Another factor for ML to train and automatically speed up to the juicy bits. There's a lot of room to waste even in 1 minute formats. Though TikTok itself is mostly time wasting and this remark is more an indictment of my attention span.

Is it possible to scrub videos / skip around in tiktok? I've tried googling it but to no avail.

Re: TikTok reveals details of how its algorithm works

#108

Post by novok »

jakear wrote: post #21
xnx wrote: post #18

Earlier quoted context omitted.

There's so much good meta-data (likes, comments, duration, sound used, views, like/view ratio, skips, loops, subscribes, etc.) that I'd be surprised if they were digging into the contents of the video at all right now.

They could also be digging only into audio, doing speech recognition on it, then clustering the text. Augment that with the text users have put into the video directly using the in-app editor and you have some pretty solid data.

I would also start doing image recognition in the video frames, to extract things like gender, objects, etc.

Re: TikTok reveals details of how its algorithm works

#109

Post by panopticon »

undefined1 wrote: post #73

TikTok is like a highly compressed version of YouTube. YouTube incentivizes creators to make artificially long videos, resulting in a huge amount of filler. So you get a lot of videos where you can skip the first 20%+ and not miss anything. The content is buried and spread out. 1 minute of content surrounded by 9 minutes of filler. TikTok removed those 9 minutes, so it feels very refreshing in that respect. There's n…

> YouTube incentivizes creators to make artificially long videos, resulting in a huge amount of filler.

Given how popular obnoxious jump cuts are on YouTube, I would assume this wasn't the case. What part of YouTube incentivizes this?

I always attributed the increased intros/outros/ad spot/filler lengths to the general trend of YouTube getting more "professional" than it did 10 years ago.

Re: TikTok reveals details of how its algorithm works

#110

Post by neves »

ramraj07 wrote: post #55
xnx wrote: post #9

Earlier quoted context omitted.

More data beats smarter algorithms any day, and TikTok gets a lot of data because its videos are so short and interaction rate so high. There are tons of signals it can use as inputs: How much of a video did you let play before swiping next? How many times did you let the video loop? Did you like? Did you comment? Did you like a comment? Did you click through to the profile? Did you view other videos from the profile…

I don't buy this argument. YouTube ostensibly has a metric ton of information like this and even if tiktok has more training data now, I'm fairly sure their training data in principle was smaller than what YouTube historically had given their decades long presence and their ubiquity in the internet. This is on top of the documented effort by YouTube to perfect their recommendation algorithm using the best ML minds th…

I just watched the sumptuous Gandhi movie and thought "Nice, now I'll search for some movies about the real Gandhi". After typing "Gand" in the remote control, there was Gandhi in the 5th, after searches about a more important historical figure: Gandalf.

The first video was titled in big letters "Was Gandhi a racist who spanked women?".

Make no evil. Duh!

Post reply on HN