Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Clickbaitification of Netflix

slate.com

Post reply on HN
Print view View original

131–140 of 313 posts

Re: The Clickbaitification of Netflix

#131

Post by pdinny »

personjerry wrote: post #7

I worked at Facebook for some time, and did a bunch of data work. We had this culture of building something, looking at the results between the experiment groups, and then choosing the statistically more successful one -- i.e. a newsfeed algorithm that had better engagement. This sounds great at first, and certainly is straightforward if you want a promotion. But behind the scenes some of us had this thought that our…

> The employees are incentivized to do so and this is what they measure, so this is what they build

I also work in data teams and this is directly my observation too.

Closely related are Conway's Law (you ship your org chart) and Goodhart's Law (commonly paraphrased as 'When a measure becomes a target, it ceases to be a good measure.')

In Netflix's case I certainly feel the shipped product is too close of a reflection of org structure and incentives and user experience suffers as a result.

This feels particularly evident in the treatment of thumbnails and title descriptions. Often I find myself clicking through on a thumbnail in order to read the description only to find that the I've already read the description and didn't want to watch that title, but the thumbnail has since changed.

I'm sure there is some team in Netflix whose sole purpose is increasing thumbnail clickthrough rates. And they are probably succeeding in that respect by changing the thumbnails. They get to win at their portion of some funnel, even if the net result is a lousy user experience.

Re: The Clickbaitification of Netflix

#132

Post by kqr »

dr_dshiv wrote: post #74
tomnipotent wrote: post #32

Earlier quoted context omitted.

> The data/engineering/product loops at tech companies favor boosting short-term metrics I think this hits the nail on the head. Leadership is hard, and executives can be many layers removed from the frontline. It seems that A/B tests and family have proven a really effective vehicle to communicate across leadership levels what has happened, why it happened, and the impact it had on the business. I'm definitely guilt…

How might a streaming service be designed to increase user happiness and wellbeing? From a positive design perspective, could certain streaming features give users a greater sense of purpose or goal accomplishment (e.g., mastering a genre) or help users meaningfully connect to others (e.g., seeing what friends have recommended lately)? While measuring the effects of a feature on wellbeing outcomes is substantially ha…

> How might a streaming service be designed to increase user happiness and wellbeing?

The rational part of my brain would like

- Suggestions for difficult content (this tends to be things you either love or hate, which is why Netflix does not suggest it, preferring safe bets);

- Making it impossible to binge on one, or a small set, of things, instead forcing diversification;

- Encourage spending short amounts of time -- e.g. by automatically breaking up movies into episodes, not autoplaying the next episode, disincentivising stupid cliffhangers, etc.

I realise many of these things are actively user hostile and a service implementing them would be dumped in favour of the alternatives faster than they have time to rollback the commit. Including by myself, when I just want a low-effort way to pass the time with my wife until it's socially acceptable to go to bed.

And that's sort of, I guess, the point. In order to increase user well being, a service like this probably has to encourage the user to spend less time with it. But that's not a business model that generates any money.

Re: The Clickbaitification of Netflix

#133

Post by mdoms »

avnigo wrote: post #60

Veritasium talks about thumbnail and title experimentation on YouTube, and argues for the need to do so on platforms that surface up and recommend content to users: https://www.youtube.com/watch?v=S2xHZPH5Sng

Christ that guy comes across as an arrogant prick. Here, "Veritasium", let me rephrase your shitty, overly long, condescending video: "I deceive my viewers because it makes me more money".

Re: The Clickbaitification of Netflix

#134

Post by bertil »

1123581321 wrote: post #49
darkwizard42 wrote: post #6

I believe Netflix has talked about thumbnail experimentation at length in some of their engineering blogs [1]. To me, it seems perhaps Netflix has figured out a running theme in the author's viewing habits... perhaps the scandalous, soapy stuff is what they keep clicking. Same kind of reaction I get when a guy friend tells me they keep getting ads for women's bikinis or women's underwear in their Instagram ads. It is…

That response to complaining about Instagram is becoming a frustrating meme because it isn’t true that Instagram doesn’t offer sex appeal to the uninitiated, and rather than discussing that issue, this simple response just shuts down criticism of the social network by embarrassing the person who brings the issue up. Lest you turn the meme on me, I’ll say don’t have an account.

I don’t think this response is dismissive: it’s a technical explanation of what is happening. Understanding it helps us find a solution.

Netflix (of Facebook) rates content along embeddings, and highlights content along embeddings that users engage with. If a given user engages with clips featuring people in bikini, that system won’t pass the judgement (one that the author is passing, a judgement you dearly regret, ‘Lest…’) — but one that the commenter didn’t pass. No one in that process (at Facebook, Instagram or Netflix Tech team) is choosing to make things sluttier, or asks someone to bend over a little more (Netflix being a film and series producer too, I’ll reserve there).

Given that context, if, like the author, you believe that the content should not engage along certain dimensions then there’s several possible takes:

* a paternalistic approach to downgrade dimensions that are deemed unhelpful; that would have been popular in my Catholic school, less so on Hacker News, I’d expect;

* a empowering approach that either:

  * name the dimensions and let users downgrade them; that’s harder than one would expect for many reasons, but roughly: naming is power and embeddings are not human-legible;

  * let users downvote some recommendation, even if they’ve engaged with it.
Both Facebook and Netflix but also YouTube, Google Search engine, all have strangely avoided building those negative feedback for click-baits. It many ways, it’s a mystery to me, especially as someone who built so many recommendation systems. I’ve always used implicit or explicit negative feedback, to just engagement and the recommendations were meaningfully better.

I’d love to hear from people who have explored that approach if there have encountered issues implementing them. I can share two details that are not sufficient but I hope are relevant:

* The biggest problem at Facebook has always been that Likes were performative: your friends saw them, by design; people refused to use them as a way to improve their recommendation because of that. When we discussed a drop in posting, or when the five other reactions where debated, there were several suggestion to make a parallel form of Like invisible. I’ve seen those ideas being shot down repeatedly; I was never told why. I assumed there was a fear those would cannibalise the public Likes, and those drove a lot of engagement opportunities, but that was speculation on my part.

* Most effort to understand recommendations (“Why am I seeing this?”) had an abysmal engagement rate. So much that maintaining it wasn't worth the effort.

Re: The Clickbaitification of Netflix

#135

Post by bertil »

slowmotiony wrote: post #62
darkwizard42 wrote: post #6

I believe Netflix has talked about thumbnail experimentation at length in some of their engineering blogs [1]. To me, it seems perhaps Netflix has figured out a running theme in the author's viewing habits... perhaps the scandalous, soapy stuff is what they keep clicking. Same kind of reaction I get when a guy friend tells me they keep getting ads for women's bikinis or women's underwear in their Instagram ads. It is…

A friend of mine was hired to do design work for Netflix - he ended up having to create tens of different thumbnails for every show, which netflix would then run through their algorithms and decide which one to show to which user. I've looked at my netflix page and it seems that some silicon valley psychologist decided that they should be showing me thumbnails of guys with sad faces looking at something far away. Alm…

Does your friend work with User experience researcher? I know it sounds ridiculous to delegate “this looks creepy” to an expert, but that’s where those people are valuable: they can enrich an impression with detailed feedback.

Also, if your friend has any sway, can you beg him to suggest a “Stop recommending this to me” button, if they haven’t. The platform is better at removing movies I’ve stoped watching from the “Continue watching” selection after a week but… that’s still a bit long. I just want the agency of telling the machine No on occasion.

And if that works, thankfully, we can get a “stop showing me the same image” button too.

Re: The Clickbaitification of Netflix

#136

Post by thomasahle »

kqr wrote: post #132
dr_dshiv wrote: post #74

Earlier quoted context omitted.

How might a streaming service be designed to increase user happiness and wellbeing? From a positive design perspective, could certain streaming features give users a greater sense of purpose or goal accomplishment (e.g., mastering a genre) or help users meaningfully connect to others (e.g., seeing what friends have recommended lately)? While measuring the effects of a feature on wellbeing outcomes is substantially ha…

> How might a streaming service be designed to increase user happiness and wellbeing? The rational part of my brain would like - Suggestions for difficult content (this tends to be things you either love or hate, which is why Netflix does not suggest it, preferring safe bets); - Making it impossible to binge on one, or a small set, of things, instead forcing diversification; - Encourage spending short amounts of time…

> I just want a low-effort way to pass the time with my wife until it's socially acceptable to go to bed.

Is it really social pressure that prevents you from going to bed earlier?

Re: The Clickbaitification of Netflix

#137

Post by piva00 »

vasco wrote: post #8
personjerry wrote: post #7

I worked at Facebook for some time, and did a bunch of data work. We had this culture of building something, looking at the results between the experiment groups, and then choosing the statistically more successful one -- i.e. a newsfeed algorithm that had better engagement. This sounds great at first, and certainly is straightforward if you want a promotion. But behind the scenes some of us had this thought that our…

They don't seem to have fallen off, are still the most widely used platforms in the world with billions of users in total. It is popular to think that A/B testing leaves you in local maxima, but what about alternatives? The only thing I've seen work is something like Pixar's brain trust - you do testing but you also rely on the good judgement of a small group of people with a strong sense of vision. Judging good judg…

> It is popular to think that A/B testing leaves you in local maxima, but what about alternatives?

The alternative involves creativity to create a vision to achieve. A/B testing are pretty good tools to evaluate differing opinions on some vision, as the only deciding tool for what is worth to keep or not it's pretty short-sighted.

Is it really that much better? I lived through the rise of A/B testing, professionally I've been responsible for implementing platforms for experimentation in at least 5 different companies and over the years I saw it transition from a tool to a crutch. Everything now is "data-based" which means incessant A/B testing, it still depends on your average PM deciding the parameters of the experiment, leading to the current state of affairs: a bunch of uninspired features that are solely judged on the data produced, no soul, no vision behind most of it.

And it feels exactly like that, I can feel how tech products just became that, an incessant stream of features being tested, implemented or tossed.

I'd much rather have data-supported intuition than this statistical machine in motion.

Re: The Clickbaitification of Netflix

#138

Post by snakeboy »

karmasimida wrote: post #55
keiferski wrote: post #24

Earlier quoted context omitted.

This is the consequence of lack of vision. What is Netflix's long term goal? Ideally, it would be to fund and broadcast fantastic cinema. Yet, I really don't get the impression that contributing to the art form of film is very high up on the metrics chart.

> contributing to the art form of film Has netflix ever committed to this vision? I would be hell bored if Netflix is filled with Oscar-ish films. One thing to note here is, the tech community is inherently elitist, in a way many envision tech as a tool to doctrine its audience, telling them what is good or bad. I don't find this mindset, either helpful nor necessary. Recommendation engine is the new soda machine, ch…

> Has netflix ever committed to this vision? I would be hell bored if Netflix is filled with Oscar-ish films.

I think it's a bit of straw man to conflate "the art form of film" with mainstream oscar bait.

Re: The Clickbaitification of Netflix

#139

Post by account42 »

krige wrote: post #37
matheusmoreira wrote: post #12

Earlier quoted context omitted.

Copyright infringement makes everything ever created available for $0 a month. If streaming services can't do the same, they suck and are failing to compete.

At the core it's a matter of convenience. When there's only one service to pay for or even two or three most people don't have a problem with paying a fee to have access to everything they want to watch. But when there's fifteen services to pay for, even if they collectively cost less than the above example, many simply give up and turn back to piracy.

> At the core it's a matter of convenience.

Conveninece also means different things for different people. For me is includes being able to re-watch my favorite content whenever I want and on whatever device I want - no exeptions - even if the the internet is down, even if the creator no longer wants to make it available. The only thing that can guarantee that is something that gives me DRM-free files that I can manage however I want. Unfortunately for movies and shows noone provides this - best non-piracy option are physical discs but even those are so filled with anti-piracy measures (and other questionable technical decisions) which only serve to make the experience worse than piracy.

Re: The Clickbaitification of Netflix

#140

Post by denton-scratch »

pdinny wrote: post #131
personjerry wrote: post #7

I worked at Facebook for some time, and did a bunch of data work. We had this culture of building something, looking at the results between the experiment groups, and then choosing the statistically more successful one -- i.e. a newsfeed algorithm that had better engagement. This sounds great at first, and certainly is straightforward if you want a promotion. But behind the scenes some of us had this thought that our…

> The employees are incentivized to do so and this is what they measure, so this is what they build I also work in data teams and this is directly my observation too. Closely related are Conway's Law (you ship your org chart) and Goodhart's Law (commonly paraphrased as 'When a measure becomes a target, it ceases to be a good measure.') In Netflix's case I certainly feel the shipped product is too close of a reflectio…

> and title descriptions

To my mind, it's the title descriptions that are the most clickbaity. I get the sense that Netflix are beding over backwards to ensure that you can't form an impression of the flic from that description.

I cancelled my Netflix subscription. Less and less of the titles are well-known 3rd-party productions; more and more are Netflix productions that I have found to be rather dull. Their ability to "divine my preferences" was never much good, but it's got worse.

Of course, they don't really care about my preferences; what matters for Netflix is what they would prefer me to watch, apparently.

Youtube's ability to discern my preferences also seems to have nosedived over the last year. I watched one programme about a certain WWII aircraft; YT then started pushing at me an unending diet of aircraft restoration docs and amateur historian channels.

Post reply on HN