How useful was the Netflix Prize challenge for Netflix? • Hacker News Board

Re: How useful was the Netflix Prize challenge for Netflix?

#21

Post by alberth » Wed, Jul 07, 2021, 1:42 AM UTC

squarecog wrote: post #11

alberth wrote: post #2

What’s odd is that the person responding in Quora who’s speaking as if he has authority on this matter started at Netflix in 2011, 2 years after the contest ended. The contest started in 2006 and was awarded in 2009. https://en.m.wikipedia.org/wiki/Netflix_Prize

Why is this odd? If he started two years later and there was not a trace of the Prize work at the company, that would be an indicator that the competition was not important. If he started and could still see knock-on effects from the competition, that's an indicator that it was important. Plus, he didn't just start at Netflix. He "took over the small team that was working and maintaining the rating prediction algorit…

Could it not be something more simple like, Netflix didn’t originally have profiles. So my child watching kid shows and me watching action shows were all feeding into the same recommendation system resulting in subpar results. That and they originally had a 5 star rating system which they then dropped.

Its very possible Netflix realized they needed to course correct the UX and as a result the winners algorithm was solving for a problem that no longer applied because it was using assumptions (rating system & no existence of different profiles) that were no longer relevant.

Re: How useful was the Netflix Prize challenge for Netflix?

#22

Post by alberth » Wed, Jul 07, 2021, 1:44 AM UTC

dimitrios1 wrote: post #18

alberth wrote: post #17

Earlier quoted context omitted.

His LinkedIn profile says otherwise. 2011-2017 was his time at Netflix. https://www.linkedin.com/in/xamatriain

Everyone knows you can't lie on your LinkedIn profile, too.

Why would the person lie about working at another company (Telefonica) from 2007-2011.

Re: How useful was the Netflix Prize challenge for Netflix?

#23

Post by spicyramen » Wed, Jul 07, 2021, 1:48 AM UTC

Back in 2010 my wife was doing her master's and Netflix recruited MBA students to help them with their business model for streaming. The task was to understand if DVD made sense and if streaming was too risky. It panned out pretty well and the rest is history

Re: How useful was the Netflix Prize challenge for Netflix?

#24

Post by trutannus » Wed, Jul 07, 2021, 1:49 AM UTC

alberth wrote: post #2

What’s odd is that the person responding in Quora who’s speaking as if he has authority on this matter started at Netflix in 2011, 2 years after the contest ended. The contest started in 2006 and was awarded in 2009. https://en.m.wikipedia.org/wiki/Netflix_Prize

Quora's culture is odd in general. Writing "authoritative" (even they're just LARPing as an expert) is basically why people go there in the first place. It's not uncommon to read something egregiously inaccurate written as indisputable fact. Good examples can be found in anything to do with history.

On a funny note, Jordan Peterson used to be a well known Quora answer writer prior to his current career as an internet celebrity. Source: https://www.quora.com/profile/Jordan-B-Peterson

Re: How useful was the Netflix Prize challenge for Netflix?

#25

Post by Supermancho » Wed, Jul 07, 2021, 1:52 AM UTC

BB212 wrote: post #20

I think the discussion misses the most important part: The goal of the Netflix prize wasn't to come up with the best algorithm - it was to make the Netflix brand exciting and legitimate to engineers. At the time, Netflix wasn't super high-tech and I'm sure it was hard for them to get the top talent they needed. It seems silly in retrospect now, but I'm certain the reason this was approved was because they wanted the…

> At the time, Netflix wasn't super high-tech

I will admit that it was interesting to see what algorithms were poised to be cutting edge in media recommendation. The result was rather disappointing to me.

Netflix STILL isn't that exciting from anything but a compensation standpoint. The problems at netflix are about programming, while the technical challenges are droll at best.

Re: How useful was the Netflix Prize challenge for Netflix?

#26

Post by verst » Wed, Jul 07, 2021, 1:55 AM UTC

BB212 wrote: post #20

I think the discussion misses the most important part: The goal of the Netflix prize wasn't to come up with the best algorithm - it was to make the Netflix brand exciting and legitimate to engineers. At the time, Netflix wasn't super high-tech and I'm sure it was hard for them to get the top talent they needed. It seems silly in retrospect now, but I'm certain the reason this was approved was because they wanted the…

Source?

It was my understanding that there was significant business value in improving the accuracy of personalized movie recommendations. Recall that this was at a time where the majority of the business was DVDs sent via mail. A poor choice of movie created significant risk to customer satisfaction and hence retention.

Re: How useful was the Netflix Prize challenge for Netflix?

#27

Post by mikeyouse » Wed, Jul 07, 2021, 1:59 AM UTC

trutannus wrote: post #24

alberth wrote: post #2

What’s odd is that the person responding in Quora who’s speaking as if he has authority on this matter started at Netflix in 2011, 2 years after the contest ended. The contest started in 2006 and was awarded in 2009. https://en.m.wikipedia.org/wiki/Netflix_Prize

Quora's culture is odd in general. Writing "authoritative" (even they're just LARPing as an expert) is basically why people go there in the first place. It's not uncommon to read something egregiously inaccurate written as indisputable fact. Good examples can be found in anything to do with history. On a funny note, Jordan Peterson used to be a well known Quora answer writer prior to his current career as an internet…

Ha - nice find. All sorts of the typical sad self-promotion you find on Quora.

Re: How useful was the Netflix Prize challenge for Netflix?

#28

Post by setr » Wed, Jul 07, 2021, 2:00 AM UTC

Supermancho wrote: post #25

BB212 wrote: post #20

I think the discussion misses the most important part: The goal of the Netflix prize wasn't to come up with the best algorithm - it was to make the Netflix brand exciting and legitimate to engineers. At the time, Netflix wasn't super high-tech and I'm sure it was hard for them to get the top talent they needed. It seems silly in retrospect now, but I'm certain the reason this was approved was because they wanted the…

> At the time, Netflix wasn't super high-tech I will admit that it was interesting to see what algorithms were poised to be cutting edge in media recommendation. The result was rather disappointing to me. Netflix STILL isn't that exciting from anything but a compensation standpoint. The problems at netflix are about programming, while the technical challenges are droll at best.

IMO the recommendations are no good because they fundamentally take the wrong approach — rather than ask the user what they like, they try to guess what you like based on usage (which really doesn't correlate well — I watch a lot of garbage because I can’t find things I like, and I don’t have anything better to do.)

And they don’t ask because users don’t provide useful answers.

But users don’t provide useful answers, because rating things doesn’t do anyone any good.

I’m of the belief that if you can make ratings useful (catalogue all movies, including not on Netflix; give useful ways to view/update your lists; have direct relationships to recommendations), you would have dramatically better recommendations for dramatically less effort/complexity.

I don’t think you’ll ever get to “good” recommendations based on usage. The data is fundamentally garbage.

Of course, the other side is that Netflix isn’t interested in recommending things I like; their goal is to recommend things I’ll put up with. They just need 1 show worth watching and subscribing for every now and then, and N shows to keep me mildly amused to stop me from dropping it between good ones

Re: How useful was the Netflix Prize challenge for Netflix?

#29

Post by IncRnd » Wed, Jul 07, 2021, 2:03 AM UTC

I was a solver on a different team for the netflix challenge. Our team didn't win the grand prize.

I would have expected that a blog post would discuss how this was structured. Netflix contracted with innocentive.com, which is a website for solvers, and contracting to that website expanded Netflix's reach to a greater available pool of solvers. As far as I recall, all the allowed solvers for the netflix challenge _had_ to go through innocentive. I'm not sure if they would have been able to get the same level of improvement if they had not contracted with a set of potential solver teams like that.

The original challenge listing for Netflix is no longer listed at innocentive.com, but an industrious person may be able to find it on archive.org or somewhere similar.

Re: How useful was the Netflix Prize challenge for Netflix?

#30

Post by d0lphin » Wed, Jul 07, 2021, 2:12 AM UTC

verst wrote: post #26

BB212 wrote: post #20

I think the discussion misses the most important part: The goal of the Netflix prize wasn't to come up with the best algorithm - it was to make the Netflix brand exciting and legitimate to engineers. At the time, Netflix wasn't super high-tech and I'm sure it was hard for them to get the top talent they needed. It seems silly in retrospect now, but I'm certain the reason this was approved was because they wanted the…

Source? It was my understanding that there was significant business value in improving the accuracy of personalized movie recommendations. Recall that this was at a time where the majority of the business was DVDs sent via mail. A poor choice of movie created significant risk to customer satisfaction and hence retention.

It's a quote from the linked post