Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Real World Recommendation System

blog.fennel.ai

Post reply on HN
Print view View original

21–30 of 156 posts

Re: Real World Recommendation System

#21

Post by chuckcode »

fmakunbound wrote: post #5

With all of this technology applied, I am still disappointed by Netflix's recommendations – to the point of just giving up and doing something else.

Do you think part of this is that Netflix has assumed zero effort from user model? My experience has been that Netflix does an ok job of recommendations, but fails at overall discovery experience. There is no way for me to drive or view content from different angles easily. I end up googling for expert opinions or hitting up rotten tomatoes to get better reviews. Netflix knows a ton about me and their content, but seems to do a poor job of making their content browseable/discoverable overall. I do like their "more like this" feature where I can see similar titles.

Re: Real World Recommendation System

#23

Post by BubbleRings »

Want to dive in to all this stuff but can't find a starting point? Start with reading my patent!

I was smart enough to see what collaborative filtering (CF) could be early on, and to file a patent that issued. I wasn't smart enough to make it a complicated patent, or to choose the right partners so I could have success with it.

But the patent makes a good way to learn how to get from "what are your desert island 5 favorite music recordings?" over to "here is a list of other music you might like". Basic CF, which is at the core of a lot of this stuff. Enjoy!:

https://whiteis.com/whiteis/SE/

Re: Real World Recommendation System

#25

Post by 1minusp »

nikhilgarg28 wrote: post #19
habibur wrote: post #13

> a machine learning model is trained that takes in all these dozens of features and spits out a score (details on how such a model is trained to be covered in the next post). This part was the one I was interested in. As most of the rest are obvious.

(Disclaimer: I'm the author of the post) Good feedback, noted. Will get the next post focused on training within the next couple of days.

Also, how is this article different or more informative compared to others that deal with the challenges of model deployment/management at scale?

Re: Real World Recommendation System

#27

Post by TheRealDunkirk »

dinobones wrote: post #18

How FAANG actually builds their recommendation systems: Millions of cores of compute, exabyte scale custom data stores. Good recommendations are expensive. If you try to build a similar system on AWS, you will spend a fortune. Most recommender models just use co-occurrence as a seed, this can actually work pretty well on it’s own. If you want to get fancy then build up a vectorized form of the document with something…

> Millions of cores of compute, exabyte scale custom data stores. Good recommendations are expensive. If you try to build a similar system on AWS, you will spend a fortune.

And then it will be gamed, and become as useless as every other recommendation system already going.

Re: Real World Recommendation System

#30

Post by ocrow »

These recommendation systems take control away from individuals over what content they see and replace that choice with black box algorithms that don't explain why you are seeing the content that you are or what other content was excluded. All of the companies who have deployed these content selection algorithms could have also given you manual choice over the content that you see, but chose instead to let the algorithm solely determine the content of your feed, either removing the manual option entirely or burying it so thoroughly that no one bothers to use it.

These algorithms are not benign. They make choices about what information you consume, whose opinions you read, what movies you watch, what products you are exposed to, even which politicians messages you hear.

When people complain about the takeover of algorithms, they don't mean databases or web interfaces. They mean this: content selection or preference algorithms.

We should be deeply suspicious. We should demand greater accountability. We should require that the algorithms explain themselves and offer alternatives. We should implement better. Give control back to the users in meaningful ways

If software engineering is indeed a profession, our professional responsibilities include tempering the damaging effects of content selection algorithms.

Post reply on HN