Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building the Next New York Times Recommendation Engine

open.blogs.nytimes.com

Post reply on HN
Print view View original

11–19 of 19 posts

Re: Building the Next New York Times Recommendation Engine

#11

Post by ThomPete »

volaski wrote: post #10
ThomPete wrote: post #7

The way I see it, the primary thing to solve for any recommendation engine is to optimize for serendipity. I.e. allowing you to get information you didn't know you wanted. This means basically also finding ex. articles that are not written by NYT. Newspapers problem is that their primarily omnibus approach to whats relevant isn't really doing the waste amount of insightful information available that exist out there.…

Maybe you had no idea but what you describe already exists in the form of native ads and recommendation widgets like outbrain. And here's what I do when I run into them: I rarely click them. When I'm on NYT, I don't want to click out to some "recommended" website that doesn't have high journalism integrity as NYT (Let's not get into a needless argument of whether that itself is correct or not). My point is, I disagre…

I don't think we are talking about the same thing.

To get sense of where I am coming from i would like to refer to some of my writing on the subject.

http://000fff.org/#/slaves-of-the-feed-this-is-not-the-realt...

and

http://000fff.org/#/how-to-think-like-facebook-and-twitter

It's about something slightly different than what you seem to imply, sorry if that was imprecise.

Re: Building the Next New York Times Recommendation Engine

#12

Post by sumitviii »

volaski wrote: post #10
ThomPete wrote: post #7

The way I see it, the primary thing to solve for any recommendation engine is to optimize for serendipity. I.e. allowing you to get information you didn't know you wanted. This means basically also finding ex. articles that are not written by NYT. Newspapers problem is that their primarily omnibus approach to whats relevant isn't really doing the waste amount of insightful information available that exist out there.…

Maybe you had no idea but what you describe already exists in the form of native ads and recommendation widgets like outbrain. And here's what I do when I run into them: I rarely click them. When I'm on NYT, I don't want to click out to some "recommended" website that doesn't have high journalism integrity as NYT (Let's not get into a needless argument of whether that itself is correct or not). My point is, I disagre…

ThomPete is talking about categories as silos probably. Just like most people come to HN for mostly startups and programming, but stay for nautil.us too. And serenpidity here is refreshing.

His idea makes sense.

Re: Building the Next New York Times Recommendation Engine

#13

Post by flashman »

I'm really impressed that NYT took the time to document this. It's always interesting to see the different recommendation models evaluated and applied to real-world situations.

I've been pursuing a collaborative filtering approach to product recommendation lately ('people who bought this also bought that'), but perhaps LDA would let me model our products based on their metadata ('people who bought products broadly like this also bought products broadly like that').

Re: Building the Next New York Times Recommendation Engine

#14

Post by eshamukhamedov »

ThomPete wrote: post #7

The way I see it, the primary thing to solve for any recommendation engine is to optimize for serendipity. I.e. allowing you to get information you didn't know you wanted. This means basically also finding ex. articles that are not written by NYT. Newspapers problem is that their primarily omnibus approach to whats relevant isn't really doing the waste amount of insightful information available that exist out there.…

These silos are the foundation of their business. No news organization is going to have the resources to cover every piece of information on the internet, so they must specialize. The alternative to this model is to have one organization control news syndication for a large number of independent contributors. This is the intent of Facebook Newsfeed, but there's little incentive for news outlets to publish to it. No matter how much money Facebook gives them, they can make more by building up their own "silo."

Re: Building the Next New York Times Recommendation Engine

#15

Post by ThomPete »

eshamukhamedov wrote: post #14
ThomPete wrote: post #7

The way I see it, the primary thing to solve for any recommendation engine is to optimize for serendipity. I.e. allowing you to get information you didn't know you wanted. This means basically also finding ex. articles that are not written by NYT. Newspapers problem is that their primarily omnibus approach to whats relevant isn't really doing the waste amount of insightful information available that exist out there.…

These silos are the foundation of their business. No news organization is going to have the resources to cover every piece of information on the internet, so they must specialize. The alternative to this model is to have one organization control news syndication for a large number of independent contributors. This is the intent of Facebook Newsfeed, but there's little incentive for news outlets to publish to it. No m…

Sure I am well aware of why they need to do it. But that need is also what is hindering them from providing actual value and why things like Facebook, Pulse, Twitter etc are popular news sources and newspapers aren't

Re: Building the Next New York Times Recommendation Engine

#16

Post by muktabh »

We make a contextual recommendation engine as a service for online publishers at our startup ParallelDots. We discovered the problem of tags not really working well for recommendations on our clients websites too. We ended up using unsupervised word embeddings and auto encoders on top of them to solve the problem. We dont still use it for personalization though, just contextually similar articles. Great seeing some of similar problems being solved at New York Times too. :)

Re: Building the Next New York Times Recommendation Engine

#18

Post by bydamn989 »

doppenhe wrote: post #17

We have built this and anybody can use it https://Algorithmia.com/recommends . 2 lines of js to implement.Currently serving the geekwire.com recs. You can also modify it further (see blog.Algorithmia.com). The article is awesome though good on NYT.

Nice product you have there. Not only did it take 10 minutes to run, it also managed to return no results.

Re: Building the Next New York Times Recommendation Engine

#19

Post by doppenhe »

bydamn989 wrote: post #18
doppenhe wrote: post #17

We have built this and anybody can use it https://Algorithmia.com/recommends . 2 lines of js to implement.Currently serving the geekwire.com recs. You can also modify it further (see blog.Algorithmia.com). The article is awesome though good on NYT.

Nice product you have there. Not only did it take 10 minutes to run, it also managed to return no results.

That should not be the case, sorry you had a not great experience. Please share the url you used at Diego at Algorithmia for com so we can debug and get back to you.
Post reply on HN