Why You Should Not Build a Recommendation Engine

Ive been looking for an engine that can handle content-based recommendation

Directly giving the users predictions that are based on latent factors has a huge problem -- is there ever really a case where the latent factors have very high prediction ability, but an analyst can't simply see what's driving the preferences by looking at those factors, and then create something better by targeting that interaction directly?

Better to analyze the latent factors, and then use that analysis to gain engineering-type insights into the structure of the problem, and target specifically what's driving people to like something. Exactly as Netflix has been doing in recent years. Exactly as people have done when writing books or developing other entertainment content for centuries.

Just dumping unsupervised clustering on the end users is a poor technique that should stay in the 1990's where it belongs.

I think the key insight in this article is "A recommendation engine is a feature (not a product)". As a side project I built an e-commerce recommendation engine part time over 2-3 months. I didn't want to build out a massive product database and I didn't want to ask users to fill out lengthy profiles so I leveraged existing product and social APIs to handle both. It wasn't very difficult to get something that worked reasonably well - though competing solutions were mostly far worse so maybe I'm selling myself short. In the end it was decently well received (had some press, good user feedback, thousands of initial users) but it failed as a standalone business. Cost of customer acquisition vs customer lifetime value just wasn't there. I've seen no one else succeed in the space so I think the net takeaway is what I referenced at the top - it's really more of a feature than a standalone product. Building the world's best recommendation engine is really just a piece of the puzzle.

Algorithmic Recommendation system builder here.

I would include recommendation systems based on 'human generated lists'. See: http://www.google.com/patents/US8108417 I have developed these and they are far more powerful and relevant to the subjective user tastes involved with light-weight approaches related to collaborative filtering. An ensemble approach is absolutely the best while also keeping in mind that one user might think they've struck gold in terms of a result while another may not - recommendation systems are highly subjective. However, we had 50mil MAUs and 250mil searches every month and found out that there are ways to get around that. Some are psychological, for example, some users do not want to be 'told' or recommended something by someone else much less an algorithm - they would rather 'Discover' something. This depends on the product context. Is the product space related to music, shoes, books, dates etc. Labeling your recommendations as 'discoveries' works better in some cases, it depends on the user context and the product context.

Everything is a recommendation engine. Mimicking the way humans recommend things to other humans/friends is the ultimate way to build algorithms and architecture for recommendation systems.

One more thing, never forget to include a great spellchecking system, it's icing on the cake here.

I disagree. My company (ePantry) has only hundreds, not thousands, of distinct products, but making recommendations is incredibly important. Showing users the right product at the right time ensures they get the items they want, and it moves the needle on average basket size.

I tried a number of off-the-shelf recommendation systems, none of which worked for the comparatively dense matrix we have (since we only have a few hundred products). I finally rolled one in house which took a few days of work (thanks, redis!) and has a bunch of custom filters that OTS systems wouldn't ever have.

I hate building stuff like this when OTS is available, but everything was overkill, or too hard to configure, or initially gave bad results and required too much black-box tweaking. It's been a success and was way easier than I thought.