Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Simple recommendation system written in Ruby

otobrglez.opalab.com

Post reply on HN
Print view View original

11–20 of 21 posts

Re: Simple recommendation system written in Ruby

#11

Post by jbranchaud »

mazelife wrote: post #9
sixtosugarman wrote: post #2

You may use Levenshtein Distance to get better results by taking word variations into account. And also you can enhance it by using semantic similarity scores for strings.

If you're willing to get into actual NLP, then semantic similarity would certainly be one way to go. Is there any equivalent to Stanford (Java) or NLTK (Python) in Ruby land? But I'm not sure that Levenshtein will necessarily get you better results than the bag-of-words approach the author is taking with Jaccard distance, if all you're doing is document classification.

As far as NLP libraries in Ruby land, there is both [treat](https://github.com/louismullie/treat) and [ruby bindings to the Stanford Core NLP](https://github.com/louismullie/stanford-core-nlp).

Re: Simple recommendation system written in Ruby

#12

Post by otobrglez »

jbranchaud wrote: post #11
mazelife wrote: post #9

Earlier quoted context omitted.

If you're willing to get into actual NLP, then semantic similarity would certainly be one way to go. Is there any equivalent to Stanford (Java) or NLTK (Python) in Ruby land? But I'm not sure that Levenshtein will necessarily get you better results than the bag-of-words approach the author is taking with Jaccard distance, if all you're doing is document classification.

As far as NLP libraries in Ruby land, there is both [treat]( https://github.com/louismullie/treat ) and [ruby bindings to the Stanford Core NLP]( https://github.com/louismullie/stanford-core-nlp ).

I've used OpenNLP with jRuby for my NLP experiment. Check it out https://github.com/otobrglez/politiki-ner to get an idea how to mix it.

Re: Simple recommendation system written in Ruby

#13

Post by mck- »

manish_gill wrote: post #3

Programming Collective Intelligence is an excellent book for learning these sort of things. First chapter is a recommendation engine! :)

Great book indeed -- they talk about Collaborative filtering mostly, which suffers from the cold-start problem. If you need to build a recommendation algorithm that uses expert-knowledge (numerical features) you could use a simple kNN algorithm. [1] and [2] are two libraries I've written for this purpose.

[1] https://github.com/axiomzen/Alike [2] https://github.com/axiomzen/Look-Alike

Re: Simple recommendation system written in Ruby

#14

Post by jamra »

Here is a cool approach to the subject by Linked In. http://engineering.linkedin.com/open-source/cleo-open-source...

Here is my HN-obligatory, self-written golang version: https://github.com/jamra/gocleo

I went with this author's approach to use Jaccard to rank the results, however, I like this approach better: https://neil.fraser.name/writing/patch/ They basically take the distance to the beginning of the text into account.

Re: Simple recommendation system written in Ruby

#15

Post by snkcld »

for an approach using neo4j, check out cadet! (my project) cadet is more just a jruby wrapper around neo4j, but one can use it to interact with neo4j (and thus come up with recommendations without touching a line of java, or even cypher )

still in progress, and id love any input! http://github.com/karabijavad/cadet

http://github.com/karabijavad/congress-graph

Re: Simple recommendation system written in Ruby

#17

Post by wfn »

Good stuff, and a nice writeup/explanation!

To impudently hijack the thread: for a very similar approach (jaccard similarity coefficient, ruby) which has a nice abstracted implementation for background workers, take a look at David Celis 'recommendable' - here's him introducing the same system: http://davidcel.is/blog/2012/02/07/collaborative-filtering-w... and the gem itself: http://davidcel.is/recommendable/ I believe it's been discussed on HN before.

Redis is used to store the binary votes, and to compute similarity coefficients. Since redis is very good with set operations (intersections on multi-million-member sets (and more) are crazy fast), it's quite the natural choice for the db backend. One of the cases where a NoSQL solution seems to be the right tool for the job, as a matter of fact!

I've used recommendable (incl. in production code) in the past, it works very well, is reliable, robust, and easily hackable for whatever needs. (e.g. it's meant to integrate with Rails, but it's quite simple to make it work on barebones ruby, with (e.g.) Sinatra as a lightweight web app exposing vote functionality, and so on.)

Re: Simple recommendation system written in Ruby

#18

Post by otobrglez »

wfn wrote: post #17

Good stuff, and a nice writeup/explanation! To impudently hijack the thread: for a very similar approach (jaccard similarity coefficient, ruby) which has a nice abstracted implementation for background workers, take a look at David Celis 'recommendable' - here's him introducing the same system: http://davidcel.is/blog/2012/02/07/collaborative-filtering-w... and the gem itself: http://davidcel.is/recommendable/ I beli…

Thanks for the reference. David's post looks awesome!

Re: Simple recommendation system written in Ruby

#19

Post by tungwaiyip »

manish_gill wrote: post #3

Programming Collective Intelligence is an excellent book for learning these sort of things. First chapter is a recommendation engine! :)

Yes. And this is a recommendation system implementing the book's collaborative filtering algorithm in 9 lines of code

http://tungwaiyip.info/2012/Collaborative%20Filtering.html

Post reply on HN