Munich? cool, I'm probably not so alone :)
Embarrassing Flaws That Made My ‘Weekend’ App Possible
61–69 of 69 posts
Re: Embarrassing Flaws That Made My ‘Weekend’ App Possible
#62Munich? cool, I'm probably not so alone :)
Call in at http://combinat56.com sometime; there's always a couple of devs doing interesting stuff there.
Re: Embarrassing Flaws That Made My ‘Weekend’ App Possible
#63Re: Embarrassing Flaws That Made My ‘Weekend’ App Possible
#64Earlier quoted context omitted.
+1 to this... I've had very good luck using SVD/LSA-ish approaches for building recommendation engines. HOWEVER- the big catch with naïve (i.e., simple to implement and understand) approaches to LSA are that every time you add a new datum, you have to recompute the SVD of your data- which is time consuming, and raises some interesting logistical challenges when done in the context of a web app (i.e., how you persist…
I'd love any resources you could point to that were helpful to you when builder your recommendation engines (also, do you have links to the actual engines?). I've been going through a tutorial to understand how SVD works [1], but it would be nice to find something that will help me deal with implementation details. I thought I might just use Gensim [2] for now, but the authors of that tool themselves say that Gensim…
Sorry for the delay, I just now saw that somebody had replied. Regarding links to the systems- the only one that was ever publicly accessible was for a Facebook app that I wrote some years back that was let researchers post links to their publications on their profiles, and also featured a recommendation system to find other users of the app that had "similar" publications. It's unfortunately fallen into a pretty sorry state of disrepair, as I haven't had any time to devote to maintenance in more than two years and Facebook's APIs have changed quite a bit since then. I described the underlying system a little bit in a conference paper: http://view.ncbi.nlm.nih.gov/pubmed/18999247
The other systems I've used the approach for have all been along either bibliometric/bibliographic lines, or have been relating to content-based image retrieval. It's a pretty robust approach, but can take a little bit of tuning to get just right- coming up with a good evaluation strategy is important to getting the most out of it, I've found.
As far as references that were useful:
Ilya Grigorik has a very accessible getting-your-feet-wet tutorial on his site: http://www.igvita.com/2007/01/15/svd-recommendation-system-i...
It might be a little dated w.r.t. specific libraries or APIs, but the basic technique is there. For a more comprehensive look at the SVD-IR approach, take a look at:
Berry et al. Using Linear Algebra for Intelligent Information Retrieval. SIAM Review (1995) vol. 37 (4) pp. 573-595
The SVD approach falls in the same family as Latent Semantic Analysis, which is a whole black art unto itself- I'd actually suggest going back to the early papers by Landauer, Dumais, etc. if you're really interested- those guys did a great job writing up what was at the time really novel stuff.
My contact info should be in my profile, drop me an email if you have any more questions (or to let me know what you end up doing!).
Re: Embarrassing Flaws That Made My ‘Weekend’ App Possible
#65Nice writeup. Did you consider using Google's Prediction API before rolling your own recommendation system? http://code.google.com/apis/predict/
No. I thought about Directed Edge, but playing with recommendation algorithms was the raison d'etre of the project. I didn't want to make a web site and throw some amazon referral links at it, I wanted to play data miner!
That's great! http://techcrunch.com/2010/09/15/hunch-taste-graph-business-... is another alternative. Maybe you could try tweaking your algo continuously and comparing with how well hunch recommends stuff. The advantage of the hunch api is that it is able to use a lot more info about the visitor to make more awesome predictions. Just curious to see how far it goes!!! http://hunch.com/developers/v1/
Re: Embarrassing Flaws That Made My ‘Weekend’ App Possible
#66Earlier quoted context omitted.
Call in at http://combinat56.com sometime; there's always a couple of devs doing interesting stuff there.
I know about combinat56 but their website and their self presentation turned me a bit off.
Shame, it's a really nice place to work. Nice Sumo sacks, too. There's a couple of HNers work there; I'm around most afternoons.
Re: Embarrassing Flaws That Made My ‘Weekend’ App Possible
#67Earlier quoted context omitted.
Do you find that it helps? The closest thing I got that looks like what you got above there are my git log messages!
Seconded. Using distributed RCS, you commit often and therefore have a pretty good log of your daily activities.
I don't commit anything with
"Talked to client" "Wrote email" "Made blog entry" "Spent 60 minutes wasting time doing X"
Also, the gaps in the timeline are just as telling. I always feel that little bit guilty returning from HN and seeing a 1hr gap where I could have worked.
Re: Embarrassing Flaws That Made My ‘Weekend’ App Possible
#68In the very oldest version of digg to be found on the internet archive, if you view source there's an sql query inside an html comment. Embarrassment, trapped in amber forever.
digg could remove themselves from archive.org using robots.txt if they wanted to
Re: Embarrassing Flaws That Made My ‘Weekend’ App Possible
#69Earlier quoted context omitted.
+1 to this... I've had very good luck using SVD/LSA-ish approaches for building recommendation engines. HOWEVER- the big catch with naïve (i.e., simple to implement and understand) approaches to LSA are that every time you add a new datum, you have to recompute the SVD of your data- which is time consuming, and raises some interesting logistical challenges when done in the context of a web app (i.e., how you persist…
I'd love any resources you could point to that were helpful to you when builder your recommendation engines (also, do you have links to the actual engines?). I've been going through a tutorial to understand how SVD works [1], but it would be nice to find something that will help me deal with implementation details. I thought I might just use Gensim [2] for now, but the authors of that tool themselves say that Gensim…
heya binomial, don't let gensim's disclaimer deter you! it's been used by many people, both academically and commercially, and it's fairly mature.
having said that, i'm always happy to hear about success and failure stories from users. so if there's something you find broken/missing in gensim, just let me know.