Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Appsaurus (YC 07): A Smarter Recommendation Tool Than App Store Genius

wired.com

Post reply on HN
Print view View original

1–10 of 14 posts

Re: Appsaurus (YC 07): A Smarter Recommendation Tool Than App Store Genius

#6

Post by dschobel »

amichail wrote: post #3

Have the recommendations of Appsaurus been demonstrated to be better than the app store Genius?

Any such demonstrations would be totally anecdotal though.

There's no way to say its recommendations are quantitatively better, you just have to try it and see whether their machine learning stuff is better than Apple's ML stuff.

Re: Appsaurus (YC 07): A Smarter Recommendation Tool Than App Store Genius

#7

Post by amichail »

dschobel wrote: post #6
amichail wrote: post #3

Have the recommendations of Appsaurus been demonstrated to be better than the app store Genius?

Any such demonstrations would be totally anecdotal though. There's no way to say its recommendations are quantitatively better, you just have to try it and see whether their machine learning stuff is better than Apple's ML stuff .

Any such demonstrations would be totally anecdotal though.

Why not conduct a proper experiment complete with statistical significance calculations?

Re: Appsaurus (YC 07): A Smarter Recommendation Tool Than App Store Genius

#8

Post by pgbovine »

amichail wrote: post #7
dschobel wrote: post #6

Earlier quoted context omitted.

Any such demonstrations would be totally anecdotal though. There's no way to say its recommendations are quantitatively better, you just have to try it and see whether their machine learning stuff is better than Apple's ML stuff .

Any such demonstrations would be totally anecdotal though. Why not conduct a proper experiment complete with statistical significance calculations?

who should be the judges, though?

Re: Appsaurus (YC 07): A Smarter Recommendation Tool Than App Store Genius

#9

Post by emmett »

amichail wrote: post #7
dschobel wrote: post #6

Earlier quoted context omitted.

Any such demonstrations would be totally anecdotal though. There's no way to say its recommendations are quantitatively better, you just have to try it and see whether their machine learning stuff is better than Apple's ML stuff .

Any such demonstrations would be totally anecdotal though. Why not conduct a proper experiment complete with statistical significance calculations?

Yes, you could definitely gather non-anecdotal, statistical evidence, but I think you're missing dschobel's main point. I think he's making a point about the inherently subjective nature of "quality of recommendation", not suggesting the obviously stupid idea that it's impossible to gather statistical data on a recommendation engine.

And to that point, I have to agree with him. At least, most of the obvious choices for a quality measurement seem quite flawed. Here are the first few I thought of:

- Apps downloaded (this is more about how good you are at getting people to donwload an app, and recommendations are only a small part of that)

- Subjective direct rating (averaging subjective experience doesn't tell you what's good, it just tells you what people perceive as good)

- Length of usage (measures how much fun you made the rating "game", not how good your recommendations are)

This isn't an accident. Recommendation quality is inherently tricky, because if you knew how to figure out what you should recommend to any given person...that would be your recommendation engine! So you're always using a proxy for goodness, rather than a direct measurement, and that abstraction tends to leak.

Re: Appsaurus (YC 07): A Smarter Recommendation Tool Than App Store Genius

#10

Post by amichail »

emmett wrote: post #9
amichail wrote: post #7

Earlier quoted context omitted.

Any such demonstrations would be totally anecdotal though. Why not conduct a proper experiment complete with statistical significance calculations?

Yes, you could definitely gather non-anecdotal, statistical evidence, but I think you're missing dschobel's main point. I think he's making a point about the inherently subjective nature of "quality of recommendation", not suggesting the obviously stupid idea that it's impossible to gather statistical data on a recommendation engine. And to that point, I have to agree with him. At least, most of the obvious choices f…

How did Netflix evaluate submissions to its competition?
Post reply on HN