He addresses this in his article, but you really can't evaluate competitors to your own service in such a manner. Not only were the testing criteria highly subjective, the fact that songs he considered 'WTF' in the lists were the negative marks. I know music is a very subjective subject, I really think he could have been more objective about the whole thing. Maybe using other peoples music collections and getting the…
How good is Google’s Instant Mix?
11–20 of 30 posts
Re: How good is Google’s Instant Mix?
#12He addresses this in his article, but you really can't evaluate competitors to your own service in such a manner. Not only were the testing criteria highly subjective, the fact that songs he considered 'WTF' in the lists were the negative marks. I know music is a very subjective subject, I really think he could have been more objective about the whole thing. Maybe using other peoples music collections and getting the…
Nitpick: an empty playlist has no songs out of place, so it should score 0 WTFs (I don't understand how you got length(empty)=24). Giving Apple some WTFs instead of giving it 0 is clearly more in the spirit of the test than following the criterion to the letter would have been.
Re: How good is Google’s Instant Mix?
#13He addresses this in his article, but you really can't evaluate competitors to your own service in such a manner. Not only were the testing criteria highly subjective, the fact that songs he considered 'WTF' in the lists were the negative marks. I know music is a very subjective subject, I really think he could have been more objective about the whole thing. Maybe using other peoples music collections and getting the…
While the author could have been more objective about his criteria, ironically enough, I think he missed the more salient point that he raised by implication: that music engines should be mapping the user's behavior patterns vis-a-vis the songs in his collection, and not so much objective connections between songs. This is what Genius does and has always tried to do, and it's why Genius seems to work better for the author. Genius doesn't focus as heavily on attempts to forge objective links between songs, so much as it focuses on attempts to draw links in behavior patterns w/r/t songs by likeminded users.
When listening to songs in a collection, our brain maps out its own connections between songs, as reflected in the way we compile our own lists consciously or subconsciously. Sometimes those connections make objective sense (i.e., "I want to listen to '70s funk, so I'm going to pick ten '70s funk songs in a row."). Sometimes those connections make little objective sense (i.e., "I am listening to a track by Lady Gaga, and afterward, I feel like listening to a track by J.S. Bach."). A good mixing engine figures out the idiosyncracies and subjectivity of our brains, as reflected statistically by the choices we've made in the past.
Re: How good is Google’s Instant Mix?
#14He addresses this in his article, but you really can't evaluate competitors to your own service in such a manner. Not only were the testing criteria highly subjective, the fact that songs he considered 'WTF' in the lists were the negative marks. I know music is a very subjective subject, I really think he could have been more objective about the whole thing. Maybe using other peoples music collections and getting the…
Nitpick: an empty playlist has no songs out of place, so it should score 0 WTFs (I don't understand how you got length(empty)=24). Giving Apple some WTFs instead of giving it 0 is clearly more in the spirit of the test than following the criterion to the letter would have been.
From my perspective, I'd say an empty playlist is worth 24 WTFs, in the sense of "WTF, I expected 24 similar songs and got 0!"
Re: How good is Google’s Instant Mix?
#15Re: How good is Google’s Instant Mix?
#16He addresses this in his article, but you really can't evaluate competitors to your own service in such a manner. Not only were the testing criteria highly subjective, the fact that songs he considered 'WTF' in the lists were the negative marks. I know music is a very subjective subject, I really think he could have been more objective about the whole thing. Maybe using other peoples music collections and getting the…
I think that if you survey the researchers in the field, the "WTF test" would be considered fairly reasonable - especially for a quick-and-dirty evaluation. Can you point to any specific songs that he said were WTF's and you think aren't, or vice/versa? If not then it would appear to meet the objectivity criteria.
Using his own music collection might be slightly more suspect. Changing that might have flipped the outcome of iTunes vs EchoNest, but wouldn't have changed the real news here: Google does really, really badly.
Re: How good is Google’s Instant Mix?
#17I have two thoughts on this: 1-As has been mentioned multiple times on HN recently, many of the best minds of our generation are working on the algorithms that go into products like genius. They are basically the same as recomendations, online advertisements, etc. Its not surprising that theyre really damn good at this point. 2-Google probably doesnt care about the product itself, more likely it just wants to mine th…
To what end? Unless Google is trying to make sure that when its software becomes sentient and takes over the world it has good taste in music (heh) I can't see much use for the data beyond building a recommender.
Re: How good is Google’s Instant Mix?
#18I have two thoughts on this: 1-As has been mentioned multiple times on HN recently, many of the best minds of our generation are working on the algorithms that go into products like genius. They are basically the same as recomendations, online advertisements, etc. Its not surprising that theyre really damn good at this point. 2-Google probably doesnt care about the product itself, more likely it just wants to mine th…
Re: How good is Google’s Instant Mix?
#19Earlier quoted context omitted.
Fair, but a few points: - Genius was much better than this when it launched. It's even better now, but it wasn't as bad back then as Instant Mix is today. - You don't think google has more data about music than Apple did when it launched Genius? YouTube, Music Onebox, search traffic. - (severe bias alert) EN's playlist APIs are the same excellent quality today as the day it launched (sept '10.) We're roughly 0.10% Go…
I'll absolutely agree that Google Music Instant Mix is nowhere where it needs to be.. not even close, but I still can't shake the feeling that this review is more of a "Please Buy Us" post... Just curious, in regard to the quality of EN since launching.. You guys we're clearly working on your algorithm for several years prior to launch[1] in late 2010.. That's sort of a warm-up period, no? [1] second last paragraph o…
Why do they think it is so important to get in this market ASAP? Perhaps under Page there is additional pressure to launch fast and early a la start-ups?
With that said, I have no idea if the quality of EN is as good as he claims.
Re: How good is Google’s Instant Mix?
#20Could this be because this service is so new that they haven't yet had the benefit of gathering lots of data and stats from users?
Let Google build their product first, it is still a beta, for a reason. Music recommendation is close to impossible without lots of data - and it becomes unfair to compare Google Music with iTunes Genius, which has been on the market in 3 years.