Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If You Liked This, Sure to Love That

nytimes.com

Post reply on HN
Print view View original

21–27 of 27 posts

Re: If You Liked This, Sure to Love That

#21

Post by ctkrohn »

dangoldin wrote: post #17

If anyone is interested in some Singular Value Decomposition type of work - take a look at Principal Component Analysis. http://en.wikipedia.org/wiki/Principal_components_analysis

PCA is an incredibly useful technique. At work we've been using it to model the structure of the yield curve, i.e. the graph of interest rates vs. maturity. Turns out you can decompose most daily movements of the yield curve into three components: parallel shift up/down, steepening/flattening, and a "bow" where 2s5s flattens, 5s10s steepens, and 10s30s flattens.. It would be interesting to build an interest rate model that evolves these three components forward in time... it would probably be most useful for short time scales where the principal components are unlikely to change.

Re: If You Liked This, Sure to Love That

#22

Post by kurtosis »

ctkrohn wrote: post #21
dangoldin wrote: post #17

If anyone is interested in some Singular Value Decomposition type of work - take a look at Principal Component Analysis. http://en.wikipedia.org/wiki/Principal_components_analysis

PCA is an incredibly useful technique. At work we've been using it to model the structure of the yield curve, i.e. the graph of interest rates vs. maturity. Turns out you can decompose most daily movements of the yield curve into three components: parallel shift up/down, steepening/flattening, and a "bow" where 2s5s flattens, 5s10s steepens, and 10s30s flattens.. It would be interesting to build an interest rate mode…

unrelated question: I've been recently been reading a lot about wavelets and multiscale analysis. My application area is in text processing and topic models for legal document analysis. Wavelet transforms or statistical modeling in the wavelet domain seems like the kind of thing that would have been tried many times over in finance. Do you know of any instances when it turns out to be useful useful for time series?

Re: If You Liked This, Sure to Love That

#23

Post by ctkrohn »

kurtosis wrote: post #22
ctkrohn wrote: post #21

Earlier quoted context omitted.

PCA is an incredibly useful technique. At work we've been using it to model the structure of the yield curve, i.e. the graph of interest rates vs. maturity. Turns out you can decompose most daily movements of the yield curve into three components: parallel shift up/down, steepening/flattening, and a "bow" where 2s5s flattens, 5s10s steepens, and 10s30s flattens.. It would be interesting to build an interest rate mode…

unrelated question: I've been recently been reading a lot about wavelets and multiscale analysis. My application area is in text processing and topic models for legal document analysis. Wavelet transforms or statistical modeling in the wavelet domain seems like the kind of thing that would have been tried many times over in finance. Do you know of any instances when it turns out to be useful useful for time series?

I've heard people talk about it, but never seen any concrete applications to finance. If you know of any papers or introductory material, I'd be thrilled to check it out. I know nothing about wavelets or multiscale analysis -- I couldn't even define them if you asked -- but I have a decent math and statistics background so I'd love to take a look.

Re: If You Liked This, Sure to Love That

#24

Post by kirse »

olavk wrote: post #19
jfornear wrote: post #7

Earlier quoted context omitted.

I would classify those all as Indie mainstream, stuff that is popular with the scenester, psuedo-intellectual, pretentious crowd.

No, it must be more complicated than that. If they were just consistently popular with a specific demographic, it would be easy to predict - the problem seem to be that even if you like Lost in Translation, you might still hate Kill Bill, or Fahrenheit 9/11.

After signing up for Netflix last month, I realized that I am one of those users that the algorithm writer's probably hate. I usually vote things "1" or "5", because a movie I did not like meant I just wasted 2 hours of my time. On the other hand, a movie I did like was an enjoyable and relaxing 2 hours.

So I rate everything to the polar ends, and also tend to have pretty varied choices for what movies I like. Usually the only genre I avoid is Horror/Psycho/Chainsaw Death films.

Re: If You Liked This, Sure to Love That

#25

Post by staunch »

randomwalker wrote: post #9
quan wrote: post #3

I worked on the Netflix Challenge last year but did not come far and gave up after submitting a few mediocre results. Actually, I was proud of myself for my result did not explode but was within the range of the Cinematch's algorithm. The experience has given me a lot of respect for companies that are developing recommendation algorithms. There should be more great competition like this. At this point, I'd say the be…

I saw an informal study on the economics of prizes. Apparently, the monetary value of the effort that is put into solving the, the media publicity, etc. added together exceeds the value of the prize itself by anywhere between a factor of 10 and 50, depending on the prize.

Sounds like a very informal study.

Re: If You Liked This, Sure to Love That

#26

Post by zupatol »

jfornear wrote: post #7
DaniFong wrote: post #2

That's hilarious. The list of movies that are hard to classify reads like a list of my favorites... [Like] “Napoleon Dynamite” — culturally or politically polarizing and hard to classify, including “I Heart Huckabees,” “Lost in Translation,” “Fahrenheit 9/11,” “The Life Aquatic With Steve Zissou,” “Kill Bill: Volume 1” and “Sideways.”

I would classify those all as Indie mainstream, stuff that is popular with the scenester, psuedo-intellectual, pretentious crowd.

I try hard to avoid judging people by their tastes.

I try to judge films by how I react to them, but this is difficult to do if at the same time I'm asking myself how others will react to my opinion of the film.

There are films I don't want to see and books I don't want to read because I heard statements like yours which I know would cloud my judgment and spoil my enjoyment.

Luckily, I already saw and liked enough of the films you dismiss to be able to safely ignore your condemnation of those I haven't seen yet.

Re: If You Liked This, Sure to Love That

#27

Post by jaytee_clone »

My first thought was - the rating system itself creates a ceiling to the accuracy of the prediction. And it seems that every one knows that. (Netflix or the 30,000 hackers.)

Then why spend so much resource to improve 10% of the existing rating system instead of experiment with new kinds of rating system. (I'm sure someone can come up with something clever yet simple.) Yes it's costly to change the infrastructure. But if you don't do it, some startup will come out and beat them to it.

The worst part is that Netflix is paying people to think inside-of-the-box (the rating system).

I read about Bertoni a while ago and was inspired by his out-of-the-box approach (Behavioral Economics). Wouldn't that give Netflix a hint - "Yo Netflix, here's this dude who's getting the fastest-growing result by extracting more qualitative information out of the quantitative rating system. Maybe you should just design a new rating system that better orgnizes these qualitative information? Just Maybe."

Or maybe I'm missing the point here?

Post reply on HN