Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unsupervised machine learning with basket clusters

hackernoon.com

Post reply on HN
Print view View original

21–30 of 39 posts

Re: Unsupervised machine learning with basket clusters

#21

Post by surrey-fringe »

justadeveloper2 wrote: post #17
leereeves wrote: post #5

If this approach works, is it wise to be sharing it publicly? Is there a danger than whatever gains you were able to capture with a unique insight will end up being shared with competitors?

Here's how this always goes: Someone thinks they have a killer algorithm and it works and makes them money...for a time. Then it suddenly doesn't work anymore and they lose more than they gained. It is not possible to beat the market over the long term unless you are trading on insider information, somehow manipulating markets, front-running orders, or have some other edge that is not universally available. There is…

Why is it impossible to beat the market over long term without insider information?

Re: Unsupervised machine learning with basket clusters

#22

Post by sidlls »

eximius wrote: post #6
justadeveloper2 wrote: post #4

Earlier quoted context omitted.

It doesn't matter if you can do it once. You have to do it consistently over years, which is not possible. I know about Renaissance Technologies, but I suspect they have insider information or some other angle and it's not all about their algorithms.

That is a rather large accusation. Can you restate that as something constructive or helpful to the conversation?

I didn't interpret it as an accusation. I interpreted it as he wrote: a suspicion that something other than RT's algorithms is a non trivial contributor to their success. He gave an example of insider trading, but it could be something else ranging from the equally sinister lying about outcomes (e.g. by creative accounting of what counts as a return) to the less sinister better access or timing to information anybody else has that makes the algorithms more effective and far less relevant, and many other things.

His other point about the rarity of regularly beating the market certainly supports some amount of skepticism of RT, in my view. It's extraordinary, and it's an extraordinary claim that suddenly the particular talent at computational trading at RT is better than the legion of super genius other quants and techies who have been doing this for years.

Disclaimer: I haven't a dog in the fight with respect to supporting or taking RT down. As I see it, being a small time player, my trading strategies are necessarily constrained to the point where RT's methods, legit or not, aren't applicable. So whether they win or lose doesn't make a difference to me.

Re: Unsupervised machine learning with basket clusters

#23

Post by alextheparrot »

justadeveloper2 wrote: post #18
justadeveloper2 wrote: post #14

Earlier quoted context omitted.

Do you understand investing at all? People have been attempting to "beat the market" for decades and nobody has been successful. There is always the "reversion to the mean" issue in every case. People with algorithms think they are beating the market then something changes and their algorithm no longer works. Except for Renaissance--they have consistently been beating the market, or so it appears. Does that help?

AlexTheParrot--then home come they are the only one?

I'm unsure what to make of this.

Re: Unsupervised machine learning with basket clusters

#24

Post by amrrs »

rushabh wrote: post #2

So the author was able to beat S&P500 by 10% over a period from June-2016 to June-2017 using this solver. The most important question is whether the same underlying relationships will hold true for 2017-18. This seems like a classic example of hindsight. There are many things one can tell on hindsight when the results are out. The key is if they will hold for the future too. Am I missing something?

This is very much of how NNT taunts some reporter trying to explain the causality of Intra-day fluctuations of a Share. Humans obsession with establish wrong attribution in an attempt to express their domain expertise never expires! Now Data Science fuels it!

Re: Unsupervised machine learning with basket clusters

#25

Post by scottlegrand2 »

dajohnson89 wrote: post #19
justadeveloper2 wrote: post #14

Earlier quoted context omitted.

Do you understand investing at all? People have been attempting to "beat the market" for decades and nobody has been successful. There is always the "reversion to the mean" issue in every case. People with algorithms think they are beating the market then something changes and their algorithm no longer works. Except for Renaissance--they have consistently been beating the market, or so it appears. Does that help?

No, it doesn't help all. You have yet to provide any evidence for your accusation of insider trading (or "something else"). You can't really accuse RT of wrongdoing, just because you don't understand their methodology. Also -- if I were a quant that managed to beat the market consistently, I would shut up and go straight to RT, for two reasons. To avoid toxic dubious comments like yours, and because RT can pay me bet…

Except that Renaissance trading doesn't really want to hire quants. They wish to hire scientists and mold them in the Renaissance way, which is something you'd know if you were a quant.

http://www.reuters.com/article/simons-hedge-idUSN21355752200...

Re: Unsupervised machine learning with basket clusters

#26

Post by scottlegrand2 »

zenkat wrote: post #9
square90 wrote: post #8

I'm skeptical of the starmine dataset[0] being used: "The dataset is based on relationships between elements in the periodic table and public companies." This description is a bit vague. [0] http://starmine.ai/datasets/dataset_builder.html

I'm having trouble deciding if this whole post is a sly commentary on how easy it is to get machine learning wrong. Beat the market by 10% with k-means clustering and a feature set derived from companies and chemical elements! Hahahaha no.

I know, right? Now mapping it to the sequences coming from the numbers stations? Totally different story!

Re: Unsupervised machine learning with basket clusters

#27

Post by wumbolad »

surrey-fringe wrote: post #21
justadeveloper2 wrote: post #17

Earlier quoted context omitted.

Here's how this always goes: Someone thinks they have a killer algorithm and it works and makes them money...for a time. Then it suddenly doesn't work anymore and they lose more than they gained. It is not possible to beat the market over the long term unless you are trading on insider information, somehow manipulating markets, front-running orders, or have some other edge that is not universally available. There is…

Why is it impossible to beat the market over long term without insider information?

Because by definition in order to make money you are abusing a market inefficiency. If you use this inefficiency, you're effectively shrinking the inefficiency to closer to nothing every time its used.

Re: Unsupervised machine learning with basket clusters

#28

Post by IshKebab »

rushabh wrote: post #2

So the author was able to beat S&P500 by 10% over a period from June-2016 to June-2017 using this solver. The most important question is whether the same underlying relationships will hold true for 2017-18. This seems like a classic example of hindsight. There are many things one can tell on hindsight when the results are out. The key is if they will hold for the future too. Am I missing something?

Yeah an the fact that he starts the post with "I finally beat the S&P 500 by 10%." suggests he has basically just tried a lot of techniques on historical data until he found one that worked. Almost certainly luck.

Re: Unsupervised machine learning with basket clusters

#29

Post by scottlegrand2 »

lightbyte wrote: post #12
leereeves wrote: post #5

If this approach works, is it wise to be sharing it publicly? Is there a danger than whatever gains you were able to capture with a unique insight will end up being shared with competitors?

Sharing it publicly is the author saying it does not actually work as he claims it does. You don't write a blog post on a method you created to make guaranteed money, you use it.

A couple years back there were some traders talking about trying to implement a really fast version of K-means for clustering stocks that are transiently correlated on some sub-reddit. I wonder if this guy is the one who started that thread.

Re: Unsupervised machine learning with basket clusters

#30

Post by srean »

alextheparrot wrote: post #15
justadeveloper2 wrote: post #14

Earlier quoted context omitted.

Do you understand investing at all? People have been attempting to "beat the market" for decades and nobody has been successful. There is always the "reversion to the mean" issue in every case. People with algorithms think they are beating the market then something changes and their algorithm no longer works. Except for Renaissance--they have consistently been beating the market, or so it appears. Does that help?

He simply asked you to backup your claim, which you don't seem to be able to otherwise you would have. Throwing around a "yeah they can do it, but they must be cheating" isn't exactly constructive.

I have no dog in the fight, but generally it is the extraordinary claims that need backing up.

Consistently beating the market would indeed be an extraordinary claim, because we havent seen that a lot (if at all).

Post reply on HN