The Quants Run Wall Street Now • Hacker News Board

Re: The Quants Run Wall Street Now

#81

Post by dsacco » Mon, May 22, 2017, 2:36 AM UTC

yourapostasy wrote: post #54

SirLJ wrote: post #49

Everyone can try to do it, but you need to spend the time and educate yourself like in any other profession or hobby and the first step is to go to eBay and search for historical stock market data, you can buy 20 years of data for less than $100 and you can test all trading ideas for free and without losing a single penny...the barrier for entry is very low, some Python knowledge + Linux machine and the data and off…

Why isn't this data out on torrents? Is it somehow copyrighted?

There are a few reasons.

1. Almost all buyers are institutional funds or individuals with the means to trade as well-informed investors. To put it succinctly, they're in it as serious business, because it's incredibly expensive. They have no incentive to make the edge they just purchased for five - six figures public.

2. These vendors go to various lengths to protect the data, including steganographic "trap streets" to identify the account the data belongs to.

3. There are significant legal liabilities inherent to releasing the data without a redistribution license. You agree to these explicitly by buying the data.

Re: The Quants Run Wall Street Now

#82

Post by SirLJ » Mon, May 22, 2017, 2:36 AM UTC

dsacco wrote: post #75

SirLJ wrote: post #49

Everyone can try to do it, but you need to spend the time and educate yourself like in any other profession or hobby and the first step is to go to eBay and search for historical stock market data, you can buy 20 years of data for less than $100 and you can test all trading ideas for free and without losing a single penny...the barrier for entry is very low, some Python knowledge + Linux machine and the data and off…

Dude, seriously...you have to stop peddling this ebay recommendation. No one is going to be competitive with data purchased on an auction website. In case anyone is wondering where they should actually get data to be competitive, I can personally speak for these two: * Nanex NxCore * CBOE Livevol I haven't purchased data from here, but I have heard it well recommended from people I trust: * TickData * QuantQuote Ther…

This the cheapest and most accurate I can find, if you have better source for $100 or less, please share with us...

Re: The Quants Run Wall Street Now

#83

Post by aswanson » Mon, May 22, 2017, 2:37 AM UTC

HappyTypist wrote: post #78

tptacek wrote: post #72

Earlier quoted context omitted.

There's nothing handwavy about the liquidity argument. Improving liquidity reduces the cost of trading for almost everyone in the market and thus makes almost everyone just a little wealthier. That means ordinary people, like schoolteachers and custodians, retire with a bit more money, and are a bit more comfortable; it means lots of people can donate just a bit more money to charitable causes without crossing whatev…

Not everyone at Google works on adtech. There are many people working on browsers that have sped up and innovated the web for hundreds of millions of users, or operating systems that are the foundations of many people's lives.

Also, one could say the value that Android has added as an open-source alternative to Ios has a value to society in the tens of billions.

Re: The Quants Run Wall Street Now

#84

Post by dsacco » Mon, May 22, 2017, 2:40 AM UTC

SirLJ wrote: post #82

dsacco wrote: post #75

Earlier quoted context omitted.

Dude, seriously...you have to stop peddling this ebay recommendation. No one is going to be competitive with data purchased on an auction website. In case anyone is wondering where they should actually get data to be competitive, I can personally speak for these two: * Nanex NxCore * CBOE Livevol I haven't purchased data from here, but I have heard it well recommended from people I trust: * TickData * QuantQuote Ther…

This the cheapest and most accurate I can find, if you have better source for $100 or less, please share with us...

My point is that data is not competitive. If you give me an example of a strategy you use with that data, I'll illustrate the issues with using data at the resolution ebay sells (to say nothing of things like survivorship bias...).

If you insist on data at this price point, why not just scrape Yahoo Finance? There are literally GitHub projects that obviate this for you.

Re: The Quants Run Wall Street Now

#85

Post by dsacco » Mon, May 22, 2017, 2:42 AM UTC

dluan wrote: post #65

My question for these new would-be quants is, how do you see your work? I know there are certain types of smart whizzes who see working for Google/Facebook/Amazon as some sort of intellectual step down, or where in finance the value to society is some handwavy "market-making" argument (depending on how philosophical you get), but presumably these types going into this are also tenure-track, research producing scienti…

Someone else mentioned pandas and AQR; I'll also mention BeakerX (Beaker Notebook) which was developed by Two Sigma: https://github.com/twosigma/beakerx

You can read more about it here, too: http://beakernotebook.com

Jane Street is also pretty awesome - they open sourced their in-house standard library for OCaml, called Base: https://github.com/janestreet/base (they use a lot of OCaml).

Re: The Quants Run Wall Street Now

#86

Post by tptacek » Mon, May 22, 2017, 2:43 AM UTC

HappyTypist wrote: post #78

tptacek wrote: post #72

Earlier quoted context omitted.

There's nothing handwavy about the liquidity argument. Improving liquidity reduces the cost of trading for almost everyone in the market and thus makes almost everyone just a little wealthier. That means ordinary people, like schoolteachers and custodians, retire with a bit more money, and are a bit more comfortable; it means lots of people can donate just a bit more money to charitable causes without crossing whatev…

Not everyone at Google works on adtech. There are many people working on browsers that have sped up and innovated the web for hundreds of millions of users, or operating systems that are the foundations of many people's lives.

I didn't say Google, I said adtech.

Re: The Quants Run Wall Street Now

#87

Post by SirLJ » Mon, May 22, 2017, 2:45 AM UTC

dsacco wrote: post #84

SirLJ wrote: post #82

Earlier quoted context omitted.

This the cheapest and most accurate I can find, if you have better source for $100 or less, please share with us...

My point is that data is not competitive. If you give me an example of a strategy you use with that data, I'll illustrate the issues with using data at the resolution ebay sells (to say nothing of things like survivorship bias...). If you insist on data at this price point, why not just scrape Yahoo Finance? There are literally GitHub projects that obviate this for you.

Please, if you'd like to help the community, just share a good data source... I would never share my winning strategy with some "know it all" on a public forum :-)

Re: The Quants Run Wall Street Now

#88

Post by whoisninja9 » Mon, May 22, 2017, 2:46 AM UTC

tptacek wrote: post #72

dluan wrote: post #65

My question for these new would-be quants is, how do you see your work? I know there are certain types of smart whizzes who see working for Google/Facebook/Amazon as some sort of intellectual step down, or where in finance the value to society is some handwavy "market-making" argument (depending on how philosophical you get), but presumably these types going into this are also tenure-track, research producing scienti…

There's nothing handwavy about the liquidity argument. Improving liquidity reduces the cost of trading for almost everyone in the market and thus makes almost everyone just a little wealthier. That means ordinary people, like schoolteachers and custodians, retire with a bit more money, and are a bit more comfortable; it means lots of people can donate just a bit more money to charitable causes without crossing whatev…

i agree with you and i think it is a good point but just one doubt i have that liquidity argument is correct assuming that this doesn't effect the fundamental pricing and cause bubbles (for ex.[oversimplified one] if lot of people followed trend following - it keeps driving bubbles in the long run - although i know counter argument is markets actually become efficient with more trading, but it is hard for me to buy into that 100% - i hope i'm wrong though). Also I think a decent gauge would be how much open-source has been contributed back and how much of it is valuable. Other things are hard to quantify like the example you gave, but I can't prove that you're wrong either.

Also when you develop something like physics, once proved - it always works with very good precision(reason we can launch satellites accurately). Or let's say a software or database that is supposed to work a certain way. What's being done at quant funds - I can't think of something on those lines that could be useful. Lot of these things don't always work because you're still speculating with large inaccuracy but a slight edge statistically.

Some interesting links:

http://fortune.com/2016/05/11/warren-buffett-hedge-fund-bet/

http://www.marketwatch.com/story/why-99-of-trading-is-pointl...

Re: The Quants Run Wall Street Now

#89

Post by dsacco » Mon, May 22, 2017, 2:52 AM UTC

whoisninja9 wrote: post #88

tptacek wrote: post #72

Earlier quoted context omitted.

There's nothing handwavy about the liquidity argument. Improving liquidity reduces the cost of trading for almost everyone in the market and thus makes almost everyone just a little wealthier. That means ordinary people, like schoolteachers and custodians, retire with a bit more money, and are a bit more comfortable; it means lots of people can donate just a bit more money to charitable causes without crossing whatev…

i agree with you and i think it is a good point but just one doubt i have that liquidity argument is correct assuming that this doesn't effect the fundamental pricing and cause bubbles (for ex.[oversimplified one] if lot of people followed trend following - it keeps driving bubbles in the long run - although i know counter argument is markets actually become efficient with more trading, but it is hard for me to buy i…

I'm not going to comment on the liquidity argument (it's a dead horse in my opinion), but another commenter and I mentioned a few examples of open source contributed by "quant" funds.

Re: The Quants Run Wall Street Now

#90

Post by dsacco » Mon, May 22, 2017, 2:54 AM UTC

whoisninja9 wrote: post #74

dluan wrote: post #65

My question for these new would-be quants is, how do you see your work? I know there are certain types of smart whizzes who see working for Google/Facebook/Amazon as some sort of intellectual step down, or where in finance the value to society is some handwavy "market-making" argument (depending on how philosophical you get), but presumably these types going into this are also tenure-track, research producing scienti…

none, their goal is to optimize profits. They haven't gotten to a point where investing in cutting edge tech only research is seen by them as a value add. Also for such projects they stay away from open source collaboration with outside world - so that would be really slow to develop something useful. Hope that will change in future.(very few exceptions - as someone commented pandas - pandas creator at two sigma actu…

To be honest with you, I consider most examples of companies releasing open source software to be commoditizing their complements and streamlining recruiting, not altruism.

The result can be altruistic, I guess, but my point is that judging intentions with regards to open source on the part of large companies is a fool's errand in my opinion.