Live data from Hacker News

The Quants Run Wall Street Now

wsj.com

71–80 of 165 posts

Re: The Quants Run Wall Street Now

#71
post #62
post #49

Everyone can try to do it, but you need to spend the time and educate yourself like in any other profession or hobby and the first step is to go to eBay and search for historical stock market data, you can buy 20 years of data for less than $100 and you can test all trading ideas for free and without losing a single penny...the barrier for entry is very low, some Python knowledge + Linux machine and the data and off…

If you are going to use machine learning on the data, though, make sure you know what you are doing unless you are just using someone else's complete package. It's really easy to screw up machine learning. I recall an example given in a class I took. (I may be misremembering the details, though). Some people were trying to apply machine learning to currency trading. They had a bunch of data. They normalized the data…

I've read about a similar scenario (possibly the same one?), and you got the important bits right.

You cannot normalize the entire dataset at once; the normalization contains some parity effectively allowing the algorithm to cheat and see the future... but in real life we can't see the future.

The simple example that eliminates some of the "black magic" is that normalizing against the entire set lets the algorithm know what the highest and lowest points across the entire data set are - and knowing the lowest and highest lets the algorithm buy low/sell high for all of the known data.

Re: The Quants Run Wall Street Now

#72
post #65

My question for these new would-be quants is, how do you see your work? I know there are certain types of smart whizzes who see working for Google/Facebook/Amazon as some sort of intellectual step down, or where in finance the value to society is some handwavy "market-making" argument (depending on how philosophical you get), but presumably these types going into this are also tenure-track, research producing scienti…

There's nothing handwavy about the liquidity argument. Improving liquidity reduces the cost of trading for almost everyone in the market and thus makes almost everyone just a little wealthier. That means ordinary people, like schoolteachers and custodians, retire with a bit more money, and are a bit more comfortable; it means lots of people can donate just a bit more money to charitable causes without crossing whatever financial safety line they have, &c.

It's hard to make the same kind of argument about adtech.

Re: The Quants Run Wall Street Now

#73
post #65

My question for these new would-be quants is, how do you see your work? I know there are certain types of smart whizzes who see working for Google/Facebook/Amazon as some sort of intellectual step down, or where in finance the value to society is some handwavy "market-making" argument (depending on how philosophical you get), but presumably these types going into this are also tenure-track, research producing scienti…

pandas was first developed at AQR Capital.

http://pandas.pydata.org/community.html

Re: The Quants Run Wall Street Now

#74
post #65

My question for these new would-be quants is, how do you see your work? I know there are certain types of smart whizzes who see working for Google/Facebook/Amazon as some sort of intellectual step down, or where in finance the value to society is some handwavy "market-making" argument (depending on how philosophical you get), but presumably these types going into this are also tenure-track, research producing scienti…

none, their goal is to optimize profits. They haven't gotten to a point where investing in cutting edge tech only research is seen by them as a value add. Also for such projects they stay away from open source collaboration with outside world - so that would be really slow to develop something useful. Hope that will change in future.(very few exceptions - as someone commented pandas - pandas creator at two sigma actually is pushing for open-source, i think it is great but i hope it is not just a gimmick to attract better talent).

Re: The Quants Run Wall Street Now

#75
post #49

Everyone can try to do it, but you need to spend the time and educate yourself like in any other profession or hobby and the first step is to go to eBay and search for historical stock market data, you can buy 20 years of data for less than $100 and you can test all trading ideas for free and without losing a single penny...the barrier for entry is very low, some Python knowledge + Linux machine and the data and off…

Dude, seriously...you have to stop peddling this ebay recommendation. No one is going to be competitive with data purchased on an auction website.

In case anyone is wondering where they should actually get data to be competitive, I can personally speak for these two:

* Nanex NxCore

* CBOE Livevol

I haven't purchased data from here, but I have heard it well recommended from people I trust:

* TickData

* QuantQuote

There are likely others, but bear in mind that the higher the data quality, the higher the price. For most strategies you'll likely want vetted intraday data, preferably at minute resolution or lower. Tick data is better, but it's going to be huge (my drives are near 100TB). You can reformat it into custom bar sizes if you have data on the actual trades/quotes.

Re: The Quants Run Wall Street Now

#76
post #73
post #65

My question for these new would-be quants is, how do you see your work? I know there are certain types of smart whizzes who see working for Google/Facebook/Amazon as some sort of intellectual step down, or where in finance the value to society is some handwavy "market-making" argument (depending on how philosophical you get), but presumably these types going into this are also tenure-track, research producing scienti…

pandas was first developed at AQR Capital. http://pandas.pydata.org/community.html

Cool, thanks, that's what I was hoping to learn about.

Re: The Quants Run Wall Street Now

#77

Anyone have hard numbers on if/by how much quants outperform old fashioned techniques like flags and finding stocks that tend to go the opposite direction from the one you're interested in? I don't know the terminology for all this but I know there are limits due to uncertainty so even the best algorithms may not do much better that someone guessing. Do quants do 10% better, 2x better, 10x better? My Dad and I had a…

> Anyone have hard numbers on if/by how much quants outperform old fashioned techniques

From the article, "In the past five years, quant-focused hedge funds gained about 5.1% a year on average. The average hedge fund rose 4.3% a year in the same period."

Re: The Quants Run Wall Street Now

#78
post #72
post #65

My question for these new would-be quants is, how do you see your work? I know there are certain types of smart whizzes who see working for Google/Facebook/Amazon as some sort of intellectual step down, or where in finance the value to society is some handwavy "market-making" argument (depending on how philosophical you get), but presumably these types going into this are also tenure-track, research producing scienti…

There's nothing handwavy about the liquidity argument. Improving liquidity reduces the cost of trading for almost everyone in the market and thus makes almost everyone just a little wealthier. That means ordinary people, like schoolteachers and custodians, retire with a bit more money, and are a bit more comfortable; it means lots of people can donate just a bit more money to charitable causes without crossing whatev…

Not everyone at Google works on adtech. There are many people working on browsers that have sped up and innovated the web for hundreds of millions of users, or operating systems that are the foundations of many people's lives.

Re: The Quants Run Wall Street Now

#79
post #49

Everyone can try to do it, but you need to spend the time and educate yourself like in any other profession or hobby and the first step is to go to eBay and search for historical stock market data, you can buy 20 years of data for less than $100 and you can test all trading ideas for free and without losing a single penny...the barrier for entry is very low, some Python knowledge + Linux machine and the data and off…

What about options? So for I haven't been able to find any good source other than CBOE and their resellers, which are all quite expensive.

I trade options using Livevol data. Good historical options data doesn't come cheaper than about $3,000/year (I'm willing and grateful to be proven wrong, but I've looked).

Re: The Quants Run Wall Street Now

#80
post #66

Earlier quoted context omitted.

Why isn't this data out on torrents? Is it somehow copyrighted?

I have a hard time seeing how it could be copyrighted, at least if we are talking about comprehensive stock listings organized in the obvious way (e.g., a table of prices organized by date, where the stocks included are chosen by some straightforward criteria). In the United States I'd expect this to be covered by Feist Publications, Inc., v. Rural Telephone Service Co., 499 U.S. 340 (1991). [1] https://en.wikipedia.…

Nondisclosure agreements.
Post reply on HN