Live data from Hacker News

Building AI Trading Systems

dennybritz.com

41–50 of 109 posts

Re: Building AI Trading Systems

#41
post #10

readers may also be interested in Benter's paper "Computer Based Horse Race Handicapping and Wagering Systems: A Report" -- https://www.gwern.net/docs/statistics/decision/1994-benter.p... > This paper examines the elements necessary for a practical and successful computerized horse race handicapping and wagering system. Data requirements, handicapping model development, wagering strategy, and feasibility are addresse…

The parent comment is the most useful one in the thread so far for anyone who seriously wants to learn about quantitative trading.

Sports betting is essentially the same thing as proprietary trading in financial markets. The paper gives a good summary of a technique that was very successful in its day.

There is very little publicly available material on quantitative techniques that are useful for proprietary trading. Lo and Mackinlay's "non-random walk down wall st" was good, but that's 20 years old.

The mathematical literature on gambling is a lot more accessible. It's also probably easier to consistently make at least small money gambling, because the barriers to entry are lower.

Re: Building AI Trading Systems

#42
post #36
post #23

Earlier quoted context omitted.

+1 for indexes. This is the route most successful traders seem to take - alternating between indexes and bonds. You won't see Warren Buffet buying stocks on Robinhood.

Warren Buffett doesn't buy stocks on Robinhood because they couldn't handle his volume. I saw him quoted somewhere not long ago saying that if he was working with a "small portfolio", a few hundred thousand dollars or maybe it was even a few million, that he was quite certain he could return in excess of 100% annually. But yes, picking stocks is probably not as effective for most people as buying an index fund.

He said the bit about returning 100% in a very old Berkshire meeting (90s). He's recently commented that the market has become either too volatile or crowded to allow the same type of value investing that got him to where he is

Re: Building AI Trading Systems

#43
post #37

Just a reminder: nobody ever wrote about their super successful trading strategy. Its just never happened. If you have the wherewithal to research and build a trading system that works, then you're smart enough to know that the moment you reveal your edge to the world - it disappears. Even if you dont discuss the innards of your strategy, but you talk about your process or the system youre strategy is built on, you'v…

I agree except if your strategy is something everyone uses then it becomes a self-fulfilling prophecy of winning.

If thats the case, more often than not you wont be in the high-frequency realm. In other words, the "self-fulfilling" trades are usually short ideas that hedge funds highly publicize or maybe a penny stock someone wants to pump (illegally) - and everything in between.

Where higher frequency trades do become "self fulfilling" can be in intraday technical analysis. For example, there are many people that follow RSI signals in options. Youll see retail people trade this, and then market makers will step in and bring things back in line - because options dont trade on technicals...

Re: Building AI Trading Systems

#44
post #37

Just a reminder: nobody ever wrote about their super successful trading strategy. Its just never happened. If you have the wherewithal to research and build a trading system that works, then you're smart enough to know that the moment you reveal your edge to the world - it disappears. Even if you dont discuss the innards of your strategy, but you talk about your process or the system youre strategy is built on, you'v…

With the notable exception of Ed Thorp, who managed to write Beat the Market first, and then start a hedge fund to exploit the strategy 7 years later, and only when a reader proposed they go into business together.

Though it helped that the period was 1967 to 1974. The piranhas were a little slower back then

Re: Building AI Trading Systems

#45
post #8

I've never tried the AI trading path but I imagine that you can't get huge gains with public data, unless you find a way to extract "hidden" information by processing real time news. I wonder nevertheless if there's a sweet spot where you can build a simple AI trading algorithm and get modest earnings from it.

[deleted]

Re: Building AI Trading Systems

#46
post #34

Earlier quoted context omitted.

Friend, if you are up on Hackernews making comments like this, I can guarantee with near 100% certainty that you are not capable of sustained outperformance in the markets. Whether you realise it now, later or never is no skin off my nose, but sooner would be cheaper. Comments like this are why these "beat-the-market" threads are evergreen on HN. They do a disservice to the community.

The low beta anomaly has persisted for since we have data. If you consider the S&P 500 the market, then yes, it's not difficult to beat it by investing in low beta stocks and leveraging up so your beta is 1. This is a classic and time-tested strategy that will probably always outperform, on a risk-adjusted basis.

You're arguing on HN that you have a long-term strategy that can reliably beat the market. Such a strategy would be worth billions of dollars. You probably don't have such a strategy.

Good luck with that though.

Re: Building AI Trading Systems

#47
post #33

Earlier quoted context omitted.

Friend, if you are up on Hackernews making comments like this, I can guarantee with near 100% certainty that you are not capable of sustained outperformance in the markets. Whether you realise it now, later or never is no skin off my nose, but sooner would be cheaper. Comments like this are why these "beat-the-market" threads are evergreen on HN. They do a disservice to the community.

I'm really glad these sort of comments were made around 2013 on this community and I started trading cryptocurrency. His comments make sense to me, and I can guarantee you with near 100% certainty there is another millionaire trader reading this thread.

You didn't need to trade to make millions in cryptocurrency if you started in 2013. Just buy and hold.

Re: Building AI Trading Systems

#48

Writing ai trading systems is the coding I do for fun since 2012. I'm a little under break even so far but I keep at it because find it so interesting. Since I started every single week I have learned a new way of thinking about a problem I encountered or a new approach to problems that still stand in my way. Questions like, how do you choose a stoploss? Well you can pick it statistically based on history or you can…

It sounds like a lot of fun! I love the idea that there’s one metric ($) to measure the effectiveness of your strategy/code. Any recommendations or hints on where to get started (assuming I’m decent with python/pandas etc)?

Yes!

1. I would start in the numerai tournament, I did this for 3 years after the first two years of me by myself on the market. It's useful because they provide ml ready data, and you can iterate very quickly. If you do not have ML experience numerai will teach you about many different types of overfitting and the many correct and incorrect ways to deal with them. An example would be some ML people always apply dropout, but when you have a small signal to begin with, dropout can dropout the signal, and then there is only noise left for the model to fit and of course it will then perform poorly. The other thing it will help with is the hopelessness that you will encounter from hitting a wall(hitting a wall is common in ML, and should be expected), the scoreboard shows individuals who have broken through that wall so you can know it is possible. I stopped participating after stabilizing in the top 20 because they change the format of the tournament every so often and I wanted my Saturdays back. You don't need to reach top 20, I hit a wall around rank 100 back when they used actual bitcoin to pay people. You just need to do well in one of the rounds where everyone else fails so you can go through the process of 'what did I do that I'm not aware of that made me succeed where everyone else failed'

2. Read Advances in Financial Machine Learning by Marcos Lopez de Prado. This goes over the false assumptions that outsiders make, and then outlines rookie mistakes(I made many of the mistakes described in the book, then read this book when it came out). It also will break you out of the thinking that leads to typical approaches and why it is unrealistic to expect them to work.

3. Become familiar with retail trader mistakes like overtrading, improper sizing, and emotions as well as the fact that you cannot rely upon regulating bodies to prevent fraud from occurring, they only act after it has occurred.(This is for scenarios where your models says short this stock, then you see that the stock is fraudulent but it continues to exist.) Learning blackjack probabilities + sizing helps with developing a strategy. Things like, do you want a trading system that has 60% accuracy and 10% profit each time, or one that has 45% accuracy but 200% profit each time. It's interesting because even if you have a 50% accurate 200% profit/50% loss strategy, you still need to calculate the probability of what number of losses you will see in a row that will still bankrupt you if you have the wrong size. In college for me this was covered under the Discrete Math Class.

After steps 1+2+3 I think people who have some level of control over their emotions have the right foundation to code a system. There are people that should not trade because they don't have the right personality profile.

4. Find a way to fit the data you encounter into a DB. Early on I had to pay 100 a month to get daily csvs for stock data. I wrote code that answered questions for me from the csvs. This was wasted time, because you can write SQL to answer so many questions. Keep this DB on a separate computer from the one you do ML dev on. Because the computer that ML dev happens on inevitably gets wiped(it will happen to you).

Then for you its a matter of just leveraging python+pandas etc to code a solution that meets your criteria. There are three categories that you have to operate across, infrastructure+forecasting+trading strategy. When you see one of your models predictions become true it really is a different feeling. But to ease my conscience I should warn you, if you are the curious type and you try this once, you will always be curious about it.

For timeseries data im currently using iextrading even though it has downsides(they only have data for trades that route through their exchange). I used to use kibot, alphavantage,scrape yahoo, download stock data csvs from ebay,and save etrade realtime quotes. For placing trades I am currently using alpaca.(I've used IB,etrade,and robinhoods private api before they blocked it).

Re: Building AI Trading Systems

#49
post #46
post #34

Earlier quoted context omitted.

The low beta anomaly has persisted for since we have data. If you consider the S&P 500 the market, then yes, it's not difficult to beat it by investing in low beta stocks and leveraging up so your beta is 1. This is a classic and time-tested strategy that will probably always outperform, on a risk-adjusted basis.

You're arguing on HN that you have a long-term strategy that can reliably beat the market. Such a strategy would be worth billions of dollars. You probably don't have such a strategy. Good luck with that though.

There are a number of persistent factors. They are well known in academia and to financial practitioners. There are structural reasons for their existence. We are talking about value, fama french size, etc. These are classic factors that everyone knows about.

https://www.aqr.com/Insights/Research/Journal-Article/Bettin...

https://en.wikipedia.org/wiki/Low-volatility_anomaly

I think your attitude of dismissing decades of academic research by Nobel prize winners is a little myopic.

Also I don't think you understand what "risk-adjusted" means.

Re: Building AI Trading Systems

#50

Don't do this. It's the programmers Moby Dick. You are better off self learning stats/ML skills in your free time and joining a quant fund than to try and do it yourself.

Agreed. It's a goose chase, the house always wins and even a winning system works one week and not the next.
Post reply on HN