Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Case study: Algorithmic trading with Go

polygon.io

Post reply on HN
Print view View original

31–40 of 311 posts

Re: Case study: Algorithmic trading with Go

#31

Post by jjtheblunt »

eshnil wrote: post #16
czbond wrote: post #11

Earlier quoted context omitted.

@WestCoastJustin I've been really wanting to use Go, but as you say, much of the community is Python due to the data analysis strengths. To the detriment of the other things Python does do poorly. Can you give some thoughts with your experimentation on the following from a Go perspective. 1. Supported TA libraries in Go. I'm familiar with TAlib (python), bloom, etc. - certain forks tailored to real time rather than h…

Why not switch to Mojo lang for this? It's Python-compatible with Golang like performance from what I hear.

It's not available yet outside of hosted environments provided by Modular.

Re: Case study: Algorithmic trading with Go

#32

Post by _zkyx »

k9jd883jb wrote: post #20
_zkyx wrote: post #2

Happy to answer any questions about this. It's been a side project that turned into a full blown obsession. There is nothing too secret about the system since it's more about having a solid platform that you can plug your strategies into. I'd probably even open source it but I'd have to clean up all my hacks :)

Awesome write up. I have a similar project in Go myself, although I just use minute bar data instead of realtime ticks. Can you share your approach for plugging in various strategies? I quickly learned that having a pluggable strategy system is tricky as it could span across multiple layers of the system. Also, with backtesting, are you storing and replaying all the quote/tick data? or just using the historical aggre…

I wish this was better, but honestly, I'm just hard coding them right into the BUY loop. So, I need to restart the app anytime I want to change something. That's why I build this logic to dump and reload the state into a gob file (go memory dump essentially). Ideally, you'd have some type of format to write out your algorithm, and then have a way of hot loading it or something. But, I don't change these enough to really do that yet.

For back testing, I download all raw trades and quotes, and put them into 1 file sorted by time (1 file per day). Then, I compress them using lz4. This allows me to sort of replay the entire market and build up all my intraday backtesting from the source. This took me a long time to figure out and build but has been so worth it. So, I have an off-line script that basically, loops through these files, and replays the market, and makes simulated trades, and then spits out what would have happened. There is a GUI for that too so you can go in an explore the trades and see what triggered the buy and sell. I have seen nothing that goes backtesting for intraday like this.

This is super inefficient but I'm just building the aggregates on the fly. I could probably cache them somewhere but it takes maybe 4-5 minutes to replay a days worth of trades/quotes and build all this so I haven't bothered yet.

Re: Case study: Algorithmic trading with Go

#33

Post by pqdbr »

_zkyx wrote: post #27
nemothekid wrote: post #22

Earlier quoted context omitted.

1. I think buying and holding is more probable to have a higher return 2. Sometimes you get a cool api and think wow this would be fun, and next thing you know you've lost thousands on boneheaded trades. I did something similar during the pandemic with Rust with the Polygon API (and instead of interactive brokers, I used tradier). Eventually I learned I actually had more fun building the thing than actually trying to…

I have won and lots thousands for sure. Haha. When stocks were on a rip the bot was making lots of money just because everything was going way up. Then, in 2022 when everything went way down, like tons of tech stocks, my bot sucked. So, I really need to add shorting or something. I'm still exploring things on the strategy side.

Isn't the trend detection equally applicable to shorting as well, with swapped signals?

Or do you believe there are more fundamental changes needed so your app can trade in shorting as well?

Re: Case study: Algorithmic trading with Go

#34

Post by the__alchemist »

_zkyx wrote: post #25
makestuff wrote: post #18

This is really interesting. Have your strategies out performed buy and holding index funds, or are you mainly just doing this with a small amount of capital to learn how the markets work? I have always wanted to try algorithmic trading to learn about it, but I have always read it is a fools errand to think you will beat just buy and hold.

This is going to sound crazy given all the scams out there. But I was interested in testing the idea of small compounding returns. Like, could you get a daily 0.5% compounding return. Sure, you could go all in on TSLA for example and get a 1% daily return. But, could you do that with automation, using lots of small bets, across the entire market. You can, but there is a scale issue here. In that you need to make expo…

> Like, could you get a daily 0.5% compounding return

It seems that this is the key to your approach. How is this part achieved?

Re: Case study: Algorithmic trading with Go

#36

Post by mhh__ »

makestuff wrote: post #18

This is really interesting. Have your strategies out performed buy and holding index funds, or are you mainly just doing this with a small amount of capital to learn how the markets work? I have always wanted to try algorithmic trading to learn about it, but I have always read it is a fools errand to think you will beat just buy and hold.

Out perform relative to a number or your expectation?

You might want returns that aren't correlated just to an index - this is a major reason to look to invest in (say) a hedge fund.

Re: Case study: Algorithmic trading with Go

#37

Post by mhh__ »

_zkyx wrote: post #25
makestuff wrote: post #18

This is really interesting. Have your strategies out performed buy and holding index funds, or are you mainly just doing this with a small amount of capital to learn how the markets work? I have always wanted to try algorithmic trading to learn about it, but I have always read it is a fools errand to think you will beat just buy and hold.

This is going to sound crazy given all the scams out there. But I was interested in testing the idea of small compounding returns. Like, could you get a daily 0.5% compounding return. Sure, you could go all in on TSLA for example and get a 1% daily return. But, could you do that with automation, using lots of small bets, across the entire market. You can, but there is a scale issue here. In that you need to make expo…

https://en.m.wikipedia.org/wiki/Martingale_(betting_system)

Re: Case study: Algorithmic trading with Go

#38

Post by mhh__ »

makestuff wrote: post #26
_zkyx wrote: post #25

Earlier quoted context omitted.

This is going to sound crazy given all the scams out there. But I was interested in testing the idea of small compounding returns. Like, could you get a daily 0.5% compounding return. Sure, you could go all in on TSLA for example and get a 1% daily return. But, could you do that with automation, using lots of small bets, across the entire market. You can, but there is a scale issue here. In that you need to make expo…

That makes sense to me and lines up with what buffet says about there are alot of people on wall street who can average 50% returns with 100k but once it gets into the millions it is much harder to find alpha.

One of the issues arbitrageurs have is that as they get good and bigger they can run out of things to arbitrage.

Re: Case study: Algorithmic trading with Go

#39

Post by blobbers »

Regarding open sourcing: don't be afraid to show your flaws; it doesn't have to be lonely if it is open source.

Obviously whatever trading bot you're running separate from the actual trading engine itself is somewhat proprietary, but it would be great for the community to get more of this type of software in the hands of other hackers.

Quantopian / Robinhood tried and failed, and the numerous clones since then have been somewhat sub par.

Re: Case study: Algorithmic trading with Go

#40

Post by _zkyx »

czbond wrote: post #11
_zkyx wrote: post #2

Happy to answer any questions about this. It's been a side project that turned into a full blown obsession. There is nothing too secret about the system since it's more about having a solid platform that you can plug your strategies into. I'd probably even open source it but I'd have to clean up all my hacks :)

@WestCoastJustin I've been really wanting to use Go, but as you say, much of the community is Python due to the data analysis strengths. To the detriment of the other things Python does do poorly. Can you give some thoughts with your experimentation on the following from a Go perspective. 1. Supported TA libraries in Go. I'm familiar with TAlib (python), bloom, etc. - certain forks tailored to real time rather than h…

> Supported TA libraries in Go. I'm familiar with TAlib (python), bloom, etc. - certain forks tailored to real time rather than historical (eg: no re-compute on ticks)

I've been basically, just manually coding the algorithms from python into Go. ChatGPT is amazing at this. I really only just about 4 so it was a one time thing.

> Data storage (article mentioned you're all in memory). I've been using S3 & ArticDB

Yeah, I ran into issues and then was like what would be the fastest, then just went in-memory. I download all raw trades/quotes each night and store then into gob+lz4 compressed files. Then for backtesting and stuff I can load these in and build the aggrogate bars on the fly.

> If your in-data memory is treating you well for multiple TA calculations (example: in Python, you can compute & save pickled dataframes - and re-read those over longer time periods)

Yeah, I have a historical lookup table that I build nightly too. This gives me a reference point when I'm doing % change calculations and stuff. I should probably have mentioned that.

Post reply on HN