Live data from Hacker News

HRT's Python fork: Leveraging PEP 690 for faster imports

hudsonrivertrading.com

31–40 of 103 posts

Re: HRT's Python fork: Leveraging PEP 690 for faster imports

#31
post #15

Earlier quoted context omitted.

when milliseconds mean millions

If that was the case, why use Python in the first place?

It's not uncommon to have a fast core and then an API that alpha / research teams feed signals into

if you are someone like HRT I presume the bulk of their money comes at very short holding periods so you have e.g. fast signals that work short term and then mid frequency alpha signals that spit out a forecast over a few timeframes i.e. it might not be that they buy (aggressively) really quickly but rather than someone sells to them and then they hold onto the position for longer than they would if they have no opinions.

Similarly this shapes where you post your orders e.g. if you really want it then you want to be top of the book

Re: HRT's Python fork: Leveraging PEP 690 for faster imports

#32

Interviewed with HRT awhile back. While I didn't get past the final round, their Python internals interview (which I did pass) was an absolute blast to prepare for, and required a really deep dive into implementation specific details of CPython around things like exactly how collisions are handled in dict, details about memory management, etc. Pretty much had to spend a few weeks in the CPython source to prep, and wa…

>Python devs at HRT really know their stuff.

Its a finance firm - i.e scam firm. "We have a fancy trading algorithm that statistically is never going to outperform just buying VOO and holding it, but the thing is if you get lucky, it could".

Scammers are not tech people. And its pretty from their post.

> In Python, imports occur at runtime. For each imported name, the interpreter must find, load, and evaluate the contents of a corresponding module. This process gets dramatically slower for large modules, modules on distributed file systems, modules with slow side-effects (code that runs during evaluation), modules with many transitive imports, and C/C++ extension modules with many library dependencies.

As they should.

The idea that when you type something in the code and then the interpreter just doesn't execute it is how you end up with Java like services, where you have dependency injection chains that are so massive that when the first time everything has to get lazily injected the code takes a massive amount of time to run. Then you have to go figure out where is the initialization code that slows everything down, and start figuring out how to modify your code to make that load first, which leads to a mess.

If your python module takes a long time to load, this is a module problem. There is a reason why you can import submodules of modules directly, and overall the __init__.py in the module shouldn't import all the submodules by default. Structure your modules so they don't do massive initialization routines and problem solved.

Furthermore, because of pythons dynamic nature, you can do run time imports, including imports in functions. In use, whether you import something up at the top and it gets lazily loaded or you import something right when you have to use it has absolutely no difference other than code syntax, and the latter is actually better because you can see what is going on rather than the lazy loading being hidden away in the interpreter.

Or if you really care, you can implement lazy work process inside the modules, so when you import them and use them the first time it works exactly like lazy imports.

To basically spend time building a new interpreter with lazy loading just to be able to have all your import statements up at the top just screams that those devs prefer ideology over practicality.

Re: HRT's Python fork: Leveraging PEP 690 for faster imports

#33
post #15

Interviewed with HRT awhile back. While I didn't get past the final round, their Python internals interview (which I did pass) was an absolute blast to prepare for, and required a really deep dive into implementation specific details of CPython around things like exactly how collisions are handled in dict, details about memory management, etc. Pretty much had to spend a few weeks in the CPython source to prep, and wa…

when milliseconds mean millions

Those days are all over btw.

Most trading firms are past the whole "beat the other guys to buy". Established large investment firms already have all that on lockdown in terms of infrastructure and influence to the extent where they basically just run the stock market at this point (i.e Tesla posts horrible quarter numbers, but stock goes up).

Most of the smaller firms basically try to figure out the patterns of the larger firms and capitalize on that. The timescales have shifted quite a bit.

Re: HRT's Python fork: Leveraging PEP 690 for faster imports

#34

Interviewed with HRT awhile back. While I didn't get past the final round, their Python internals interview (which I did pass) was an absolute blast to prepare for, and required a really deep dive into implementation specific details of CPython around things like exactly how collisions are handled in dict, details about memory management, etc. Pretty much had to spend a few weeks in the CPython source to prep, and wa…

>Python devs at HRT really know their stuff. Its a finance firm - i.e scam firm. "We have a fancy trading algorithm that statistically is never going to outperform just buying VOO and holding it, but the thing is if you get lucky, it could". Scammers are not tech people. And its pretty from their post. > In Python, imports occur at runtime. For each imported name, the interpreter must find, load, and evaluate the con…

> "We have a fancy trading algorithm that statistically is never going to outperform just buying VOO and holding it, but the thing is if you get lucky, it could".

You wish lol. How do you think they pay for all the developers?

Firms like HRT don't even take outsider money, they don't really need to.

And besides, we don't get paid for beating stocks, a lot of funds will do worse than equities in a good year for the latter, the whole point is that you're benchmarked to the risk free rate because your skill is in making money while being overall market neutral. So you rarely take a drawdown anywhere near as badly as equities.

As a service this is often a portfolio diversification tool for large allocators rather than something they put all the money into.

It is true however that some firms are basically just rubbish beta vehicles that probably should in an ideal world shut down.

Re: HRT's Python fork: Leveraging PEP 690 for faster imports

#35
post #15

Earlier quoted context omitted.

when milliseconds mean millions

Honestly if you're a millisecond too slow you might as well not trade at all. From my own experience with trying to get Python to go fast for crypto trading, you can get it pretty fast using Cython - single digit microseconds on an average AWS instance for a simple linear regression was my proudest moment. They're probably pushing it even faster because nanoseconds are where the money's at. Many HFT firms are down in…

For crypto you pay the miners to put your transaction first. You don't need millisecond precission reaction time.

Re: HRT's Python fork: Leveraging PEP 690 for faster imports

#36

Interviewed with HRT awhile back. While I didn't get past the final round, their Python internals interview (which I did pass) was an absolute blast to prepare for, and required a really deep dive into implementation specific details of CPython around things like exactly how collisions are handled in dict, details about memory management, etc. Pretty much had to spend a few weeks in the CPython source to prep, and wa…

>Python devs at HRT really know their stuff. Its a finance firm - i.e scam firm. "We have a fancy trading algorithm that statistically is never going to outperform just buying VOO and holding it, but the thing is if you get lucky, it could". Scammers are not tech people. And its pretty from their post. > In Python, imports occur at runtime. For each imported name, the interpreter must find, load, and evaluate the con…

> Its a finance firm - i.e scam firm. "We have a fancy trading algorithm that statistically is never going to outperform just buying VOO and holding it, but the thing is if you get lucky, it could".

HRT trades their own money so if it didn't beat VOO then they'd just buy VOO. There are no external investors to scam.

Re: HRT's Python fork: Leveraging PEP 690 for faster imports

#37

Interviewed with HRT awhile back. While I didn't get past the final round, their Python internals interview (which I did pass) was an absolute blast to prepare for, and required a really deep dive into implementation specific details of CPython around things like exactly how collisions are handled in dict, details about memory management, etc. Pretty much had to spend a few weeks in the CPython source to prep, and wa…

>Python devs at HRT really know their stuff. Its a finance firm - i.e scam firm. "We have a fancy trading algorithm that statistically is never going to outperform just buying VOO and holding it, but the thing is if you get lucky, it could". Scammers are not tech people. And its pretty from their post. > In Python, imports occur at runtime. For each imported name, the interpreter must find, load, and evaluate the con…

[flagged]

Re: HRT's Python fork: Leveraging PEP 690 for faster imports

#38

Interviewed with HRT awhile back. While I didn't get past the final round, their Python internals interview (which I did pass) was an absolute blast to prepare for, and required a really deep dive into implementation specific details of CPython around things like exactly how collisions are handled in dict, details about memory management, etc. Pretty much had to spend a few weeks in the CPython source to prep, and wa…

>Python devs at HRT really know their stuff. Its a finance firm - i.e scam firm. "We have a fancy trading algorithm that statistically is never going to outperform just buying VOO and holding it, but the thing is if you get lucky, it could". Scammers are not tech people. And its pretty from their post. > In Python, imports occur at runtime. For each imported name, the interpreter must find, load, and evaluate the con…

Runtime imports are a maintenance nightmare and can quickly fragment a codebase. Static analysis of imports is so desirable that it is almost always worth the initialization performance hit. Tradeoffs.

Re: HRT's Python fork: Leveraging PEP 690 for faster imports

#39

Interviewed with HRT awhile back. While I didn't get past the final round, their Python internals interview (which I did pass) was an absolute blast to prepare for, and required a really deep dive into implementation specific details of CPython around things like exactly how collisions are handled in dict, details about memory management, etc. Pretty much had to spend a few weeks in the CPython source to prep, and wa…

>Python devs at HRT really know their stuff. Its a finance firm - i.e scam firm. "We have a fancy trading algorithm that statistically is never going to outperform just buying VOO and holding it, but the thing is if you get lucky, it could". Scammers are not tech people. And its pretty from their post. > In Python, imports occur at runtime. For each imported name, the interpreter must find, load, and evaluate the con…

> "We have a fancy trading algorithm that statistically is never going to outperform just buying VOO and holding it, but the thing is if you get lucky, it could".

How they make $8B/y underperforming VOO?

Reference: https://www.businessinsider.com/hudson-river-trading-hrt-8-b...

Post reply on HN