Live data from Hacker News

Show HN: Investorsexchange.jl – parse trade-level stock market data in Julia

github.com

51–60 of 71 posts

Re: Show HN: Investorsexchange.jl – parse trade-level stock market data in Julia

#51

When looking at the linked archives [1] it seems that they sit at around 4GB/day. Is that really all of the data that the US stock market generates a day? I thought it would be at least 10x that. [1] https://iextrading.com/trading/market-data/

Its only data for flows going through IEX, less than 3% market volume

https://iextrading.com/stats/

Re: Show HN: Investorsexchange.jl – parse trade-level stock market data in Julia

#53
post #32

One file has a size of nearly 5 GB... Every day two of these files get released... So nearly 10 GB every day. So if we download the raw data every day for one year, we would have 3650 GB just for one year... It would be interesting how much reduced the size of the processed data is compared to the raw data. You say that you have 50 GB of data spanning multiple years. How many years exactly?

Ah, this is a nuanced point I totally left off the README. Each raw file is ~5GB, but the raw files are a dump of network traffic from the firehose feed that tracks not just trades, but also updates to orders that do not result in trades. If you skip all of the algorithmic bots' constant updates to their bid/offer spreads and look at just the trades that clear, you can store years of raw trades in under 100GB.

Re: Show HN: Investorsexchange.jl – parse trade-level stock market data in Julia

#54
post #2

Investor Sex Change?

Domain names of the old internet: - penisland.net (sells pens) - expertsexchange.com (renamed to experts-exchange.com) - therapistfinder.com (find a therapist) - whorepresents.com

- rim.jobs (Research in Motion (Blackberry) job postings)

Re: Show HN: Investorsexchange.jl – parse trade-level stock market data in Julia

#56
post #2

Investor Sex Change?

HN is special, and you all here in this comment tree are the best of the best -- love the fact that everyone is having fun with this in such a civil way and historically knowledgeable way.

For anyone who wants the naming backstory, InvestorsExchange.jl was originally IEXTools.jl, but Julia's package registration automatic name checks didn't like it ("Name does not meet all of the following: starts with an uppercase letter, ASCII alphanumerics only, not all letters are uppercase. Name is not at least five characters long") [1]. So to Wikipedia I went to find the non-acronym name of the IEX exchange, which is "Investors Exchange" [2]. Thank you all for helping me understand why IEX goes by IEX in all of their branding.

[1] https://github.com/JuliaRegistries/General/pull/27989

[2] https://en.wikipedia.org/wiki/IEX

Re: Show HN: Investorsexchange.jl – parse trade-level stock market data in Julia

#57
post #28

Earlier quoted context omitted.

I too am here for this reason also. I would change the name. It is an issue. Put a dash in it or an underscore... just something.

Underscore is (was?) illegal in domain names.

It's not illegal in domain names. Take a look at the SRV type in the domain name system. It's underscores galore.

However, for host names or for a URL's host component, you're limited to hyphens.

In this case though it's just the name of a Julia package[1], which isn't bound by either the rules of domain names or host names as .jl is not a TLD.

[1]: https://pkgdocs.julialang.org/v1/creating-packages/#Package-...

Re: Show HN: Investorsexchange.jl – parse trade-level stock market data in Julia

#58

Earlier quoted context omitted.

Domain names of the old internet: - penisland.net (sells pens) - expertsexchange.com (renamed to experts-exchange.com) - therapistfinder.com (find a therapist) - whorepresents.com

My employer did some work early in Dotcom for a company that owned the domain name "manufacturersexchange", when I pointed out that was a terrible name for a business they said "no business person would notice that". Business never got off the ground.

I wonder what their argument would have been. "Business people just aren't observant enough to notice things in general shrug" ?

Re: Show HN: Investorsexchange.jl – parse trade-level stock market data in Julia

#59
post #22

It's good that the actual file is InvestorsExchange.jl, because without the E capitalized, that's definitely not how my brain parses it.

"Invest or sex change" is how my brain initially parsed it, lol. Was your initial reaction the same?

Re: Show HN: Investorsexchange.jl – parse trade-level stock market data in Julia

#60

Reminds me of the time the company that owns BlackBerry decided to post some listings on the new-at-the-time .jobs TLD...

I interned at RIM back in the day and every single person I went to school with asked how I liked my RIM job.

Maybe that's why they were single.
Post reply on HN