Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Finding Signal in the Noise: Machine Learning and the Markets (Jane Street)

signalsandthreads.com

Post reply on HN
Print view View original

121–130 of 157 posts

Re: Finding Signal in the Noise: Machine Learning and the Markets (Jane Street)

#121

Post by [deleted] »

godelski wrote: post #39
twoodfin wrote: post #13

Earlier quoted context omitted.

If it didn’t provide value, nobody would pay for it. The alternative to highly technical, agile quantitative trading is fat middle men, wide spreads, and capital sitting in 8%* stupider places than it would otherwise. That’s a pretty big deal even if the observable effects are extremely diffuse. * Made up number, but if we woke up on Monday with nothing but the tech we used to trade in 1984 it would probably hit much…

Lots of things provide no value that people pay for. The inverse is true too: lots of things provide value that no one pays for. I find it shocking that anyone that programs would think this way considering how widespread and common open source tooling is. I don't know how you can get through your day without using OSS or even free websites like stack overflow.

[deleted]

Re: Finding Signal in the Noise: Machine Learning and the Markets (Jane Street)

#122

Post by edanm »

jakewins wrote: post #116
edanm wrote: post #86

Earlier quoted context omitted.

I'm intrigued - I didn't listen to the post but wonder what kind of best practices you're talking about. Do you have this written down anywhere or on video anywhere? I'd love to learn more about what you mean.

I’m intrigued as well.. My experience is notebooks struggle as a format for production code. We encourage people who work heavily in notebooks to use them for exploratory work, but choose other tools when it comes time to ship. When you are exploring something, experimenting, showing.. it’s great; train-of-thought structure, APIs like Pandas optimised for writing and terseness etc. But when you have a piece of code t…

I write production code, I also work lots in Jupyter notebooks.

Personally, I think the fact that notebooks are usually easier/funner for me to work with is a big problem. I'm by no means a Clojure expert, but I did do a semi-large project in Clojure a few years ago, and some of the ideas of true REPL-driven development that exist there are things I wish that Python supported.

It's hard to explain without actually learning it for real (and most Python devs mistakenly think Python has REPL-driven development; I sure did before learning Clojure!). But once you get used to being able to interact with your actual source code, and at any point just being able to write new code and immediately print out its value, then with one shortcut make it part of the regular codebase... that just blurs the distinction that exists between Jupyter Notebooks and production code in a way that makes everything much better.

Re: Finding Signal in the Noise: Machine Learning and the Markets (Jane Street)

#123

Post by alfiedotwtf »

Barrin92 wrote: post #24
bblcla wrote: post #16

Earlier quoted context omitted.

Tyler Cowen did an 'ask me anything' at Jane Street when I interned in 2016. One of the interns asked him exactly this: "What do you think of the fact that we all work here instead of, I don't know, curing cancer?" He replied with, roughly, "Those of you who work here probably couldn't do anything else other than perhaps math research. Arguably, working here is the economically efficient use of your time." I think ab…

General intelligence is overstated sometimes, but it is a thing. Someone who is smart enough to work for Jane Street probably could at least be an intelligence analyst or software developer at the NSA contributing to national security. (Jim Simmons literally was a code breaker during the Vietnam War) I don't think there's a gene for playing esoteric minigames on the options market while you literally suck at everythi…

I would rather people work in trading than for the NSA

Re: Finding Signal in the Noise: Machine Learning and the Markets (Jane Street)

#124

Post by thijson »

TheAlchemist wrote: post #83
Workaccount2 wrote: post #75

Earlier quoted context omitted.

My actual gripe is with where that value comes from, and what it's true cost to society is. I'd argue that pulling these people out of moving society forward is just another form of externalized cost. The stanford guy who figured out how to halve the cost of solar in 2012 never got to realize it because $750k/yr to do stochastic modeling of cattle feed to (secretly) overcharge farmers for futures contracts was just t…

The way I see HFT, is that previously this money was going to brokers and banks. Now, most of this money is going to HFT shops, but you could also make an argument that some of it stays with the investors, since spreads are much smaller now. There is 0 cost to society, maybe even a small gain. As for moving society forward, I don't know. If not in finance, most of these guys would work for big tech companies trying t…

One benefit I've seen is that commission free trades wouldn't be possible without payment for order flow. I remember the old days when it cost a lot to do a trade. Also the stock price tick increments used to be higher. That's been steadily eroded.

Re: Finding Signal in the Noise: Machine Learning and the Markets (Jane Street)

#125

Post by FabHK »

throwaway2037 wrote: post #111
FabHK wrote: post #58

Earlier quoted context omitted.

I think the liquidity argument is overrated. The market has two functions: 1. Incentivizing convergence of the price towards fundamental value, to support proper asset allocation decisions. 2. Supporting buying and selling (ie "liquidity") to shift consumption in time (and enable productive investments with the delayed consumption). Suppose a retirement fund holds their investments over a 20 year period on average, g…

This comment is weird to me. Are you saying that the "financial sector" (incredibly vague term) makes 30% of US GDP? Not even close. A trivial Google search proves that it is way off base. According to this source: https://www.statista.com/statistics/248004/percentage-added-... > finance, insurance, real estate, rental, and leasing ... is 20.7% of US GDP. Also, most of the GDP in the "financial sector" is in commerci…

The finance sector contributes only 7% of GDP, correct. One of the sources I had in mind [0] cites 31%, but that was revenue as a proportion of GDP, which doesn't really make sense. However, the financial sector takes about 30% of US profits [1].

With respect to trading only once a day, I don't think your ad absurdum counterarguments hold water.

The HK stock exchange used to trade 4.5 hours a day, from 10 to 12, and then after a generous 2 hour lunch break from 2 to 4:30. How is trading 8 or 12 hours a day better for pension funds?

Price discovery with a limit order book that's cleared once a day might work just as fine as when it's spread out over hours, maybe even better.

Think about some major event affecting a company happening on the weekend. Then everyone can put in buy/sell orders with adjusted prices, and they'll be cleared during the morning auction. How is that worse than if the event happens intraday, and only the most switched on automatic HFT will pick off people still quoting at the old price, then the professional traders in hedge funds will pick off people?

As I said, the difference would be that the gains of the fast movers would instead be distributed among those on the "wrong side" of the news. Why should price discovery be worse?

Finally, what makes you think that liquidity and bid-ask spread concentrated in a minute a day would be worse than spread out over many hours a day?

[0] https://www.investopedia.com/ask/answers/030515/what-percent...

[1] https://conversableeconomist.com/2022/09/13/financial-servic...

Re: Finding Signal in the Noise: Machine Learning and the Markets (Jane Street)

#126

Post by objektif »

philosopher1234 wrote: post #72
objektif wrote: post #33

Earlier quoted context omitted.

Would you rather people spend their time on centering divs? Or come up more ways to make people click on ads. I see this type of view often on HN from big tech employees. Get off your high horses people.

You haven’t contested the substance of the argument at all. It’s as though you agree.

There is nothing to contest here. This is HN forum and there is wide held belief that capital markets tech work is somehow below what the folks are doing in SV.

Re: Finding Signal in the Noise: Machine Learning and the Markets (Jane Street)

#127

Post by MichaelRo »

ustad wrote: post #105
MichaelRo wrote: post #85

Earlier quoted context omitted.

It's fun to see how every time a finance topic pops up, discussion steers towards lamenting on how "talent is wasted" by doing this and not that. See, if you're a fishmonger and someone comes to you and say "why don't you trade flowers instead", you ignore them because they are not your customer. They won't trade fish with you, and in fact they wouldn't trade flowers either. They're just useless relative to your trad…

At the end of the day, it’s fine to ignore the critics or people who don’t understand your work, but dismissing the conversation entirely without considering the underlying point might close off an opportunity for meaningful reflection. Just my two cents.

Reflect on what? Some utopian cvasi-religious bullshit which has no real applicability except virtue-signalling? Why don't we all do nice things instead of being caught in the capitalistic rat race to make a living? (Not that other systems fared better). Why do we have to work shit, meaningless jobs instead of doing the grand things of life, engineering, arts, etc?

Well first of all because all those niches are already full beyond capacity. All the engineering, arts, cancer research, you name it, that the society can pay for is stuffed to bursting point and in no lack of pipelines of fresh wannabe recruits. Also cancer research is a deeply unprofitable enterprise overall, much in contrast to a domain like finance. One can labor a lifetime and get nothig, at least in finance they'll shovel some money from one pocket to the other and pocket the commission themselves.

Please leave people alone and stop suggesting alternate careers, that you yourselves did not choose. That's the uttermost hypocrisy. I don't wanna see anymore programmers with fat paychecks jumping on whatever latest fad like flies on a fresh laid turd doing AI and crypto and advertising and just bullshit pretend work at FAANG shedding crocodile tears on how the finance guys should be poor and suffering. Quit your high paying jobs yourselfes and go starve while doing biology PhDs then work as baristas while competing with the other 10,000 highly skilled Phds hoping to catch one of the 10 currently open positions in the world that pay a miserable salary with no stability in the future that does cancer research.

Re: Finding Signal in the Noise: Machine Learning and the Markets (Jane Street)

#128

Post by anonym29 »

philosopher1234 wrote: post #71
anonym29 wrote: post #14

Earlier quoted context omitted.

Can you codify the difference? I seem to be fundamentally misunderstanding the difference between wealth creation and wealth extraction if voluntary market activity constitutes extraction. I'd definitionally describe all voluntary transactions free of coercion to imply the buyer values the utility of what they're buying (i.e. true wealth - piles of currency are not true wealth, they're what you exchange for true weal…

Sorry but this is a shit definition. You’ve defined scamming people, a totally voluntary error, as creative not extractive. You’ve also defined building roads as extractive not creative. I think you’re latching onto some very misguided principles.

Building toll roads is creation of value. People voluntarily pay to drive on them because they offer superior qualities to garbage public roads. If I could exclusively drive on toll roads, I would.

Public roads, OTOH, are extractive because someone is taking money from you to build them even if you don't consent to it and deliberately avoid them at every opportunity.

Re: Finding Signal in the Noise: Machine Learning and the Markets (Jane Street)

#129

Post by rangestransform »

andrepd wrote: post #118
pinkmuffinere wrote: post #51

Earlier quoted context omitted.

Ya, I’m very surprised by the argument “you’re some of the best at numerical analysis and high frequency trading, and you’d be bad at anything else”, lol. That said, I think there are better reasons to work at such a place. Providing liquidity to the market is a good thing, and has real world value, it’s just hard for us to connect it to concrete outcomes. But as a simplified “toy” example, when they orchestrate a tr…

I.e., they work to increase the return on capital. Since I believe one of the biggest problems in the world today is economic inequality stemming from an increasing gulf in the remuneration of labour and the remuneration of capital, I don't think this is a good thing. Let alone a good use of stupendous amounts of talent and money.

The only reason software engineers are able to be paid so much at all is because of “capitalism” and “free trade” I.e. vacuuming up money from the rest of the world, we sell them ads and “SaaS” and “intellectual property” while they sell us food and clothes. Would be a bad day for me personally and our profession in the US when that stops

Re: Finding Signal in the Noise: Machine Learning and the Markets (Jane Street)

#130

Post by achierius »

MichaelRo wrote: post #127
ustad wrote: post #105

Earlier quoted context omitted.

At the end of the day, it’s fine to ignore the critics or people who don’t understand your work, but dismissing the conversation entirely without considering the underlying point might close off an opportunity for meaningful reflection. Just my two cents.

Reflect on what? Some utopian cvasi-religious bullshit which has no real applicability except virtue-signalling? Why don't we all do nice things instead of being caught in the capitalistic rat race to make a living? (Not that other systems fared better). Why do we have to work shit, meaningless jobs instead of doing the grand things of life, engineering, arts, etc? Well first of all because all those niches are alrea…

Nobody is saying finance people should be poor and starve. The same people critiquing the finance industry in this thread have been happy to critique ad tech as well, and I agree that they're both largely parasitic 'faux frais'. But regardless of which industries you choose to put in that category, the point isn't to make individuals choose to work in one or another -- that's not how serious change happens. It's to build support and hopefully eventually consensus on the idea that our economic system is broken and in serious need of change.
Post reply on HN