Ask HN: Physicists of HN, what are you working on these days?

I've spent a lot of time over the past year thinking about what my next move should be. In this time I've also become hooked on coding, mostly python. My current plan is to pivot into (you guessed it) data science. I expect that this will lead to better pay, vastly more potential employers, and allow me to get back to working on interesting projects.

In hindsight, if I could do everything over I'm not sure I would go into physics. What I've seen is that physics is mostly a field of niches that are filled by specialists. To go far in physics you need to become a specialist, however this can really limit your options later. I think this is why many physicists eventually find themselves going into other fields like data-science.

Went commercial rather than academic, as the job market was insanely crowded, and I didn't have "enough" differentiation in my opinion, to land a tenure track, at a lower tier school given the huge influx of high quality talent from the former soviet union (FSU). I was around during the whole Young Scientists Network days (early 90s) where we collectively did deep soul searching on whether or not Physics as an academic career actually made sense.

I joined a supercomputer maker in 1995 ABD, and finished writing up (my third rewrite, first was in 1994, final accepted one was in 1997) and defending . I stayed with them for 6 years, until I saw that they had no real hope of long term survival.

During my time in school, I'd been a consumer of Supercomputing systems across the US, and in my department. I decided that was the career direction I'd go, with the idea that I'd become an entrepreneur after watching successful companies develop and grow. Learn from them, not just their successes, but their failures.

Needless to say, my first job I learned a great deal. My thesis advisor was still trying to push me to postdocs with her former classmates, and I was tempted, but my wife and I decided to start a family, and that kind of nuked that direction.

I left the first place, and was recruited to help another company bootstrap an HPC division. That was fun, I got experience in all the non-technical side of businesses as well as the tech.

They had a financial crisis, and I took a small package and started my own HPC company. I ran that for 14.5 years. It was a wild ride, and I learned a tremendous amount (e.g. I failed in many non-fatal ways). Unfortunately, the last learning experience was in fact, fatal. I joined a cloud company and have been helping to build a "next generation" cloud.

My thesis advisor just retired, and she's been sending me things about Astronomy and Physics openings. They are mostly adjunct teaching things. I love teaching, I get a real blast out of it. But the adjunct life is a massive pay cut, at a time I cannot afford one. I don't have enough spare time to do a good job either.

Recently, my alma mater has an opening in the CS department for HPC-like people, which is definitely up my ally. I've got a real interest in ML and its connections to statistical mechanics, quantum computing, numerical simulations, etc. But I am undecided as to whether or not I should look into this more.

[1] https://scalability.org/about-me/

[2] https://academictree.org/physics/tree.php?pid=743767

For myself at least I would add job security as another benefit. I struggled in vain for 6-7 months during my lectureship applying to any and all postdocs I could find, so the prospect of cycling through 2-3 year postdocs in various locations eventually overwhelmed me. I loved the research but realised I didn't love it enough to potentially live apart from my partner and regularly go through months-long application processes.

On the flip side, for me freedom is actually something I miss from academia. Admittedly that's partly because I'm a regular employee rather than a freelancer. Outside of teaching, my time was basically my own and I was free to work from home or anywhere else. I'm somewhat resentful of having to justify what I'm spending my time on day by day. Not that my manager would particularly mind if I spent a day researching something slightly blue-sky, but the fact that I might have to respond to questions about it creates this background tinge of anxiety that I find prevents deep creative thought.

My background in physics and math really helped in CS classes I've taken since graduating, and the general technical background and comfort with math has been very helpful in general. I do wish I had done more research and programming in college, as I ignored them in favor of experience that would line up better with work in DC.

Working on data science/ML now.

Started undergrad in 2008 and had to decide between CS and engineering physics. Went with the latter because the school had a particularly strong program.

Applied for PhD programs and national lab positions in 2012. The PhD route seemed more interesting, so I went with that. Ended up spending 6 years on materials simulation.

My peer group ended up going to academia, software engineering, and data science. Technology companies seemed exciting so I interviewed for software engineering and data science roles. Ended up with a company that predicts accident risk from driving data.

I think that physicists are particularly well suited to the machine learning space. I was trained to work systematically work through problems with no analytic solution through clever approximations. This tenacity can help with tackling problems in the machine learning space.