I Regret Quitting Astrophysics

I'm just happy I get to see a terminal screen for a living.

If you are an astrophysicist curious about moving into the datascience profession, I would just like to point out that satisfaction with a move like that depends on a large number of factors, and that you should consider not only Marcel's experiences (which I think are important and shared by many) but your own personal circumstances. There are pro's and con's for every person.

Let me offer another datapoint:

* I left after my PhD, which ended absolutely terribly. I knew years before finishing that I was just not cut out for a life in academia, and positioned myself in the best way I knew how for a life in industry, but I did not anticipate just how truly painful the experience would be. I stayed to finish my PhD out of a nagging obligation to follow through with a lifelong dream of mine. I thought I would deeply regret leaving after receiving my masters degree. Whether or not that is true, I really don't (and will never) know. But I was not happy.

* My personal reasons for unhappiness are not uncommon but are definitely not universal. I did not excel at the entrepreneurial and social aspects of astrophysics that are crucial for success. Without strong social and professional ties to close collaborators within the community, and without an innate ability to thrive in an environment with very little structure, add to that a predisposition for mental health problems, I burnt out quickly. I found myself withdrawing from social settings as burnout set in, and the hill was steeply downward from there. When I don't socialize, my ideas stagnate, my work suffers, and my perspective warps until it's easy to submit to delusions like "I am doing great" when really I was (obviously, in retrospect) not doing great at all.

* The transition to datascience was on one hand fantastic. I have a structure that (1) forces me to socialize daily, (2) forces me to confront my productivity levels frequently. Even when completely remote these past 9-10 months, I find my productivity is far more consistent than it was in graduate school, and my life is far more sane. I have a true work-life balance that I could only fantasize about before.

* The stress is much lower (in my current role). My work no longer defines who I am as a person. Doing what you love sounds great on paper but really what it means are higher highs and lower lows with each small success and failure. While in the beginning of my graduate school experience, I was reasonably well-rounded and rational, as I dived deeper into my own thesis work, I started noticing that I became far less receptive to criticism. My thesis work was so integral to my identity that it became harder and harder to be objective about it.

* Financially I am far more secure, and that is incredibly important to me. I'm not earning anywhere near as much as a $500k AI/ML researcher at FAANG, but I'm also not earning a $30k grad student salary or a $50-60k postdoc salary. I can live extremely comfortably while still building a savings (and am no longer in credit card debt). I also can find a job in the location I want. I don't need to write 20 detailed applications (all during "application season") for roles around the world, wait months to hear replies, and then pack my bags to go wherever I'm accepted. That's a hell of a perk.

* Like Marcel, however, I do find myself missing many aspects of the world of astrophysics. I miss a community completely made up of people at the top of their professional game, making groundbreaking contributions on existentially important things, where everyone seems to share the same "datascience" mindset. I miss symposiums and lunch talks and people with deep passion for what they do. However I have found it incredibly satisfying to work alongside people with other strengths; programmers, product, designers, and managers. These people may or may not have a deep passion for what they do (for some its a job, for others its a passion), but they all have incredibly valuable expertise to share and perspectives to offer. So in that sense, it's not clear to me which world is better.

* As time wears on, I'm finding myself losing some of the knowledge I once had, and that's a bummer. Datascience allows me to keep the statistics channels of my brain somewhat active, but I'm no longer doing things like graphical models and MCMC. I'm trying (slowly) to build a blog where I can do these things as a hobby, but it's difficult to motivate a lot of times; I find that a lot of times it takes me back to the stressful mindset of grad school. I still have three unpublished papers from my graduate school days but the longer time goes, the less of a chance I think these will ever get published, but they stay in their current state because the last time they were touched I was almost losing my mind from stress and I just avoid teleporting myself back to that terrible time.

I apologize for the rant but my point is: your mileage may vary; Marcel makes some very good points but consider all the pros and cons for yourself. As someone else commented, a sufficiently motivated data scientist can definitely get back in the academic game (though it will be more challenging).

Having been a tenured professor who left (sort of, not completely), these issues are really salient to me. I was surprised when I left how many stories I heard of others doing the same. They just aren't mentioned for many reasons, including institutional ego.

Academics in many fields is in a crisis from my perspective. It's not obvious because there's always amazing advancements like the new mRNA vaccine, but it comes at a huge human cost that is rarely covered (pieces like the recent Atlantic article notwithstanding, and some stuff in places like STAT). Also, those advancements are often misleading, because the volumes of stuff that are more typical (like the hydroxychloroquine story) you don't hear about, without the skeptical eye of the world waiting for a much-needed and very-real product.

At least in my field, I suspect the author would have problems returning only because they don't fit the narrative that's required for positions. Again, this depends on field, but a lot of academic hiring, at least at the level of faculty, is about branding of sorts, of building up a story of them being on a meteoric trajectory of genius so brilliant it can't be contained by fate or circumstances of any sort. I've read it referred to as the madonna-whore complex, which is quite accurate. It's hard to explain unless you've been on the side of faculty hiring, but it's very much about looking like the next super-Einstein-Nobel Winner in a room full of people who will never attain anywhere near that much fame (but who think they have), in a scientific process that is really about incremental contributions anyway.

It's all heavily location-dependent too. The author talks about nomadic lifestyles which is accurate, but there's also an element of that which is important to keep in mind: if your nomadic wanderings leads you to a great place, it's great, but if that place is dysfunctional, it's horrible. It's hard for me to say it's a case of grass-is-greener only because it really depends on all sorts of things, including that individual, their connections, and a lot of luck.

I think the pathology of modern academics only increases as you go up the ladder, to be honest, unless you're in a good place. I can't stress enough how much of an oversupply there is at the moment of extremely talented labor, and dysfunctional, corrupt systems that are broken and outdated. There's plenty of academics that is cutting edge and open-minded, but there's also a lot of it that is supremely dated and behind the times, years behind industry due to narrow-mindedness, reinforced by distorted incentive systems.

I keep thinking something has to give but so far there's no end in sight, at least in the US.

I miss the money, of course, but doing research makes me much happier.