Live data from Hacker News

Consider working on genomics

claymcleod.dev

151–160 of 299 posts

Re: Consider working on genomics

#151
post #98

Look, I did this at multiple places for a number of years. The issue is that you often form an adversarial relationship with the scientists. They don't really want you there. They are perfectly happy just organizing everything by hand with post its and excel spreadsheets. They do not want you to mess with their flow with your software, even if it would help them to be more efficient.

Can you elaborate with some anecdotes? Why is their current workflow wrong? Why would an organization hire someone to build software if they can achieve goals with spreadsheets?

The fact they renamed human genes because they were importing it in Excel in a way it thought they were dates and was changing them says a lot.

Can you really trust the scientific results if they depend on software made by people who don't care about code quality?

Re: Consider working on genomics

#152
These should be separate in academia.

SWEs cannot write code that maps equations that may change daily completely due to modeling / assumptions change.

Too much focus on modularizing, premature optimization, useless unit testing etc. Who cares about all these if the underlying model is wrong?

If things are stable enough to go into production then the code should leave academia and be re-written properly by SWEs, not by clueless bio phds.

Re: Consider working on genomics

#153

20 years ago I got interested in "bioinformatics." I loved learning something about molecular biology, after all those years of hearing about DNA and not understanding it. And "Molecular Biology of the Cell" is, hands down, the greatest textbook ever written. That said: a lot of the comments are spot on. You're working in a field where the hard scientists and business people rule and you're a helper. Maybe they're gr…

Molecular Biology of the Cell got me extremely excited about genetics and bioinformatics, highly, highly recommend this book to any software person I meet who is interested in biology.

As to the work environment, it seems to be extremely varied depending on the lab and team your on. I came from a number of years doing web development in marketing and finance before joining an R1 university research lab, and in many ways the day-to-day is quite similar in both fields. You are not the 'go-to' person for most things, but with that said, even as an individual contributor I feel my voice is heard on technical decisions where appropriate. As for pay, it's the biggest aspect that will make me leave at some point. If you do not have a PhD, or even a degree in my case, you can't expect to get paid a lot. As to the speculation on the satisfaction of the work, it is indeed deeply satisfying!

I got to have a conversation with one of the hero donors that gave a kidney biopsy after a life-saving transplant. It's hard to overstate just how impactful your work feels when talking to someone like that. Even as a small cog in the larger machine (our lab is around 50 strong with many people being at the top of their sub-fields), the end results of the effort will be massive improvements in individuals quality of life, this alone makes it quite easy to get out of bed in the morning.

Re: Consider working on genomics

#154
This is tangentially related to what I'm currently doing.

I basically work in EdTech. The company is not an EdTech company, it's a education services company. I was hired on to develop software that we couldn't find in the market[0].

I'm the process of building this thing, we've been attending and speaking at conferences in our industry. And I'm seeing a lot of the same stories: academia is trying to do research, the research fundamentally requires software to make the research happen, the quality of the software can have a huge impact on results, but because software development is tangential to the research goals, there's little to no allocation to software developers. This leaves the researches to cobble together a solution that maybe kinda fulfills their need, not corky, and certainly not perpetually (a lot of reliance on trial software and services).

We would love to offer our software to researchers in our field. We've gotten feedback from several that what we are building is exactly the sort of thing they need. But they have no money, and even if we were in a position to give it away for free, we can't even make those connections come to fruition.

So I don't know what to do. I really am thinking of starting to give it away for free, because at least we'd benefit from more research results in our field pricing the efficacy of our approach. But that's a really slow burn.

[0] Specifics don't matter, but if you're curious, I make a VR environment for foreign language training emphasizing culture.

Re: Consider working on genomics

#155
When considering software roles in science organizations, forget assumptions you might make about a typical tech job, joining a bunch of other software and hardware people -- or you'll risk accidentally ending up on the other side of a distorted status system (not the side that normally pampers techbros).

You need to feel out the particular person you'll be reporting to on how well they personally respect and understand the role, and also whether they'll have clout/funding and have your back if the org turns out to be rough (think AMZN). And also try to feel out respect within the organization, and some of the people/teams with whom you'll be collaborating.

You also need to check compensation, so you don't wind up a low-paid person who later discovers they're competing for local house offers with others in the org who are getting big-bucks TC (plus consulting on the side).

You also probably have to be OK with never being the star (like you hypothetically could someday be in a software company). Supporting actors should still get respect and get paid.

Find the right science situation, and you might have much more positive impact on the world than you could have in a software company, while also being happy and comfortable.

Some more quick of-the-cuff comments about this (sorry for run-ons, but I need to get back to my weekend)...

* RESPECT -- Whether or not the organization is university-affiliated, a lot of the researchers and administrators might have only worked in academia-like environments before. Academia is very hierarchical, software engineering might be considered commodity technician or support staff, and the high-status people almost certainly don't understand your discipline, though they might think they do. (They often think software is relatively easy grunt work, and that software people just have oversized egos, which has some truth to it, but not that much.)

(Some real-life instances of this I've heard of include: someone with no understanding overriding software engineering technical decisions, because a colleague from their academic caste made an offhand comment, and they assume an academic who hasn't even looked at the system knows more than an experienced practitioner developing it; not wanting to include people who made key software contributions as coauthor on a paper for a software system, but making sure professors who had near-zero involvement were included; scientists openly speaking of the software people as having commodity interchangeable skillsets, in way they'd never speak about peers in their domain; getting an unsalvageable monstrosity of pasted-together incompatible frameworks and Stack Overflow posts done by a summer intern, dumped on software engineer to "clean up" or "extend", and being unable to convince that this is orders of magnitude harder to fix than to just make a viable system in the same time the intern took; in an academic environment, a grad student being higher status than key software people, and bossing them around with bad decisions, while treating their own obligations like homework they were trying to sneak past a grader rather than as a system that has to actually work.)

* COMPENSATION -- Related to the above. If you're very experienced and marketable in tech, and would be making key enabling contributions, are you getting paid like it?

(The most recent life sciences software engineering opportunity I talked with, with a high-profile organization, they needed FAANG-like Staff/Principal experience in multiple areas, all-in-one person, for key bespoke computational infrastructure on which a lot was riding. When we got to salary, it was capped at less than a new grads were getting offered elsewhere, and despite being in a top HCOLA city. The recruiter half-heartedly argued about it being for the science, etc. I said, if they're thinking of this as an academic non-profit, that would be OK, so long as everyone there is making this level of money. But that wasn't the case: the science domain people were considered the valuable assets, making good money, and software was seen as more a commodity support skill by whomever set the pay grade. Maybe within a decade that will agree with the market, everyone will decide that someone who can learn organic chemistry should get paid more than someone who doesn't seem to do much more than fingerpaint in a Web framework builder and type nonsense in Jira, :) and maybe then most software people will be thankful for any job at all, but not yet.)

(I did actually look at a science company with a strong software tech company influence. But, though they claimed to be rethinking how the tech company did things, they seemed to carbon-copy the single most obvious bad side of that company. Talking with colleagues after I withdrew my application, the gossip was that they were getting lots of software people who'd burnt out on the tech company. So I guess maybe the rethinking was on what had been bothering those people, who were already at the tech company, and so who weren't entirely representative of the talent pool that included people for whom the tech company had showstoppers.)

Re: Consider working on genomics

#156

I've worked on genomics, at the Broad. Can confirm there is a ton of toxicity. There's also a lot of smart and great people, if you can find a team specifically dedicated to software I would recommend it. Also the pay is good...for a non-profit. More broadly...shortages like this aren't because SWEs just love ad-tracking and hate health improvements. People need to be willing to pay for these services (case in point;…

Hmm as an ex-Broad employee (and now in another genomics center), what did you find really toxic about the Broad? FWIW, I really loved Broad the people, my direct line manager and co-workers. The management was horrible and the management at DSP (not the line folks/managers) were the worst.

Since my post was upvoted a tad... I'll give more feedback about the Broad.

When I joined, it was running really like an academic center. Like literally in my lab, if I wanted to go into the lab and pipet and do library prep, the wet lab scientist would teach me and vice versa. It was lit. a place where anybody could pivot their career to anything. We worked on NIAID/NIH grants and went to conferences even as SWE's and I felt we were doin' important research - not just pipeline monkeys but actually performed important analysis like RNASeq differential analysis, ChIPSeq peak calling, metagenomics etc. on publications along with PhD scientists even if I didn't have the academic credentials. Groups within the Broad was running courses for Software Engineers to learn Biology... and you could literally take off middle of day to go across the street to Stata Center to attend lectures on ML or audit Comp Bio at MIT. Nobody would bat an eye. 75% of my group got a Masters degree on the job where we spent more time some months on classes than actual work.

The culture somewhere shifted around 2018-2019... where they brought in new management to run DSP (Data Science Platform where most SWE's likely end up). The DSP management (not the chief guy) but lieutenants ran the "tech playbook"... get PM, Scrum/Agile coaches in; make software and comp biologists line workers.A lot of my fav. people either left or got pushed out. A lot of intuitional knowledge about sequencing and biology got lost, self-driven people left and line workers to work on Portal web development and Data pipeline management came in. To the point where I presented once to the software engineers of DSP and nobody in the room even knew the basic's like what is a long or short reads is. I left soon afterwards to a place where I wouldn't be silo'd. I wouldn't recommend the Broad to anybody now... unless you're working for an academic group. Avoid platform groups (DSP mostly; other platforms are still good) if you want to learn & grow.

Re: Consider working on genomics

#157
post #95
post #73

I recently switched from software engineering on ads and web performance at a FAANG to (meta)genomics at a nonprofit startup; happy to answer questions

From a genomics layperson with decent dev skills: A. What are the broad and medium goals you work on? B. What are your daily activities? How do they fir into (A)? C. What does nonprofit genomics vs for profit look like from a revenue standpoint? D. What specific technologies/stacks are you using? E. The CRUD frontend+backend+database to serve users (and sell ads) is pretty ubiquitous in 'tech', with some branches. Ho…

A: The goal of my current project is to identify novel pandemics, even if they're caused by something we've never seen before, most likely by looking at growth patterns. At a broader scale, I'm trying to learn enough about working in this field that I'll be able to contribute on whatever future projects seem most appropriately important to me.

B: unlike my previous work, I'm back to being an individual contributor. Very few meetings, mostly coding and analysis. Current thing is trying to understand what drives per-sample variability in wastewater sequencing data.

C. Our group is currently philanthropically funded, and is focused on determining whether/how this is possible/practical.

D. I'm mostly working in (bio)python, with a few bits written in C and gluing things together with bash.

E. I was most recently working on (a) JS infrastructure to fetch and render ads and (b) working with browsers on platform features that would improve privacy, security, and efficiency.

Re: Consider working on genomics

#159

What is the opportunity here -- writing new algorithms, implementing them accurately, optimizing them for special execution architectures, or just building more usable tools? I remember Manolis Kellis sprinkled some pretty interesting genomic questions into his Algorithm class's problem sets. There were a number of cool problems about optimally aligning strings, searching within text, etc. This was like 15 years ago…

We're only starting to see the age of genomics accelerated by GPUs. I think it's still early if you have the technical background.

Edico developed FPGA-based processing solution for common bioinformatics processing tasks (e.g. dna/rna mapping, variant calling) and the company was bought by Illumina.

The product (Dragen) has been around for a few years and now will be integrated in the new generation of sequencers. Extremely impressive technology and a better fit for the niche compared to GPU-based solutions I have seen. More downstream processing and analytics is sometimes closer to traditional ML and naturally there are lots of GPU-based algos.

Re: Consider working on genomics

#160
post #129
post #39

Earlier quoted context omitted.

In general salaries would be lower than what you can get in a standard software role. Some of the ones on that list allow remote work, others are more limited. The tradeoff is that you're working towards the betterment of humanity. Whether or not that's worth the tradeoff for you is a personal decision.

> you're working towards the betterment of humanity If only this was actually true. In reality, you're being taken advantage of and it's very manipulative and deceptive to make this claim.

Agree that not everyone will buy this. But it is why many people are working in positions that are otherwise worse on paper. It's up to them to decide if its real or not.
Post reply on HN