Live data from Hacker News

Consider working on genomics

claymcleod.dev

261–270 of 299 posts

Re: Consider working on genomics

#261

> Of course, this would not be the fault of the individuals who maintain the software, who are often brilliant: it's just simply not fair to expect individuals to ensure this consistency using their own, ad-hoc processses I think this is a little generous. Engineers of all stripes should take responsibility for their work. If they say, "Yes I can add methylation analysis in three weeks," then they should make sure th…

> "Yes I can add methylation analysis in three weeks," virtual or reprogram the robotic arm?

Hah. Virtual : - )

Re: Consider working on genomics

#262
Somewhat self-interested plug here: consider working in metabolomics as well. Metabolomics is where sequencing was in ~2008. The physics and chemistry are pretty well worked out (though many improvements are surely coming in the same way that 454 gave way to Illumina, PacBio, Nanopore, etc.). The software and computational workflows are truly awful, like hard to describe bad. The company that figures out metaboloimcs well is going to command a much larger market than genomics - genomics tells you what's possible, metabolomics tells you actually what's happening.

Re: Consider working on genomics

#263
post #219

Earlier quoted context omitted.

> just juggling various bioinformatics file formats Your other points are spot on. This one I want to address specifically. The file formats. Academics love their incredibly over-engineered file formats. MARC. SGML. DICOM. HL7. RDF. Those are just the ones I know. Universally, they try to cover every corner case that anyone could ever imagine. Academics absolutely love their ontologies. Just implementing one of them…

Actually I think genomics / bioinformatics is a counterpoint there. One of the things I like about the field is nearly every file format is under-engineered. It's TSV all the way down and if you need compression gzip it. If you need to index that, sort it (literally often with unix sort command) and block-gzip it. Anything more engineered arose specifically because the above failed and something more is actually need…

Oh, then eventually they'll get a committee together and after a few years they'll produce a unified file format that somehow manages to cover all the cases in the different existing formats (or at least the ones used by well-funded PIs) and is a hellscape of optional properties and required elements so poorly specified that it's impossible for any two implementations to communicate.

Re: Consider working on genomics

#264
post #114

Usually, scientific oriented companies or organizations have little regard for software as a domain, craft, etc. It’s just a thing that gets in the way, despite being vital. It’s almost just a utility to them rather than a differentiator and active component of the advanced work going on. For example, the Broad Institute is super interesting, but having applied there several times, they are esoteric, to say the least…

> Good luck trying to use a functional-first language Good luck trying to use a functional-first language at any company (be in bioinformatics or otherwise).

it happens :)

and the coming years will be interesting, rust is placing a lot of functional bits on the map, just like closures were an obscure thing 10 years ago, there might be a rise in abstraction in the mainstream

Re: Consider working on genomics

#265
post #106

Earlier quoted context omitted.

I think your phrase here sums up how many people feel: > why would I choose to work at an organization that respects my craft so little they haven't bothered to maintain their software for a decade This is changing in my experience, albeit slowly. And really, this is what I'm calling on us, as a community, to do better on. The reason you _would_ work at these organizations is because (1) the subject-matter is really…

How are we "as a community" going to be able to improve this? It's ultimately down to the cultural norms of the field, as well as the realities of academic funding. I was a research software engineer (RSE) for the best part of a decade. The best thing that happened to me was being made redundant when my funding ran out, and being forced to work in industry. What a difference (and wholly for the better). The reasons y…

I've seen a posted phd position that was extremely weak academically, because they just wanted someone with a CS degree to implement their pre-existing ideas, but didn't want to pay a developer's salary.

The position kept being posted multiple times over a couple of years. Then I moved on and don't know what happened.

Re: Consider working on genomics

#266
post #39

Several of the job boards linked don't have any job listings, most don't provide a salary range, several require advanced degrees, and none specify whether remote work is possible. If I can get a better salary and working conditions at some crappy no-name startup, why would I choose to work at an organization that respects my craft so little they haven't bothered to maintain their software for a decade?

In general salaries would be lower than what you can get in a standard software role. Some of the ones on that list allow remote work, others are more limited. The tradeoff is that you're working towards the betterment of humanity. Whether or not that's worth the tradeoff for you is a personal decision.

> The tradeoff is that you're working towards the betterment of humanity.

You're working to give some pharma company something to patent and make millions of $

Re: Consider working on genomics

#267

These should be separate in academia. SWEs cannot write code that maps equations that may change daily completely due to modeling / assumptions change. Too much focus on modularizing, premature optimization, useless unit testing etc. Who cares about all these if the underlying model is wrong? If things are stable enough to go into production then the code should leave academia and be re-written properly by SWEs, not…

Umm..... software weather modeling systems map equations that may change daily completely due to modeling / assumptions change.

Funny you mention weather because I have worked in the field. We had to make a change in the discretization scheme, and pretty much all of the assumptions of the super optimized parallelized production version had to be thrown away, to the point that we had to abandon it completely.

Software engineering needs well defined boundaries to design between them and test edge cases. When we are doing fundamental research we don’t have this. Literally anything can change in the logic, the inputs, the outputs.

Re: Consider working on genomics

#268

Earlier quoted context omitted.

Hmm as an ex-Broad employee (and now in another genomics center), what did you find really toxic about the Broad? FWIW, I really loved Broad the people, my direct line manager and co-workers. The management was horrible and the management at DSP (not the line folks/managers) were the worst.

Since my post was upvoted a tad... I'll give more feedback about the Broad. When I joined, it was running really like an academic center. Like literally in my lab, if I wanted to go into the lab and pipet and do library prep, the wet lab scientist would teach me and vice versa. It was lit. a place where anybody could pivot their career to anything. We worked on NIAID/NIH grants and went to conferences even as SWE's a…

> respect of a lab assistant – unless you have a PhD and a postdoc.

This applies to even those with those with Biology backgrounds, as an undergrad that entered the Industry after an expensive and precarious 5 years of University and exiting during the aftermath of the financial crisis with tons of debt I knew I was never going to enjoy or like my time there within the first months.

I had aspirations to be CLS (you need to be sponsored by a corporation for the training/licensing process) but the truth is the Industry is rife with petty political rivalries where you can get sucked into for no other reason than being assigned to someone's lab that didn't cite them years back--you and your career can easily become collateral damage as result or some other bitter rivalry.

I found most in that Industry to be passive-aggressive cowards who would never confront an issue with anyone or anything and would rather create and foster this toxic atmosphere where it's typical that unless you did a PhD or a Post-doc you might as well be a mindless drone who carries out the edicts of your superiors who graduated in the 70s or 80s.

I will offer this advice: don't enter the Industry unless you get paid extraordinarily more to do so than any other offer you get, and if you love the life/health sciences (as I once did) please find some other outlet because the Industry will quickly steal any passion and leave you without much recourse.

Work in Genomics is promising, as is most Health Sciences in the 21st Century, but it is in DIRE need of a cultural shift (most boomer aged researchers need to die or retire already) and since the best ones are bio-hackers for a reason despite the lack of funding, there are other options albeit not lucrative ones.

> The culture somewhere shifted around 2018-2019

Your experience sounds like the brochure version of what we were sold as an undergrad in the Health Sciences, the ability to have on the job cross-discipline job experience, the reality was way more toxic, we didn't have agile or PM back then but we had Lab Directors and the thumb of corporate which in my view was way more hostile towards such an environment. Anything that deviated from your workload was seen as a unnecessary distraction and misuse of company resources.

I'm glad I made the pivot to tech when I did despite the turmoil to get there, but sadly now that I'm focused on AI/ML in order to come back to tech industry outside of my narrow displine, it's now imploding on itself with mass layoffs or hiring freezes and it seems that the recession will be used a reason to up-end the many reasons why tech was better than the health sciences, where apparently it's already becoming more normal for even a role as an intern for a YC backed company to require a Masters/PhD student!

Re: Consider working on genomics

#269
post #114

Usually, scientific oriented companies or organizations have little regard for software as a domain, craft, etc. It’s just a thing that gets in the way, despite being vital. It’s almost just a utility to them rather than a differentiator and active component of the advanced work going on. For example, the Broad Institute is super interesting, but having applied there several times, they are esoteric, to say the least…

Current Broad SWE with 5 years’ tenure. Feel free to ask any questions. I’m in the “bunch of software people together” department so it’s not as insular or PI driven as working in a lab. I still mostly like the role but it has become more generic over the years as the department acquiesced to the working ways & programming languages of outside private funders.

1. Could you share a bit about your stack? I'd be specially interested about the data engineering side of things if possible.

2. As a SWE, how deep into biology/genetics concepts have you had to go during your tenure?

Re: Consider working on genomics

#270
post #92

What would I need to get started on an open source genomics program? Where do they live? What do they do? Like, do you need a genome interpreter? Does one exist? Are there any open source products used by the field currently? I know the names of the programs and items I'd look at to get started in AI, for example. But for genomics, it's a total mystery.

Some of the software mentioned in these Debian pages might be relevant:

https://wiki.debian.org/DebianGenomics https://blends.debian.org/med/tasks/ https://blends.debian.org/science/tasks/

Post reply on HN