Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Consider working on genomics

claymcleod.dev

Post reply on HN
Print view View original

241–250 of 299 posts

Re: Consider working on genomics

#241

Post by sargstuff »

clusterhacks wrote: post #23

I work in this field at a large medical research institution. There is a significant amount of genomics analysis that occurs here on a day-to-day basis. The genomic processing pipeline work all falls directly into my group. There is next to zero demand for tool development internally. I do it on the side of "normal" IT data management because I love high performance computing, algorithms, and multithreaded hackery. B…

Ah, sounds so much like history of programming. At the 50's stage of straight up statistical manipulation.

DNA base units not viewed as base 4 binary number system that can be transformed into an abstract software language, where can select abstraction level of choice to use. Much like musical notation not viewed as numeric system.

Although, most software engineers don't view systems as numerical language development, too.

Re: Consider working on genomics

#242

Post by geoffjentry »

laidoffamazon wrote: post #234
bmitc wrote: post #114

Usually, scientific oriented companies or organizations have little regard for software as a domain, craft, etc. It’s just a thing that gets in the way, despite being vital. It’s almost just a utility to them rather than a differentiator and active component of the advanced work going on. For example, the Broad Institute is super interesting, but having applied there several times, they are esoteric, to say the least…

I live next to Broad's offices and see people leaving/entering the office at odd hours on Saturday and Sunday. That (and the fact that they pay about 75% what I made as a new grad) prevented me from ever applying there.

Keep in mind that there are wetlabs with experiments being conducted in them. Lab techs will be coming and going at all hours.

Re: Consider working on genomics

#243

Post by cratermoon »

pestatije wrote: post #15
dankle wrote: post #2

> There is a significant gap between how software is currently developed in this space versus how it should be developed. The vast majority of genomics-related software is not written with speed or reliability in mind. True, but working in academia is very VERY different working in a tech/product company.

> This state of affairs makes it difficult for anyone other than the original author to contribute to these code bases, further cementing the one-maintainer policy. Who wants to fix other peoples code mess? This is a no-no if you want to promote a job opening.

I do. It's my bread-and-butter. I call myself a code janitor. I live by books like "Working Effectively With Legacy Code" and "Kill it With Fire". But I have my limits. Academic code has.. coded, in the medical sense, and it can't be revived. Put a DNR on it.

Re: Consider working on genomics

#244

Post by geoffjentry »

bmitc wrote: post #231
noname123 wrote: post #156

Earlier quoted context omitted.

Since my post was upvoted a tad... I'll give more feedback about the Broad. When I joined, it was running really like an academic center. Like literally in my lab, if I wanted to go into the lab and pipet and do library prep, the wet lab scientist would teach me and vice versa. It was lit. a place where anybody could pivot their career to anything. We worked on NIAID/NIH grants and went to conferences even as SWE's a…

This is a great read. Thanks for the information. What you originally described is basically my dream job: software engineers working alongside scientists and engineers, where the software engineers become domain knowledgeable if not experts in certain areas. I had a job similar to that at a similar places (actually places), but I ended up leaving because I was a one man team and got burnt out. Writing software for s…

A really hard aspect to this is that there's a massive impedance mismatch between the research & production side of things. Working in the research side is pretty straightforward - although software development practices are going to be a lot looser & faster. Working in a production environment is straightforward, it's like any other software job. But - working at the confluence of those two states is incredibly difficult.

Re: Consider working on genomics

#245

Post by teekert »

tonto wrote: post #187
teekert wrote: post #168

I work in genomics, this is very true. I know of some modernization efforts, ie by companies working with new file formats, like GenomSys [0] with mpeg-genomics [1]. It feels like it’s going very slowly though. The field just really depends on their Unix philosophy tools, there is a lot of gzipped text files that are piped through bash scripts and tool like awk and grep. It works, mostly, but there is a lot of weirdn…

mpeg-g to me is probably bad for the field. sam/bam/cram is the way of the present and mpeg-g offers little over these formats and is patent encumbered. xref samtools developer blog http://datageekdom.blogspot.com/2018/09/mpeg-g-ugly.html?m=1

Hmm I agree with James. I bring the same points to my employer actually, they sort of listen but they like IP.

Re: Consider working on genomics

#246

Post by njbooher »

attractivechaos wrote: post #150
olalonde wrote: post #103

Earlier quoted context omitted.

Is it feasible to do any meaningful work in this field without joining a team? (e.g. as a solo hobbyist/entrepreneur)

Difficult but possible. For example, Robert Edgar [1] works alone and is one of the most productive developers in this field. [1] http://drive5.com

I don't work in this space anymore, but just want to say kseq (and the rest of klib) is such an awesome time saver. Thank you.

Re: Consider working on genomics

#247

Post by sargstuff »

sargstuff wrote: post #241
clusterhacks wrote: post #23

I work in this field at a large medical research institution. There is a significant amount of genomics analysis that occurs here on a day-to-day basis. The genomic processing pipeline work all falls directly into my group. There is next to zero demand for tool development internally. I do it on the side of "normal" IT data management because I love high performance computing, algorithms, and multithreaded hackery. B…

Ah, sounds so much like history of programming. At the 50's stage of straight up statistical manipulation. DNA base units not viewed as base 4 binary number system that can be transformed into an abstract software language, where can select abstraction level of choice to use. Much like musical notation not viewed as numeric system. Although, most software engineers don't view systems as numerical language development…

difference in view between qualatative & quantitative usage; NP vs. P type problem(s).

Re: Consider working on genomics

#248

Post by tejtm »

ninala wrote: post #214
YouWhy wrote: post #119

I am a career SW engineer that has worked on genomics in a startup. The field is genuinely exciting. The endemic disease of the field is the leadership. A leadership made out of Principal Investigators forged in academia, appear simply incapable of producing any item which is not articles (or equivalents thereof).

Do you think that's true of pharmaceuticals/biotechs as well? Or just academia?

Decades ago my very very bright HCI prof commissioned a psyc study for a database we were building for some biologists next door, you know so we could better address their needs in ways that would be useful to them. Details are pretty fuzzy anymore but they proved correct many times over.

Things the study said would not work never worked i.e. biologists wanted "temporarily" private data, say until till published as psyc predicted they would never freely share it.

but the biggest thing I will try to paraphrase:

Biology is an observational, the work is in interpreting which lends to group dynamics and politics, leaders ect.

Which is at odds with Math/CS which is constructive where if something can be proved then that is that.

So when a CS person states a fact from their perspective a biologist might see it as just another opinion subject to hierarchical ranking.

So I would argue it is a function the individuals proclivities and correlated training in the cultural environment they end up in.

So a healthy work environment could value both fact and opinion where each has a complementary role whether academic or industry.

But as a longtime academic, I am now sadly looking towards industry.

Re: Consider working on genomics

#249

Post by sargstuff »

debacle wrote: post #11

Science programming jobs suck. You get all the bad parts of academia, including less money, plus you're seen as a janitor rather than an engineer, and you get to deal with scientists all day. Tooling roles in SWE in every other field are highly regarded. Why not here?

Well, math / computational power for simple, static protein modeling is horendus.

Re: Consider working on genomics

#250

Post by boldlybold »

331c8c71 wrote: post #159
boldlybold wrote: post #142

Earlier quoted context omitted.

We're only starting to see the age of genomics accelerated by GPUs. I think it's still early if you have the technical background.

Edico developed FPGA-based processing solution for common bioinformatics processing tasks (e.g. dna/rna mapping, variant calling) and the company was bought by Illumina. The product (Dragen) has been around for a few years and now will be integrated in the new generation of sequencers. Extremely impressive technology and a better fit for the niche compared to GPU-based solutions I have seen. More downstream processin…

I'm more excited about NVIDIA's acquisition of Parabricks and the version 4.0 of the software that makes it free to use, than I am about DRAGEN. At the very least it's good to have some competition in the space, Illumina's stuff is always SO expensive. We'll have to see what hardware will win in the end.
Post reply on HN