Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Consider working on genomics

claymcleod.dev

Post reply on HN
Print view View original

221–230 of 299 posts

Re: Consider working on genomics

#221

Post by theGnuMe »

Couple of things I know.

Bioinformaticians come in two flavors. Those that studied biology and then took up coding and then the even rarer computer scientists who learned biology. The latter are so rare that they are almost all professors or founders or work at Deep Mind etc... Then, there are the biomedical engineers, etc...

The computer scientists will go off a solve protein folding when the bioinformaticians and chemists worked on it for years.. I am exaggerating a little here, I imagine there were plenty of bioinformaticians on the Alpha Fold team, but the fundamental breakthrough was DNNs.

Re: Consider working on genomics

#222

Post by jltsiren »

mherdeg wrote: post #136

What is the opportunity here -- writing new algorithms, implementing them accurately, optimizing them for special execution architectures, or just building more usable tools? I remember Manolis Kellis sprinkled some pretty interesting genomic questions into his Algorithm class's problem sets. There were a number of cool problems about optimally aligning strings, searching within text, etc. This was like 15 years ago…

> But is there still algorithmic low hanging fruit?

Algorithmic bioinformatics has become a separate research field, because there are so many low-hanging fruit. Biotech companies create new instruments producing new kinds of data, researchers find new uses for the data, and new algorithmic problems emerge all the time. There is also a steady migration of people from theoretical computer science to bioinformatics, because it's often easier to get research funding for something bioinformatics-related than for pure CS.

Re: Consider working on genomics

#223

Post by danking00 »

I don't have any funding to hire right now, but I'm always happy to chat about the industry and my experience building Hail (https://hail.is, https://github.com/hail-is/hail), a tool widely used by folks with large collections of human sequences.

The other posters are not wrong about compensation. Total compensation is off by a factor of two to three.

However, it is absolutely possible to work with a group of top-notch engineers on serious distributed systems & compilers in service of an excellent scientific-user experience. I know because I do. We are lucky to have a PI who respects and hires a diversity of expertise within his lab.

I enjoy being deeply embedded with our users. I do not have to guess what they need or want because I help them do it every day.

I also enjoy enmeshing engineering with statistics, mathematics, and biology. Work is more interesting when so many disciplines conspire towards the end of improved human health.

Re: Consider working on genomics

#224

Post by sargstuff »

clmcleod wrote: post #197
aWidebrant wrote: post #134

"From my experience, what works incredibly well is a partnership between biologists and software engineers: the biologists first come up with the first concept of the tool, which is purely focused on ensuring good results. After this first iteration is completed, engineers then come in and rewrite the tool using modern engineering practices with things like speed and reliability in mind." Like others have pointed out…

Yeah I think this is fair enough after reading it back. However, that was not exactly my intention here, and I think this is a case of me needing to be more careful in my wording. When I said that software engineers add in the speed and reliability, I didn't mean they _only_ add in the speed and reliability: just that these two tenants of good software engineering where accounted for in this "correct" way of doing th…

software engineer provides/developes the appropriate level of abstraction for the non-software engineer to make use of.

Which if there's no standard for field, and working outside of a given field, makes writing grant(s) without paring up with someone who can develop field standards to be included in grant necessary. Hard to find/compete for scarce applicants using limited resources.

aka startups vs. big company funding for pure research lab (bell labs, xero parc, etc)

Re: Consider working on genomics

#225

Post by kzuberi »

aWidebrant wrote: post #134

"From my experience, what works incredibly well is a partnership between biologists and software engineers: the biologists first come up with the first concept of the tool, which is purely focused on ensuring good results. After this first iteration is completed, engineers then come in and rewrite the tool using modern engineering practices with things like speed and reliability in mind." Like others have pointed out…

> this really makes the engineer's end of the bargain sound like janitorial work

I don't think you should interpret it that way. Another take would be that its like collaborating with a domain expert outside your specialization.

Important is that your potential impact as an engineer can grow as you become more knowledgeable in the relevant bio. Most of the scientists I've worked with were happy to teach background (and some were just exceptional, fun times if you also found the field interesting as I did!). Obviously some allowance must be made for differences in culture from org to org, and that likely accounts to some of the disappointed voices - but I'm not convinced this is endemic to the field as opposed to organization specific. Just like with an opportunity with any particular company, do your research.

Incidentally, working on a well defined engineering+optimization problem, if you are lucky enough to bump into one, is just candy for lots of engineering types. Ok quick & simple one: a scientist I worked with was doing some analysis that involved intersecting piles of genomic intervals with each other, which was taking many hours for a single run - super painful to tweak and re-execute. Our team showed them how to use interval-trees and made these available integrated in our internal tools, and the problem transformed into ~10 min execution runs. See, a wee a bit of comp-sci where suddenly you're the domain expert. And appropriately appreciated!

Re: Consider working on genomics

#226

Post by zmmmmm »

wesleywt wrote: post #135

The code is bad because transient Phds and Post-docs are writing it. If there was money in it then the best software developer would already be working on it. Sadly there is none.

yep ....

One of the borderline fraudulent aspects of the field is the pretense that method publications are real software.

That is, you come up with a break through statistical or algorithmic method, you get it to run exactly once based on whatever random walk of exploratory code got you to a result that looks better than competing/prior methods, and then you dump your workspace into a script and put it on Github and pretend this is something anybody else could or should responsibly use in your Tier 1 publication. The minute the publication is approved there is zero benefit to the authors in maintaining the software, and in fact its better if nobody can run it because that way they can't disprove your results. Then naturally nobody can get this to work afterwards and 50% of software engineering time and effort is trying to run code that can/never will work outside the context it was created in - but you have to try because this is now the accepted best practice method of doing X or Y based on its publication.

The bigger problem is that this whole cycle actually shapes the view of software engineering by academics to the point where they really do think that most software engineering is a waste of time. A small number of 10x engineers manage to prosper in the environment, but it's mainly because they have the sheer technical capability to deal with ALL of that while still doing something useful, and it actually makes the problem worse because the academics then see that as the baseline for software engineering capability.

Re: Consider working on genomics

#227

Post by neilv »

bmitc wrote: post #114

Usually, scientific oriented companies or organizations have little regard for software as a domain, craft, etc. It’s just a thing that gets in the way, despite being vital. It’s almost just a utility to them rather than a differentiator and active component of the advanced work going on. For example, the Broad Institute is super interesting, but having applied there several times, they are esoteric, to say the least…

> I once saw a Python signature (function name and arguments) spill over 10-20 lines,

Quote I liked (can't find attribution; maybe Alan Perlis?):

"If your function has 10 arguments, you're missing some."

Re: Consider working on genomics

#228

Post by sargstuff »

theGnuMe wrote: post #221

Couple of things I know. Bioinformaticians come in two flavors. Those that studied biology and then took up coding and then the even rarer computer scientists who learned biology. The latter are so rare that they are almost all professors or founders or work at Deep Mind etc... Then, there are the biomedical engineers, etc... The computer scientists will go off a solve protein folding when the bioinformaticians and c…

biologist / chemist will take the architecture studio approach, then develop math to shorten the write-up.

research software engineer will develop the mathematics to describe things, then use the numerical system to write software to determine things.

Re: Consider working on genomics

#229

Post by hobofan »

jltsiren wrote: post #213
hobofan wrote: post #93

> the biologists first come up with the first concept of the tool, which is purely focused on ensuring good results. After this first iteration is completed, engineers then come in and rewrite the tool using modern engineering practices with things like speed and reliability in mind. I think that is already accepted as good practice, and the way most people in the field work, which is part of the reason why the field…

> As with most other kinds of software, the biologists should be treated as customers (or trained up to be skilled-enough engineers), as it is done in other disciplines. To create good accounting software you also wouldn't propose to have the accountant write the initial version of the software, would you? Accounting is a bit different, because it has already been invented. There are standards and best practices for…

Accounting is not a static thing, and is also constantly changing with new legislation and financial instruments popping up. Most bioinformatics tasks nowadays are not any more "creative" in their research. Specifically in the last few years a good chunk of the research is just okayish application of ML research to their field of research.

For many specific problem sets in the natural science informatics disciplines, you can just stay up-to-date on ML trends and release a new paper that applies them every few years, in an almost automatable way.

Re: Consider working on genomics

#230

Post by sargstuff »

jonathanyc wrote: post #203
rafiki6 wrote: post #124

Earlier quoted context omitted.

I wish more fields would just start adopting the product/engineer partnership that Software companies have perfected. Engineers are very good at what they do. Product people are very good at what they do. They need each other to build things. Sure, engineers might know enough about product to get by and product people might know enough about coding to get by, but the reason it works is because each one is an expert i…

I have heard from a friend who's a doctor that in hospitals there's a very adversarial relationship between doctors and MBAs. The MBAs see the doctors as a cost center, and the doctors resent people without MDs being above them. Your comment reminds me to be thankful that at many software companies engineering, product, and design do respect each other as equal partners. I totally agree that to do otherwise is busine…

to very opposing philosopies:

MD's -> patient interest comes first

MBA's -> company interest comes first

Post reply on HN