Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Perl Saved the Human Genome Project

drdobbs.com

Post reply on HN
Print view View original

11–20 of 28 posts

Re: How Perl Saved the Human Genome Project

#11

Post by chromatic »

rmoriz wrote: post #4
pasbesoin wrote: post #2

In the same vein, though sometimes with less detail: http://oreilly.com/pub/a/oreilly/perl/news/success_stories.h... O'Reilly also published some of these in at least two folded/stapled pamphlets that were handed out for free e.g. at conferences. I recall a finance-centered application where the Perl prototype far outperformed the subsequent implementation and ended up taking over the production role. It looks like m…

iirc: O'Reilly stopped maintaining perl.com content around that time and finally handed the domain over to the perl foundation a few weeks ago… http://www.perl.com/pub/2010/07/relaunching-perlcom.html

Minor nit: the abandoning occurred in 2008.

Re: How Perl Saved the Human Genome Project

#12

Post by hackermom »

elblanco wrote: post #10
c1sc0 wrote: post #9

Earlier quoted context omitted.

Perl was the only language on the block with strong built-in text-processing capabilities. For many a biologist the Camel book was the only programming book they read before moving on to solve real biological problems instead of fiddling with programs.

It's also performant enough that it wasn't worth the time to learn a faster performing language.

Without going off on a limb: back then, if you knew Perl, you knew C.

Re: How Perl Saved the Human Genome Project

#13

Post by westbywest »

Many moons ago, I worked on an FPGA-based platform that was among several research projects targeted at the Genome Project. The general idea was to offload BLAST-style sequence alignment to purpose-compiled FPGAs, such that sequencing across the entire dataset could be performed in order of magnitude less time. It really wasn't all that complex (I just implemented Smith-Waterman directly, as a demonstration), only intended to perform fuzzy matches at Gbps speeds to winnow the working dataset down to a size more palatable to a desktop workstation.

My understanding is that all these projects (mine included) were cast adrift when the funding for them evaporated in the post-9/11 climate. In the intervening years, I was aware that Perl was being picked rapidly at the Genomics labs in the nearby university hospital (i.e. since we never delivered them the FPGA platform), and I'm happy to read Perl has risen to fill this niche.

Re: How Perl Saved the Human Genome Project

#14

Post by chronomex »

p3ll0n wrote: post #6

In addition to Lincoln's thoughts I think one of the main reasons bioinformaticians are attracted to Perl is because it is forgiving. Biological data is often incomplete, fields can be missing, or a field that is expected to be present once occurs several times (because, for example, an experiment was run in duplicate), or the data was entered by hand and doesn't quite fit the expected format. Perl doesn't particular…

A paragraph very similar to this one occurs in the article.

Re: How Perl Saved the Human Genome Project

#15

Post by sprout »

c1sc0 wrote: post #9
adorton wrote: post #8

Earlier quoted context omitted.

Also, remember that when the project started, Python and Ruby didn't exist yet. Perl still wasn't the only dynamic scripting language on the block, but it probably the most mature and best-suited to this problem domain. I wonder if perl would still be used if the project was started today.

Perl was the only language on the block with strong built-in text-processing capabilities. For many a biologist the Camel book was the only programming book they read before moving on to solve real biological problems instead of fiddling with programs.

From the bit of Bioinformatics work I did in college, that actually seemed a problem rather than a boon. Genes are not ascii sequences, and Perl is not really made to manipulate them.

Perl is however an excellent scripting language, and though it's ugly, it's just as easy as Python to pick up and use. Thus its danger.

Re: How Perl Saved the Human Genome Project

#16

Post by dstorrs »

The part that made me smile was when he said "In all, between one and TERAbytes of data would generated!!!!" [exaggerated emphasis mine]

I've got 3-4 terabytes of storage within a dozen feet of me as I type this; it really drives home the pace of change in computing.

Re: How Perl Saved the Human Genome Project

#17

Post by hartror »

hackermom wrote: post #12
elblanco wrote: post #10

Earlier quoted context omitted.

It's also performant enough that it wasn't worth the time to learn a faster performing language.

Without going off on a limb: back then, if you knew Perl, you knew C.

I learnt Perl and used it on projects long before I learnt C.

Re: How Perl Saved the Human Genome Project

#18

Post by pjscott »

chronomex wrote: post #14
p3ll0n wrote: post #6

In addition to Lincoln's thoughts I think one of the main reasons bioinformaticians are attracted to Perl is because it is forgiving. Biological data is often incomplete, fields can be missing, or a field that is expected to be present once occurs several times (because, for example, an experiment was run in duplicate), or the data was entered by hand and doesn't quite fit the expected format. Perl doesn't particular…

A paragraph very similar to this one occurs in the article.

From the article:

"Perl is forgiving. Biological data is often incomplete, fields can be missing, a field that is expected to be present once occurs several times (because, for example, an experiment was run in triplicate) or the data gets entered by hand and doesn't quite fit the expected format. Perl doesn't particularly mind if a value is empty or contains odd characters. Regular expressions can be written to detect and correct a variety of common errors in data entry. Of course, this flexibility can also be a curse, as I'll discuss in more detail later."

A few words are different. The article says triplicate, and p3ll0n says duplicate, for example. But they are similar enough to use as testing input to a diff algorithm.

EDIT: Also from this guy's comment history:

http://news.ycombinator.com/item?id=1456105

Some of the phrasing looks to have been copied and pasted from this article by Jonathan Ellis:

http://www.rackspacecloud.com/blog/2009/11/09/nosql-ecosyste...

I bet if you could make a bot to do this -- go out and find relevant information, and summarize it -- you could actually provide a serious public service. As long as you cited your sources, so it's not a plagiarism-bot.

Re: How Perl Saved the Human Genome Project

#19

Post by ben1040 »

draegtun wrote: post #3

Slightly related video presentations Curing Cancer with Perl by David Dooling of the Washington University Genome Center: * part 1 - http://blip.tv/file/1997719/ * part 2 - http://blip.tv/file/1998152 * part 3 - http://blip.tv/file/2000983/

Wow, I filmed that video!

I cannot speak officially for the Genome Center, but I'll throw out there that the ORM that powers much of the GC's analysis platform is out on Github and CPAN.

It's actually more than an ORM in that it also supports features like automated creation/smart rewriting of class files based on database tables, quick and easy command modules that get turned into hierarchical command-line tools for free, and an automated test harness that can even parallelize onto an LSF cluster if you've got one.

Github http://github.com/sakoht/UR

CPAN w/ documentation http://search.cpan.org/dist/UR/lib/UR.pm

Post reply on HN