A Fake Shell for Pangenomics
cs.cornell.edu
A Fake Shell for Pangenomics
1–6 of 6 posts
Re: A Fake Shell for Pangenomics
#2I really like the IR-based approach, it solves something that's always bothered me about shell pipelines: you're forced to think in terms of serializing bytes, even when both ends of the pipe are the same program and could just share memory. Flash makes that optimization explicit and easy to compose with the rest of the pipeline. One question, though: have you run into any issues with the "opportunistic" binary format substitution (the .flatgfa fallback) when scripts are shared across machines where some files have already been converted and others haven't?
Re: A Fake Shell for Pangenomics
#3> It might seem odd to prefer shell scripting over a full-featured dynamic scripting language, but shell scripts like this have some material advantages over Python:
And thus 99% of bioinformatics pipelines are shell at their heart... You need 10 packages, written in 4 different programming languages, and the common interfaces are files and pipes.
And for that matter, this could use a named pipe rather than a file (assuming `odgi depth` only uses streaming access):
odgi depth -i chr8.pan.og -r chm13#chr8 | \
bedtools makewindows -b /dev/stdin -w 5000 > chm13.chr8.w5kbps.bed
odgi depth -i chr8.pan.og -b chm13.chr8.w5kbps.bed --threads 2 | \
bedtools sort > chr8.pan.depth.w5kbps.bed
odgi depth -i chr8.pan.og -b --threads 2 chr8.pan.depth.w5kbps.bed
Re: A Fake Shell for Pangenomics
#4> It might seem odd to prefer shell scripting over a full-featured dynamic scripting language, but shell scripts like this have some material advantages over Python: And thus 99% of bioinformatics pipelines are shell at their heart... You need 10 packages, written in 4 different programming languages, and the common interfaces are files and pipes. And for that matter, this could use a named pipe rather than a file (a…
I added a shebang to a readme once (written in literate style) so the poor engineers on the other side wouldn't have to deal with the multi-step monstrosity within.
Re: A Fake Shell for Pangenomics
#5> It might seem odd to prefer shell scripting over a full-featured dynamic scripting language, but shell scripts like this have some material advantages over Python: And thus 99% of bioinformatics pipelines are shell at their heart... You need 10 packages, written in 4 different programming languages, and the common interfaces are files and pipes. And for that matter, this could use a named pipe rather than a file (a…
The "bash is 235x faster than Hadoop" article pops up on HN every so often and this is another great time to link it here:
https://adamdrake.com/command-line-tools-can-be-235x-faster-...
Re: A Fake Shell for Pangenomics
#6> It might seem odd to prefer shell scripting over a full-featured dynamic scripting language, but shell scripts like this have some material advantages over Python
Nothing strange. Shell is the most natural dynamic language. It's a shame we don't have better shells.