Good points, although I agree that a lot of them boil down to similar statements. I have settled on a style of doing "functional-like" programming in Python and C++. It's more about the high level architecture than low level coding details. It means being very paranoid and rigorous about state -- but still having great tools to express it! For example: Using ZERO mutable globals, and passing state in as a parameter t…
I suggest giving some form of parser combinators a try for a more functional approach to parsing/lexing.
I've looked at dozens of parsers, both for shells and for other programming languages, and none of them use this technique. I've studied many forms of parsing, and written my own lexing and parsing library/language. This included going on a pretty deep dive into PEGs.
I'm going to be a bit rude and say that parser combinators look appropriate for toy programs only. shell is the opposite of a toy language... it's fairly big, and has a very precise spec with all the hairs and scars of evolution.