Perl Startups: Lacuna Expanse

What language prevents TIMTOWTDI? And don't say Python. I'm not talking about the language motto. Because that's the nature of using these things called programming languages: http://www.paulgraham.com/langdes.html

Python has Pep8. Perl has Perl::Critic. C# has FxCop. Etc.

I've not yet seen any language possess the qualities that prevent it from being unreadable. It's an amazing open question though, isn't it?

I think you hit the point with "code maintenance can become a huge headache due to someone not understanding the idioms used by the person who originally wrote the code". But this isn't a language thing. This is a people thing.

This is true, BUT there is a law of diminishing returns. The value of a large library scales nonlinearly. The added packages are more likely to be unrelated to whatever you are doing, and so become progressively less likely to be useful.

That said a culture of testing all of the time makes the Perl packages less likely to have serious conflicts. Which is a huge value proposition. (For the Ruby fans who jump up to object, do you run unit tests by default every time you go to install gems? Do you have people volunteering lots of machines with different things installed for the purpose of running unit tests on everything in Rubygems? No? Then you don't know what a "culture of testing all the time" is.)

I have some experience in this area. My first startup: Grazr (http://tcrn.ch/aUqiIb), the back end was mostly Perl with some custom C modules. Grazr was started in 2005 when Perl was already out of fashion.

So why did we choose Perl, practical reasons mostly. My co-founder had previously been the CTO of the company that owned Slashdot and he had a team of top-notch Perl people that he thought would want to join us. I had worked a lot with Perl in a previous life so it seemed like a natural fit.

Perl has many strengths. You can "just get stuff done" with Perl. The module library is obviously a huge strength. The first version (effectively a public prototype) of Grazr went live very quickly (days). If you’re careful with it, you can also build some fairly complex projects using Perl. The machinery for Grazr processed a huge amount of data (for the time) multiple terabytes of feed data and had a lot of advanced features like: processing of normalized feeds, an embedded scripting language, cross link analysis, a search engine, and most of the heavy lifting was Perl. Think of the system as an advanced form of the feed API that Google Reader eventually released, nearly real-time (unlike the Reader API we updated feeds upon request) with the addition of a scripting layer on top. I'm very proud of what we built, technology-wise. We failed in other ways (UX, marketing and sales mainly) but the underlying system was quite impressive.

When the feed application language and processing engine stopped being the focus of the company, the Grazr system ran on only a few virtual servers, responsively and smoothly for over a year with steadily growing traffic. That was with the whole company ignoring it and focusing on other 'pivots' (before pivot was the term-du-jour for that). Why the company ignored a service that was steadily growing traffic with absolutely no input from the team/company is a rant for another day. :)

When I co-founded my next company, Smarterer (http://smarterer.com/), I vowed gone-with-the-wind-style that as God as my witness, I would never write a big startup-project in Perl again. Smarterer is written mainly in Python.

Why the Perl hate? First, I still think Perl is a fine language. You can get a lot done with it. But as other's have pointed out, one of the primary design philosophies of the language make it painful in some cases. Large, evolving projects with multiple contributors seems to be the axis of evil for Perl. Startups exacerbate this problem because they exhibit most of these factors and compound it with unrealistic functional time pressure.

What I observed as problems were:

First, with Perl there is always a temptation to be clever. It's a lot of fun, but one of the underlying philosophies of Perl, the "Tim Toady" thing, opens up a lot of temptation to show just how clever you are, usually too clever by half. That always leads to the WTF moments as you’re reading any code you haven’t written in the last week. I started to feel each WTF moment as a tax on our tech team.

When you have a big Perl project with several contributors, coming up with coding standards is a must but usually in a startup environment getting something working is almost always the first priority. Also, coding standards only get you so far with Perl because it's so flexible. If you spend any reasonable amount of time, the unwritten standards of how people write Perl code subtly shifts over time. Usually the code get better as contributors influence each others coding styles, but this makes going back to read working code that’s 6 months to a year old a massive pain in the ass.

Lastly, having coding standards doesn’t protect you against other peoples modules. If you have a large and complex enough system, you’re going to start tripping over the bugs and corner cases in other people’s modules. Perl has the strength of CPAN, but because you can’t enforce a coding standard across all of Perl, debugging into other people’s modules can be an exercise in frustration. When some bug is tracked down to a third-party module, treating it as a black box is no longer an option.

When faced with these problems you can usually work through each case. In the best case it just takes a few minutes of working through the cognitive dissonance, but in other cases you end up breaking out the print statements and/or debugger to instrument code just to figure out what the heck was the intent here.

As a startup guy, the energy required to “re parse” code that’s already been written in subtly different ways starts to drive you crazy because that’s time and energy that could be being applied to solving the bug and tackling the new problem at hand. That re-read and re-parse tax gets worse and worse the large the project becomes and the more people that are involved.

That’s, generally, why I wouldn’t recommend Perl as a language for a startup project, but every case is different. If you're not in the axis-of-evil for Perl, you have incredible discipline and you only rely on a limited number of third-party modules you may be OK.

Just a side note: somehow the official Perl website is one of the fastest websites I've come across on the web. Is there a reason for this ?

No. If it were, I wouldn't even consider using it.

Switching between multiple interpreter installations is pretty easy (a drop down in a preference pane) -- it will find every interpreter on your path, and can index anything defined in $PERL5LIB for auto-complete goodness.

The debugger is great -- I love being able to set conditional breakpoints using a possibly complex expression in the native language (not just simple var=val type things).