Clojure-powered Startups

Clojure-powered Startups
infoq.com

This I think is the key bit of the story:

"At this point, we were still focused on using Ruby/Eventmachine as a cornerstone of our technology stack. But here we hit a snag. Yep. I’m gonna beat up Ruby, because it was a mistake.

...it doesn’t scale. ::ducks::

OK, that’s not completely fair. Ruby does scale. But it doesn’t scale well, or easily, and in benchmarking and stress testing, I was seeing that we were going to have a use a small truck load of resources at AWS, or spend a bunch of preciousssss developer time making it scale. It was too expensive. I wanted high user::machine density, and I didn’t want to have my developers do handstands to get it.

Yeah, I could’ve done lots of things. I didn’t have time to do that shit. Not with a four person team. Not with cash running out, and promises to keep, and no time to work for Ruby. I needed something that would work for me. This thing had to be FAST. It had to drive hardware to the limit without driving us crazy.

The bottom line is that it was too much work to make Ruby go fast enough.

...A little background on our application might help. Our job is to give prospective hotel bookers a view into what their options are. At the time we were making the decision to migrate from Ruby to Clojure, the system was using so-called “realtime” rates and availability checks with hotel suppliers to get that information. That meant that when a visitor conducted a search, we would spin up dozens of individual HTTP requests to hotel supplier sites to get rates and availability data at that very moment. We’d parse the responses, collate them, and present them back to the UI in just a few seconds.

You might ask why we didn’t cache that information, but suffice it to say there were significant business drivers for that decision.

Managing this concurrent (and long-running) IO was a major theme for us, and Ruby did so reasonably well using EventMachine. However, we had to normalize the returned data into a single unified data model, and none of our partners had simple (or compact) XML representations of the data, so not only did we have IO issues, but CPU-bound processing issues as well. The combination of the two made the EventMachine implementation suffer from less-then-stellar throughput, and because of Ruby’s green threads implementation and global interpreter lock, we had to run oodles of Ruby processes on each box to achieve reasonable throughput.

Perhaps just as importantly, the reactor pattern’s upside-down flow of control style of programming was (and is) a pain in the ass. It was hard to read, hard to maintain, and generally obstreperous.

..I can already year you Ruby folks protesting “Fibers!” and so on. Heh. Have fun storming that castle.

Clojure was a whole different story, and addressed these issues admirably, for all the reasons you’ll discover when you look into it further. "

http://www.colinsteele.org/post/23103789647/against-the-grai...

[1] http://jan.rychter.com/blog/2010/7/5/clojure-w-fablo.html

So instead of using DataOutputStream, you have to use BinaryOutputStream.

Clojure doesn't wrap much, so that means very little code written for one VM runs on the other. Plus you don't have lein, which is a bit problem tooling-wise