Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Our experience using Clojure to speed up Beanstalk

blog.beanstalkapp.com

Post reply on HN
Print view View original

1–10 of 75 posts

Re: Our experience using Clojure to speed up Beanstalk

#2

Post by zackzackzack »

I wonder how much of the speed change came from the difference in languages versus the difference in experience when writing both. The ruby version was written prior to the clojure one and so anything that was learned about git/programming during the ruby writing would haven be available during the clojure writing. I can believe that clojure would be faster; using concurrency well guarantees this somewhat. I still wonder what would have happened if they had written the clojure version first without having written the ruby version. Or wrote a ruby version after the clojure version.

Re: Our experience using Clojure to speed up Beanstalk

#3

Post by dsabanin »

zackzackzack wrote: post #2

I wonder how much of the speed change came from the difference in languages versus the difference in experience when writing both. The ruby version was written prior to the clojure one and so anything that was learned about git/programming during the ruby writing would haven be available during the clojure writing. I can believe that clojure would be faster; using concurrency well guarantees this somewhat. I still wo…

Well, once I was done with writing code to cache a Subversion repo, it was clearly visible how much faster Clojure is.

Basically same loop to go through all revisions in a repo was 15-20x times faster in Clojure than in Ruby. Very similar calls, pretty much same algorithm. But something that Ruby does inside the bindings was not very efficient.

For Git the problem was a bit different. Internally Grit (the Git API library for Ruby) tries to read huge pack files (potentially hundreds of MBs) in pure Ruby. This just can't be fast.

There's also additional overhead of not using any ORM for operating with our caches in Clojure version. That probably contributed to overall performance improvement, but 20x for svn / 40x for git measurement was taken even before we got to saving the results into DB and it stayed that way later on.

Re: Our experience using Clojure to speed up Beanstalk

#5

Post by balac »

EternalFury wrote: post #4

There seems to be a race towards the most exotic or revivalist programming languages. As if programming languages were magic bullets of some sort.

What is your contention to their choice here? It seems that are simply choosing the best tool for the job, it is not as if they are rewriting their whole web frontend in Clojure too.

Re: Our experience using Clojure to speed up Beanstalk

#6

Post by asparagui »

zackzackzack wrote: post #2

I wonder how much of the speed change came from the difference in languages versus the difference in experience when writing both. The ruby version was written prior to the clojure one and so anything that was learned about git/programming during the ruby writing would haven be available during the clojure writing. I can believe that clojure would be faster; using concurrency well guarantees this somewhat. I still wo…

According the article, they were using ruby to bridge to a svn module that had git capability. They replaced that module with a native git library, called from Clojure.

In summary: benchmark, profile, find hotspots, optimize. Works in every language. >:3

Re: Our experience using Clojure to speed up Beanstalk

#7

Post by DanI-S »

I've been playing with 4clojure[1] in my off time for weeks, and it's been a great introduction to the language, although it obviously doesn't help with learning how to package and deploy an application or service. How did you make the first steps from playing around in the REPL to writing production code?

[1] http://www.4clojure.com/

Re: Our experience using Clojure to speed up Beanstalk

#9

Post by dsabanin »

DanI-S wrote: post #7

I've been playing with 4clojure[1] in my off time for weeks, and it's been a great introduction to the language, although it obviously doesn't help with learning how to package and deploy an application or service. How did you make the first steps from playing around in the REPL to writing production code? [1] http://www.4clojure.com/

Have you seen Leiningen[1]? It pretty much solves packaging and deploying problem. Awesome tool.

Also if you're into Web development, I can recommend playing with Noir[2] framework and Korma[3] for SQL abstraction. Heroku also support deploying Clojure apps out of the box, so you can easily use a free tier to get something out there.

[1] https://github.com/technomancy/leiningen

[2] http://webnoir.org/

[3] http://sqlkorma.org/

Post reply on HN