Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ruby is too slow for programming competitions

blog.clifreeder.com

Post reply on HN
Print view View original

221–230 of 254 posts

Re: Ruby is too slow for programming competitions

#221

Post by vidarh »

timr wrote: post #195
vidarh wrote: post #157

Earlier quoted context omitted.

I have tried it. My MSc dissertation required a lot of computation (weeks of runtime after I'd rewritten critical portions in C++), and I'd do it that way again. For many types of problems, putting the data into an efficient format before passing it to your C/C++ code is trivial in terms of both difficulty and time spent. I prototyped everything in Ruby, profiled, and then rewrote a handful of the critical parts in C…

I've done it too, and I think most of the responses to the GP are missing the point. Can it be done? Sure. But it's a pain in the ass. It's not nearly as trivial as people suggest. It's basically only worthwhile when you know you have the resources to build an infrastructure of re-usuable fast modules that get coupled with a scripting language interface. Even when you give up on writing true "hybrid" code (i.e. "just…

I didn't find it hard at all, nor a pain in the ass. I wrote my code. Profiled it. Found the few, small bits that needed to be fast and treated the Ruby code as an executable spec for the C, and rewrote it very quickly.

There is SWIG support for Ruby, as well as the Ruby FFI implementation and RubyInline, offering various degrees of simplicity, but none of them are hard. The Ruby extension interface is more work, but even that is fairly simple to work with. I've taken C libraries and wrapped them in Ruby in minutes using SWIG for example.

Overall I often these days choose to write in Ruby first and replace even for applications that I intend to write the finished version of entirely in C. Prototyping in Ruby and rewriting in C once I have a working version is faster to me than iterating over a C version.

Re: Ruby is too slow for programming competitions

#222

Post by hsitz »

ClifReeder wrote: post #189
gnuvince wrote: post #82

I'm sorry, but is the Go program correct? I ran it on my machine, and for Case #1, it reports a solution of 1, while there are actually 19. EDIT: I modified the Go program to use int64 instead of int (the upper bound of the first case is not a valid 32-bit integer), and the execution time is now much higher: 2 minutes 41 seconds on my laptop.

Welp, I had no idea about the int64 business. Probing on this, but it does seem like it changes things quiet a bit. UPDATE: After updating go from 1.0.2 to 1.1, this matches my results - it now takes about 2 minutes, 47 seconds to run through the input (and Code Jam confirms the output is correct). This seems like a significant detail to me, so I'll update the post.

Modifying your Go code to use int64's the program (compiled with Go v.1.0.3) takes about 20 minutes for the L1 problem on my Thinkpad x200 (core duo 2.4GHz).

More importantly, the argument that Ruby is "too slow" seems to hedge on the L1 input for Fair and Square problem being solvable with "fast" languages using same unoptimized algorithm as can be used with the S small input. If you'll look at the Codejam results this is simply false. In fact, a large percentage of Ruby users passed Fair and Square L1 problem, higher than many "fast" languages:

Language % of S input solvers successful on F&S L1 input

Ruby ---------- 29.6 (112 L1, 378 S)

C ------------- 20.4 (199 L1, 973 S)

Java ---------- 27.6 (1183 L1, 4285 S)

Go ------------ 25.4 (15 L1, 59 S)

(from http://www.go-hero.net/jam/13/languages/0 )

I think this means you need to rethink your conclusions.

In fact algorithm contests are usually designed so that the small inputs are solvable by brute force algorithms, but you need to optimize them to solve larger inputs, harder problems. It's not just people using Ruby and other "slow" languages who have to rethink their approach.

Re: Ruby is too slow for programming competitions

#223

Post by daGrevis »

RyanZAG wrote: post #77

A lot of people are talking about the problem itself - which is interesting - but very often you run into real problems which cannot be simplified so easily (NP hard problems, many different other cases). In addition, when you need to do important calculations on a server for your product and you're using Ruby, you may very well end up requiring 10x as many servers. This is pretty massive when you consider an AWS ser…

I would put Python in the same category as Ruby, if we are talking about performance. Sure, there is PyPy, but for Ruby there is Topaz.

Re: Ruby is too slow for programming competitions

#224

Post by msy »

RyanZAG wrote: post #77

A lot of people are talking about the problem itself - which is interesting - but very often you run into real problems which cannot be simplified so easily (NP hard problems, many different other cases). In addition, when you need to do important calculations on a server for your product and you're using Ruby, you may very well end up requiring 10x as many servers. This is pretty massive when you consider an AWS ser…

I think the project you're looking for is Topaz, which has a close ecosystem relationship with PyPy: http://docs.topazruby.com/en/latest/blog/announcing-topaz/

Re: Ruby is too slow for programming competitions

#225

Post by timr »

vidarh wrote: post #221
timr wrote: post #195

Earlier quoted context omitted.

I've done it too, and I think most of the responses to the GP are missing the point. Can it be done? Sure. But it's a pain in the ass. It's not nearly as trivial as people suggest. It's basically only worthwhile when you know you have the resources to build an infrastructure of re-usuable fast modules that get coupled with a scripting language interface. Even when you give up on writing true "hybrid" code (i.e. "just…

I didn't find it hard at all, nor a pain in the ass. I wrote my code. Profiled it. Found the few, small bits that needed to be fast and treated the Ruby code as an executable spec for the C, and rewrote it very quickly. There is SWIG support for Ruby, as well as the Ruby FFI implementation and RubyInline, offering various degrees of simplicity, but none of them are hard. The Ruby extension interface is more work, but…

"I wrote my code. Profiled it. Found the few, small bits that needed to be fast and treated the Ruby code as an executable spec for the C, and rewrote it very quickly."

The fact that the speed-critical parts of your problem could be reduced to a "few, small bits" indicates that we're talking about entirely different things.

I was in a problem domain where writing the time-critical pieces in a different language meant ensuring that huge amounts of working-set data were available to that optimized code. Efficient access to that data meant storage in native format, which meant that supporting two languages would require a nasty I/O layer for the interpreted language. Not worth it.

And that's just for the problems where it was possible to consider using Perl/Python at all. For some problems, the memory overhead of Perl/Python data structures ruled out use on all but the smallest problems. Again, not worth it.

It's nice that the speed-critical parts of your problem were reducible to a few lines of code, but that's not often the case. For most real problems, I/O is a huge concern, and can't be dismissed just by pre-processing the input.

Re: Ruby is too slow for programming competitions

#226

Post by davidw »

geedy wrote: post #172
davidw wrote: post #130

Earlier quoted context omitted.

> would you call University education dicking around as well? That's a big debate! For a lot of people, I would, though, yes. There are serious economists who posit that universities are mostly about signalling, rather than actually learning much. I don't think I agree with that entirely, and believe it depends a lot on what's studied. It also depends on the opportunity costs, which may not be that high for many peop…

> That's a big debate! For a lot of people, I would, though, yes. There are serious economists who posit that universities are mostly about signalling, rather than actually learning much. I don't think I agree with that entirely, and believe it depends a lot on what's studied. It also depends on the opportunity costs, which may not be that high for many people that age. Fair enough. For myself, it was what lead me to…

There are plenty of things that aren't "productive", but strike me as still being more creative/interesting/useful than churning out the same answer as everyone else to a made-up problem. Making your own programming language or OS is not 'productive', but a great learning experience, creative, and lots of fun. And who knows, maybe someone will get some use out of it. I once made a fairly simple programming language, and it did end up seeing some niche adoption and use in industry, which was cool.

Re: Ruby is too slow for programming competitions

#227

Post by JulianMorrison »

vidarh wrote: post #219
yasth wrote: post #193

Earlier quoted context omitted.

Well for one Go beats Ruby in every benchmark that I am aware of, and generally by quite some distance. Also realistically 60-70% of all libraries created for other languages are bitrotted to junk or pointlessness by this point. If you can't see why an app that fits within the worked problemspace of Go (and even with the current libraries plenty of things are wholly doable in Go) and will require much less server res…

The point is if you need the performance, C is the more natural choice for a lot of it at this point. I looked through one of the Go tutorials a while back, and it was WTF after WTF. If it works for you, fine, but Go has a long way to go to be interesting for a lot of us.

C will give you 2 times the performance with 1/2 the memory usage of Go for 10 times the development effort, if you are skilled at optimizing C.

Re: Ruby is too slow for programming competitions

#228

Post by espadrine »

nfm wrote: post #155
frehpt wrote: post #128

Earlier quoted context omitted.

Option A: -Write your application in Go Option B: -Write application in Ruby -Profile Ruby application to identify code to be rewritten in C -Learn C if you don't already know it -Rewrite various parts of your application in C -Run Valgrind to search for memleaks -Ensure that your target systems all have required libs installed Option A seems much simpler...

That's a ridiculous set of alternatives.

The bottom line is: the time necessary to optimize Ruby to run below 8 minutes was bigger than he could afford, and apparently Go, once written (which apparently gave the OP some issues) needn't be optimized.

That said, Go wasn't necessarily the better choice, because of the subtlety that made the Go program overflow silently. Writing the solution in Scala or even in JS would have probably given him less of an issue.

Re: Ruby is too slow for programming competitions

#229

Post by thomasz »

msy wrote: post #224
RyanZAG wrote: post #77

A lot of people are talking about the problem itself - which is interesting - but very often you run into real problems which cannot be simplified so easily (NP hard problems, many different other cases). In addition, when you need to do important calculations on a server for your product and you're using Ruby, you may very well end up requiring 10x as many servers. This is pretty massive when you consider an AWS ser…

I think the project you're looking for is Topaz, which has a close ecosystem relationship with PyPy: http://docs.topazruby.com/en/latest/blog/announcing-topaz/

It was announced two months ago. Would you trust it enough to use it in production in the next two to five years?

Re: Ruby is too slow for programming competitions

#230

Post by dmpk2k »

camus wrote: post #218
ksec wrote: post #206

I am risking myself getting insanely downvoted for writing this. But I hope it at least gets some of the attention. I really hope the Ruby Communities do actually admit Ruby is Slow. Because until you actually admit there is a problem, there will never be any notion of wanting to fix it. Most of the time, they will come up with Rails Examples or serving dynamic web pages, and the bottleneck being in Database. And the…

PHP and PYTHON are slow too. Like them Ruby can take advantage ) of C extensions. You cant write everything in ruby like PHP developpers use native drivers for database communication , compression , image manipulation ,etc ... that's normal. Scripted languages should be used for what they are for , a thin layer on the top of other more performant languages. Like Twitter , Ruby/Rails allowed them to bootstrap their id…

Yes, but I think what ksec was driving at is there isn't an intrinsic reason what you said should be true.

Why should Ruby (and scripting languages) be used for RAD and not performance? That's because implementations range from really slow to somewhat slow. Self, LuaJIT, V8 and IonMonkey, and increasingly PyPy are ample evidence that this need not be the case, and that's just listing the dynamically-typed languages.

The bar has been raised the past few years. Increasingly we can pick three of: productivity, libraries, performance. Ruby lacks the last, which makes it a less interesting prospect for future greenfield projects compared to its competition.

Post reply on HN