Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Which programming languages are the fastest?

shootout.alioth.debian.org

Post reply on HN
Print view View original

11–20 of 23 posts

Re: Which programming languages are the fastest?

#11

Post by angusgr »

wccrawford wrote: post #7
angusgr wrote: post #5

As the page says, "No. Which programming language implementations have the fastest benchmark programs?" Which begs the question - aside from macho bragging rights, exactly what good is this kind of information? Assume I did have a specific problem case for which performance in terms of memory, time or LoC mattered. As a first step, I'd write a specific benchmarkable implementation of my own problem (or one very like…

It matters sometimes. A while back, I wrote a program to solve a puzzle. The more complex the puzzle, the longer it took to solve. Today, I might come up with a better algorithm, but back then, all I could find was a brute-force approach. The most complex puzzle I had seen at the time was taking hours to run. The program was written in VB6, and it was just wasn't mean for CPU-intensive programs like that. If I were t…

Today, I might come up with a better algorithm, but back then, all I could find was a brute-force approach. The most complex puzzle I had seen at the time was taking hours to run.

I bet that you could come up with a better algorithm now, even in VB6 if you had to, to get the run time down. I've seen people take "optimised" C solutions that run in hours and make them run in minutes, just by rewriting the algorithm and rethinking basic assumptions.

Re: Which programming languages are the fastest?

#12

Post by igouy »

angusgr wrote: post #5

As the page says, "No. Which programming language implementations have the fastest benchmark programs?" Which begs the question - aside from macho bragging rights, exactly what good is this kind of information? Assume I did have a specific problem case for which performance in terms of memory, time or LoC mattered. As a first step, I'd write a specific benchmarkable implementation of my own problem (or one very like…

(I often get this wrong: "beg the question - assume the truth of a proposition needing proof.")

> ... I would take the most human-readable program over the most fast, lean or concise one

And some people do look at the source code for 2 dozen languages.

http://shootout.alioth.debian.org/help.php#why

Re: Which programming languages are the fastest?

#13

Post by igouy »

angusgr wrote: post #9
njharman wrote: post #6

Earlier quoted context omitted.

Exactly, I can't believe (what seems like a) majority of developers are still talking about language performance (and optimization). It matters for a small subsegment of our industry. The rest, speed/cost to implement and maintain far outweigh raw language performance.

Yes! Also, even when you do care about performance, general performance isn't ever the same as specific performance. You have to optimise to your use case , not some arbitrary general use case like "render a Mandelbrot set" or "calculate pi". If I have a slow On^2 Erlang algorithm, then I'm a lot better off looking at ways to rewrite it as Olog(n) instead of scrapping Erlang and moving to Java because it had a better…

> general performance isn't ever the same as specific performance

http://shootout.alioth.debian.org/dont-jump-to-conclusions.p...

> scrapping Erlang and moving to Java

"Most (all?) large systems developed using Erlang make heavy use of C for low-level code, leaving Erlang to manage the parts which tend to be complex in other languages, like controlling systems spread across several machines and implementing complex protocol logic."

http://www.erlang.org/faq/introduction.html#id49850

Re: Which programming languages are the fastest?

#14

Post by angusgr »

igouy wrote: post #12
angusgr wrote: post #5

As the page says, "No. Which programming language implementations have the fastest benchmark programs?" Which begs the question - aside from macho bragging rights, exactly what good is this kind of information? Assume I did have a specific problem case for which performance in terms of memory, time or LoC mattered. As a first step, I'd write a specific benchmarkable implementation of my own problem (or one very like…

(I often get this wrong: "beg the question - assume the truth of a proposition needing proof.") > ... I would take the most human-readable program over the most fast, lean or concise one And some people do look at the source code for 2 dozen languages. http://shootout.alioth.debian.org/help.php#why

You're entirely correct on both counts. In fact, all 3 of those reasons given on that page are good ones.

However, this was still posted under the title "Which programming language is the fastest?" - so I think it's worth exploring in exactly what ways that is or isn't useful information. Just as their own reason #3 says - "To show how difficult it can be to make meaningful comparisons"

I'd also question exactly how useful reading toy benchmark solution code is to real problems. The code to the benchmark problems is often nicely written, but if I have a problem to solve then wouldn't I be better off looking for samples of readable code that solve problems related to my specific problem domain?

Re: Which programming languages are the fastest?

#15

Post by angusgr »

igouy wrote: post #13
angusgr wrote: post #9

Earlier quoted context omitted.

Yes! Also, even when you do care about performance, general performance isn't ever the same as specific performance. You have to optimise to your use case , not some arbitrary general use case like "render a Mandelbrot set" or "calculate pi". If I have a slow On^2 Erlang algorithm, then I'm a lot better off looking at ways to rewrite it as Olog(n) instead of scrapping Erlang and moving to Java because it had a better…

> general performance isn't ever the same as specific performance http://shootout.alioth.debian.org/dont-jump-to-conclusions.p... > scrapping Erlang and moving to Java "Most (all?) large systems developed using Erlang make heavy use of C for low-level code, leaving Erlang to manage the parts which tend to be complex in other languages, like controlling systems spread across several machines and implementing complex p…

Most (all?) large systems developed using Erlang make heavy use of C for low-level code, leaving Erlang to manage the parts which tend to be complex in other languages, like controlling systems spread across several machines and implementing complex protocol logic.

I know, when programming in Erlang I have unsurprisingly done exactly this. If I had an On^2 solution in Erlang that I knew I couldn't rewrite in Erlang to be faster, then that's my last resort - I'd write a C port to perform my particular inner loop performance-limited code, and leave the rest in Erlang.

However, I wouldn't either (a) go directly to C without trying to optimise the solution in Erlang first, or (b) look at a list of other languages and wonder which one I could move my entire program to.

Re: Which programming languages are the fastest?

#16

Post by igouy »

angusgr wrote: post #14
igouy wrote: post #12

Earlier quoted context omitted.

(I often get this wrong: "beg the question - assume the truth of a proposition needing proof.") > ... I would take the most human-readable program over the most fast, lean or concise one And some people do look at the source code for 2 dozen languages. http://shootout.alioth.debian.org/help.php#why

You're entirely correct on both counts. In fact, all 3 of those reasons given on that page are good ones. However, this was still posted under the title "Which programming language is the fastest?" - so I think it's worth exploring in exactly what ways that is or isn't useful information. Just as their own reason #3 says - "To show how difficult it can be to make meaningful comparisons" I'd also question exactly how…

> worth exploring in exactly what ways that is or isn't useful information

Now consider how you could use the varying performance on different tasks, and the varying performance on the same task of programs written in the same programming language, as examples (as a resource) for exploring (and showing to others) in what ways you think that is or isn't useful information.

If we just consider those "Which programming language is the fastest?" pages, then I think - simply getting someone to look away from a single number ranking (the median) and notice that for some language implementations the performance range is relatively small but for others the range is enormous - is useful.

> I'd also question exactly how useful reading toy benchmark solution code is to real problems.

Someone's told me that n-body pretty much is what they get paid to work on. Someone's told me that k-nucleotide and reverse-complement are pretty much is what they get paid to work on. The question as usual is who's "real" problems?

> wouldn't I be better off looking for samples of readable code that solve problems related to my specific problem domain?

Perhaps - and yet for some people what they see in those tiny tiny programs will be enough for a personal opinion about whether X is "the most human-readable".

Re: Which programming languages are the fastest?

#17

Post by hitechsites »

angusgr wrote: post #5

As the page says, "No. Which programming language implementations have the fastest benchmark programs?" Which begs the question - aside from macho bragging rights, exactly what good is this kind of information? Assume I did have a specific problem case for which performance in terms of memory, time or LoC mattered. As a first step, I'd write a specific benchmarkable implementation of my own problem (or one very like…

While one specific benchmark may not sway your decision on which programming language to use for your own projects, it should certainly be a factor when designing a large web application for hundreds of thousands of users. Unless of course you say that benchmarks are irrelevant and do not measure anything that corresponds to real world performance. But that would be like saying it does not matter whether you use a bicycle, a car, or a jet engine to travel from Point A to Point B, since intrinsic speed of your choice mode of transportation is difficult to measure, and is irrelevant.

Re: Which programming languages are the fastest?

#18

Post by angusgr »

hitechsites wrote: post #17
angusgr wrote: post #5

As the page says, "No. Which programming language implementations have the fastest benchmark programs?" Which begs the question - aside from macho bragging rights, exactly what good is this kind of information? Assume I did have a specific problem case for which performance in terms of memory, time or LoC mattered. As a first step, I'd write a specific benchmarkable implementation of my own problem (or one very like…

While one specific benchmark may not sway your decision on which programming language to use for your own projects, it should certainly be a factor when designing a large web application for hundreds of thousands of users. Unless of course you say that benchmarks are irrelevant and do not measure anything that corresponds to real world performance. But that would be like saying it does not matter whether you use a bi…

While one specific benchmark may not sway your decision on which programming language to use for your own projects, it should certainly be a factor when designing a large web application for hundreds of thousands of users.

Sure, but the benchmark has to fit the use case. If you're designing a large web application, find benchmarks for large web applications or write benchmark cases yourself that match the use cases you're going to see and think you need to optimise for.

Re: Which programming languages are the fastest?

#19

Post by angusgr »

igouy wrote: post #16
angusgr wrote: post #14

Earlier quoted context omitted.

You're entirely correct on both counts. In fact, all 3 of those reasons given on that page are good ones. However, this was still posted under the title "Which programming language is the fastest?" - so I think it's worth exploring in exactly what ways that is or isn't useful information. Just as their own reason #3 says - "To show how difficult it can be to make meaningful comparisons" I'd also question exactly how…

> worth exploring in exactly what ways that is or isn't useful information Now consider how you could use the varying performance on different tasks, and the varying performance on the same task of programs written in the same programming language, as examples (as a resource) for exploring (and showing to others) in what ways you think that is or isn't useful information. If we just consider those "Which programming…

These are all good points. I must confess, when I first looked at some shootout problem code a few years ago I remember being in violent disagreement about how some of the solutions were implemented (ie thinking "If I were solving this problem in a non-toy way I wouldn't do it that way, that's a total hack for benchmark purposes"), but it looks like they've evolved a fair bit and I can't find the same complaints I had then - so kudos for that. I need to recalibrate some of my opinions on that front. :)

Given that this sounds like its your own project, I have a vague idea that you might find interesting: Is there a way to present the results in an additional way which is framed initially as "look at the different ways to solve subproblem X" rather than "look at the graph of performance differences"? It could be an interesting approach to have an alternative additional interface that makes browsing the sample code (maybe with annotations?) front and centre, with a focus on programming techniques as a major differentiator, with scores as an adjunct. At present the visitor is encouraged to start from benchmark scores (either for everything, for a problem, or for a language) and from there move to source code later, but there might be a way to frame it as "let me flip through solutions to problems like X and tag the ones I like the look of, then let's see how they performed"

It's a pretty vague idea, and I don't have much of a handle on how it'd work from a UI perspective, but its seems like it might head off some of the kneejerk responses you get from kneejerkers like me ;), plus it could help encourage some of the uses you're describing in your post.

Re: Which programming languages are the fastest?

#20

Post by igouy »

angusgr wrote: post #19
igouy wrote: post #16

Earlier quoted context omitted.

> worth exploring in exactly what ways that is or isn't useful information Now consider how you could use the varying performance on different tasks, and the varying performance on the same task of programs written in the same programming language, as examples (as a resource) for exploring (and showing to others) in what ways you think that is or isn't useful information. If we just consider those "Which programming…

These are all good points. I must confess, when I first looked at some shootout problem code a few years ago I remember being in violent disagreement about how some of the solutions were implemented (ie thinking "If I were solving this problem in a non-toy way I wouldn't do it that way, that's a total hack for benchmark purposes"), but it looks like they've evolved a fair bit and I can't find the same complaints I ha…

> browsing the sample code (maybe with annotations?)

At present, even from the home page, source code is only 2 or 3 clicks.

Who would write those annotations? Currently there are 1100 programs, but they change as new programs are contributed and others removed.

Who would do the work to create a series of blog posts discussing the development of each and every program?

http://adventuresinmercury.blogspot.com/2011/04/shootout-spe...

Post reply on HN