Their code is on github [1]. I am a C++ programmer so I'm happy that C++ came out on top, but the comparison is obviously flawed (as these sorts of comparisons always are). The problem is that the implementers don't have the same level of skill in each language and who knows how much time they spent analyzing and refining each version. See Russ Cox's response [2] to a Google paper [3] of a similar nature comparing C+…
Given that their code is on github and many people criticized the unidiomatic nature of their code, wouldn't it make sense to set up a competition to improve the code? Everyone can submit pull requests with their improved codes and the authors should simply rerun the codes on a standard machine to get a benchmark. Obviously, a couple of rules should apply: 1. when you improve the code, do not move any elements outsid…
A comparison of programming languages in economics
71–80 of 83 posts
Re: A comparison of programming languages in economics
#72Their code is on github [1]. I am a C++ programmer so I'm happy that C++ came out on top, but the comparison is obviously flawed (as these sorts of comparisons always are). The problem is that the implementers don't have the same level of skill in each language and who knows how much time they spent analyzing and refining each version. See Russ Cox's response [2] to a Google paper [3] of a similar nature comparing C+…
>The problem is that the implementers don't have the same level of skill in each language and who knows how much time they spent analyzing and refining each version One of they authors addresses this in the comments on the OP: > As we explain in the paper, this was a “design” parameter. We did not want to “adapt” the algorithm to each language to improve speed, because it will open the door to too many “uncontrolled…
We did not want to “adapt” the algorithm to each language to improve speed, because it will open the door to too many “uncontrolled variables” (how good a programmer in that language you are and so on). The only partial exception was with Mathematica for reasons explained in the paper.
This is rather half-baked reasoning. The only ways I can think of to control for programmer skill or at least account for it is to
1. Use "idiomatic" code for all languages.
2. Use "optimized" code for all languages.
3. Use "real" code for all languages with a large and detailed sample set.
Re: A comparison of programming languages in economics
#73Earlier quoted context omitted.
It is unfortunate that the authors of this paper didn't reach out to experts in each programming language in order to write high-quality code against which they could have run the analysis. This seems like the perfect use-case for open-source: academic study, subject-matter experts in one area who are stepping outside their areas of expertise and an interesting subject matter. They should have crowd-sourced the code…
From the paper... Second, to make the comparison as unbiased as possible, we coded the same algorithm in each language without adapting it to the peculiarities of each language (which could reflect more about our knowledge of each language than of its objective virtues). They have invited others to fork the code on github and write the fastest possible for each language.
This is not "as unbiased as possible" -- it is biased towards the initial form of algorithm description.
To make the comparison "as unbiased as possible" would be to code the same algorithm but adapt it to the peculiarities of each language implementation.
Re: A comparison of programming languages in economics
#74Earlier quoted context omitted.
Fair enough. My point was supposed to be more limited, saying that a JIT'ed solution is not a cure all for performance considerations. Sure, it is great that there is a JIT compiler helping out. It just is not necessarily enough on its own to even the performance gap between native and not.
It is enough. You can perf test the same non-trivial program (such as a biz application) in C and Java, and given you allow the JVM to warm up, discover hot spots, and JIT your code... they will run just about the same performance level... every time. The JVM is impressive.
JVM is impressive until closures are involved, in which case when inlining is not done (either because it cannot be or ever-changing heuristics are not met), you can drop back to virtual calls and performance suffers a great deal.
Compare to C++ where closures can be used with virtually no penalty and aggressively optimized. Even C with its trick of passing an extra *void for environment comes out shining performance-wise in comparison.
Edit: There's a good discussion of the problems that JVMs have in this area at http://www.azulsystems.com/blog/cliff/2011-04-04-fixing-the-....
"The Problem is a hard one to solve; (i.e. it’s not the case that we’ll wave our wands and do a quick fix & rebuild HotSpot and the problem will go away). John Rose, Doug Lea and I all agree that it’s a Major Problem facing JVMs with long and far reaching implications."
Maybe progress will be made soon though, with more pressure caused by increasing use of closures in the new streams library and with anonymous function syntax available now.
Re: A comparison of programming languages in economics
#75Earlier quoted context omitted.
Do you have any specific questions? I teach in a PhD-granting department and do research in energy economics. (I'm an associate editor of one of the energy field journals.)
Huh, some of your papers sound like ideas I sketched out in my commonplace book (particularly the ones on oil prices in the context of the larger economy), looks like I'm a few years behind. I guess my biggest concern is the viability of pursuing a PhD in economics without having a previous degree in econ. I have a BS and MS in environmental science and an MPA in energy policy/economics. I think I have stronger scien…
"I guess my biggest concern is the viability of pursuing a PhD in economics without having a previous degree in econ."
It's hard to say without knowing more about your background, but many have done it before. A degree in econ is useful but by no means necessary.
You should probably worry more about your math training (have you taken real analysis or another course with real proofs?) than your econ training. I had three calculus classes and linear algebra. The first week of classes the professors were talking about quasiconcavity and the Bellman equation. I was lost. I worked 80-90 hours a week the first year in order to catch up.
"Also, it seems to me that the general market for energy economists is very strong, and will remain that way for quite some time. Do you think that's accurate?"
I have no specific data, but yeah, the market for economists in most fields is strong. Our program is not highly ranked, yet almost all of our PhD's are able to get tenure track jobs with reasonable teaching loads. To my knowledge environmental/energy policy is a hot field right now, particularly if you can bring in grants.
"as far as I can tell economics is the field that most closely aligns with this in the context of energy and the environment"
You should also consider PhD programs in agricultural economics or environmental economics. Highly-ranked economics programs often have a theoretical/mathematical focus that doesn't fit with what you want to do. I know from experience that energy isn't fully respected in economics departments.
Re: A comparison of programming languages in economics
#76Earlier quoted context omitted.
Huh, some of your papers sound like ideas I sketched out in my commonplace book (particularly the ones on oil prices in the context of the larger economy), looks like I'm a few years behind. I guess my biggest concern is the viability of pursuing a PhD in economics without having a previous degree in econ. I have a BS and MS in environmental science and an MPA in energy policy/economics. I think I have stronger scien…
"I guess my biggest concern is the viability of pursuing a PhD in economics without having a previous degree in econ." It's hard to say without knowing more about your background, but many have done it before. A degree in econ is useful but by no means necessary. You should probably worry more about your math training (have you taken real analysis or another course with real proofs?) than your econ training. I had th…
Thanks a bunch!
I actually have a friend in Cornell's PhD economics program right now, and he's warned me about math (and he majored in math) and that Real Analysis in particular is crucial. That's what most concerns me, as I've leaned on my physics-PhD significant other in the past for some math-related help.
That's a really good point about field-specific economics programs, I hadn't really considered that, but it makes a lot of sense given my interests and current abilities. It seemed to me that energy was weirdly neglected when looking through programs (even though it has so many neat peculiarities and inefficiencies), so it's good to know I'm not the only person who has noticed that.
Thankfully, I'm not looking at it for the immediate future as my current position will last a few years (we have boatloads of funding and bipartisan support), and I'm at possibly the easiest national lab to get a staff position and advance without a PhD (Oak Ridge). Down the line though, I know I love academia and teaching, so I'll probably go back, and your advice is super helpful to think about in the interim, thanks again!
Re: A comparison of programming languages in economics
#77Earlier quoted context omitted.
Fair enough. My point was supposed to be more limited, saying that a JIT'ed solution is not a cure all for performance considerations. Sure, it is great that there is a JIT compiler helping out. It just is not necessarily enough on its own to even the performance gap between native and not.
It is enough. You can perf test the same non-trivial program (such as a biz application) in C and Java, and given you allow the JVM to warm up, discover hot spots, and JIT your code... they will run just about the same performance level... every time. The JVM is impressive.
Only if you do an impressive job of designing the data in such a way that you aren't constantly thrashing. Which, more and more, means non idiomatic Java code. Consider, even with massive advances in JIT capabilities, you do get performance benefits from an array over an ArrayList.
Now, can this be improved? Of course. Scala, for example, does a good job of not requiring primitive arrays. This is done with compile time tricks, though, not runtime (JIT) ones.
Re: A comparison of programming languages in economics
#78Earlier quoted context omitted.
I don't know about other languages, but the authors did reach out to us to sort out performance issues with the Julia code.
That is very good to know. That reassures me a little. I was really impressed with the speed difference they noted between Python and R and the fact that Numba wasn't too much slower than C++ implementations. However, given the state of the R code I became wary of accepting the benchmarks at face value. I haven't made the jump from R to Python for statistical applications yet. So, this article gave me something to th…
I've used Numba to speed up some tight numerical computations written on NumPy arrays and it's incredibly good - about as fast as rewriting it from scratch in Cython, but much, much faster.
The flipside is that it only really results in a speed up for pure numerical code - if you have to use any Python objects, then you lose everything.
Re: A comparison of programming languages in economics
#79Earlier quoted context omitted.
"I guess my biggest concern is the viability of pursuing a PhD in economics without having a previous degree in econ." It's hard to say without knowing more about your background, but many have done it before. A degree in econ is useful but by no means necessary. You should probably worry more about your math training (have you taken real analysis or another course with real proofs?) than your econ training. I had th…
Thanks a bunch! I actually have a friend in Cornell's PhD economics program right now, and he's warned me about math (and he majored in math) and that Real Analysis in particular is crucial. That's what most concerns me, as I've leaned on my physics-PhD significant other in the past for some math-related help. That's a really good point about field-specific economics programs, I hadn't really considered that, but it…
Best of luck whatever you end up doing.
Re: A comparison of programming languages in economics
#80Another interesting comparison from the github repo[1] is the proportions of lines that it took in each language. [1] https://github.com/jesusfv/Comparison-Programming-Languages-...
Programs for some languages are tall and narrow, while comparable programs for other languages are short and wide.