Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Energy Efficiency Across Programming Languages

sites.google.com

Post reply on HN
Print view View original

131–140 of 143 posts

Re: Energy Efficiency Across Programming Languages

#131

Post by igouy »

etiene wrote: post #128
igouy wrote: post #126

Earlier quoted context omitted.

> One should not compare a JIT … with interpreters Because ?

Because they're "different families" of language implementation. A Just In Time compiler is a compiler. It looks like an interpreter, but it compiles the code to machine code and runs it.

And obviously the performance of programs using those language implementations on the same computer under the same workload may be measured and compared.

Why should "different families" of language implementation not be compared?

Re: Energy Efficiency Across Programming Languages

#132

Post by etiene »

igouy wrote: post #130
etiene wrote: post #129

Earlier quoted context omitted.

Oh yes, that is true. But the person making the benchmarks should be aware of that and either look for better matching algorithms or implement them themselves.

Why should a comparison be restricted just because the best Lua programs-contributed fail to use multi core? (Tail wagging the dog). If someone who didn't know Lua wrote those programs, then you'd complain that the programs weren't written by an expert. If Lua wasn't included then I think you'd complain about that too.

If your intention is not to make a good comparison, then sure. I've seen other papers on language implementation and benchmarks, and researchers were way more careful than this, individually inspecting the algorithms they were using.

I happened to spot a problem on this one on the Lua part and also on some other parts concerning how they classify things, because it happens to be things I know more. I'm not going out of my way to look for problems in the rest of the paper because I'm not a reviewer and I have better things to do.

But if I was the one way conducting this academic research, then I would want the results not to be bogus. The difference here is this is not a blog post about some bullshit comparison someone is making. If that was the case, then alright, if public contributed algorithms failed to represent exactly what was being looked for, why not go for whatever is out there. I just expect a more rigorous procedure from academics.

Re: Energy Efficiency Across Programming Languages

#133

Post by etiene »

igouy wrote: post #131
etiene wrote: post #128

Earlier quoted context omitted.

Because they're "different families" of language implementation. A Just In Time compiler is a compiler. It looks like an interpreter, but it compiles the code to machine code and runs it.

And obviously the performance of programs using those language implementations on the same computer under the same workload may be measured and compared. Why should "different families" of language implementation not be compared?

Firstly, the paper separates languages into compiled languages and interpreted languages to begin with. So I was expecting it would follow this pattern correctly.

Secondly, why would you compare the performance of C and Ruby, for example? Ruby is bound to be slower for design reasons. Doing it to say "Look, C is fast!" doesn't mean anything. There are different reasons one would compare C and Rust, or C and Ruby. The way the results are displayed and what it tries to convey is very important.

If you're not making a baseline, or being very clear, etc. then it's just misleading to have a benchmark portraying Javascript as faster than Ruby, Python and Lua. Either you put those into their correct classifications clearly, or you compare it with LuaJIT, PyPy, Julia etc.

Re: Energy Efficiency Across Programming Languages

#134

Post by igouy »

etiene wrote: post #132
igouy wrote: post #130

Earlier quoted context omitted.

Why should a comparison be restricted just because the best Lua programs-contributed fail to use multi core? (Tail wagging the dog). If someone who didn't know Lua wrote those programs, then you'd complain that the programs weren't written by an expert. If Lua wasn't included then I think you'd complain about that too.

If your intention is not to make a good comparison, then sure. I've seen other papers on language implementation and benchmarks, and researchers were way more careful than this, individually inspecting the algorithms they were using. I happened to spot a problem on this one on the Lua part and also on some other parts concerning how they classify things, because it happens to be things I know more. I'm not going out…

> I just expect a more rigorous procedure from academics

Are they likely to be expert in all those programming languages?

----

If Lua wasn't included would you complain ?

( Like this:

https://news.ycombinator.com/item?id=15255427

https://news.ycombinator.com/item?id=15251242

https://news.ycombinator.com/item?id=15251144

)

Re: Energy Efficiency Across Programming Languages

#135

Post by igouy »

etiene wrote: post #133
igouy wrote: post #131

Earlier quoted context omitted.

And obviously the performance of programs using those language implementations on the same computer under the same workload may be measured and compared. Why should "different families" of language implementation not be compared?

Firstly, the paper separates languages into compiled languages and interpreted languages to begin with. So I was expecting it would follow this pattern correctly. Secondly, why would you compare the performance of C and Ruby, for example? Ruby is bound to be slower for design reasons. Doing it to say "Look, C is fast!" doesn't mean anything. There are different reasons one would compare C and Rust, or C and Ruby. The…

> … the paper separates languages into…

The paper, both shows tables that include all the results (ordered by Energy consumed) and shows separate charts for what the authors classify as "either a compiled, interpreted, or virtual-machine language".

What is "an interpreted language" ?

"Although we refer to Lua as an interpreted language, Lua always precompiles source code to an intermediate form before running it. … The presence of a compilation phase may sound out of place in an interpreted language like Lua. However, the distinguishing feature of interpreted languages is not that they are not compiled, but that any eventual compiler is part of the language runtime and that, therefore, it is possible (and easy) to execute code generated on the fly."

p57 "Programming in Lua" (2003)

> … why would you compare the performance of C and Ruby, for example? Ruby is bound to be slower…

Except when the C program is measured to be 50x slower than the Ruby program --

http://benchmarksgame.alioth.debian.org/u32/compare.php?lang...

Re: Energy Efficiency Across Programming Languages

#136

Post by etiene »

igouy wrote: post #134
etiene wrote: post #132

Earlier quoted context omitted.

If your intention is not to make a good comparison, then sure. I've seen other papers on language implementation and benchmarks, and researchers were way more careful than this, individually inspecting the algorithms they were using. I happened to spot a problem on this one on the Lua part and also on some other parts concerning how they classify things, because it happens to be things I know more. I'm not going out…

> I just expect a more rigorous procedure from academics Are they likely to be expert in all those programming languages? ---- If Lua wasn't included would you complain ? ( Like this: https://news.ycombinator.com/item?id=15255427 https://news.ycombinator.com/item?id=15251242 https://news.ycombinator.com/item?id=15251144 )

> Are they likely to be expert in all those programming languages?

No, but again, I take paper results very seriously. Misleading results are bad for science, period. Here is an example of more rigorous research: https://arxiv.org/pdf/1602.00602.pdf

> If Lua wasn't included would you complain ?

Maybe I would, who knows. I did also mention in another comment it would be nice if Julia was there. It depends how relevant they are for the research being conducted. As a language used very often in microcontrollers, Lua is very relevant for energy efficiency research. I don't know anything about Forth, but I wouldn't just dismiss people saying it should be there with "then contribute forth algorithms to the benchmarks game yourself".

Re: Energy Efficiency Across Programming Languages

#137

Post by etiene »

igouy wrote: post #135
etiene wrote: post #133

Earlier quoted context omitted.

Firstly, the paper separates languages into compiled languages and interpreted languages to begin with. So I was expecting it would follow this pattern correctly. Secondly, why would you compare the performance of C and Ruby, for example? Ruby is bound to be slower for design reasons. Doing it to say "Look, C is fast!" doesn't mean anything. There are different reasons one would compare C and Rust, or C and Ruby. The…

> … the paper separates languages into… The paper, both shows tables that include all the results (ordered by Energy consumed) and shows separate charts for what the authors classify as "either a compiled, interpreted, or virtual-machine language". What is "an interpreted language" ? "Although we refer to Lua as an interpreted language, Lua always precompiles source code to an intermediate form before running it. … T…

Lua is not compiled to machine code. It compiles to an intermediate bytecode, which is then interpreted. So you could say it is a VM. Also bear in mind, LuaJIT is not Lua. It's a different implementation.

I guess you misinterpreted my comment. I didn't imply there are no reasons to compare C and Ruby. I said the reason is important and the way you portray that, and the way you portray your results, changes things.

Re: Energy Efficiency Across Programming Languages

#138

Post by igouy »

etiene wrote: post #137
igouy wrote: post #135

Earlier quoted context omitted.

> … the paper separates languages into… The paper, both shows tables that include all the results (ordered by Energy consumed) and shows separate charts for what the authors classify as "either a compiled, interpreted, or virtual-machine language". What is "an interpreted language" ? "Although we refer to Lua as an interpreted language, Lua always precompiles source code to an intermediate form before running it. … T…

Lua is not compiled to machine code. It compiles to an intermediate bytecode, which is then interpreted. So you could say it is a VM. Also bear in mind, LuaJIT is not Lua. It's a different implementation. I guess you misinterpreted my comment. I didn't imply there are no reasons to compare C and Ruby. I said the reason is important and the way you portray that, and the way you portray your results, changes things.

> Lua is not compiled to machine code…

Do you think Roberto Ierusalimschy is confused about that ?

Again, here's what the creator of Lua says -- "… the distinguishing feature of interpreted languages is not that they are not compiled, but that any eventual compiler is part of the language runtime…"

EDIT:

Is that not what you mean by "interpreted language" ?

Re: Energy Efficiency Across Programming Languages

#139

Post by etiene »

igouy wrote: post #138
etiene wrote: post #137

Earlier quoted context omitted.

Lua is not compiled to machine code. It compiles to an intermediate bytecode, which is then interpreted. So you could say it is a VM. Also bear in mind, LuaJIT is not Lua. It's a different implementation. I guess you misinterpreted my comment. I didn't imply there are no reasons to compare C and Ruby. I said the reason is important and the way you portray that, and the way you portray your results, changes things.

> Lua is not compiled to machine code… Do you think Roberto Ierusalimschy is confused about that ? Again, here's what the creator of Lua says -- "… the distinguishing feature of interpreted languages is not that they are not compiled, but that any eventual compiler is part of the language runtime…" EDIT: Is that not what you mean by "interpreted language" ?

> Do you think Roberto Ierusalimschy is confused about that ?

No, of course not. I think you are. You are clearly misinterpreting his words. I will repeat, (PUC)Lua is not compiled to machine code, but to an intermediate bytecode form. Those are completely different things, and what I said does not come in conflict at all with the quote you took from PiL, so I don't know what you are confronting me about.

Re: Energy Efficiency Across Programming Languages

#140

Post by igouy »

etiene wrote: post #136
igouy wrote: post #134

Earlier quoted context omitted.

> I just expect a more rigorous procedure from academics Are they likely to be expert in all those programming languages? ---- If Lua wasn't included would you complain ? ( Like this: https://news.ycombinator.com/item?id=15255427 https://news.ycombinator.com/item?id=15251242 https://news.ycombinator.com/item?id=15251144 )

> Are they likely to be expert in all those programming languages? No, but again, I take paper results very seriously. Misleading results are bad for science, period. Here is an example of more rigorous research: https://arxiv.org/pdf/1602.00602.pdf > If Lua wasn't included would you complain ? Maybe I would, who knows. I did also mention in another comment it would be nice if Julia was there. It depends how relevant…

> … I take paper results very seriously…

"disgusts me" & "pure dishonesty" don't seem like a serious response.

> … an example of more rigorous research…

Which again uses programs that were contributed to the benchmarks game.

> … wouldn't just dismiss people…

I wouldn't just dismiss them: I'd tell them that others have every right to present what seems important to them, and exclude what seems less important -- without being accused of dishonesty.

There's only so-much time & money.

Post reply on HN