Experiment: Unit testing isn't enough; You need static types, too • Hacker News Board

Re: Experiment: Unit testing isn't enough; You need static types, too

#211

Post by davesims » Wed, Jun 20, 2012, 7:41 PM UTC

CodeMage wrote: post #209

davesims wrote: post #143

Earlier quoted context omitted.

>it says nothing even remotely resembling "this benefit applies to all languages with static typing". That is precisely what it says, and that is reiterated later: "...the conclusion can be reached that...in practice [unit testing] is an inadequate replacement for static type checking." I'm not sure what you're reading, but there's no qualifications in the language used here regarding the idea of 'static type checkin…

> That is precisely what it says This is a very strong claim and it's false. The article doesn't say that anywhere. You interpret it that way. I would hazard a guess that presenting your own interpretation as fact is what brought on those downvotes you complain about below.

Can you show how I've misinterpreted the plain language of the conclusion section?

I'm under the (perhaps mistaken) assumption that in academic papers people tend to mean what they say and choose their language carefully, particularly in the conclusion section.

If the following are not in fact broad, strong claims about the nature of static and dynamic languages in general, then won't you please explain to me how I should interpret them?

Here are the quotes from the conclusion of the paper (emphasis mine):

"The translation of these four software projects from Python to Haskell proved to be an effective way of measuring the effects of applying static type checking to unit tested software."

"Based on these results, the conclusion can be reached that while unit testing can detect some type errors, in practice it is an inadequate replacement for static type checking."

Re: Experiment: Unit testing isn't enough; You need static types, too

#212

Post by papsosouid » Wed, Jun 20, 2012, 7:43 PM UTC

davesims wrote: post #206

papsosouid wrote: post #199

Earlier quoted context omitted.

>So? Folks don't use the "potential", they use the real. Haskell is real. >They're asking questions like "should I use Java or Python". That's wonderful, but it has nothing to do with the subject at hand, which was the question "can static typing reduce the number of bugs?". If you want an answer to a different question, don't complain about the answer given for this question, go find someone answering the question y…

>Why can't anyone follow a simple line of reasoning without resorting to fallacies? Indeed. The conclusion C was out of scope with the premises A and B. C is wrong, but that doesn't mean A and B cannot infer useful, more modest conclusions. What I don't understand about every one of your responses is that you seem to think false equivalence applies in only one direction. You seem to think it's fine for OP to infer br…

>I continually insist that such conclusions are out of the scope of the data given:

Yes, clearly you have some serious issues to work through.

Re: Experiment: Unit testing isn't enough; You need static types, too

#213

Post by papsosouid » Wed, Jun 20, 2012, 7:48 PM UTC

x1 wrote: post #207

zopa wrote: post #196

Earlier quoted context omitted.

> "The only thing the compiler will yell at you is if you passed a type that is not of a Query type. The compiler will not yell at you for getting the current session directly or creating your own jdbc driver for that matter." In Haskell, I'd have a module, Database, that held all my db code. That module would export functions something like query :: Query -> DBResult update :: Query -> DBAction -> DBResult (read tho…

> In the rest of my program, those functions would be the only way to talk to the database. There's your guarantee. Honest question. Take these pseudo sql calls: //Bad Person username = "lastname'; drop table user--" //Good Programmer query = "select * from users where name like %[username]%"; input = {"username":"frank"}; result = execute(query,input); //Bad Programmer query = "select * from users where name like '%…

In your static example, "Bad Programmer" would be fine, because the Query constructor does escaping. You could do this in a dynamically typed language too, but notice that you don't, you just use strings. The difference between static and dynamic is that with static typing, you can't compile your incorrect program. With dynamic typing, you find out at run time that you forgot to escape the string (turning it into a Query), when that code actually runs.

Re: Experiment: Unit testing isn't enough; You need static types, too

#214

Post by papsosouid » Wed, Jun 20, 2012, 8:03 PM UTC

davesims wrote: post #185

davesims wrote: post #143

Earlier quoted context omitted.

>it says nothing even remotely resembling "this benefit applies to all languages with static typing". That is precisely what it says, and that is reiterated later: "...the conclusion can be reached that...in practice [unit testing] is an inadequate replacement for static type checking." I'm not sure what you're reading, but there's no qualifications in the language used here regarding the idea of 'static type checkin…

From the downvotes I can only conclude that many of you wish the study didn't claim what it claims and are merely shooting the messenger. If anyone can point out rhetoric within the study that qualifies it in such a way as to make comparisons of other statically typed languages with other dynamically typed languages out-of-bounds or expressing a false equivalence within the scope of the conclusions of the study itsel…

You would do well to consider the very real possibility that it is in fact you who is misguided, and not the rest of the world. You come off sounding childish when you refuse to even consider the possibility that you are simply misinterpreting the purpose and conclusion of the study. The only reason most people can think of to explain your behaviour is that you have an axe to grind and just want to shoot down anything that paints static typing as a positive thing.

Re: Experiment: Unit testing isn't enough; You need static types, too

#215

Post by JustinJ70s » Wed, Jun 20, 2012, 8:05 PM UTC

pimentel wrote: post #9

How can you translate from python to a static language, when the code is written for the interfaces, and not types? How willl you translate a function receiving a (possibly custom) iterable, when the function doesn't care about the type, but just whether it implements a next() method?

Scala's type system can handle this as it support type safe duck typing.

Re: Experiment: Unit testing isn't enough; You need static types, too

#216

Post by davesims » Wed, Jun 20, 2012, 8:13 PM UTC

papsosouid wrote: post #202

davesims wrote: post #162

Earlier quoted context omitted.

> Yeah, no shit. Stop drinking warm coke. Fine, then don't claim something like "All cokes in all contexts at all temperatures are better than all pepsis in all contexts at all temperatures." This is equivalent to what the study does with static vs. dynamic. Your argument, if you actually had a point, would be something along the lines of, "wait I'm talking about this boutique hand-crafted cola (Haskell) I get at Who…

>Fine, then don't claim something like "All cokes in all contexts at all temperatures are better than all pepsis in all contexts at all temperatures." He didn't. He said "coke tasted better than pepsi". I've explained this to you several times already. You are the only one saying anything about "all the time in every context". You. Not the author, not his paper. You.

Get me, still waiting over here for a relevant quote from the paper. I've given mine. Where are yours?

Re: Experiment: Unit testing isn't enough; You need static types, too

#217

Post by CodeMage » Wed, Jun 20, 2012, 8:15 PM UTC

davesims wrote: post #211

CodeMage wrote: post #209

Earlier quoted context omitted.

> That is precisely what it says This is a very strong claim and it's false. The article doesn't say that anywhere. You interpret it that way. I would hazard a guess that presenting your own interpretation as fact is what brought on those downvotes you complain about below.

Can you show how I've misinterpreted the plain language of the conclusion section? I'm under the (perhaps mistaken) assumption that in academic papers people tend to mean what they say and choose their language carefully, particularly in the conclusion section. If the following are not in fact broad, strong claims about the nature of static and dynamic languages in general , then won't you please explain to me how I…

Honestly, at this point I can no longer tell whether you're misinterpreting or misrepresenting the conclusions. I'll make an honest attempt to argue, nevertheless.

"Static type checking" and "unit testing" are two concepts. There are numerous concrete implementations of these two concepts. The former is implemented in several languages, including C++ and Java and Haskell. The latter is implemented in several frameworks/tools, such as TestNG and PyUnit.

The article concludes that unit testing, as a technique for discovering and/or preventing defects, cannot wholly replace static type checking.

Apart from mentioning the concrete implementations of abstract techniques that the author used, the article does not conclude anything about the benefits of using specific languages, frameworks or tools.

What you have claimed so far is that:

1. there is a "hidden assumption is that all static and dynamic typing are created equal, i.e., since Haskell is statically typed and Haskell appears to have caught Python bugs that unit tests did not, therefore Java will catch bugs in a Ruby codebase, C++ will catch bugs in a JavaScript codebase, etc."

If anyone jumped to this conclusion, it was you. The only thing I can conclude from the article is that static typing checks such as those implemented in Haskell catch bugs that were not caught by unit testing logic such as that used in Python projects within the study. To conclude anything more I would need the data not present in the article, such as exactly what types of errors we caught or missed, etc.

2. the conclusion of the study "makes an unwarranted equivalence of all languages that have 'static type checking'"

It doesn't. The conclusion about the static type checking vs. unit testing might not be backed by enough solid data, but the conclusion makes no claims about languages, beyond specifying which languages were used in the study.

3. the claim that "this benefit applies to all languages with static typing" is "precisely what" the conclusion "says".

No occurrence of any phrase even remotely resembling the quote can be found in the article. Saying "this is precisely what it says" means "you'll find that phrase or one very similar to it in the text". Maybe you were trying to claim that "this is precisely what it means", but it's definitely what it "says".

All in all, the sweeping generalization about the concrete languages was introduced by you. My guess is that this is because you were, like me, frustrated by the vagueness of the article. I would have loved seeing more concrete data. Saying "X types of errors were found" is not as good as saying "the following types of errors were found" and that's just the start.

Re: Experiment: Unit testing isn't enough; You need static types, too

#218

Post by dllthomas » Wed, Jun 20, 2012, 8:34 PM UTC

debacle wrote: post #204

dllthomas wrote: post #191

Earlier quoted context omitted.

Even if you're a good programmer, typing and interfaces make refactoring quicker and typos easier to find. This is why so many people have problems writing in JavaScript.

Typing makes refactoring quicker only if you're relying on it. If you're not, then it will actually make refactoring slower. Interfaces will not make refactoring faster, just easier.

> Typing makes refactoring quicker only if you're relying on it.

As opposed to?

Re: Experiment: Unit testing isn't enough; You need static types, too

#219

Post by evincarofautumn » Wed, Jun 20, 2012, 8:43 PM UTC

dllthomas wrote: post #189

evincarofautumn wrote: post #115

Earlier quoted context omitted.

Dynamicity or dynamicness .

What's wrong with "dynamism"?

I considered it, but dynamism refers to personality and philosophy, while dynamicity is just the condition of being dynamic.

Re: Experiment: Unit testing isn't enough; You need static types, too

#220

Post by tikhonj » Wed, Jun 20, 2012, 8:58 PM UTC

anamax wrote: post #195

papsosouid wrote: post #154

Earlier quoted context omitted.

Yes, it was you. Why do you think the comparison should be "really bad static type system" vs "really good dynamic type system"? In what way does that make the test more useful? Allow me to say this again, as I do not know how to be any clearer: You do not test the potential of something by using the worst possible example of it. The only point of your desire is to reinforce the strawman that java = static typing. A…

> You do not test the potential of something by using the worst possible example of it. So? Folks don't use the "potential", they use the real. They're asking questions like "should I use Java or Python". > do airbags help prevent deaths" would be a very poor test if it used anything other than the best possible airbag technology. That's not how things actually work. You decide between what's available. The performan…

And the answer to "should I use Java or Python" is: no! Use Haskell ;). If you're entirely tied to Java (and, in that case, Python would probably not be ideal), you can still use Scala.

The question the study was asking was not "what language should I use for my lowest-common-denominator workforce" but rather "can a static type system catch more errors than unit tests and can statically typed code be as expressive as dynamically typed code".

In other words, it was asking for existential quantification: "does there exist some type system such that..." rather than "forall type systems..." or even "forall average systems...".