Experiment: Unit testing isn't enough; You need static types, too • Hacker News Board

Re: Experiment: Unit testing isn't enough; You need static types, too

#241

Post by jaylevitt » Wed, Jun 20, 2012, 10:31 PM UTC

papsosouid wrote: post #202

davesims wrote: post #162

Earlier quoted context omitted.

> Yeah, no shit. Stop drinking warm coke. Fine, then don't claim something like "All cokes in all contexts at all temperatures are better than all pepsis in all contexts at all temperatures." This is equivalent to what the study does with static vs. dynamic. Your argument, if you actually had a point, would be something along the lines of, "wait I'm talking about this boutique hand-crafted cola (Haskell) I get at Who…

>Fine, then don't claim something like "All cokes in all contexts at all temperatures are better than all pepsis in all contexts at all temperatures." He didn't. He said "coke tasted better than pepsi". I've explained this to you several times already. You are the only one saying anything about "all the time in every context". You. Not the author, not his paper. You.

> He said "coke tasted better than pepsi"

I think he actually said "coke tastes better than pepsi". That verb tense has very different implications.

Re: Experiment: Unit testing isn't enough; You need static types, too

#242

Post by jaylevitt » Wed, Jun 20, 2012, 10:46 PM UTC

papsosouid wrote: post #213

x1 wrote: post #207

Earlier quoted context omitted.

> In the rest of my program, those functions would be the only way to talk to the database. There's your guarantee. Honest question. Take these pseudo sql calls: //Bad Person username = "lastname'; drop table user--" //Good Programmer query = "select * from users where name like %[username]%"; input = {"username":"frank"}; result = execute(query,input); //Bad Programmer query = "select * from users where name like '%…

In your static example, "Bad Programmer" would be fine, because the Query constructor does escaping. You could do this in a dynamically typed language too, but notice that you don't, you just use strings. The difference between static and dynamic is that with static typing, you can't compile your incorrect program. With dynamic typing, you find out at run time that you forgot to escape the string (turning it into a Q…

I'm admittedly ignorant of any type system newer than C++. In a modern static language, how would you design Query such that any SQL injection is caught at compile-time?

On the dynamic side, Rails (in Ruby) doesn't currently catch SQL injections, but it does catch HTML-escaping injections. It (roughly) tags all strings as tainted by default, and when you send them to the browser, it escapes them. If you want to send literal ampersands, angle brackets, etc., you have to mark them as explicitly safe. Since most of your literal HTML is generated by templates (which themselves distinguish variables from static HTML), you end up with run-time safety unless you actively try to break out of it.

Re: Experiment: Unit testing isn't enough; You need static types, too

#243

Post by eta_carinae » Wed, Jun 20, 2012, 10:55 PM UTC

thebluesky wrote: post #83

eta_carinae wrote: post #77

The advantage of statically typed languages has a lot less to do with tests and everything to do with tools. IDE's can do very little when no type information is available, and most automatic refactorings require human supervision when performed on dynamically typed languages (read this for details: http://beust.com/weblog/2006/10/01/dynamic-language-refactor... ).

Tooling and performance are two largest advantages. Disadvantage is more verbosity, but it's a trade-off. With dynamically typed languages I find that you still need to worry about types, but you have to trace through the code to figure out what type a particular variable is (esp. if you're not the only one working on a project). In a statically typed language that information is readily available.

> Disadvantage is more verbosity

Type inference cuts down the verbosity tremendously.

Re: Experiment: Unit testing isn't enough; You need static types, too

#244

Post by pyre » Wed, Jun 20, 2012, 11:14 PM UTC

tikhonj wrote: post #238

pyre wrote: post #168

Earlier quoted context omitted.

> "NoneType has no attribute '...'" C is statically typed, but I can still attempt to dereference a null pointer. Static typing doesn't save me here, nor does the compiler, as it's possible for these issues to happen at runtime. This may be something that Haskell doesn't allow, but it's not something inherent to static-typing.

Intercal is dynamically typed, but it doesn't save me any development time or make my code any shorter! (Well, maybe compared to Java :)). On top of this, the C type system is not really about correctness at all. My understanding is that it primarily helps with performance, memory management (e.g. you know the size of stuff) and not accidentally using a non-pointer as a pointer. I'm not a C person, but C does not giv…

I'm just 'arguing' that pitting static typing against dynamic typing using specific languages as examples isn't necessarily the whole picture. Saying that static typing will save you from attempting to call methods on None in Python is a fallacy. Saying that Haskell's static type system will save you, is possibly correct.

My original point was that it's possible that we (programmers) focus more on issues that could be with static typing (of some implementation) just because it seems like a group of problems that could be 'easily' solved. I.e. 'the grass is always greener'

Re: Experiment: Unit testing isn't enough; You need static types, too

#245

Post by dllthomas » Wed, Jun 20, 2012, 11:32 PM UTC

evincarofautumn wrote: post #219

dllthomas wrote: post #189

Earlier quoted context omitted.

What's wrong with "dynamism"?

I considered it, but dynamism refers to personality and philosophy, while dynamicity is just the condition of being dynamic.

The Wikipedia disambiguation for Dynamism (last edited in March) includes:

"Dynamism (computing), When any process in computer is using Dynamic management methods for its processing/computing/memory management/parallelism handling for being able to give more user friendly work that are more easy to interact and modify."

... for whatever that's worth.

Re: Experiment: Unit testing isn't enough; You need static types, too

#246

Post by zopa » Wed, Jun 20, 2012, 11:55 PM UTC

x1 wrote: post #207

zopa wrote: post #196

Earlier quoted context omitted.

> "The only thing the compiler will yell at you is if you passed a type that is not of a Query type. The compiler will not yell at you for getting the current session directly or creating your own jdbc driver for that matter." In Haskell, I'd have a module, Database, that held all my db code. That module would export functions something like query :: Query -> DBResult update :: Query -> DBAction -> DBResult (read tho…

> In the rest of my program, those functions would be the only way to talk to the database. There's your guarantee. Honest question. Take these pseudo sql calls: //Bad Person username = "lastname'; drop table user--" //Good Programmer query = "select * from users where name like %[username]%"; input = {"username":"frank"}; result = execute(query,input); //Bad Programmer query = "select * from users where name like '%…

> "Right now I see no difference between the good and bad"

You're building a new query string each time you create a Query object, and concatenating the string onto that. With that approach, each time you build a Query object you have a fresh opportunity to mess up. So you're right that there's no difference between your to cases.

Let's drop my off-the-cuff example and look at how a real library, postgresql-simple, handles the issue:

  query :: (ToRow q, FromRow r) => Connection -> Query -> q -> IO [r]

Usage example

  query conn "select x from users where name like ?" (Only username)

Do you see the difference? Instead of sticking the username into the SQL query by hand, we use a query function that takes three parameters: a database handle, a Query with a '?' character, and a thing you want to use in the query. The function takes care of properly escaping the username during interpolation. (The "Only" is just a wrapper to make sure we're handing in a datatype we can query with.)

Notice that because Query is a distinct type from String, just doing

  query conn ("select x from userse where name like" ++ username)

doesn't typecheck. Bad Programmer would have a hard time screwing this up.

The full documentation for postgresql-simple is here: http://hackage.haskell.org/packages/archive/postgresql-simpl...

Re: Experiment: Unit testing isn't enough; You need static types, too

#247

Post by zopa » Thu, Jun 21, 2012, 12:05 AM UTC

CodeMage wrote: post #217

davesims wrote: post #211

Earlier quoted context omitted.

Can you show how I've misinterpreted the plain language of the conclusion section? I'm under the (perhaps mistaken) assumption that in academic papers people tend to mean what they say and choose their language carefully, particularly in the conclusion section. If the following are not in fact broad, strong claims about the nature of static and dynamic languages in general , then won't you please explain to me how I…

Honestly, at this point I can no longer tell whether you're misinterpreting or misrepresenting the conclusions. I'll make an honest attempt to argue, nevertheless. "Static type checking" and "unit testing" are two concepts. There are numerous concrete implementations of these two concepts. The former is implemented in several languages, including C++ and Java and Haskell. The latter is implemented in several framewor…

> "Saying "X types of errors were found" is not as good as saying "the following types of errors were found" and that's just the start."

The blog post is vague, but the paper (also available at the link) isn't. It identifies the particular errors found.

Re: Experiment: Unit testing isn't enough; You need static types, too

#248

Post by swah » Thu, Jun 21, 2012, 12:07 AM UTC

dochtman wrote: post #109

cageface wrote: post #10

I'm convinced that dynamically typed languages are a transitional technology that will be superseded once we develop type systems that are both usefully strict but also flexible. After over ten years working in dynamic languages I'm very happy to have a compiler on my side again.

Seems to me that that's what static typing proponents have been saying for years (if not decades), and it still isn't true. That said, I'm currently hacking on a compiler to see if I can come up with a design for a static language that's almost as pleasant to use as a dynamic one, so I'm not completely hopeless. But it certainly seems like the data point that until now no such superior type system (strict but flexibl…

So we're talking about type inference - which way are you going?

Re: Experiment: Unit testing isn't enough; You need static types, too

#249

Post by zopa » Thu, Jun 21, 2012, 12:21 AM UTC

jaylevitt wrote: post #242

papsosouid wrote: post #213

Earlier quoted context omitted.

In your static example, "Bad Programmer" would be fine, because the Query constructor does escaping. You could do this in a dynamically typed language too, but notice that you don't, you just use strings. The difference between static and dynamic is that with static typing, you can't compile your incorrect program. With dynamic typing, you find out at run time that you forgot to escape the string (turning it into a Q…

I'm admittedly ignorant of any type system newer than C++. In a modern static language, how would you design Query such that any SQL injection is caught at compile-time? On the dynamic side, Rails (in Ruby) doesn't currently catch SQL injections, but it does catch HTML-escaping injections. It (roughly) tags all strings as tainted by default, and when you send them to the browser, it escapes them. If you want to send…

See http://news.ycombinator.com/item?id=4139798

Re: Experiment: Unit testing isn't enough; You need static types, too

#250

Post by ScottBurson » Thu, Jun 21, 2012, 1:30 AM UTC

davesims wrote: post #48

The sample size of this study is statistically too insignificant to draw the conclusions given. The counterfactuals to the claims of dynamic type advocates were already true and provable, so even if the sample size of the codebases studied were statistically significant (not to mention vetted for quality of unit test as well as coverage ) the conclusions are nevertheless trivial. In addition, the hidden assumption is…

I see both sides of this argument. The OP -- at least in this blog post; I haven't read the paper -- spends most of his time talking about how he's demonstrated the insufficiency of unit testing. For the purpose of that argument, it really doesn't matter that he used Haskell as opposed to some other type checker.

It's only in the last two sentences of his "Conclusion" section that he turns the argument around, and here is where he oversteps:

While unit testing does catch many errors it is difficult to construct unit tests that will detect the kinds of defects that would be programatically detected by static typing. The application of static type checking to many programs written in dynamically typed programming languages would catch many defects that were not detected with unit testing[...]

Clearly, this is overbroad. For starters, he should have used "could" in place of "would". And it wouldn't have been a bad time to remind the reader that Haskell's type system differs from those of other statically typed languages with which the reader may be more familiar.

I don't quite agree, though, that the conclusion is "trivial". Maybe I'm just out of touch, but I wasn't aware of a good test of how true the dynamic argument was in practice, as opposed to theory -- particularly claim #2.