Study of 49 programmers: static type system had no effect on development time • Hacker News Board

Re: Study of 49 programmers: static type system had no effect on development time

#141

Post by jfoutz » Fri, Jun 08, 2012, 2:29 PM UTC

quadhome wrote: post #80

jfoutz wrote: post #3

so, by swiching from python to haskell, you can get code that's 20x faster for no extra development time? sounds like a win to me. (cherry picked shootout data - http://shootout.alioth.debian.org/u32q/which-programming-lan... )

Please read the paper and not the HN title. The paper's title is casting doubts on the positive impact of statically typed languages as they had a negative impact on development time. (though not quality)

Here's my pet unprovable theory. Development time is dominated by a programmer's understanding of a problem. Static languages force programmers to say a lot about their understanding of a problem.

Dynamic code can be very fast, but in lisp, after algorithms, that means actually going and finding all of the calls to elt and replacing them by calls to nth. With a static language the compiler has a much better chance of specializing the code for you.

Re: Study of 49 programmers: static type system had no effect on development time

#142

Post by kaffeinecoma » Fri, Jun 08, 2012, 2:36 PM UTC

phamilton wrote: post #33

I recently talked to a friend of mine who expressed the frustration of not knowing what a function returns because it's a language without static typing. It was something I hadn't thought of before personally. I thought about it a bit and realized that as a Ruby programmer, I depended on two things to figure out what a function returns, and neither is the source or documentation (most of the time at least): 1) a REPL…

> a REPL. Just give it a whirl and see what you get.

In a large system you might have to jump through a lot of hoops to get an accurate result from the REPL. What if the thing you're trying to puzzle out is 20 call levels deep? As a new Ruby dev I struggle with this.

Re: Study of 49 programmers: static type system had no effect on development time

#143

Post by rbanffy » Fri, Jun 08, 2012, 2:41 PM UTC

okamiueru wrote: post #121

rbanffy wrote: post #110

Earlier quoted context omitted.

The part quoted said it clearly In the first case, the use of a statically typed programming language had a significant negative impact Considering the size of the sample, I'd guess the difference is rather large to considered significant. By looking at the numbers quickly, it seems to be around 25%. I'm a bit shocked, in fact, because in my own experience, the difference is much larger, but this experiment controls…

Yes, I was able to read that in both your comments, as well as in the paper. I suspect you are not understanding my point, or what I said about the meaning of the use of the word "significance" when used in statistics. edit: Say you flip a loaded coin that is 50.1% likely to be heads. Now you want to test whether this is loaded, and flip the coin a certain number of times and count the outcomes. If the number of time…

Yes, but I'd risk that a 25% deviation is significant when the sample is 49 students. The smaller the sample, the larger the deviation must be to be significant, but 25% is quite a difference.

Also, it's worth to notice they controlled for language - they used the same language in two flavors - to isolate the typing system difference. It's not a Lisp vs. C thing.

Re: Study of 49 programmers: static type system had no effect on development time

#144

Post by jshen » Fri, Jun 08, 2012, 2:41 PM UTC

ozataman wrote: post #119

jshen wrote: post #23

Earlier quoted context omitted.

I work for a fortune 20 company and my last employer was a fortune 10 company. My experience is the opposite of yours. I've never seen these consistent wins you talk about, despite my years at some of the largest corporations.

Have you also used dynamic languages extensively as part of large teams at your employer? Dealing with lots of people over long periods of time is messy, no matter what language/tool you use. I think the claim is just that static typing gives you better results than dynamic typing. All relative..

Yes, we used ruby with over 100 programmers at the office I was in.

Re: Study of 49 programmers: static type system had no effect on development time

#145

Post by jshen » Fri, Jun 08, 2012, 2:44 PM UTC

tmarthal wrote: post #40

jshen wrote: post #23

Earlier quoted context omitted.

I work for a fortune 20 company and my last employer was a fortune 10 company. My experience is the opposite of yours. I've never seen these consistent wins you talk about, despite my years at some of the largest corporations.

I don't think the size (or the revenue) of the company matters as much as the size of the projects that you're working on within those companies. Someone mentioned maintenance, so I will also ask how old the codebases that you're working on are? For dynamic typed languages, as the code grows older (with more bug fixes and add-ons and one-offs) then it gets harder and harder to maintain as 100% error free (let alone d…

Large projects, at least 4 years old.

"I will also throw in the fact that doing software library version upgrades on dynamically typed systems is a pain in the ass."

This is very true, but you gain a lot with dynamic languages in other areas. The last company I worked for had about 300 programmers in the office I was in. Roughly half did Java and the other half did Ruby. I was one of the few that moved between both ruby and java projects. I did not see these great benefits of static typing that people always insist exist for large teams at large companies.

Re: Study of 49 programmers: static type system had no effect on development time

#146

Post by klodolph » Fri, Jun 08, 2012, 2:46 PM UTC

jbranchaud wrote: post #111

tikhonj wrote: post #53

Earlier quoted context omitted.

I've only scanned the comments, but the chief reaction seems to be exactly what you want for a scientific approach--people are arguing about the scope and methodology of the study. A paper like this is not some broad-reaching conclusion--it's very specific and based on some potentially flawed methodology. You want people to qualify exactly how specific it is and talk about potential flaws in its approach. That's how…

"Almost no CS papers I've read have contained empirical studies." You aren't reading the right papers then. At least in Software Engineering, you can't get into the main conferences (ICSE and FSE) without a pretty significant empirical study.

Yes, but that's the main difference between computer science and software engineering.

Re: Study of 49 programmers: static type system had no effect on development time

#147

Post by rbanffy » Fri, Jun 08, 2012, 2:50 PM UTC

bunderbunder wrote: post #126

rbanffy wrote: post #109

Earlier quoted context omitted.

I agree dynamic languages result in smaller codebases, but that's only because they tend to be so much more concise than most statically typed languages. Also, as for large and complex projects done by distributed teams, I don't have to point much further than Django or Plone to prove dynamic typing works well in that context.

The plural of anecdote is not proof. Heck, it's not even data. Heck, the examples you give aren't even anecdotes. They're just name-dropping. One would have to be pretty familiar with the codebases in question and the history of their development in order to be able to give a clear assessment of what, if any, impact dynamic typing might have had on them.

Fair enough. They prove large projects built by distributed teams are possible in at least one dynamic language.

There was a time I was familiar with both codebases (I have some catching up to do) and that's why I mentioned them. Both projects carry heavy heritage and are experiencing huge pressures to evolve and both are doing very well (from what I hear on the dev lists).

Re: Study of 49 programmers: static type system had no effect on development time

#148

Post by dpritchett » Fri, Jun 08, 2012, 2:51 PM UTC

10098 wrote: post #54

phamilton wrote: post #33

I recently talked to a friend of mine who expressed the frustration of not knowing what a function returns because it's a language without static typing. It was something I hadn't thought of before personally. I thought about it a bit and realized that as a Ruby programmer, I depended on two things to figure out what a function returns, and neither is the source or documentation (most of the time at least): 1) a REPL…

that's one aspect of dynamic typing that i strongly dislike. you can't figure out what a function does by looking at its signature. if documentation is absent (which is almost always the case), a coding task turns into a reverse engineering task.

A good repl that automatically prints out the function's heredocs is usually enough for me. I like bpython particularly because it shows the argument list and the heredoc as soon as you type the opening parenthesis for a function call. In other REPLs you can usually call a docs(fn) or a source(fn) to handle the rest.

Re: Study of 49 programmers: static type system had no effect on development time

#149

Post by dleimbach » Fri, Jun 08, 2012, 2:55 PM UTC

tl;dr embrace polyglot :-)

If static typing wasn't useful, I don't think anyone would bother with types. At the same time some static type systems are restrictive while coding, so you're seeing more and more type inference in languages.

Go does it by allowing one to declare a variable with an expression " a := 1".

C++ 2011 has it with "auto"

Haskell is pretty darned good at it, and a lot of people don't write the types of their functions, though I think it's good practice to make sure you and the compiler agree on what you've written.

That said there's languages like Clojure which are dynamically typed in a sense, and I believe pay a slight performance penalty because they have to use reflection to behave that way. There's ways to annotate types in Clojure to get around that. This is an interesting case because Lisp is typically a dynamically typed language.

At the same time there's awesome languages and environments like Racket that have statically typed and dynamically typed versions of their scheme dialects.

My feeling is that a ruling on whether types are useful or not is sort of a pointless discussion. It's not even true that "the jury is out". You're just choosing tools with different properties and some make sense for some situations and others for others.

Re: Study of 49 programmers: static type system had no effect on development time

#150

Post by bunderbunder » Fri, Jun 08, 2012, 2:59 PM UTC

silverlake wrote: post #133

fusiongyro wrote: post #32

The study may or may not be flawed, but what's really interesting to me is the reaction. We need more science in our computer science, which means more experiments and more results like this. We should also be open to the truth that we use the tools we like because we like them rather than because they're technically superior, even though we pimp them ad nauseum as though they are. I once read an article about a tech…

No study can control all the variables enough to convince a fanboy that his favorite language isn't the greatest ever. However, the current state of PL research is as close to astrology as you can get. Why are we working on type systems and modules and concurrency primitives et. al. without a scrap of evidence that any of it contributes to programmer productivity? There's no science there. In fact, everyone here shou…

Well, one thought that comes to mind is that the topic is essentially a subset of ergonomics, so perhaps experimental protocols should take a few more cues from ergonomics research.

For example, controlling variables has to be done by constructing artificial systems from the ground up. You can't just pull two commercial products off the shelf and then pretend you're examining the impact of only one of the hundered different things that differs between the two.

Similarly, if we wanted to compare the impact of dynamic vs. static typing, we'd have to make sure that that is the only variable. Which means you basically have to construct a new programming language from the ground up, so that you can easily create new dialects of it that differ in only one very specific characteristic.