Study of 49 programmers: static type system had no effect on development time • Hacker News Board

Re: Study of 49 programmers: static type system had no effect on development time

#171

Post by aidenn0 » Fri, Jun 08, 2012, 5:35 PM UTC

cageface wrote: post #166

rbanffy wrote: post #151

Earlier quoted context omitted.

> I've heard of big Ruby projects with test suites that take 20+ hours to run. I'd assume those tests go well beyond type checking.

Sure but remember most static languages are 20+ times faster than Ruby so even with compile times you might still come out way ahead.

Ah, but a lot of bugfixes are developed by making small changes to understand better what is going on internally during the bug symptom, and there the compile time is more important than the compile/test time.

Re: Study of 49 programmers: static type system had no effect on development time

#172

Post by aidenn0 » Fri, Jun 08, 2012, 5:37 PM UTC

Lednakashim wrote: post #26

Don't we are about the ability of the compiler to optimize? Does anybody compile code nowadays?

Meh, dynamically typed systems are within an order of magnitude of statically typed systems, even on benchmarks that greatly benefit from the static type information.

Re: Study of 49 programmers: static type system had no effect on development time

#173

Post by bunderbunder » Fri, Jun 08, 2012, 5:46 PM UTC

silverlake wrote: post #164

bunderbunder wrote: post #150

Earlier quoted context omitted.

Well, one thought that comes to mind is that the topic is essentially a subset of ergonomics, so perhaps experimental protocols should take a few more cues from ergonomics research. For example, controlling variables has to be done by constructing artificial systems from the ground up. You can't just pull two commercial products off the shelf and then pretend you're examining the impact of only one of the hundered di…

Did you even read the linked paper? They did exactly what you described.

As the linked paper points out in the very first sentence of the section describing the language, this hasn't been a very popular approach so far.

I suppose I should have nodded to that. Got me there.

Re: Study of 49 programmers: static type system had no effect on development time

#174

Post by bunderbunder » Fri, Jun 08, 2012, 5:55 PM UTC

rbanffy wrote: post #147

bunderbunder wrote: post #126

Earlier quoted context omitted.

The plural of anecdote is not proof. Heck, it's not even data. Heck, the examples you give aren't even anecdotes. They're just name-dropping. One would have to be pretty familiar with the codebases in question and the history of their development in order to be able to give a clear assessment of what, if any, impact dynamic typing might have had on them.

Fair enough. They prove large projects built by distributed teams are possible in at least one dynamic language. There was a time I was familiar with both codebases (I have some catching up to do) and that's why I mentioned them. Both projects carry heavy heritage and are experiencing huge pressures to evolve and both are doing very well (from what I hear on the dev lists).

Sure, but even then one merely has an observation that there are these two large projects that are written in dynamic languages and are being successfully maintained.

That doesn't imply that they are being successfully maintained because they are written in dynamic languages, or despite being written in dynamic languages. And of course there's also the possibility that static vs. dynamic is a wash and doesn't really have an impact at all. Or that the potential impact of going with static or dynamic is heavily influenced by other factors - does the static language have type inference, does the dynamic language support duck typing, stuff like that.

Long story short, correlation does not, in and of itself, imply causation.

Re: Study of 49 programmers: static type system had no effect on development time

#175

Post by jdlshore » Fri, Jun 08, 2012, 6:02 PM UTC

barik wrote: post #72

edanm wrote: post #57

Earlier quoted context omitted.

I think your point is important, and well made. The only problem is, people have been doing these experiments for 30 years, and do you know what the net effect it's had on the world of programmers: none at all. Saying "no no, this time really listen to this study " seems to be having no effect. There are a lot of causes for this, not least of all the things you mention (nobody cares about science, people like or disl…

"In fact, one of the few studies that people actually believe is the "some programmers are 10x better" study, and that was actually fairly well conducted - many students were given identical tasks, and a fairly large amount of time to do them." Can you by chance point me to this paper? I'd like to add it to my paper collection, since most of the studies I've seen concerning programmer variability use members of the w…

There is no such study. It's a well-regarded and popular myth.

Laurent Bossavit does a masterful job of researching the origins of this myth (and others) in his new book The Leprechauns of Software Engineering, available on LeanPub [1].

[1] http://leanpub.com/leprechauns

Re: Study of 49 programmers: static type system had no effect on development time

#176

Post by [deleted] » Fri, Jun 08, 2012, 6:18 PM UTC

thomasz wrote: post #100

This is an interesting paper, but I'd dismiss the findings as a strong argument in the dynamic vs static typing debate. I think the setup is a little bit skewed as it's plays into the strengths of dynamic languages: * The whole api consisted of 14 classes, which makes the conclusion that static typing doesn't help with API discoverability somewhat moot. * We are not talking about Haskell, Scala or even Java/C++ flavo…

[deleted]

Re: Study of 49 programmers: static type system had no effect on development time

#177

Post by jdlshore » Fri, Jun 08, 2012, 6:18 PM UTC

thomasz wrote: post #100

This is an interesting paper, but I'd dismiss the findings as a strong argument in the dynamic vs static typing debate. I think the setup is a little bit skewed as it's plays into the strengths of dynamic languages: * The whole api consisted of 14 classes, which makes the conclusion that static typing doesn't help with API discoverability somewhat moot. * We are not talking about Haskell, Scala or even Java/C++ flavo…

Final paragraph of section 3.2, page 4: "...we have to make sure that the programming task given to the subjects cannot be solved by the subjects completely..."

Re: Study of 49 programmers: static type system had no effect on development time

#178

Post by rbanffy » Fri, Jun 08, 2012, 8:01 PM UTC

bunderbunder wrote: post #174

rbanffy wrote: post #147

Earlier quoted context omitted.

Fair enough. They prove large projects built by distributed teams are possible in at least one dynamic language. There was a time I was familiar with both codebases (I have some catching up to do) and that's why I mentioned them. Both projects carry heavy heritage and are experiencing huge pressures to evolve and both are doing very well (from what I hear on the dev lists).

Sure, but even then one merely has an observation that there are these two large projects that are written in dynamic languages and are being successfully maintained. That doesn't imply that they are being successfully maintained because they are written in dynamic languages, or despite being written in dynamic languages. And of course there's also the possibility that static vs. dynamic is a wash and doesn't really…

In any case, it's mostly an apples to oranges comparison. Observing from my heavily biased toward web applications experience, there aren't as many Python codebases as there are Java ones at least in part because you can do a lot more with a small Python codebase than you can do with a similarly sized Java codebase.

Re: Study of 49 programmers: static type system had no effect on development time

#179

Post by sampo » Fri, Jun 08, 2012, 8:45 PM UTC

okamiueru wrote: post #121

rbanffy wrote: post #110

Earlier quoted context omitted.

The part quoted said it clearly In the first case, the use of a statically typed programming language had a significant negative impact Considering the size of the sample, I'd guess the difference is rather large to considered significant. By looking at the numbers quickly, it seems to be around 25%. I'm a bit shocked, in fact, because in my own experience, the difference is much larger, but this experiment controls…

Yes, I was able to read that in both your comments, as well as in the paper. I suspect you are not understanding my point, or what I said about the meaning of the use of the word "significance" when used in statistics. edit: Say you flip a loaded coin that is 50.1% likely to be heads. Now you want to test whether this is loaded, and flip the coin a certain number of times and count the outcomes. If the number of time…

There is no reason to quarrel about the meaning of 'significance' in statistical testing vs. in natural language. Just look at figs. 4 and 5 in the paper, and see that mean times spent for the scanner task were:

5.17 hours (dynamical typing) 7.71 hours (static typing)

And that the difference in statistically significant (p=0.04, Mann-Whitney U-test). Whether 5 vs. 8 hours is significant in the natural language sense, everyone can decide for themselves.

Re: Study of 49 programmers: static type system had no effect on development time

#180

Post by weavejester » Fri, Jun 08, 2012, 9:19 PM UTC

akkartik wrote: post #170

weavejester wrote: post #157

Earlier quoted context omitted.

How does static typing help that?

By limiting the damage a single crappy programmer can do. Everywhere people talk about 'large' teams maintaining a large codebase in this thread, substitute 'mediocre' teams stuck with a poor bloated codebase that is the vehicle for their ambitions. It's just not worth anybody's time to understand its unique needs in detail, especially since any improvement you make today risks being messed up tomorrow.

I can understand using static typing if you already have a large codebase that's statically typed, but is there any reason you'd start a project with a statically typed language?