Live data from Hacker News

Study of 49 programmers: static type system had no effect on development time

cs.washington.edu

121–130 of 192 posts

Re: Study of 49 programmers: static type system had no effect on development time

#121

Earlier quoted context omitted.

Statistically speaking, 'a significant negative impact' doesn't mean the same as it does in regular English. It doesn't say much about how large the difference is, only that they have enough data to show that there _is_ a difference. When an abstract summarizes that the difference wasn't much to talk about, that does mean a lot more than you give credit. Now, judging from their test setup, and also from the very low…

The part quoted said it clearly In the first case, the use of a statically typed programming language had a significant negative impact Considering the size of the sample, I'd guess the difference is rather large to considered significant. By looking at the numbers quickly, it seems to be around 25%. I'm a bit shocked, in fact, because in my own experience, the difference is much larger, but this experiment controls…

Yes, I was able to read that in both your comments, as well as in the paper.

I suspect you are not understanding my point, or what I said about the meaning of the use of the word "significance" when used in statistics.

edit: Say you flip a loaded coin that is 50.1% likely to be heads. Now you want to test whether this is loaded, and flip the coin a certain number of times and count the outcomes. If the number of times you flip the coin is too low, you won't be able to say the coin is 'significantly loaded'. It might be either way, you don't have enough data. If you flip it enough times, you will be able to say something about it -- i.e. that either it significantly is, or significantly isn't loaded.

However, in vernacular English, you would still say that the difference isn't very significant. Who cares if it is 50% or 50.1%.

Re: Study of 49 programmers: static type system had no effect on development time

#122

I think there are several flaws in this study, but alas, its subject is quite inflammatory and it is easy for people to gloss over the details and start drawing conclusions. Here are a few things to keep in mind, IMO: - Good/capable/flexible static languages are much much harder to design than dynamic ones. Case in point is Haskell, where hundreds (thousands?) of researches have been working for years to widen the ar…

Real world tasks or math tasks?

Re: Study of 49 programmers: static type system had no effect on development time

#123
post #73

Earlier quoted context omitted.

You are hitting exactly the main point. Dynamic languages are very good for prototyping or small scale projects. But they fail to address the context of programming large scale applications with teams distributed across multiple sites.

I'm betting youtube is bigger than all of your sites combined, and they use python.

Let's see, each project usually has:

- Multi-site development at least in 3 countries; - Sometimes up to 100+ developers - CI systems - Source code of several MB of source code - Enterprise like infrastructures for Fortune 500 companies - Different skill sets from the "just out of the university" to the "top developer"

Maybe YouTube beats this, but Google only has Phd guys able to crack out crazy algorithms/data structures in minutes. Not typical in most software houses.

Re: Study of 49 programmers: static type system had no effect on development time

#124

Looking over the comments, WOW, please please PLEASE read the paper! This title on HN is very misleading. A static type system had no POSITIVE effect on development time. The study is casting doubts on POSITIVE impact of static type systems. To quote: We measured two different points in the development: first, the development time until a minimal scanner has been implemented, and second the quality of the resulting s…

This study is way to small and limited to show much of anything... but many of the proponents and historical engineering studies that advocate static typing don't claim that it reduces "development time" so much as reduce the times/costs throughout the entire life cycle of the product. In fact, the "development time" or in this case "prototype time" hasn't been all that terribly interesting.

What would be interesting is to take these same projects, and same team members and add some features a year later and see what that looks like and then take the same project and give it to a new team after 6 months and have them add some minor feature or fix a bug and measure that. Or double the performance of the project and measure what that costs, if it's possible. etc.. It's fun to argue and advocate for what you like though.

Re: Study of 49 programmers: static type system had no effect on development time

#125

In my own experience: if I write dynamic code I'm still thinking of types. Especially in Python.. What's mutable, what isn't... This is what my professors at school say too. Less typing (pressing keys) doesn't make you think faster!

Duck typing requires more mental effort to track the variable types. And in python some type errors are detected only in runtime, like when you call a function with a wrong parameter count.

Re: Study of 49 programmers: static type system had no effect on development time

#126
post #73

Earlier quoted context omitted.

You are hitting exactly the main point. Dynamic languages are very good for prototyping or small scale projects. But they fail to address the context of programming large scale applications with teams distributed across multiple sites.

I agree dynamic languages result in smaller codebases, but that's only because they tend to be so much more concise than most statically typed languages. Also, as for large and complex projects done by distributed teams, I don't have to point much further than Django or Plone to prove dynamic typing works well in that context.

The plural of anecdote is not proof. Heck, it's not even data.

Heck, the examples you give aren't even anecdotes. They're just name-dropping. One would have to be pretty familiar with the codebases in question and the history of their development in order to be able to give a clear assessment of what, if any, impact dynamic typing might have had on them.

Re: Study of 49 programmers: static type system had no effect on development time

#127

The study may or may not be flawed, but what's really interesting to me is the reaction. We need more science in our computer science, which means more experiments and more results like this. We should also be open to the truth that we use the tools we like because we like them rather than because they're technically superior, even though we pimp them ad nauseum as though they are. I once read an article about a tech…

Reasoning from basic principles is a valuable tool to evaluate conclusions. Sure, it has strong weaknesses (hidden assumptions which are wrong, insufficient imagination about what could happen, rationalizing one's biases). But it is useful, when you dont have complete trust in the quality or the scope of the experiment. For instance, claims about quantum computing solving NP complete problems are legitimately held in doubt because of theoretical reasons. Also, whenever there are short term positives hiding a long term negative, like say unsustainable financial or ecological behavior, the negatives might be only seen by a chain of reasoning and not by direct experiments.

I agree in the sense that we see so many cases in the other direction - reasoning full of holes being trusted over empirics. The interesting thing is in any given situation, how much trust to give to the different tools that we have to evaluate a claim.

Re: Study of 49 programmers: static type system had no effect on development time

#128
The static versus dynamic typing "battle" is kind of from the kindergarten. You need both, similarly as to how both boys need their fathers even though both of them get declared the strongest in the world in turns.

With dynamic typing it's much more difficult to create large, reliable codebases. There's too much implicitness among the interactions of different actors, modules, and programmers.

With static typing it's much more difficult to create any codebase. The exploratory nature of programming means that you will have to constantly change and redefine the static types, signatures and interfaces.

What you want is a language where you can start prototyping dynamically and later on when the APIs stabilize gradually you can then stamp static type checking on top of them for cementation without actually having to rewrite anything, possibly in another language. When you can selectively force a freeze on a certain piece of code or API then you can gradually move from dynamic to "static enough" typing as the part of your development process.

Re: Study of 49 programmers: static type system had no effect on development time

#129
I wonder whether they are actually measuring the right thing here. I think the real value to static typing is not so much the initial development of small (tiny, in this case) programs, but rather large systems that have long lifetimes and maintenance periods involving multiple developers.

I just started diving into Ruby. I'm about a month into it, after having spent over a decade doing Java and other static languages. The productivity boost for new development in Ruby is very real; I converted a 10K LoC java program to In dynamic languages it's very hard to navigate existing code and libraries. In Java, you can click on a variable and instantly navigate to its declaration and class definition. In a well-configured environment (say Intellij + Maven) you can even click on imported libraries and drill right down into their source, even the JRE. It's totally natural to know exactly what a method takes as arguments, and what it returns.

In Ruby it always feels like a mystery. When you ask the IDE to jump into a library, it pops up a dialog of 20 choices and makes you guess which one to open. It feels like the old days of running find/grep over the source tree. And if you somehow manage to guess correctly and find the code you're looking for, you often have to read the comments (if they exist) to figure out what it expects/returns. I'm sure a lot of this is due to my inexperience, but code navigation is really painful in a dynamic environment. I can only imagine how hard it would be in a non-trivial project after a couple of years with a few different devs.

Of course refactoring is way better in a static environment. Even if you throw out the IDE, you can more or less instantly find where a piece of code is used by simply renaming it and recompiling to see what breaks. In Ruby you have to rely on tests for that kind of thing, instead of getting it for free from the compiler.

Don't get me wrong- I'm excited about Ruby, and I think dynamic languages will play a major role in the rest of my career. I just wish that code navigation could be better.

Re: Study of 49 programmers: static type system had no effect on development time

#130

This is an interesting paper, but I'd dismiss the findings as a strong argument in the dynamic vs static typing debate. I think the setup is a little bit skewed as it's plays into the strengths of dynamic languages: * The whole api consisted of 14 classes, which makes the conclusion that static typing doesn't help with API discoverability somewhat moot. * We are not talking about Haskell, Scala or even Java/C++ flavo…

The scope of the study also seems suspect to me. From the paper:

"The introduced study is a relatively large study with 49 subjects and required for each subject approximately 27 hours pure development time. Including the teaching time for each subject, between 43 and 45 hours working time was required per subject."

If I were working on a software project where I expected 30 hours of development work, I'm not sure static vs dynamic typing would matter at all. I'd pick the language that would get the job done in 30 hours.

I'm surprised this was accepted in a peer-reviewed conference. I'd give this study more credence if this involved the same number of subjects working on or inheriting a much larger, longer-lived code base. Though, based on the subtitle of the paper, it sounds like the author could be convinced otherwise.

Post reply on HN