Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Study of 49 programmers: static type system had no effect on development time

cs.washington.edu

Post reply on HN
Print view View original

101–110 of 192 posts

Re: Study of 49 programmers: static type system had no effect on development time

#101

Post by thomasz »

batista wrote: post #34
cromwellian wrote: post #13

Studying individuals is the wrong experiment. Static typing benefits tooling most when you have larger groups of programmers who much collaborate, often asynchronously. It's easy to hold a mental model of the code you write in your head and minimize mistakes, it's harder to hold a mental model of a larger program consisting of code written by many programmers. I'd like to see them hand a pre-written codebase of say,…

> Studying individuals is the wrong experiment. Static typing benefits tooling most when you have larger groups of programmers who much collaborate, often asynchronously. Absolutely. And also: static typing benefits is not about "development time" but about maintenance, adding stuff, refactoring etc. If anything, dynamic typing would be expected to lead to faster development time, which is also why it's used in most…

They used a custom, minimal IDE that looks like a text editor and a class browser sewn together.

Re: Study of 49 programmers: static type system had no effect on development time

#102

Post by ExpiredLink »

fusiongyro wrote: post #32

The study may or may not be flawed, but what's really interesting to me is the reaction. We need more science in our computer science, which means more experiments and more results like this. We should also be open to the truth that we use the tools we like because we like them rather than because they're technically superior, even though we pimp them ad nauseum as though they are. I once read an article about a tech…

> We need more science in our computer science, which means more experiments and more results like this.

There is an ongoing experiment wrt the usability of programming languages, libraries, frameworks, concepts, ... It's called the market.

Re: Study of 49 programmers: static type system had no effect on development time

#103

Post by sgk284 »

fusiongyro wrote: post #32

The study may or may not be flawed, but what's really interesting to me is the reaction. We need more science in our computer science, which means more experiments and more results like this. We should also be open to the truth that we use the tools we like because we like them rather than because they're technically superior, even though we pimp them ad nauseum as though they are. I once read an article about a tech…

Agreed with your overall premise, but this particular study doesn't meet the criteria to have any conclusions drawn from its results.

The number of programmers is too small and skills not representative (49 undergrad programmers). The problem (writing a simple scanner / parser) isn't one that will really benefit from a decent type system. All you need need are ints, strings, and arrays and you're good to go.

Re: Study of 49 programmers: static type system had no effect on development time

#104

Post by webreac »

This study is flawed: the usefulness of type system is not for initial development of a small system, it is for the maintenance of huge code base.

The fact that type system had very small cost for initial development of small system should be seen as an invitation to use strong types.

Re: Study of 49 programmers: static type system had no effect on development time

#105

Post by okamiueru »

quadhome wrote: post #94
tikhonj wrote: post #83

Earlier quoted context omitted.

"In the experiments the existence of the static type system has neither a positive nor a negative impact on an application's development time (under the conditions of the experiment)." Maybe the abstract is lying, but that seems to agree with my original premise. That is, the type system had no effect on development time. Now, the paper makes no comment either way--all it says is that the type system did not have an…

Read further than the abstract. It's just an abstract. To quote: We measured two different points in the development: first, the development time until a minimal scanner has been implemented, and second the quality of the resulting software measured by the number of successful test cases fulfilled by the parser. In none of these measured points the use of a static type system turned out to have a successful impact. I…

Statistically speaking, 'a significant negative impact' doesn't mean the same as it does in regular English. It doesn't say much about how large the difference is, only that they have enough data to show that there _is_ a difference. When an abstract summarizes that the difference wasn't much to talk about, that does mean a lot more than you give credit.

Now, judging from their test setup, and also from the very low number of testers, I find it very hard to agree with what it seems to be you're taking away from this article. Static typing has more benefit in large code-bases, with multiple programmers, and for avoiding hard to find bugs related to dynamic typing. This doesn't seem to be well reflected by the setup they had.

Also, don't think you should repeat the same comments throughout the HN articles, you are not replying to individuals, but to general readers.

Re: Study of 49 programmers: static type system had no effect on development time

#106

Post by mduerksen »

akkartik wrote: post #70
mduerksen wrote: post #66

I'm still torn between dynamic and static typing. The main disadvantage of dynamic typing for me is the lack of verifiable documentation about what data a function needs, and which data comes out of it. This becomes a problem when a) the data is complex (e.g. dictionary of lists of items with certain properties) b) I haven't looked at the function for a while. In those cases, I find statically typed code easier to re…

I use tests for documentation. They seem like more work at the start, but they're more flexible. You can self-document things with tests that you cannot with just types. For example, here are some tests from the lisp interpreter I've been working on: http://github.com/akkartik/wart/blob/8a8cf96816/030.test You're right that tests aren't close to code. But that can be good or bad. Since it's not next to the code it ca…

> You can self-document things with tests that you cannot with just types

Absolutely. But I would say that is a different debate. In my case, I don't want to specify the behavior, but only the form, which is the counterpart of a static type, but more flexible (e.g. {} is likened to any structure that behaves like a map - the class doesn't have to be exactly the same).

With my like-function, I don't have to spend too much time specifying what is expected, and I get a lot of bang for the buck.

I want to do just enough so the code stays comprehensible and thus manageable.

(fn parse-int [str] ...) needs no type or unit-test to be understood.

(fn parse-appointment [str] ...) is better understood when it has {:post [(like {:id (UUID.) :name "me" :date (java.util.Date .)} %)]}

And as soon as the function gets really smart, and it's smartness isn't revealed directly by the code, it would be time for either a good comment or a Unit Test (or make the code better so that it does, which is an often forgotten option).

Re: Study of 49 programmers: static type system had no effect on development time

#107

Post by carterschonwald »

kinghajj wrote: post #27
carterschonwald wrote: post #8

Earlier quoted context omitted.

only 20x? What happens when you reevaluate it with the python code not having any pieces that are C under the covers? (my understanding that a lot of python idioms for fast code basically turn into ah "use the approach that pushes as much work into C as possible")

That's a common idiom in Haskell, too :)

Not in the same way at all. Bytestring or hmatrix may have their primops in C, but the control flow stuff code for libs like Repa or Vector are pure Haskell (and in fact current state of the art work on Repa has it matching if not beating C code).

The crucial difference is that in python, these faster python idioms are for using C as the control code

Re: Study of 49 programmers: static type system had no effect on development time

#108

Post by arihant »

dan00 wrote: post #65
arihant wrote: post #41

Development time with static languages is not an issue. Try comparing the time one needs to change the codebase written in ML and the one written in Lisp and come back. It is experimentation and change that is slow in static typing.

"It is experimentation and change that is slow in static typing." It's all about the expressiveness of the language. If you can express more with less code, than it's also easier to change, to experiment. If you change something in a dynamically typed language you still have to catch all places, where the change has consequences. Only If you want to experiment with something locally without needing to update the rest…

I do not believe so. In languages like ML, static typing is not just used for type safety. Proper use of static typing extends to be able to use it in establishing invariants all over your code. Now, your entire codebase becomes dependent on the type. No matter how expressive a language is, this causes a lot of friction.

Re: Study of 49 programmers: static type system had no effect on development time

#109

Post by rbanffy »

pjmlp wrote: post #73
cromwellian wrote: post #13

Studying individuals is the wrong experiment. Static typing benefits tooling most when you have larger groups of programmers who much collaborate, often asynchronously. It's easy to hold a mental model of the code you write in your head and minimize mistakes, it's harder to hold a mental model of a larger program consisting of code written by many programmers. I'd like to see them hand a pre-written codebase of say,…

You are hitting exactly the main point. Dynamic languages are very good for prototyping or small scale projects. But they fail to address the context of programming large scale applications with teams distributed across multiple sites.

I agree dynamic languages result in smaller codebases, but that's only because they tend to be so much more concise than most statically typed languages.

Also, as for large and complex projects done by distributed teams, I don't have to point much further than Django or Plone to prove dynamic typing works well in that context.

Re: Study of 49 programmers: static type system had no effect on development time

#110

Post by rbanffy »

okamiueru wrote: post #105
quadhome wrote: post #94

Earlier quoted context omitted.

Read further than the abstract. It's just an abstract. To quote: We measured two different points in the development: first, the development time until a minimal scanner has been implemented, and second the quality of the resulting software measured by the number of successful test cases fulfilled by the parser. In none of these measured points the use of a static type system turned out to have a successful impact. I…

Statistically speaking, 'a significant negative impact' doesn't mean the same as it does in regular English. It doesn't say much about how large the difference is, only that they have enough data to show that there _is_ a difference. When an abstract summarizes that the difference wasn't much to talk about, that does mean a lot more than you give credit. Now, judging from their test setup, and also from the very low…

The part quoted said it clearly

In the first case, the use of a statically typed programming language had a significant negative impact

Considering the size of the sample, I'd guess the difference is rather large to considered significant. By looking at the numbers quickly, it seems to be around 25%. I'm a bit shocked, in fact, because in my own experience, the difference is much larger, but this experiment controls for language and my experience doesn't.

Post reply on HN