Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

Do there exist studies done on the effectiveness of statically vs dynamically typed languages?

In particular:

  • Measurements of programmer productivity
  • Defect Rate

Also including the effects of whether or not unit testing is employed.

I've seen lots of discussion of the merits of either side but I'm wondering whether anyone has done a study on it.

Edit

Sadly, only one of the papers shown is actually a study and it does nothing but conclude that the language matters. This leads me to ponder: what if I proposed doing such a study with volunteers from this site?

share|improve this question
6  
@bigown, it doesn't seem to me that issues of productivity and defects relate to computer science theory –  Winston Ewert Oct 6 '10 at 22:04
2  
@Winston: Studying this kind of issues it's the job of computer scientists, not programmers. –  bigown Oct 6 '10 at 23:21
6  
@bigown, yes its a computer science issue but its not a computer science theory issue. CS theory essentially deals with what we can mathematically prove about about programs and computing. Issues of programmer productivity are not cs theory questions. There have been discussions of dynamic typing both here and on stackoverflow. There have been none on cstheory. –  Winston Ewert Oct 7 '10 at 0:09
4  
The question's perfectly on topic. This question discusses one of the most important properties of the tools we use to program. –  Frank Shearar Oct 15 '10 at 12:46
3  
@Winston: Typing systems do belong in CS theory, but practical studies don't. –  David Thornley Oct 15 '10 at 15:16

6 Answers 6

Here's a starting point:

The paper is challenging the commonly received wisdom that, all else being equal, programmers write the same number of lines of code per time regardless of language. In other words, the paper should serve as supporting empirical evidence that mechanical productivity (lines of code written) is not a good measure of functional productivity, and must at least be normalized by language.

share|improve this answer
4  
For the non-IEEE people, what's the basic summary? –  Frank Shearar Oct 16 '10 at 7:21
    
@Frank Shearar, the conclusion they draw is that programming language does affect productivity. They are measuring lines of code per programmer per language per year, I'm not sure thats a good measure of productivity. –  Winston Ewert Oct 18 '10 at 15:04
2  
@Winston: That's definitely a flawed metric. You'd find COBOL to be a very productive language by it: it takes a lot of lines to do anything useful, but they're fairly easy to write. –  David Thornley Oct 20 '10 at 16:07
    
Winston, David: I'm pretty sure the authors are not suggesting that lines-of-code productivity is a measure of functional productivity. Rather, the paper is challenging the commonly received wisdom that, all else being equal, programmers write the same number of lines of code per time regardless of language. In other words, the paper should serve as supporting empirical evidence that mechanical productivity (lines of code written) is not a good measure of functional productivity, and must at least be normalized by language. –  Piet Delport Oct 20 '10 at 16:26
    
I agree with that. But it doesn't serve to answer my original question. –  Winston Ewert Oct 20 '10 at 17:26

Some suggested reading:

Not exactly on static typing, but related:

Some interesting articles or essays on the subject or on static analysis of programs in general:

And for the ones who would be wondering what this is all about:

However, I doubt any of these with give you a direct answer, as they don't do exactly the study you're looking for. They will be interesting reads though.

Personally, I firmly consider that static typing over dynamic typing facilitates bug detection. I spend way too much type looking for typos and minor mistakes like these into JavaScript or even Ruby code. And when it comes to the view that Dynamic Typing gives you a boost in productivity, I think that mostly comes down to tooling. If statically typed languages have the right tools to allow for background recompilation and provide an REPL interface, then you get the benefits of both worlds. Scala provides this for instance, which makes it very easy to learn and prototype away in the interactive console, but gives you the benefits of static typing (and of a stronger type system than a lot of other languages, ML-languages aside). Similarly, I don't think I have a loss of productivity by using Java or C++ (because of the static typing), as long as I use an IDE that helps me along. When I revert to coding only with simple configurations (editor + compiler/interpreter), then it feels more cumbersome and dynamic languages seem easier to use. But you still hunt for bugs. I guess people would say that the tooling issue is a reversible argument, as if tooling were better for dynamic languages, then most bugs and typos would be pointed out at coding-time, but that reflects the flaw in the system in my opinion. Still, I usually prototype in JRuby and will code in Java later most of the things I do.

WARNING: Some of these links are unreliable, and some go through portals of various computing societies using fee-based accesses for members. Sorry about that, I tried to find multiple links for each of these but it's not as good as I'd like it to be.

EDIT: I added some more articles.

share|improve this answer
    
That bug finding - is it mainly because of mis-spelled variable names, or method names, or somewhere in between? (I loathe implicit variable declaration for precisely this reason: in Smalltalk, you declare all your variables at the top, so you immediately know when you've mistyped a variable name. (Method name typos are sometimes caught too, if the method name's never been used in the image before.)) –  Frank Shearar Oct 16 '10 at 11:53
    
Re tooling, I have to say that it depends on your language - Smalltalk has excellent tools, including a Refactoring Browser that Eclipse has (I'm told) yet to catch up to. –  Frank Shearar Oct 16 '10 at 11:55
    
@Frank Shearar, since I started with Ruby (coming from Java), I've realized that what @haylem's saying probably does not apply to Smalltalk. Nor does my mantra about automatic refactoring being impossible in dynamically-typed langs. I completely agree with haylem's "personally" section.... ignoring SmallTalk of course :) This is fair, to some extent, since SmallTalk, while not dead, is definitely not enjoying the popularity that Python or Ruby have (now in Oct 2010). –  Yar Oct 16 '10 at 14:22
2  
@haylem, personally I thank you for making me feel that I'm not the only person in the world that works in dynamic languages but, when given a choice, many times CHOOSES statically-typed languages (same case, Java vs. JRuby or Groovy). –  Yar Oct 16 '10 at 14:24
4  
It is interesting because my own preference for dynamic typing is for rather different reasons. I mean fast compiles and interactive interpreter are useful but they aren't why I like dynamic typing. I like dynamic typing because I find many situations in which the static typing languages just make it difficult to impossible to describe a particular concept. –  Winston Ewert Oct 18 '10 at 0:50

I honestly do not think that Static vs Dynamic typing is the real question.

I think that there are two parameters that should come first:

  • the expertise level in the language: the more experienced you are, the more you know about the "gotchas" and the more likely you are to avoid them / track them down easily. This is also true about the particular application/program you are working on
  • testing: I love static typing (hell I like programming in C++ :p) but there just so much that a compiler / static analyzer can do for you. It's just impossible to be confident about a program without having tested it. And I am all for fuzzy testing (when applicable), because you just can't think about all possible input combinations.

If you are comfortable in the language, you'll write code and you'll track down bugs with ease.

If you write decoupled code, and test each functionality extensively, then you'll produce well-honed code, and thus you'll be productive (because you cannot qualify as productive if you do not assess the quality of the product, can you ?)

I would therefore deem that the static vs dynamic debate with regard to productivity is quite moot, or at least vastly superseded by other considerations.

share|improve this answer
2  
If this is a counter-question, where is the question in it? :) I agree that other factors are more important then static vs dynamic typing. However, dynamic typing advocates claim better productivity and static typing advocates claim better code quality. I was wondering whether anyone had actual evidence to support their claims. –  Winston Ewert Oct 16 '10 at 18:17
    
@Winston: I removed the counter bit :p As you mentionned it's mostly claims. I think most advocates of dynamic typing are mixing ease of use with dynamic typing, while ease of use is mostly about tools. I do agree that the possibility to write quick throw-away prototypes and to experiment short commands using an interpreter are a productivity boost, but even Haskell (perhaps the language with the most impressive type system of the moment) has an interpreter for quick experimentation :) –  Matthieu M. Oct 17 '10 at 10:23
    
But until someone actually does a study that considers this question - whether methodology, tools have a larger impact than language on defect rates, productivity - we just end up comparing anecdotes. –  Frank Shearar Oct 20 '10 at 8:22

Here are a few:

  • Stefan Hanenberg. 2010. An experiment about static and dynamic type systems: doubts about the positive impact of static type systems on development time. In Proceedings of the ACM international conference on Object oriented programming systems languages and applications (OOPSLA '10). ACM, New York, NY, USA, 22-35. DOI=10.1145/1869459.1869462 http://doi.acm.org/10.1145/1869459.1869462

  • Daniel P. Delorey, Charles D. Knutson, Scott Chun, "Do Programming Languages Affect Productivity? A Case Study Using Data from Open Source Projects," floss, pp.8, First International Workshop on Emerging Trends in FLOSS Research and Development (FLOSS'07: ICSE Workshops 2007), 2007

  • Daly, M.; Sazawal, V., Foster, J.: Work in Progress: an Empirical Study of Static Typing in Ruby, Workshop on Evaluation and Usability of Programming Languages and Tools (PLATEAU) at ON-WARD 2009.

  • Lutz Prechelt and Walter F. Tichy. 1998. A Controlled Experiment to Assess the Benefits of Procedure Argument Type Checking. IEEE Trans. Softw. Eng. 24, 4 (April 1998), 302-312. DOI=10.1109/32.677186 http://dx.doi.org/10.1109/32.677186

share|improve this answer

Just yesterday I've found this study: Unit testing isn't enough. You need static typing too.

Basically the author used a tool able to convert automatically a project from a non-static typing language into a static typing one (python to haskell)

Then he selected a number of open source Python projects that also included a reasonable amount of test units, and automatically converted them to haskell.

The translation to Haskell revealed a series of errors related to the type of the variables: the errors weren't discovered by the test units.

share|improve this answer
    
Uncomfortable truth of dynamic typing. –  Den Jan 24 at 13:57
1  
"The translation to Haskell revealed a series of errors related to the type of the variables: the errors weren't discovered by the test units.": With a dynamically-typed language you have to manually test properties of your code that in a statically-typed language are automatically checked by the compiler. This is both more time-consuming and error-prone. +1 –  Giorgio Mar 30 at 16:18
  • Link to discussion of ACM paper "An Experiment About Static and Dynamic Type Systems" (2010) by Stephan Hanenberg article (referenced by Lorin Hochstein in a previous post).
  • Conclusion: Productivity for similar quality was higher in a dynamic language.
  • Potential biases/validity issues: Experimental subjects were all students. Also, limited variety of the programming tasks (subjects were asked to implement a scanner and parser).
  • ACM paper "Do Programming Languages Affect Productivity?" (2007) by Delorey, Knudson, and Chun.
  • Conclusion: JavaScript, Tcl, Perl more productive than C# C++ and Java. Python and PHP fall in the middle.
  • Potential biases/validity issues: No measure of quality (such as bugs discovered post-release). No measure of reliability (is software written in statically typed languages more dependable?). Sample bias - all projects were open taken from open source CVS repositories. Also, no distinction between weakly and strongly typed languages (i.e. pointers).
  • Thesis "Empirical Study of Software Productivity and Quality" (2008) by by Michael F. Siok
  • Conclusion: Choice of programming language does not significantly influence productivity or quality. However, it does affect labor costs and "quality within the overall software projects portfolio".
  • Potential biases/validity issues: Restricted to avionics domain. Programming languages could have all been statically typed. I didn't read the thesis, so I cannot evaluate its rigor.
    My opinion. Although there is weak evidence that dynamically typed languages are more productive, it is not conclusive. (1) There are many factors that were not controlled, (2) there are too few studies, (3) there has been little or no discussion about what constitutes an appropriate test method.
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.