Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

In No Silver Bullet, Fred Brooks makes a variety of predictions about the future of software engineering, best summed up by:

There is no single development, in either technology or in management technique, that by itself promises even one order-of-magnitude improvement in productivity, in reliability, in simplicity.

His argument is very convincing. Brooks was writing in 1986: was he right? Do developers in 2014 produce software at a rate less than 10x faster than their counterparts in 1986? By some appropriate metric -- how large has the gain in productivity actually been?

share|improve this question

closed as primarily opinion-based by MichaelT, GlenH7, Jim G., Kilian Foth, Robert Harvey Aug 13 at 18:33

Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise.If this question can be reworded to fit the rules in the help center, please edit the question.

4  
Comments are not for extended discussion; this conversation has been moved to chat. –  World Engineer Aug 12 at 18:28

8 Answers 8

up vote 58 down vote accepted

Do developers in 2014 produce software at a rate less than 10x faster than their counterparts in 1986?

I would imagine that there's been at least an order of magnitude improvement in productivity since then. But not by leveraging one single development, in either technology or in management technique.

Increases in productivity have come about by a combination of factors. Note: This is not a comprehensive list:

  1. Better compilers
  2. Vastly more powerful computers
  3. Intellisense
  4. Object orientation
  5. Functional orientation
  6. Better memory management techniques
  7. Bounds checking
  8. Static code analysis
  9. Strong typing
  10. Unit Testing
  11. Better programming language design
  12. Code generation
  13. Source code control systems
  14. Code reuse

And so on. All of these techniques combine to produce productivity gains; there isn't a single silver bullet that has ever produced an order of magnitude speedup.

Note that some of these techniques have existed since the sixties, but have only observed widespread recognition and adoption recently.

share|improve this answer
15  
Don't forget better source/version control systems. –  Doval Aug 12 at 15:28
7  
Ah, right. I would imagine that we could add another dozen (or hundred) things to this list. –  Robert Harvey Aug 12 at 15:29
44  
@user61852: Because expectations always exceed reality. –  Robert Harvey Aug 12 at 16:20
9  
@user61852 thecodelesscode.com/case/154 –  Idan Arye Aug 12 at 21:51
1  
@RossPatterson: We've basically got what we need for developer tools at this point. We're even doing stupidly wasteful things with our compiling CPU cycles these days just because we can---look at template metaprogramming. Remember we're comparing against the '80s; while I certainly wasn't a developer then, I did learn to code on a machine built in 1990. –  tmyklebu Aug 13 at 0:39

Robert Harvey's answer is good, but I think he left out what may be the biggest reason why programmers are more productive than ever: widespread availability of software libraries.

When I started my career there was no STL, .NET, QT, etc. You had raw C or C++, and that was about it.

Things that used to take days or weeks or work (XML parsing, TCP/IP connections, HTTP communciation) can now be done with a handful of lines of C# code. This is where we have gotten orders of magnitude better in terms of overall developer productivity.

Our current product uses a docking window framework that we purchased from a vendor. This let me have a fully functional docking window UI in a matter of minutes. I shudder to think of what it would take to do that in the days of using the win32 API.

share|improve this answer
19  
I would add to this the availability of documentation and assistance. Twenty years ago, you had a mailing list and your immediate colleagues. Now, the programming expertise of the world, in nearly every topic, is available instantly through a single interface. –  NWard Aug 12 at 17:26
15  
@NWard so basically stack overflow =) –  Chris Aug 12 at 18:50
2  
@tmyklebu people just found other (generally better) solutions [citation needed]. Using libraries to quickly parse "mountains" of XML is in many ways better than hand coding unique solutions. XML can certainly be overly terse and repetitive, but the libraries make using it a breeze in most situations. –  WernerCD Aug 13 at 3:32
5  
@tmyklebu As painful as it can be to parse large quantities of XML, try parsing large quantities of binary data in an undocumented proprietary format, as would have been produced by most applications written circa 1986. –  James_pic Aug 13 at 11:29
2  
@tmyklebu No one needed gigabytes of anything back in the day (or even megabytes!). What generates that amount of XML is the fact that we are storing and tracking way more data than ever. james_pic is right, XML is way better than having a bazillion proprietary binary formats kicking around. XML may be wordy, but it is cross-platform, human readable, and highly flexible. Those are all valuable traits. –  17 of 26 Aug 13 at 13:06

For the sake of argument, I disagree with the assertion of Fred Brooks.

There is an improvement in technology which allowed alone an order-of-magnitude improvement in productivity: internet, and more precisely the progressive availability of internet connection in every home in the last two decades.

Being able to find instantly an answer to nearly every problem you can encounter as a developer enables a huge boost in productivity. Don't know how to select nth element in JQuery? Google search leads to an answer of the exact question on Stack Overflow. Don't know how to use overflow in CSS? Mozilla's MDN is here. Forgot what virtual keyword means in C#? Google helps, again.

When I started programming, I didn't have internet. I had books, as well as some digital-format documentation stored locally, but searching through books and documentation was a slow process, and no matter how much books I had, there were many issues that weren't mentioned there.

More importantly, nearly every problem you encounter was already encountered by somebody else before. Internet helps finding people who had this error and successfully solved it. This is not a sort of information you find in books or official documentation like MSDN. Instead, such information is usually found:

  • On Stack Overflow, obviously. For example, I haven't seen any book which mentioned this problem.

  • In blogs. I found a lot of help on blogs when I had particular problems, would it be with WCF configuration or a proxy server which doesn't start or a cryptic bug when using a specific API on a machine with culture different from en-US.

  • In bug reports concerning open source software. For example, it was very helpful recently when I attempted to use Ubuntu's MAAS and when I used PXE. You don't find this sort of information in books either.

The importance of the immediate availability of an answer to most issues one can encounter is especially noticeable if we take in account that most of the time a team spends on a project is spent maintaining it.

  • Internet doesn't help much during architecture and design phases (Programmers.SE might help, but it would be much more valuable for an architect to read books about architecture and design, to learn design patterns, etc.).

  • It starts to be helpful during testing and implementation steps, when actual issues arise.

  • But most issues appearing during the maintenance, it's there when help from others through internet appears crucial. Basic example: a WCF service works perfectly well on my machine, but fails with no exception whatsoever when deployed in production, while it worked for the last two weeks. This happened to me, and I'm thankful to blog writers and Stack Overflow community for helping me solving the issue in a matter of hours rather than weeks.

Would it justify a x10 increase in productivity? Difficult to tell.

  • On one hand, there are cases where you find an answer immediately, while you could have spent days searching for it alone (or be forced to rewrite a large part of the application).

  • On the other hand, the overall productivity improved based on multiple factors, such as better project management (Agile, TDD etc. comes to mind), better team management (Radical Management by Stephen Denning comes to mind), better tools (debuggers, profilers, IDEs, etc.), better hardware (no, I won't be very productive if forced to write in Assembler for a slow CPU and very limited RAM), etc.

share|improve this answer
4  
"Internet" is not a single development either. –  Ben Voigt Aug 12 at 18:35
1  
@BenVoigt: I would qualify it as a technology from the Brooks' quote. But I agree, the terms may not be obvious: first, would it be Internet (early 1980s) or World Wide Web (1989)? Second, it's not just the technology itself which was essential, but its popularity. –  MainMa Aug 12 at 19:03
    
But the things we pile under the concept of "Internet" involve many different technologies. There's multiple generations of World Wide Web (DHTML/Javascript/browser). There's the fibre optic advances that make datacenter-scale connections possible. There are the CMOS process improvements that allow servers to have a terabyte of RAM and perform data mining. Algorithms to index a million questions about programming and provide the 10 closest hits in some natural language sense. –  Ben Voigt Aug 12 at 19:38
5  
People working with the systems Brooks designed didn't need Google to remember how to write JCL. These systems came with well-documented development environments that were leveraged continually / improved incrementally over the decades. Whether due to planned obsolescence or something less sinister, we’ve gotten away from that. In any case, I'm hesitant to call what we have now an improvement, and completely unwilling to declare it a "silver bullet". –  user1172763 Aug 12 at 20:19
    
Complexity is the enemy. You could hold JCL in your head and hold the manual in your hand; a single shelf could document the whole system. Now there has been a huge explosion in the size of systems and their underlying rate of change. –  pjc50 Aug 13 at 9:53

I'd say the internet is a pretty good candidate. StackOverflow and Google are a modern-day developer's most powerful tools. Instant knowledge-sharing on a global basis! These days you don't need to know the answers, you only need to know how to get to know the answers - and that is a perfectly adequate subsitute that allows developers to spend less time reading and more time coding, and thus being productive. It means that every programmer in the world has access to all of the answers, and all they need to do is know how to ask the right questions.

share|improve this answer
    
You are the only person to point out the silver bullet. Not only has it made programmers more productive by a magnitude, but actually by multiple magnitudes of order. –  Dunk Aug 15 at 14:09
    
+1000 - you can get help in a few minutes instead working for a few days on an obscure issue. –  james Aug 15 at 15:10

I would suggest that trends pulling us in the other direction (i.e. toward lower productivity) are at least as strong as the trends you asked about. The experience of doing client/server development tool like VB6 or PowerBuilder came pretty close to the "Rapid Application Development" (RAD) ideal. You got to draw your forms, and there were obvious hooks for your procedural or SQL code.

Web development, at least initially, destroyed a lot of the techniques and infrastructure that made these things possible, and many client/server developers just stopped being developers, or clung desperately to, say, VB6.

The transition to heavily client-driven Web development was yet another slash-and-burn experience; Microsoft was getting back to the RAD ideal with WebForms, and then it quickly fell out of favor. Instead developers were expected to abuse the infrastructure (e.g. XMLHttpRequest, which is seldom used for XML) and otherwise monkey around at a low level of abstraction in an effort to make their websites more interactive

There are good reasons behind all of these transitions, and it's not fair to compare a process that yielded a native .EXE (requiring installtion and management at the individual client) to Web development, nor is it completely fair to compare a process that relies heavily on postbacks with one that yields a more seamless experience. But I will say that the practices currently in vogue result in some surprisingly low-level thought processes among people who missed out on client/server, RAD, and the like. Client/server buttons just worked, and one certainly did not have to worry about the underlying data channels that got data to the event handlers that made this happen.

Conversely, more contemporary developers tend to think that those of us who did client/server development (or even postback-based Web development) have a tendency to resort to shortcuts (and mean that in a bad way). That's understandable, but I still think that the way contemporary development is done is surprisingly low level, and the days of searching for a "silver bullet" seem to have given way to the era of mocking those of us who question the wisdom of mining and smelting our own lead.

share|improve this answer
    
have you seen Meteor.js? –  strugee Aug 13 at 2:07
2  
@strugee Yes, and I think Meteor.js may be a step in the right direction. Still, though, the fact that we've essentially "started over" technologically at least 3-4 times since Brooks wrote his book (referring here to the transition to the PC, then to client/server, then to Web, and then to AJAX) is arguably what's kept us from finding the "silver bullet" or even making linear improvements in productivity. Time will tell how long the current development techniques endure (and improve). There are some reasons for optimism... everyone is now reaching for a robust, cross-platform point. –  user1172763 Aug 13 at 15:22

Technology has advanced very much and now we have all the things Robert Harvey enumerates in his answer, but:

  • The problem seems to be changing requirements. The client knows that there will not be waste of materials when changing the requirements of a software project, so they do. That kind of requirement changes almost never happen once a physical-world project like a building, is half done.

  • Another important aspect is that programming continues to be highly handiwork. Rarely if ever, RAD-generated code goes into production without being modified by hand first.

  • Code continues to be highly complex, and that complexity doesn't seem to decrease with new technologies.

  • The rate of deadlines not met or budgets exceeded continues to be greater that those in other disciplines, and often times technical debt is incurred in order to meet them, generating hidden costs.

  • Something that have, without a doubt, happened is that compartmentalization has ocurred. The sheer ammount of technologies one has to know is so that programmers have had to specialized a small number of technologies in order to become really good at them, requiring different kinds of experts to complete a large project.

  • One thing that talks about software complexity is that whereas there are literally hundreds of car makers in the world, the list of companies capable of creating and maintaning an operating system, (desktop, mobile, embedded or otherwise), can be counted with the fingers of your hands.

  • All the above has created a situation in which there are not enough people studying to be programmers, so that governments have created campaigns in order to motivate more students into taking that career path.

  • One taste of the maturity of the software industry is that software licenses continue to state "<companyX> makes no representations about the suitability of this software for any particular purpose. It is provided "as is" without express or implied warranty." Imagine a hardware maker stating that their product is not suitable for any particular purpose. That's the state of the art right now.

share|improve this answer
2  
There are certainly more than 10 companies capable of creating and maintaining their own OS, but few choose to do so, because other opportunities are more lucrative. –  Ben Voigt Aug 12 at 18:36
    
@BenVoigt Of course, creating and maintaining an OS is comparatively less lucrative out of sheer complexity, requiring decades of research and development, which they should have begun, at latest, in the nineties in order to have a mature product today. –  user61852 Aug 12 at 19:31
1  
Also the "return" side of the RoI isn't that good, because the market is already saturated. –  Ben Voigt Aug 12 at 19:48
2  
Of course, all you've done is enumerate the well-known roadblocks to effective programming that have been around since shortly after Lady Lovelace picked up her pencil. The only thing that's different vs 30 years ago is that things have become exponentially more complex. –  Daniel R Hicks Aug 13 at 17:21

A great deal of what we have learned in software engineering practice in the past 30+ years is of the form "technology X can speed up initial development of new software, but if you don't spend as much or more time thinking about how and when to use it as you saved by using it, in the long run it will turn your application into a sucking swamp of technical debt, costing you orders of magnitude more time and effort in maintenance."

Technologies that fall under this razor include: hand-coded assembly language, compilers, interpreters, procedure libraries, imperative programming, functional programming, object-oriented programming, manual memory allocation, garbage collection, static types, dynamic types, lexical scope, dynamic scope, "safe" pointers, "unsafe" pointers, the absence of pointers as a language concept, binary file formats, structured-markup file formats, macros, templates, avoidance of macros and templates, shared memory, message passing, threads, coroutines, asynchronous event loops, centralized remote services, distributed services, locally installed software, arrays, linked lists, hash tables, and trees.

The fact that many of the items in the above list come in groups which together exhaust the known solution space is very much intentional, and should in and of itself tell you something. One could argue that the only unambiguous, across-the-board improvements in praxis we've had since they first switched on the Z3 are block-structured programming (as opposed to spaghetti code) and memory protection (boy, do I ever not miss the days when a typo could take down my entire computer).

share|improve this answer

While one might argue with specific metrics (ie, have things improved by a factor of 9.98?), I (speaking as something of an old-timer) have to agree with the general sentiment of Brooks' comment.

First off, there has been very little truly new technology invented since maybe 1970. Yes, integrated circuits have gotten longer, lower, wider, and glass fiber has improved communications speeds, but for every step forward there's one back.

Compiler technology has permitted about a 10x improvement in programmer "productivity" vs 1970, when one figures function produced divided by actual coding time, but the proliferation of new or "revised" programming languages and environments means that the average programmer is spending more and more time in "catch up" mode, and less in productive activity. Apple, Google, and Microsoft all spew out new and substantially incompatible "upgrades" to their environments at a rate that is just below that that would provoke revolt among their serfs ... er, programming customers. Similarly, HTML/CSS/Javascript/whatever keeps getting more complex.

At one time the rate at which documentation could be produced and propagated would have limited and corralled all this "innovation", but, thanks to the Internet, rigorous documentation is no longer really necessary -- just spew the functions out and rely on bloggers to ferret out the details and make them available.

Added: I've been thinking about this since yesterday, and in particular thinking about the project I worked on from about 1978 until 2008. This project (the IBM System/38 and its successors) was somewhat unique in that from the start efforts were made to control the complexity of it (one being the division of the software into two roughly equal parts, with a "machine" interface between them). In the particular area where I worked, several of my coworkers similarly were dedicated to controlling complexity (though we didn't use that term much at the time). The result was a product that (at first) was quite robust and a "hit" with the customers pretty much from the git-go. And it was a pleasure to work on -- like playing in a well-trained orchestra.

Of course, over the years complexity crept in, usually at the behest of market planners and managers who had no appreciation for controlling complexity (which is somehow different from just maintaining simplicity). I don't have the feeling that this was inevitable, but it was impossible to prevent in this case without a manager (like Glenn Henry did originally) pushing back on the forces of confusion.

share|improve this answer
1  
But I am reminded of something that occurred to me (and no doubt many others) 20-30 years ago -- there is a sort of Peter Principle of programming (or perhaps a 2nd law of programming thermodynamics) that states that complexity inevitably increases to the point of incomprehensibility. Things never get simpler. –  Daniel R Hicks Aug 13 at 18:53

Not the answer you're looking for? Browse other questions tagged or ask your own question.