Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I am wondering if there are any studies that examine the efficacy of software projects in CMMI-oriented organizations. For example, are CMMI organizations more likely to finish projects on time and/or on budget than non-CMMI organizations?

Edit for clarification:

CMMI stands for "Capability Maturity Model Integration". It's developed by the Software Engineering Institute at Carnegie-Mellon University (SEI-CMU).

It's not a certification, but there are various companies that will "appraise" your organization to various levels of CMMI, such as level 2 and level 3. (I believe CMMI level 1 is an animalistic, Hobbesian free-for-all that nobody aspires to. In other words, everybody is at least CMMI level 1, even if you've never heard of CMMI before.)

I'm definitely not an expert, but I believe that an organization can be appraised for CMMI levels within different scopes of work: i.e. service delivery, software development, foobaring, etc. My question is focused on the software development appraisal: is an organization that has been appraised to CMMI Level X for software projects more likely to finish a software project on time and on budget than another organization that has not been appraised to CMMI Level X?

However, in the absence of hard data about software-oriented CMMI, I'd be interested in the effect that CMMI appraisals have on other activities as well.

I originally asked the question because I've seen various studies conducted on software (e.g. the essays in The Mythical Man Month refer to numerous empirical studies, as does McConnell's Code Complete), so I know that there are organizations performing empirical studies of software development.

share|improve this question
1  
Capability Maturity Model Integration? Anecdotally, looking at cmmifaq.info/#1, it seems to me that your question is catastrophically oversimplified. CMMI is about process improvement, which is always going to cost you more time and money in the short run. Whether you get a payback on that investment, and when, is going to depend greatly on the organization's current state of affairs and many confounding factors. –  Robert Harvey Jun 12 '12 at 15:51
1  
Since CMM (CMMi's moral predecessor) has been around since the early/mid 90s, you'd think someone would have "circled back" to check if becoming CMMi* Level X certified actually makes a difference. After all, that circling back is part of the process improvement that CMM and CMMi are supposed to promote. –  Bruce Ediger Jun 12 '12 at 16:02
1  
@BruceEdiger - I suspect three factors preventing this from happening very often. The first is that many companies take on the 'fad of the year' with respect to improvements. They do CMM for a while, then Kaizan, etc, etc. They never really understand any of it, and they never ask the hard questions because they seek a magic bullet. The second factor is that if you DO have something that works, the company may decide that it shouldn't talk for fear of giving the competition a leg up. The third is that they are so busy getting on with their work that they never bother to mention it. –  Michael Kohne Jun 12 '12 at 16:13
    
CMMI isn't a development process. It is a way for describing a development process. I've worked at multiple CMMI certified locations. They each have radically different development processes, but each have been certified. –  Sign Jun 12 '12 at 16:51
    
@Micheal Kohne: fair enough, but CMM and CMMi are promulgated by software engineering researchers. It would seem that someone whose job it is to do research would actually try to validate (or get independent validation) of the conclusions of their experiment(s). –  Bruce Ediger Jun 13 '12 at 18:33

1 Answer 1

The SEI has published some material regarding the impacts of CMMI on various aspects of software development. There are a few samples of benefits that companies have seen from adopting either CMMI or the SW-CMM. In addition, some companies have provided their success stories to the SEI. Of particular interest might be the technical report Demonstrating the Impact and Benefits of CMMI: An Update and Preliminary Results. Although these are all curated by the SEI, I tend to trust this data since the SEI is a federally-funded research and development center with significant ties to the academic and government communities rather than a commercial/for-profit entity.

You might also be interested in the January/February 2012 issue of CrossTalk, which is a journal for defense industry software engineering. This issue focuses specifically on the benefits of achieving high CMMI maturity (typically defined as Level 4 and Level 5) and reveal data from a couple of defense contractors on the impact to project cost, schedule, and quality (all positive) as they moved into the high maturity levels.

share|improve this answer
4  
SEI was also one of the major players in creating the CMMI and has quite a bit to gain in seeing it prosper. I don't think the the amount of pro-CMMI material on their website is an accident. –  Wyatt Barnett Jun 12 '12 at 17:57
2  
The same way all those "studies" touting the greatness of Windows were paid for by Microsoft so of course they are going to find that Windows is the best.., –  Wayne M Jun 13 '12 at 10:45
    
I think we got a bit off topic here. Bias aside, I don't see much in the way of empirical data in any of those links -- they are all case-study oriented. In particular, none of them have a control group that didn't implement CMMI. –  mehaase Jun 13 '12 at 19:11
    
@mehaase They are empirical. When it comes to process improvement, it's more valuable to compare your own organization before the change to your own organization after the change due to consistency across product, technology, knowledge and skills... –  Thomas Owens Jun 13 '12 at 20:24
    
Fair enough. They are, literally speaking, "empirical", in the sense that they are based on observation and not deduction, but there are still no control groups. The problem with the before-and-after approach is that there may be correlations between organizational effectiveness and CMMI level, but you don't know which way the causality runs. Maybe organizations that are truly interested in improving their ability are more likely to adopt CMMI, but such a self-motivated organization may have been capable of improving itself even without CMMI. –  mehaase Jun 14 '12 at 14:16

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.