Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

There is a lot of information about tools and techniques for improving legacy codebases, but I haven't come across any successful real world case studies. Most advice is on the micro level, and while helpful, doesn't convince many people because of a lack of evidence it can help at the macro level.

I am looking specifically for incremental improvements that have been proven to be a success in the real world when updating a large legacy codebase to meet today's quality standards, and not a complete rewrite.

Before:

  • Large: greater than 1MLOC
  • Legacy: no automated tests
  • Poor quality: high complexity, high coupling, high escaped defects

After

  • Automated tests
  • Easier updates/maintenance
  • High quality: lowered complexity, decoupled code, few escaped defects

What kind of incremental steps have been proven in the real world to update a large legacy codebase successfully to meet above quality standards, without going through a total rewrite?

If possible, include an example company or case study of a large legacy project that has gone through a "successful" quality improvement process in your answer to back it up.

share|improve this question
1  
2  
Netscape –  Karthik T Jan 29 '13 at 8:57
7  
The entire financial industry? Much of it runs on 40 year old FORTRAN code. Unlike Netscape they can't chuck it out and re-write it from scratch, so it's been gradually improving this whole time. –  MattDavey Jan 29 '13 at 9:43
2  
in my POV, Netscape can hardly used as a successful example - the project ended the company..... which at the time was a commercial for profit organization. Can't imagine the shareholders crack open the top shelf bubbly that day...... in fact there is a well know white paper along the lines of "What not to do" using Netscape as the perfect case study.... –  mattnz Jan 30 '13 at 0:56
2  
Hi @mikelong I've edited your question to try and get it reopened. Your original question asking for a list of examples, which is considered "not constructive" by StackExchange standards. Feel free to edit it further to add more details about what you mean by "high quality", or to update the wording if I've made a mistake. :) –  Rachel Jan 31 '13 at 13:35

4 Answers 4

Books like http://www.amazon.com/Working-Effectively-Legacy-Michael-Feathers/dp/0131177052 should be witness enough to how large, legacy poor quality code bases are common in the industry.

My guess at why you have not heard or seen, and, more importantly, you will never likely hear about them until you work on one of them yourself, is, nobody seems capable for various reasons, to come out clean and say that their code base was all the above without facing non-trivial repercussions.

This could explain the dearth of studies you speak of. If you read enough books, for example, Peter van der Linden's Deep C Secrets, you will read about million dollar bugs where the part about which project had them will be missing.

NOTE: I wanted to make this a comment, but it was too long. I understand this does not answer the question fully.

EDIT: C++11 & The long term viability of GCC is questioned -- if the developers refactor GCC and make it more toolable as LLVM/clang, it might provide a good example. The discussion notes that the documentation is poor at some places pushing the entry barrier for new developers higher.

share|improve this answer
    
The article on gcc is very interesting, thanks! –  mikelong Jan 31 '13 at 6:25

On February 3, 2013, Michael Meeks, one of the LibreOffice developers, is giving a talk in a couple of days entitled, "LibreOffice: cleaning and re-factoring a giant code-base, or why re-writing it would be even worse." It sounds like exactly what you're asking for: a discussion of what they've done to take " a poorly-understood, gigantic code-base extensively commented in German, with no unit tests, a tangled build infrastructure, and twenty-five years of un-paid technical debt" and modernize it.

The presentation can be streamed online, and (I think) recordings will be available at some future date.

share|improve this answer
1  
I realize it's scheduled for a few days from now, however once it's broadcasted would you be able to add a summary of the process they took to modernize their code base to your answer, in case those links ever go dead? –  Rachel Jan 31 '13 at 17:10
    
@Rachel - If I'm able to catch the broadcast, I'll definitely do so. Thanks. –  Josh Kelley Jan 31 '13 at 17:41
    
Wow, this is great - thanks a lot! –  mikelong Feb 1 '13 at 0:38

I've actually been through a fairly significant refactoring three times in my career. Code has a tendency to decay, so if your code base is around long enough, a large refactor is pretty much unavoidable. All my examples were on private code bases, which might explain why public examples are hard to find.

The first time was an application which, believe it or not, had a fundamental architecture that made it only work with dot matrix printers. When my company could no longer find a vendor to supply the ribbons, they assigned me to make it work with a laser printer.

The second time was a migration of several hundred automated test scripts from C to Java, partly because we needed better cross platform capability, and partly because it was getting difficult to hire new C developers.

The third time I am still in the middle of, which is modularizing a huge monolithic application to allow unit testing by reducing coupling, and for cross platform purposes.

I compare the effort to climbing a mountain. You have this huge goal ahead of you, but you don't tackle it at the macro level. You take it one handhold at a time, always having a close fallback position, never disconnecting the previous safety until the next one is in place. You start out just making small incremental improvements, and after a while you turn around and there's suddenly this beautiful view.

Let's say you have 60,000 files of highly coupled code, for example. You want to start putting it under unit test, but the dependencies make it impossible. How do you fix it? You decouple one file. You add automated tests. You get back to stable ground before moving on. Repeat 59,999 times.

If that sounds simple, that's because it is simple. It's not easy, but it's simple. It's hard to notice any progress at first. We are two years into what seemed an impossible refactor, and likely have years ahead of us until we're finished, but looking back we suddenly realize how much better the code has gotten already, and we have been able to continue to deliver new functionality to our customers in the mean time.

The other two times worked the same way. You find the smallest safe step you can take, and you take it, always keeping the application in a working state. You only worry about the big picture to make sure you're heading in the right direction. All your actions are small, steady, and incremental.

share|improve this answer

From personal experience working on a multi-million line code base I have found a few strategies that seem to work.

Look at all of the bugs (even closed ones) and try to break them down into categories. Specifically to try break them down by the component that they belong to. If they belong to more than one component note that they do. Once you have done this look at which bucket is the biggest and use that to determine where to start. Additionally you can look at the revision history of the files to determine what changes the most and use that as a guide of where to start. Basically what you are trying to do is to find what is most broken fix that and repeat. Additionally I have found that trying to fix everything at the same time never works it just causes more problems.

If you find that there a lot of things that belong to multiple components that is an indication of "system" issues and may point to code that is too tightly coupled or an API that needs refreshing.

Another area where I have spent a lot of time is testing the existing code base. There are multiple strategies here and all have merit but no one is a complete solution to the problem.

  • Unit testing can work but often times you are limited to what can be unit tested due to tightly coupled code. However do it where you can.
  • External testing is another avenue. I assume that you probably already have this and if not I would spend some time creating it. Additionally something that has worked for me is to add the ability to randomly inject faults/events into the system. In addition to that try to inject multiple things at the same time to try to make it fail in new ways.
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.