Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I work with a code base that is over 500K lines of code. It is in serious need of refactoring. There have been refactoring efforts identified that will take longer than the normal two week sprint. These can't be broken up into smaller tasks as I have seen suggested in other answers on this site. The product needs to work at the end of the iteration, and partial refactoring will leave the system in an unusable state as the dependency between items is horrible. So what would be the best way to approach this hurdle? Again I mention, breaking it down into smaller pieces is is not an option, that has already been done.

Update: People seem to need an explanation of why this can't be fit into a 2 week sprint. There is more involved in a sprint than just writing code. We have a policy of no code without tests. That policy did not always exist and a large portion of the codebase does not have them. Also some of our integration tests are still manual tests. The issue is not that the refactoring itself is so large. It is with the fact that small changes have an effect on many parts of the system, and we need to ensure that those parts still operate correctly.

We can't put off or extend a sprint because we have monthly hotfixes. So this change extending past a sprint cannot stop the other work being added to the hotfix.

Refactoring vs Redesign: Just because our development process is not efficent enough to handle this refactoring in a two week cycle does not warrant renaming it to a redesign. I would like to believe that in the future we could accomplish the exact same task within a two week cycle as our process improves. The code in question here has not had to change in a very long time, and is quite stable. Now, as the direction of the company is becoming more adaptable to change, we want this portion of the code base to be as adaptable as the rest. Which requires refactoring it. Based on the answers here, it is becoming apparent that there is missing scaffolding necessary to make this refactoring work in the time frame of normal sprints.

Answer:

I am going to do the branch and merge approach that Corbin March suggested the first time so we can learn more about these problem areas and how to identify the missing tests. I think moving forward, we should take the approach that Buhb suggested of identifying the areas that are missing tests and implement those first, then do the refactoring. It will allow us to keep to our normal two week sprint cycle, just like many here have been saying should always be the case for refactoring.

share|improve this question
16  
I recommend fire, and lots of it! :P –  FrustratedWithFormsDesigner Aug 24 '11 at 16:10
21  
As a side note, this is by definition large-scale redesign, not refactoring. Hope you don't take this as nitpicking - IMHO it is important to use clear terminology to avoid communication problems :-) –  Péter Török Aug 24 '11 at 16:32
9  
@Charles, "Refactoring is typically done in small steps. After each small step, you're left with a working system that's functionally unchanged." Quoted from the page I linked above. –  Péter Török Aug 24 '11 at 17:20
7  
If you are doing such large steps that you can't have a working piece of code on a regular basis, that is the very definition of when refactoring becomes redesign. Please don't get angry at people for calling you on your misuse of the word. There's nothing wrong with asking about redesign. The commenters are merely trying to indicate that you won't get answers you want because you are misusing the word, which is what has already happened. –  jprete Aug 24 '11 at 19:27
5  
I know it's a bit off topic, but you got me dead curious about what the circumstances are that makes refactoring in small steps impossible. Please share some more background regarding this. –  Buhb Aug 24 '11 at 20:30
show 7 more comments

16 Answers 16

up vote 12 down vote accepted

If you have the luxury of delaying the refactoring, I suggest focusing coming iterations on adding unit test and automated integration tests to the point where you can comfortably refactor the codebase, and then refactor in a single sprint.

share|improve this answer
add comment

My suggestion:

  1. Create a branch
  2. Merge daily from trunk to your branch and resolve conflicts.
  3. Work until it's done. Your branch may be outside core development for several sprints.
  4. Merge back to trunk.

There's no getting around the fact that it will probably get ugly. I don't envy you. In my experience, when you drastically change a project, it's easier to merge ongoing development into the new paradigm versus somehow merging the new paradigm into a now-changed trunk after everything is finished. Still, it's going to hurt.

share|improve this answer
2  
+1: very nice approach to use version control ... –  IAbstract Aug 24 '11 at 16:48
1  
We have discussed using this approach, and if no other ideas come out before we tackle it, this is what we plan to do. –  Charles Lambert Aug 24 '11 at 17:04
3  
I am afraid you will have a lot of overhead merging back and forth between the trunk and the branch if you want to perform such a thorough refactoring. –  Giorgio Aug 24 '11 at 20:42
1  
Considering the volatility your presentation seems to suggest, perhaps you should consider Git, as it facilitates this sort of speculative branching and merging. –  John Tobler Aug 25 '11 at 2:00
1  
@Giorgio: so do I, daily merge seems a bit paranoid, but then it really depends on the team size. The more people, the more changes, and the more frequently you should merge. Daily seems the bottom limit though, if you want to have time to work. –  Matthieu M. Aug 25 '11 at 11:41
show 5 more comments

Not every task can be done in an (artificial) 2-week sprint, so common sense is called for. If it can't be broken down any more, and it must be done, just get on and do it. The process is not more important than the end result, and should be seen as a guideline rather than a never-to-be-broken law.

share|improve this answer
10  
+1 for common sense! –  FrustratedWithFormsDesigner Aug 24 '11 at 20:07
3  
Your answer is very good information on the topic. I'm here to "get on and do it" as you say. I asked the question in hopes of getting some information so that I could come up with either a process to run parallel to my existing one, or somehow modify the current process to resolve my current situation. I need something to tell the developers so that they have guidelines within which to work, as this will be happening at least 2 more times. –  Charles Lambert Aug 24 '11 at 21:39
show 1 more comment

Just do a 3, 4 or 5 week sprint. Who cares? You're obviously convinced that nothing can be done in a shorter time-frame, so stop fighting it.

Just don't tell your fellows at The Royal Society of Blind Agile Adherance.

share|improve this answer
show 1 more comment

I recommend starting with the book Working Effectively with Legacy Code, by Michael Feathers. He covers a variety of techniques to reduce the scope of refactoring-style changes.

share|improve this answer
4  
I'm going to give this a +1 because it is probably the best book to start out with when you begin learning to, as the title says, work effectively with legacy code. It is relevant to someone who might ask this question and not understand what is involved with breaking things into smaller steps. –  Charles Lambert Aug 24 '11 at 21:45
show 3 more comments

In our shop, when we have large refactoring or rewriting tasks that can't be completed in a release window, we leave the functionality as-is in the system. But, we start on the new, refactored lego-block functions as time permits. Eventually, we get to a state where we have enough lego-blocks done, that a sprint provides sufficient time to plug the lego blocks into the application and turn it on for the users.

More concisely, we break up or rework large functions using new names. Then, at the end, we use our renamed, refactored work instead of the nasty old code.

share|improve this answer
show 1 more comment

Dedicate one sprint to figuring out how to keep the code functioning properly mid-refactor. This can take the form of deprecated methods and classes, wrappers, adapters, and the like. Your refactoring may make the code dirtier for a short time in order to be cleaner long-term; that's OK. Right now you're saying it can't be done. I don't think that's right - think about what your process would be like if it could be done, think about what steps you can take to make it so. Then dive in.

share|improve this answer
1  
I said nothing of the sort. I was saying we went through a process similar to what you have described and came out on the other end with some code that cannot be refactored in a single sprint. –  Charles Lambert Aug 24 '11 at 18:29
show 3 more comments

+1 on Corbin March's answer, thats exactly what I was thinking. It sounds like your code-base is a bit ugly and it's going to take more than a single sprint cycle to clean it up.
So just like Corbin said,

  1. branch it into a "refactoring project"
  2. test your branch change
  3. promote to your test environment for QA testing
  4. incrementally merge it back into the trunk until the refactoring is done.

I'm sure you wouldn't have any problems selling this to your dev manager, if your PM's are having a hard time seeing it then explain to them that Rome wasn't built in a day and cleaning up all the garbage thrown into Rome's streets isn't going to be done in a day either. The refactoring will take a while but it'll be well worth it in the end in terms of easier maintenance, quicker enhancement releases, less production tickets, and a more fulfilled SLA.

share|improve this answer
1  
I have all the ammo I need to talk everyone into it. I just need an official plan for executing it, when I present it. From the answers here, I have no doubt that I will come up with a repeatable solution. –  Charles Lambert Aug 25 '11 at 16:29
add comment

Although the redesign you really want to do is a large task, would it be possible to refactor smaller pieces to break/decouple individual dependencies? You know--when in doubt, add indirection. Each of these decouplings should be a smaller task than the mammoth one you cannot complete.

Once the dependencies are removed, you should be able to break up the remaining refactoring tasks to be obtainable within sprints.

share|improve this answer
add comment

In the project I am working for at the moment, we are using 4-week sprints. And sometimes we cannot finish a user story and we just restart it during the following sprint.

However, IMHO it should be possible to break up a refactoring into smaller stories that fit in a 4-week sprint. If you cannot bring the code into a consistent state within 4 weeks, I have the feeling that you are rewriting your application rather than refactoring it.

share|improve this answer
show 4 more comments

Start a maintenance window during which no further development is performed. Perform the redesign, then resume development sprints.

share|improve this answer
add comment

I recommend that when certain tasks will take longer than the 2 week sprint cycle, the task be slated for another time. Your team has identified the need for refactoring and that is important. Sometimes, there is no other option ...and, yes, that does suck.

When the time comes to begin the refactoring, you will simply suspend the normal sprints. You don't have a choice. Use smart version control - branch, refactor, test, merge. There is always a point in time where refactoring some large projects take priority over features. If possible, I would also attempt to separate the concerns for better flexibility.

share|improve this answer
show 2 more comments

Having recently gone through the same issue with a part of our codebase (which also is a bit larger), I hope I can share a few insights with you. In my situation, the codebase had been developed by another team, so noone from the original developers was involved in this refactoring. My experience with the codebase was about 1 year, and another developer tasked with it 2 years.

Allow me two small notes regarding the other answers here:

  • Branching will help you establish a playground for yourself, but do never merge your changes to trunk in one big changeset. You will get into serious trouble with the rest of the team.
  • You need to do it incrementally, and it can be done. No excuses. Read Working Efficiently with Legacy Code cover to cover. Read it again.
  • Thinking you can do it one big step is a fallacy. Although it seems more work, doing it incrementally is much easier to manage.

Going seemingly "off-reservation" for two weeks or more won't go unnoticed. You need to ensure you're backed by project management, and even more importantly the team. If the team isn't committed to this refactoring (and this means, do it now, not in the distant future), you will get into trouble.

Don't do it alone, use pair programming. This doesn't strictly mean you need to sit in front of the same keyboard all the time, but can handle small and narrow tasks (e.g. write tests that capture the behavior of this class) individually.

Do a Scratch Refactoring and treat it as such. (A sort of "throw-away" prototype refactoring, the "throw-away" bit is important!) Honestly, it is unlikely you know all the implications your refactoring will have. A scratch refactoring will help you in certain regards:

  • You will discover pieces of the system you never knew they existed.
  • You will have a much better view on how modules interconnect
  • You will discover a new way of grouping your system into modules, likely much different from your current understanding. Discuss insights with your partner. Try to distill your new view. Write a short architecture whitepaper (or draw a diagram) that says where things are currently and where they should logically belong.

When you have done your scratch refactoring, I hope you will discover that you can't just go and change everything all over. You will feel bad, it's just a big fat mess and you can't just flip a switch and make it work. This is what happened to me, it might be different in your situation.

However, my partner and I did have a much better understanding of our system. Now we were able to identify individual, smaller (albeit still large) refactorings/redesigns. We captured our vision of the system in a diagram and shared it with the team, together with the backlog items we came up with to implement that vision. Empowered by common consensus, we decided which items we would implement over the course of the next iteration.

One last thing that helped us was using a big whiteboard. There's too many things to keep in your head. It is utterly important you keep notes. Write yourself a de-briefing note at the end of the day, capturing what you have done today and want to do tomorrow. It does help relaxing big times, and you need a relaxed time if you want to keep up with the task. Good luck!

share|improve this answer
add comment

We have two types of works at our hand:

  1. Man-hour works
  2. Genius works

Refactoring usually consists of the first type of work, since a lot of methods are known already to developers like DRY, SRP, OCP, DI, etc. Thus when a project takes two months to be refactored, it simply takes two months, there is no way around it. Thus, my suggestion would be to not refactor the original project, and let it work on its current situation. Stop receiving new requests and requirements from stakeholders and product owner. Then let the team work on the project till it gets refactored and ready to go.

share|improve this answer
add comment

One suggestion that may help: If you have untested code such that you don't have sufficient time to refactor and retest it within the two week sprint, consider first making some other unrelated small change(s) to the code so that you can focus on writing tests for the first sprint or two. Perhaps you can identify several untested clients of the code you want to refactor; pick one client and make some other changes of some use to the business that will force you to write tests for that client. Once you're more familiar with the code, from working with it, and you have more tests, and you've possibly accomplished a few minor contributing refactorings, you will be in a much better position to accomplish the refactoring and the (now easier) testing both in one iteration.

Another approach is to make a copy of the offending code, refactor it, and then move clients one at a time to the new code. This work can be broken up across iterations.

And don't give up: Don't just accept that a large refactoring cannot be broken down into smaller steps. The easiest/fastest/best approach might take longer than an iteration. But that doesn't mean that there is no way to do iteration-sized chunks.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.