Assume you have a project that (in your opinion) is mostly decently well written, modularized, etc. and you want to keep much of it's functionality. However, an essential portion of this project is not well designed, and it can't easily be fixed with minor changes. The only way to fix it is to rip out a lot of the code; it's not so bad that you need (or should) start from scratch, but it's very pervasive.
Assume that the code is something like 20,000 lines of non-whitespace, non-single curly brace, non-comment Java, C++, C#, PHP or Ruby code. You estimate about 2000-3000 lines need to be thrown out or changed. In the case of something like Java or C#, one of your packages/namespaces will probably be entirely rewritten from scratch, and many of the references to those classes will change.
Here's the challenging part: the code that needed to change was the GLUE within the system. Here's a very simple picture that summarizes the problem:
This is the situation I was in that led to this question: What should I do when I've already waited too long between commits?
As you can see, it's not only the parts of the code that were doing the internal processing, but also the parts that were communicating with the other parts of the system. Both the APIs and the internal data structures were insufficient. Attempting to keep both sets of APIs working for incremental changes would have been woefully redundant.
I, of course, branched before I did anything, but every change I made produced a product that simply did not work. In many cases I created compile errors. This led to me continually putting off commits, because I thought "yes, I wired these pieces back together, but I still need to implement this other new interface, write this essential functionality, etc." This resulted in the cardinal sin of not checking anything in, a mistake I'd prefer not to repeat should I be in this situation again. This was mitigated by the fact that all my APIs were internal, except that I had to conform to the APIs of the outside world and third party modules, but my wrappers to those sections were changing.
One obvious alternate solution would be to just accept the fact that each change will result in a broken overall product, but since you're checking into a branch, it's okay. Especially since I am the only developer working on this code and using these APIs. So maybe that's the right thing to do. Or maybe the right thing to do is try to first refactor the API to the new one, without actually changing the functionality (if possible), even if some arguments are dummy or will eventually be removed, and then incremental changes are a lot more self contained. Or maybe there's a better way entirely?