When would one give up backward compatibility in favor of new and improved interface implementation?

Question

I have been working at the same software company for more than ten years. As a result, I have implemented a large code base using various object-oriented programming languages. I was a beginner programmer when I first started my career and I didn't know much about good interface and class design principles. I would like to think that my design skills have improved over time, but I now face more and more difficulties in improving my earlier code because of backward compatibility concerns. My code is used by a large number of customers as part of the products that my company sells.

My question is: when should one stop trying to keep backward compatibility of old interfaces and bite the bullet in favor of implementing a brand new design?

I think there comes a point where keeping backward compatibility becomes such a big burden that useful changes to interfaces become impossible. Has anyone experienced similar concerns, who can provide some feedback?

Answer 1

While 'when' is a good question, I think 'how' is even more relevant. It can be difficult to make a breaking transition in a way that doesn't make users frustrated or unhappy. Some items for consideration:

• Communicate with your clients/customers well ahead of any transition. Explain why and how it will be implemented. Citing security, performance, stability, and future flexibility as reasons are all valid.
• If you're making a clean break anyway, solicit feedback.
• If there are any 3rd party developers, make sure that you include their input as well in so much as it makes sense.
• If it's possible, provide a compatibility layer.
• Provide a comprehensive upgrade guide.
• Make the upgrade as easy and painless as possible. Minimize your users' pain, even if it means a little more work for you.
• If you charge, offer a discount for upgrades to the new version.
• Add at least a few new and useful features in a new version to incentivize the upgrade. This is especially important in making the upgrade more appealing to managers.
• If you do make a break, make it the last time you need one. That means some comprehensive planning in advance and making sure that everything is set up properly.
• If you have a UI, go easy on changes to it. Think refresh rather than redesign. For non-technical users, drastic UI changes from one version to another can be a big cause for frustration.
• Your new version should be noticeably more stable and performant than the old. Don't give your users a reason to resent the upgrade. Don't release a new version without extensive testing beforehand (unit, integration, and beta testing).
• Maintain the old version simultaneously for an extended period of time. If you do decide to phase it out and discontinue support, give at least 6-12 months notice, more if you can.
• Can you afford to maintain two versions at once in terms of both finances and manpower? (Also, from a business perspective, you shouldn't stop support of the old version until you can afford to lose the remaining clients that are clinging to the old version and don't want to upgrade. That point should be when your costs of maintaining it outweigh your profits from it.)
• Commit to doing security updates for a period of time after the old version has been discontinued if necessary.
• Again, communication is huge all throughout the process. Don't leave your users/clients/customers feeling abandoned at any point. Their loyalty and passion is the foundation of your business. Make sure that you respond to their questions on your blog or in your forums. The social factor should not be underestimated.

As for the 'when', you will probably have a better idea for your application than anyone else. In general, however, it might be time for a clean break when your technical debt and architecture completely inhibit stability, prevent reasonable performance, and make the development of new features overwhelming or needlessly difficult.

All that said, don't take this step lightly. Even done right, breaking backward compatibility is a big deal. You should strongly consider increasing your test coverage and refactoring first and if at all possible before considering a break.

Answer 2

In general I agree with James Anderson. In my experience there are, however, additional aspects that may warrant consideration and that may point towards an option that does indeed allow evolving interfaces.

This example is from one of the teams I am working with. They ship a product on a regular basis, at least once per month sometimes even weekly. Customers are encourage to upgrade as new features and new platforms are only supported on newer versions. Upgrading is easy and customers can even skip versions in between. Downgrading is not supported. In addition versions are only supported for 3 years. After that there is a grace period of one year when maintenance fees double.

As a result the vast majority - about 95% of the customers - are upgrading on a regular basis, at least once a year. This also means that you can gradually introduce new interfaces.

How about old interfaces? The technique this team uses is declaring old interfaces as 'end-of-life'. Then there is a 12 month period in which the new interface is available and the old interface has not bee retired yet. The new interfaces offer better features than the old interface so there are two incentives: Old interface end-of-life and new interface being much better.

The concrete old interface in this case was a platform specific technology that is in the process of gradually being replaced by a service interface based on standard mainstream technology.

Of course this change doesn't happen over night and it takes many years until completion. But it allowed stopping investment in the old technology and instead investing into the new technology. Customer are given assistance and have a path forward. Most of them are welcoming the move towards newer technology as well.

Keep in mind, though, that this particular approach may not work in your scenario. It certainly does work for the team that I'm working with.

Answer 3

You already got some good answers here, so I would just like to add a pointer to a very nice article from Joel Spolsky concerning this topic. He is discussing the plans of giving up backwards compatibility in IE 8, which is essentially the same as your problem:

http://www.joelonsoftware.com/items/2008/03/17.html

Answer 4

The short answer is Never!

Experience shows that dropping backward compatibility at the very least annoys customers and users, and, at worse loses them completely.

If you are asking your users to rewrite code, after they have finished the customary cursing of you and all your descendants, they will think "If I have to re-write anyway maybe I should just switch to that nice ACME library I have been reading so much about?".

The trick is to either enhance the current interface in such a way that does not break backward compatibility, or, to offer a completely new shiny and obviously superior interface while maintaining the older interface. At some point there will be features in the new interface that just won't be possible in the older interface, but, this just gives people an incentive to move, without forcing the issue.

Edit to clarify this further.

What you as a programmer think is -

"I will improve this API and make the system as good as possible and everyone will admire me".

What the users of your API will think is -

"A***e he thinks my time and schedule are not important. I have to stop what I am doing and refactor all my code for no other reason than to satisfy his ego. If I have to refactor then I am going to switch to another API just to stick it too him."

Answer 5

This is really more of a business decision than a technical one. Usually, this is done as part of a major release (e.g. going from version 3.5 to version 4.0), and often the two versions are supported in parallel for awhile. This means you'll do maintenance on the old version but all new features will only appear in the new version. The old version is supported as long as the company makes money off of it, or at least enough to offset the maintenance costs. Be prepared to pitch this to management.

Answer 6

I've been on the customer side of this one so I'd say that it really depends on what you plan to do about the transition.

We are currently upgrading our accounts system. The company doing the upgrade also supports the previous incompatible version. They plan to take the data and move it to the new system so that (in theory) all the old data will be there. We will be paying a couple of hundred pounds for the data conversion. No problem.

Compare to a previous situation at anothe rcompany I worked for. The supplier had no way to transition from the old to new systems. This put them in the same situation as every other supplier. In fact their situation was worse because we knew that they were not commited to helping us with the upgrades. They didn't get the new contract.

You've done the hard work in getting and keeping customers, how can you ease the transition?