59

First, some context (stuff that most of you know anyway):

Every popular programming language has a clear evolution, most of the time marked by its version: you have Java 5, 6, 7 etc., PHP 5.1, 5.2, 5.3 etc. Releasing a new version makes new APIs available, fixes bugs, adds new features, new frameworks etc. So all in all: it's good.

But what about the language's (or platform's) problems? If and when there's something wrong in a language, developers either avoid it (if they can) or they learn to live with it.

Now, the developers of those languages get a lot of feedback from the programmers that use them. So it kind of makes sense that, as time (and version numbers) goes by, the problems in those languages will slowly but surely go away. Well, not really. Why? Backwards compatibility, that's why. But why is this so? Read below for a more concrete situation.


The best way I can explain my question is to use PHP as an example:

PHP is loved, and hated by thousands of people. All languages have flaws, but apparently PHP is special. Check out this blog post. It has a very long list of so called flaws in PHP. Now, I'm not a PHP developer (not yet), but I read through all of it and I'm sure that a big chunk of that list are indeed real issues. (Not all of it, since it's potentially subjective).

Now, if I was one of the guys who actively develops PHP, I would surely want to fix those problems, one by one. However, if I do that, then code that relies on a particular behavior of the language will break if it runs on the new version. Summing it up in 2 words: backwards compatibility.

What I don't understand is: why should I keep PHP backwards compatible? If I release PHP version 8 with all those problems fixed, can't I just put a big warning on it saying: "Don't run old code on this version !"?

There is a thing called deprecation. We had it for years and it works. In the context of PHP: look at how these days people actively discourage the use of the mysql_* functions (and instead recommend mysqli_* and PDO). Deprecation works. We can use it. We should use it. If it works for functions, why shouldn't it work for entire languages?

Let's say I (the developer of PHP) do this:

  • Launch a new version of PHP (let's say 8) with all of those flaws fixed
  • New projects will start using that version, since it's much better, clearer, more secure etc.
  • However, in order not to abandon older versions of PHP, I keep releasing updates to it, fixing security issues, bugs etc. This makes sense for reasons that I'm not listing here. It's common practice: look for example at how Oracle kept updating version 5.1.x of MySQL, even though it mostly focused on version 5.5.x.
  • After about 3 or 4 years, I stop updating old versions of PHP and leave them to die. This is fine, since in those 3 or 4 years, most projects will have switched to PHP 8 anyway.

My question is: Do all these steps make sense? Would it be so hard to do? If it can be done, then why isn't it done?

Yes, the downside is that you break backwards compatibility. But isn't that a price worth paying ? As an upside, in 3 or 4 years you'll have a language that has 90 % of its problems fixed.... a language much more pleasant to work with. Its name will ensure its popularity.

EDIT: OK, so I didn't expressed myself correctly when I said that in 3 or 4 years people will move to the hypothetical PHP 8. What I meant was: in 3 or 4 years, people will use PHP 8 if they start a new project.

11
  • 32
    PHP is a bad example for this particular question, because more often than not you don't get to chose the version you'll work with. The majority of PHP sites are deployed on shared servers, and the owner of the server is choosing the version, not you. Lots and lots of stuff get fixed with each new version (mysql_* was deprecated in 5.5, for example), but that's irrelevant if the majority of hosting providers out there are one or even two versions back (5.3 is - unfortunately - still what the majority of providers offers).
    – yannis
    Mar 25, 2013 at 15:12
  • 5
    ...I also think you underestimate the amount of code that should be ported, the amount of things that would break, the amount of third party dependencies to adapt, etc.
    – dagnelies
    Mar 25, 2013 at 16:03
  • 12
    This great blog post joelonsoftware.com/items/2008/03/17.html of Joel Spolsky about "Martian headsets" should be mandatory for every developer who underestimates the importance of backwards compatibility.
    – Doc Brown
    Mar 25, 2013 at 16:14
  • 3
    Also, PHP has been slowly deprecating functionality with every single release - and a LOT of stuff breaks as a result. Unfortunately, PHP's stuck in a difficult spot where they have a hard time even producing deprecation warnings in a way that developers will see that won't end up disrupting sites anyway.
    – fluffy
    Mar 25, 2013 at 18:18
  • 21
    python 2.x => python 3.x is a breaking change from one well designed language to another, slightly better designed language, that has first party support for automatically changing many incompatible constructs. Porting code between them is about as easy as you could make a change between two languages. Py3k is still very slowly gaining ground.
    – Phoshi
    Mar 25, 2013 at 21:21

14 Answers 14

26

It sounds fine, but rarely works out in practice; people are extremely reluctant to change running code, and even for new, green-field projects they are very reluctant to switch way from a language/version that they already know.

Changing existing, running code that "works fine" is not something that ranks high on any project's priority list. Rather than applying effort to things that the managers thought had been paid for already, just to be able to upgrade to a newer release of a language or platform, they will decree that the developers should just stay on the old release "for now". You can try to entice your users with great features only available in the new release, but it's a gamble where you risk decreasing your user base for no clear gain for the language; cool, modern features cannot easily be weighed against the price of fragmented installation base in popular opinion, and you run the risk of getting a reputation for being an "upgrade treadmill" that requires constant effort to keep running when compared to more relaxed languages/platforms.

(Obviously, most of this doesn't apply to projects written by hobbyists just for their own pleasure. However (here be flamebait...) PHP is disproportionally rarely chosen by hackers because it's such a pleasure to write with in the first place.)

0
66

You're underestimating the impact of backwards compatibility; your estimate that all active projects would migrate in 3 or 4 years is far too optimistic.

Suppose I'm a PHP developer. PHP has flaws, but I know how to work around those flaws - that's part of the reason I get paid as a PHP developer. Now suppose that PHP 8 comes out and fixes those flaws, but it's not backwards compatible. As a result:

  • I have to spend time updating my code for PHP 8. That's time that I could spend responding to customer requests, implementing new features, keeping up with the competition.
  • Even after I've done this, there's a good chance that I've missed some corner case or unforeseen compatibility issue, introducing bugs in my code.

Given this, there's a strong incentive to never migrate to PHP 8, even if it is "better, clearer, more secure etc." It's estimated that there are still billions of lines of COBOL (!) - even though there are obviously much better technologies available, the cost of an update, combined with the risk of bugs, just doesn't make it worth it.

Second, even if I decide to migrate my own code, any nontrivial app depends on third-party libraries, and there's no guarantee that the third-party libraries will migrate. For example, Python 3 was released in December 2008, but Django (probably the leading Python web framework) didn't have stable, production-ready Python 3 support for almost five years (see here and here).

7
  • 8
    There's a surprisingly large number of COBOL positions open, especially in with older insurance companies O.o Mar 25, 2013 at 19:02
  • 1
    @Josh Kelley: I agree with you but I think that these problems only affect languages in which you cannot clearly separate legacy code from new code, e.g. Python, PHP because you need to include libraries, and C++ (templates). Languages with a different compilation model (e.g. Java, Scala, Clojure) show that it is possible to add new code (e.g. in Clojure) to legacy code (e.g. in Java) even though the two languages are not compatible at the source code level.
    – Giorgio
    Mar 25, 2013 at 19:36
  • 2
    I don't know if I should post this as a separate question or as a comment. But why couldn't they make a programming language that elevates code migration to a first class concept? In java, there's a @Deprecated annotation that just gives you a warning. Maybe another language could actually provide a macro that replaces the old code with the correct new code. If you're using the latest version it's an error to call the deprecated code, but old code gets converted to use non-deprecated new code. Just spitballin' Mar 25, 2013 at 20:49
  • 7
    @tieTYT - Some programming languages do this - see Python's 2to3 or Go's gofix (talks.golang.org/2012/splash.slide#68). It certainly helps with deprecating old features, but there are limits to how well software can understand and update other software when the language semantics change. Mar 25, 2013 at 21:12
  • 3
    @hydroparadise My aunt works as developer with banks and insurance companies, and in these days of crisis some clients of her company decided to go back to COBOL because software is cheaper! So even economy can affect the rate at which companies move to newer languages/versions.
    – Bakuriu
    Mar 25, 2013 at 22:21
17

You're making a lot of assumptions about human behavior. If you change it too much, people will evaluate your competitors, since they're going to have to spend significant effort switching anyway. For open source languages, people will just fork the old version.

Look at python for an example. 3.x has been available for four years, and still isn't widely adopted. People try to use it for brand new projects, but I think you're underestimating just how much code work is maintenance.

Of course, most people didn't consider python 2.x to be "flawed." They didn't have complaints like php users. Php is in a much more precarious position, because a lot of people only stick with it because of its large existing code base. If you lost backwards compatibility, a lot of people would take the opportunity they've been waiting for to move to python.

3
  • I think you have a good point here (+1). I think backward compatibility is a false problem, especially for compiled languages in which you can use separate compilation (see how you can integrate Scala and Clojure with Java, or C# with C++). But keeping the feeling that the new language, after all, is just an updated version of the old one is very important to avoid a fork or that people simply migrate to another language. I think these reasons are much stronger than dealing with legacy code.
    – Giorgio
    Mar 25, 2013 at 19:31
  • 1
    @Giorgio False problem? Tell that to all library writers who have to support more versions of a language at the same time.
    – svick
    Mar 28, 2013 at 19:30
  • @svick: With separate compilation you do not need to support different versions of a language. See e.g. how Scala can use Java libraries that are not compiled for Scala.
    – Giorgio
    Mar 29, 2013 at 1:10
9

For any language other than PHP I'd say, yeah, that absolutely makes sense! That's exactly what Python is doing with the switch to Python 3.

However, the problem with PHP is that there are too many flaws with the language design itself, thus what you're calling "PHP 8" would be a completely different language. And if you'd have to switch to different language, why would you stick with new PHP, rather than any of currently existing and stable alternatives?

Also PHP community is extremely slow to adapt anything new. Just look how long it took to get rid of register_globals. It's been known to be a security risk since year 2000. It's been only finally removed 12 years later. Another example, when PHP5 was introduced, it was huge improvement over PHP4, yet community did not adapt it. I took 4 years and massive actions such as GoPHP5 to jumpstart the adoption. And that even didn't have significant amount of backwards incompatible changes.

5

Disclaimer: I manage a ColdFusion user group.

ColdFusion suffers the same problems: loved by many, despised by many. In addition, tons and tons of FUD based on pre-Java versions. ColdFusion 10 came out last year, is a huge seller and last week I signed up to test the pre-release of version 11. Also, there are two major open source alternatives, one is backed by JBoss.

There are TONS of new functions in CF10 that I would love to implement, but migrating from CF 7 or 8 can be difficult depending on the size of your code base, the number of upcoming projects and the resources you have to regression test everything once you're on the latest version. I've run into a number of minor syntactical differences between 8 and 9, as well as edge cases where the code doesn't compile the same way. Once found, I documented them into our Coding Standards so they aren't used in future projects or by new developers.

That said, if ColdFusion 11 (or any programming language) were to completely deprecate certain functions and syntax, the level of effort to find and replace functionality could be enormous. The testing effort could be gargantuan. Will companies pay their developers, QA and project managers to find, replace and test all of those deprecated things? Doubtful.

If the latest version of a language is backwards compatible, but introduces a boost in performance with no code changes (CF9 is about 30% faster than CF8 and CF10 is much faster than CF9), who cares about changing function calls if they still work?

As a company, we have to worry about pleasing our clients and delivering their needs in order to bill services, build the business and recruit more clients.

FWIW, I would love to get us onto the latest version of jQuery at some point, but since certain functions were deprecated a few versions after what we use, and given the volume of JavaScript we have in the system, I don't know how we're going to pull that off.

4

There's a tradeoff here; some bugs REALLY need fixing, but some things can't be changed without breaking someone's code somewhere. I seem to remember someone stating as a "rule" that every bugfix will break someone's project, no matter how obscure or obviously-broken the bug is, someone will be using it for something. Such is the nature of programmers.

This is (to my mind) the difference between major releases, minor releases, and revisions. As a general principle:

  • Major releases are assumed to contain breaking changes.
  • Minor releases may change behaviour slightly.
  • Revisions should be pretty much cross-compatible.

For example, if I'm writing something in v2.3 of a language, I wouldn't expect to notice any difference if I upgrade to v2.3.2. If I upgrade to v2.4, then a few things might change - small syntax tweaks, some functions behave a little differently so I have to tweak logic, etc. If I upgrade to v3.0, I would not be surprised if it broke entirely - functions deprecated or missing, operations not supported or changed so much that I can't just tweak it back into line, I actually have to rewrite some functions to account for the new changes.

Edit:

Steve Vance's paper Advanced SCM Branching Strategies has this to say:

Typically, there are two to three levels of release, named by numbers connected with periods (e.g. 1.2.3). [...] In this structure the first number is associated with a major version, indicating that it has significant feature and functional enhancements from the previous; there may also be significant incompatibilities that require migration. The second number represents a minor version, which contains lesser feature and function enhancements, a significant number of bug fixes, and no incompatibilities. The third number refers to a patch level, signifying almost exclusively a collection of bug fixes; no feature or function enhancements and no incompatibilities are allowed between patch levels.

The only alteration I'd make to this is the aforementioned principle that programmers often find ways to "use" bugs, so a minor version with "a significant number of bug fixes, and no incompatibilities" might be difficult, because it's likely that those bugs will either break something that used them, or will cause a workaround to become unnecessary and start causing problems.

2
  • I'd expect 2.3->2.4 to add functionality, but not remove it. Mar 26, 2013 at 11:58
  • 1
    Coincidentally, I came across a relevant quote recently. It's a little long for a comment, so I'll edit my answer. Mar 26, 2013 at 12:08
2

It really depends on what's the target of the language - what types of applications are intended to be built with the by the language.

For example, ignoring Android, Java is mostly used in large enterprise systems and middle-ware; these types of applications tend to become very large both in size and in time. This has some implications; imagine a system with 500K+ LoC on which worker 50+ engineers in the development phase. Typically this type of system enters in maintenance after this with say 10 developers; now if the language changes and the changes are not backwards compatible the project cannot be easily migrated to a new version because the programmers that wrote some parts are gone and nobody wants to touch it. This is the smaller problem, the bigger problem consists in the fact that is kind of expensive to adapt a 500 LoC application to new language constraints. For example if generics weren't implemented with type erasure and List list = new List(); wouldn't compile millions of lines of code would need to be rewritten - which is at a great cost.

On the other hand PHP tends to be used on the web for simpler applications; usually it is developed by a single programmer or a small team. The idea is that the developers kind of know the whole project pretty well can can integrate language changes more easily. Also it's purpose is to build a site very fast, and the faster the better so if a new language feature can do this better then it is implemented even at some backwards compatibility costs.

1

One can argue Microsoft performed a similar change with ASP.NET (as successor to classic ASP) or with VB.NET (although they made so many concessions with the latter that most of the benefits of "rebooting" the language were lost).

Anyway, if anyone remembers the nightmare of Migrating VB6 code to VB.NET even with the assistance of a migration tool, they'll agree right off that language migration tools don't work very well for major language updates.

It might be possible to move the platform forward but, you should still provide support for "deprecated" APIs through at least a few revisions.

1

A lot of the "flaws" people scream about in popular programming languages aren't, they're things the screamer's favourite toy of the day has that that language lacks, THEREFORE that language is fundamentally flawed because it lacks that.
The next hype comes around, the language is suddenly flawed because it doesn't adhere to that hype.

The lack of closures in Java is a classic example. That's not a flaw in the language at all, and changing the language (as is sadly on the agenda) to include them will IMO fundamentally cripple it or at the very least make it a lot harder to read and comprehend.

What all too many people lose sight of is that each language has its strengths and weaknesses and that trying to create something that combines the strenghts of everything while avoiding every weakness will only create an utterly unusable monster that's good at nothing, incredibly unwielding, impossible to use effectively.

Add, as others have pointed out, that backwards compatibility is vital to retain existing users, many of whom are NOT going to spend the thousands of hours and millions of dollars/Euros to retrofit their million line codebases to whatever you think is "better" than the version of the language they've been using for years, and you've a host of very good arguments to leave well enough alone and if you want to play with some new overhyped idea that's supposedly the next "java killer" you'd best play with that toy rather than scream that "Java iz ded" unless it "gets fixed" to be a clone of that toy.

1

I would suggest that newer versions of a language should strive to ensure that 99.99999% of code which compiles in both the old and new versions of the language should work identically in both unless it's deliberately designed not to do so, and that most of the time when the new version refuses code that compiled under the old version, it will be because the code was--at best--dodgy, and should have been written in a different way that would compile under both the old and new compiler.

For example, if I were designing a new language similar to Java or C#, I would forbid implicit type conversions in some contexts where those languages allow them. As a simple example in C#, given

int someInt;
double someDouble;

the expression someInt.Equals(someDouble) is guaranteed to return false, regardless of the contents of the variables. It compiles because double can get converted to Object, and int has an Equals overload for that type, so the compiler does the conversion and makes the call. Were I designing a new version of C# and the .NET Framework, I would have it forbid the boxing conversion since it can't possibly do anything useful. It's possible there's some program which does such a comparison in a way which is useless but harmless, and having the compiler reject such code might break that program, but correcting or removing such useless code would be an improvement.

As a slightly less clear example, assume

float f=16777216f;
int i=16777217;

and consider the expression f==i. It's possible that some code does float/integer comparisons and works correctly, but the code should be rewritten as either f==(float)i, (double)f==i;, or (double)f==(double)i; [int to double promotion is lossless, so the latter two would be equivalent]. Some code which directly compares float and integer values may always deal with numbers which are sufficiently small that float and double comparisons would behave identically, but a compiler generally can't know that; code should make clear what kind of comparison is needed, rather than hoping the rules of the language will match the programmer's intention.

1

It's best to never break backwards compatibility.

Microsoft replaced the VB6 programming language with a new language that totally broke compatibility. So even today the 16 year old VB6 is still more popular than the dotNet version (Tiobe index August 2014). And Gartner estimates there are 14 billion lines of VB6 code still in use.

In 2014 Microsoft have again had to announce they will not update or open source VB6 despite demands from the Visual Basic programming community. But they have extended support of VB6 until 'at least' 2024, and it works fine on Windows 7 and 8. That will be over 26 years of support for the same version of VB6.

Why should existing working software have to be re-written, even Microsoft never "updated" Office to use dotNet ?

1
  • this doesn't seem to offer anything substantial over prior 14 answers
    – gnat
    Aug 17, 2014 at 12:47
1

There are a few different problems with breaking backwards compatibility. Some of the problems stem from the fact that most programming languages are also platforms (interpreters/runtimes), other problems stem from an assumption of human nature.

A. Code written in an older versions wouldn't get the benefit of new releases that improve performance, security, or features. You could mitigate this problem by supporting multiple major versions of the compiler/interpreter, but that is a huge resource drain (ie its expensive or takes a long time, and is a pain in the ass).

B. Code written for the newer versions may not be compatible with code written in older versions. You could work around this by having an interpreter/compiler that can handle multiple major versions of the language, but this is more of a pain in the ass than supporting separate interpreters/compilers (the workaround for A).

C. Major changes, if they happen too often/quickly also simply make the language harder to use, since you have more to learn and unlearn. Changes to a language might push people over the edge to switch to a new language, or might cause people to continue to use outdated versions of the language and just never switch to the new version (like has happened with python). Then again, the changes may also attract new users and excite old ones.

D. New documentation needs to be kept and maintained. Its always a rather confusing experience to look up things on google and find that you're reading the docs for a different version than you're currently using.

By and large, if you create a programming language where external modules don't have to care what version you're using, breaking backwards compatibility for the right reasons (to fix major flaws in the language) is almost definitely the right thing to do. Its likely that the major reason this isn't done is that programming language designers overestimate (to contradict someone else's answer) the costs of breaking compatibility, especially early on. The fact is, the problems of breaking compatibility can be worked around or powered through by users of that language. And this doesn't only hold for programming languages; this applies to APIs, user interfaces - really any interface in any situation.

Facebook annoys the hell out of people when it changes its UI, or its developer APIs. In the past, its made the platform difficult to work with. In some cases, APIs simply stopped working out of the blue. But people kept using it, and now the APIs and UIs are leaps and bounds better than they were 5 years ago. People will complain about change whether its good or bad for them, but that (complaining) is not a good reason to forgo that change. Unfortunately, programming language developers use this as a reason to keep the problems of their language intact.

So another couple reasons for languages not making breaking changes to improve themselves are:

E. Language developers think their users fear of change is a good reason to stagnate their language

F. Language developers kind of liked their language when they made it, and they probably think its just fine with its flaws.

G. Languages as they grow older usually cease to have a small core set of developers, and turn into more committee-built beasts. This means decisions about those languages are slow and often conservative and uncreative.

H. The last reason is that some breaking changes require significant re-evaluation of design decisions made for the interpreter/runtime. Sometimes improvements to the language simply require too much work to be feasible. I would guess this is a rarer problem than most tho.

Often, language designers aren't necessarily tool designers, and so they don't think of good solutions to this problem, or they don't execute them well. Here are some solutions I can think of to solving the breaking-changes problem:

  1. Deprecate things well ahead of when they'll be removed.

  2. Provide a good, standard converter tool. Python provided the 2to3 tool, but it wasn't well advertised, didn't come standard with python 3 as I recall, and didn't even work very well (I remember having to manually go through programs generated by 2to3 to fix issues it didn't fix). This converter tool could even be automatically run if your compiler/interpreter detects an older version. What could be easier?

5
  • The problem with the Facebook analogy is that there isn't a Facebook Legacy in use. There's no choice. Either you use the current version of Facebook, or you don't use Facebook at all. Meanwhile, there's still tons of people using Python 2 seven years after the release of Python 3 because it still exists - if it didn't, they'd grumble, but they'd port to Python 3.
    – Kevin
    Aug 15, 2015 at 16:19
  • I don't think that's a problem with the analogy, that was actually my point. Facebook has chosen the "fixing the flaws" route and has mostly eschewed the "backwards compatibility" route. Which is why they don't have a legacy version of their API. Its a perfect example of one extreme.
    – B T
    Aug 16, 2015 at 21:13
  • Breaking backwards compatibility in programming languages will just lead to people continuing to use and/or forking the old version. The old version of Facebook doesn't exist anymore; I suppose you could make a clone that supported the old API, but nobody would use it, because Facebook is a brand with a huge user-base.
    – Kevin
    Aug 16, 2015 at 22:01
  • Facebook has the advantage that, when it updates, the previous versions essentially don't exist any more. Programming languages aren't like that, and that's a relevant difference - you can use an outdated version of a programming language, such as Python 2, because it still exists.
    – Kevin
    Aug 16, 2015 at 22:02
  • I see your point. I still think its one end of two extremes. If major flaws become apparent in an unsupported version of of a language, it may be along the lines of that version ceasing to exist, because no one will want to use it.
    – B T
    Aug 17, 2015 at 0:32
0

I don't know if that's a problem for PHP code, but in many languages old, legacy code is never updated after years or, sometimes, even decades, because it works, is critical for the business running it and is too big (say millions of SLOC), so it would not make sense to rewrite it. That's a reason why java made backward-compatability an almost religious issue, despite know old problems, especially in libraries (even if they are easier to update). I guess a lot of code from the Linux kernel was not updated for decades too, despite the adoption of standards like C99 and C11.

Even in languages that are less "entreprisey", breaking old, functional code can be a problem. That's what happened with Python 2 -> 3. A whole bunch of libraries and system scripts were stable and not maintained anymore, not because they were abandoned but because they were stable and doing their work. Adapting them takes a few years. So, as a developer, you cannot necessarily jump to python 3 if your favorite library hasn't made the move yet, so your own code will not work under python 3 either, thus resulting in community fragmentation.

0
-1

The problem lies within the backwards compatibility issue. Most PHP scripts I execute are running on an older RedHat server. If I were to use the newer version of the language for future scripts, then I would have to update PHP on this server -- and run the risk of having my older scripts broken/ having to take hours to rewrite all the old code with the new standard. Plus, all my developers are used to PHP reacting a certain way (whether that way is 'broken' or not). If it no longer reacts that way, it could be a major hurdle for productivity, as developers may have to basically re-teach themselves PHP.

Not the answer you're looking for? Browse other questions tagged or ask your own question.