Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I have begun working at a large software company and was assigned to a project that is over a million and a half lines of code. It's part of a program suite that is sold to clients (not an in-house project) and the source code can be purchased if they desire (although given the extra fees associated with it, this seems rare). They've been doing software design for years and their current products are intended to be continued for the foreseeable future.

To my surprise, the million and a half lines of code are almost completely lacking in documentation. Moreover, there are some areas of code that are incredibly messy to follow or could use some refactoring to become much easier to understand (for instance, an improvement in the programming language came out 10 or so years ago that would make large portions of code much cleaner, not to mention less prone to bugs). There doesn't seem to be any efforts to rectify this and my offers to do so for the parts I'm working with have met with resistance, for which I've never really gotten a clear answer.

Are these practices common in a large business in the software industry? Or is my company unique in its lack of refactoring and documentation?

Addendum: Based on some of the comments, I'd like to clarify what I'm looking for. I understand that my company has technical debt and this is bad. I'm not looking to determine whether or not my company is worse off because of this, I am just wanting to know whether or not this lack of documentation and resistance to refactoring is a fact of life within the programming world that I'll have to deal with if I continue working in it.

share|improve this question

closed as primarily opinion-based by Michael Durrant, Jim G., MichaelT, gnat, mattnz Nov 28 '13 at 4:00

Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise.If this question can be reworded to fit the rules in the help center, please edit the question.

4  
Close. I think this will gather opinions not answers. –  Michael Durrant Nov 27 '13 at 22:19
1  
... and my opinion, from dozens of different companies is yes, that is the norm except for brand new startups with a new code written using modern (e.g. last 5 years) techniques. –  Michael Durrant Nov 27 '13 at 22:20
2  
Studies? It doesn't take a study to come to some fairly sound conclusions. All you need to do is look around you. –  Robert Harvey Nov 27 '13 at 22:23
2  
Also modern techniques include NOT writing comments - they quickly get out of date and are never well maintained, and move more in the direction of meaning class and method names (think 'pool_of_currently_available_connections' vs. 'pool' and also there are tests such as "As a user that inputs x I should see x*2 displayed", etc. –  Michael Durrant Nov 27 '13 at 22:23
3  
ime, yes, most companies are like this, you'll need to learn how to deal with it. –  GrandmasterB Nov 27 '13 at 22:46
show 9 more comments

3 Answers 3

up vote 9 down vote accepted

There are a number of reasons why companies fail to invest in making their programs "better," from a technical debt perspective:

  1. Reducing technical debt doesn't add any new features to the software. It is therefore perceived by management as having no monetary value (to a certain extent justifiably)

  2. The nature of the software business awards first-to-market, not best.

I could go on, but all the other reasons are just variants of these two. They all basically add up to short-term thinking, focusing on immediate profits instead of long-term viability.

All companies with non-trivial, active software projects have some degree of technical debt. No company is completely debt-free.

Every software project I've ever worked on has accrued Technical Debt over time -- Jeff Atwood.

Regarding documentation specifically, the less there is, the better. Dollar for dollar, you get more return by making the software simpler to use and easier to maintain than you do writing more documentation for it.

There are some informal surveys about technical debt, and how companies manage it; all you have to do is look for them. Here's one:

enter image description here

Or this one:

enter image description here

It seems fairly apparent that your company is not the only one that has this problem. It may not even be in the minority.

Further Reading
Third International Workshop on Managing Technical Debt - Overview
Managing Technical Debt in Software-Reliant Systems

share|improve this answer
    
So to answer my question, is this typical of companies? –  Thunderforge Nov 27 '13 at 22:24
    
What do you mean by "typical?" –  Robert Harvey Nov 27 '13 at 22:26
    
Do most companies behave the way that I described? I understand technical debt, but you didn't really answer whether or not this was something that happens in most companies or if this is something that only a few companies (like mine) deal with. –  Thunderforge Nov 27 '13 at 22:28
1  
Just about everyone posting has worked at less than 10 or 15 companies out of millions. So it would take a formal study to know. I know of none. Also one persons clean code is anothers spaghetti so I think unanswerable. –  Michael Durrant Nov 27 '13 at 22:29
1  
In my experience for-profit vs. non-profit, healthcare vs. consulting, boston-new york-san fran vs towns in the heartland do make a difference. ymmv –  Michael Durrant Nov 27 '13 at 23:28
show 5 more comments

A perspective which I don't believe has been covered in other answers (yet) is the cost of change in three areas:

  1. testing
  2. re-familiarization
  3. opportunity cost

A company with a large product will need to test all changes. These test procedures need to be tracked over multiple axes: different platforms, different workloads, different perspectives (performance, functional, usability, backward compatibility).

The people who work in the areas of code that you are changing will also need to then re-familiarize themselves with the code, and it becomes especially cumbersome with multiple versions being supported ("sorry, what version is that? Ahh, that's the new code and you need to speak to Bob for that"). In a similar vein, changing code just to make it pretty tends to make things like change-logs (diffs, patches, etc.) very complicated to read... and tracking issues back to a particular point in time when the bug was introduced can be very challenging if there's something in the code history that changes everything. Additionally, if a bug is long-standing (these things happen) it may be in multiple supported versions of your code, and now you have to fix the old, and the new code, in potentially very different ways.

Finally, often the decision has to be made where to spend time and money. This is more than just your time, but rather what you should rather be doing with your time (and the impact it has on down-stream resources). Would it be preferable to spend your dev-team's time on features that can produce more sales, gain new market share, and generate profit, or should the time/money be spent on things that end-users will not notice, cannot be put on marketing material ("hey, our existing code sucks, but we have made it better") and is pure cost (no profit).

Bottom line, money. Always (well, almost always).

share|improve this answer
add comment

Yes. Based on the 50 or so companies I have personally encountered, as an employee or consultant, and the 200 or so more companies I know of via colleagues.

The kind of product (1.5 million lines) you describe took years to build, and many of the original devs have moved on (or become managers).

The senior management and major continuing clients want only small incremental change: they want stability. What we technical people think are improvements are in their eyes risks. Even if we can show the new version still behaves the same as the old one, they see a risk because the old system has exactly the characteristic you describe: it's undocumented. To them, there could be undocumented behavior that clients rely upon.

From their perspective: it ain't broke, so don't fix it. You are also seeing that there is a corporate culture to that effect. The people who are resisting your "improvements" probably did the same as you when they started.

Do not expect to "win". This is a cultural issue, and cannot be changed except by change at the CEO level.

share|improve this answer
    
The senior management and major continuing clients want only small incremental change: they want stability. - I'm not so sure about that. Clients give a damn about the code and about how big or small the changes are, but they are very interested in the two magical "free"s: 100% bug free and free of charge. Oh yes, and please deliver yesterday. Most of them either never heard of Scope vs. Cost vs. Schedule, or they suddenly forget their knowledge when it comes to software. Regarding cultural issue and ain't broke, dont fix I agree, but I also do understand the business issue behind. –  JensG Nov 27 '13 at 23:21
    
To me, a product gets the way it is through the feedback loop that exists (or not) between the company and it's clients. Different clients who give stronger feedback and have "deeper" requirements either push the vendor to change or they move on to a competitor (if there is one). –  andy256 Nov 27 '13 at 23:26
    
Ultimately there's a case to say it goes beyond merely cultural. Let's say management stopped all new feature development to address the top 10% of most buggy routines, it would have to be very carefully managed to avoid becoming the labour of Sisyphus. –  James Snell Nov 27 '13 at 23:55
    
@JamesSnell: That's a good point though. I know of examples, where a specific piece of code should have been redesigned and rewritten from scratch years ago. Today, the maintainers of that code still struggle to fix the old bugs that still pop up every other week, let alone the bugs that have been introduced in the meantime by adding more features. How many time could have been saved in the long run, if only they had decided to throw away the old, badly designed, version years ago? –  JensG Nov 28 '13 at 18:49
    
@JensG I agree with you. Managing technical debt is how a well managed team/codebase should operate. But addressing the technical debt means destabilising the codebase even if the intent is to improve it - that involves risk that once you begin refactoring that it may become an unending task. The code may have needed it, but the short term cost may be enough to kill the project off before any long term benefits arrive and no manager will want to rock the boat while they're in it. –  James Snell Nov 28 '13 at 23:03
add comment

Not the answer you're looking for? Browse other questions tagged or ask your own question.