Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I recently became interested in agile practices in software development and I since have seen a lot of articles point out that these practices allow for reduced overall costs.

The logic behind that usually goes like this: if your requirements change, you can reflect this change in the next sprint backlog and this will lead to reduced cost, because designing the new feature and implementing it is close together in terms of time, so the cost goes down, according to the famous rule that the later you need to make a change to your requirements the more expensive it will be to satisfy that requirement.

But mid to big software projects are complex. A sudden change to requirements does not mean you won't have to touch other parts of your system in order to satisfy that requirement. In a lot of cases the architecture will need to be modified very significantly, which will also mean you'll need to re-implement all the features that relied on the older architecture. So the whole point of reduced costs kinda goes away here. Of course if a new requirement calls for a new independent part of the system, that's not a problem, the old architecture just grows, it doesn't need to be rethought and reimplemented.

And the opposite. If you are using waterfall and you suddenly realize that a new requirement has to be introduced, you can go and change your design. If it requires that existing architecture is altered, you redesign it. If it does not really mess with it but just introduces a new part of the system, then you go and do all the work, no problem here.

With that said, it seems to me like the only advantage agile development has is working feature complete builds between sprints, and for a lot of people and prjects this is not critical. In addition, agile seems like it results in bad software architecture overall, because features kinda get slapped one onto another, agile teams only care that a feature works, not how it works. This seems like that when systems grow in complexity with time, agile development practices actually increase chaos in the overall product architecture, thus eventually resulting in higher costs, since it will be increasingly more difficult to introduce changes, whereas waterfall allows you to perfect your architecture before you release anything.

Can somebody please point me as to where I'm going wrong here, because obviously a lot of people use agile in production environments, so I must be wrong somewhere.

share|improve this question
    
You have a point. I would like to point out that the term 'waterfall' does not have 1 universal definition (like many other terms in IT). Most people think that you are not allowed to code unless the full 2000 pages of requirements are written and signed. This does not have to be the case at all. –  Emmad Kareem May 13 '12 at 0:52
    
Yeah, by "waterfall" I mean a linear process of (most basically) functional specs -> design -> code between milestones, not sprints. And, sure, 2000 page requirements and technical specs are not a must, basic responsibilities of classes and their relations to each other are often sufficient as a top-level design. –  tux91 May 13 '12 at 1:05
3  
@EmmadKareem: Actually, in Waterfall proper, this is exactly the case. You don't start coding until every detail of the design is final. Fortunately, actual Waterfall is seldom applied in software development - mostly because it doesn't really work. –  tdammers May 13 '12 at 9:34
1  
@tdammers, thanks for your comment. This happened few times though. The waterfall method has no owner and can be applied and interpreted differently. Thee is no rule that says don't code until.... or else..., this is because it is not a methodology. When wrapped in good methodology, I think it makes great sense specially in projects where users know the core of what they need from a system. –  Emmad Kareem May 13 '12 at 9:45
    
@Emmad Kareem: I agree with you. I think agile methods are more suited for projects in which the requirements are not clear and a lot of prototyping and user feedback is needed to get the final requirements. On the other hand there are cases in which the core requirements are clear from the beginning. In these cases, I would adopt a development method that is more similar to waterfall (with some room for corrections along the way) than to an agile method. I think it really depends on the kind of project you are working on. –  Giorgio May 21 '12 at 18:39
add comment

7 Answers

up vote 5 down vote accepted

First off, the "waterfall" model was always a straw man describing how NOT to run a software development project. Look it up.

Anyhow, most "waterfall" SDLC projects entail a "Change Management" process, for when people discover that the assumptions that were written into the specifications are no longer valid (and this always happens on large complex projects). Unfortunately, the change management process is built as an exception handling measure, and is stupidly expensive, meaning the project will end up late, or of poor quality.

The idea behind Agile methodologies is that change is a given--it will happen. Therefore, you must make "change management" standard operation rather than exception handling. This isn't easy, but folks have found that if you use a lot of good design practices, you can do it.

Another major problem with front loaded design is that (most often) so much time is spent in requirements gathering and design, development and testing time suffers. Also, if testing is the last phase, and a serious problem is discovered, it is highly unlikely to get fixed within your time frame.

Perhaps one of the best features of an Agile approach is that it demands continued interaction (that is, real communication) between the team developing the solution, and the customer who needs it. Priorities are made, so that the highest priority items are done first (and if time runs out, it's the low priority items that are cut). Feedback comes quickly.

Going back to the question of dramatic changes--you really need to use methods that mitigate changes. Good SOLID OO principals can play a good part of this, as can building solid automated test suites so that you can regression test your software. You should do these things regardless of your methodology, but they become truly essential if you are trying to be agile.

share|improve this answer
    
Great thanks. For everybody who wants read up on the subject of how design works in agile and why it's not all that bad: http://jamesshore.com/Agile-Book/incremental_design.html –  tux91 May 13 '12 at 22:29
add comment

Well, there's a few advantages. The first is that customers don't read spec documents. Ever. Waterfall assumes that everyone will nicely agree to a spec at the outset and then when the software is delivered to the customer a year later matching spec, they'll be happy. In practice, customers only ever discover all the things they really need, really don't need, and really need to be something different when they are actively playing with the software itself. Agile gets a working prototype in customers hands, so these disconnects are caught immediately. Agile isn't just about reacting to changing requirements. It's also about having those changing requirements occur early, when the cost of change is low, not at the end, when the cost of change is high.

A second advantage is that because Agile has highly visible deliverables often, projects are less likely to go off the rails. With a large Waterfall project, it is all too easy to get way behind without even realizing it. Waterfall produces multimonth death marches at the end of the project. With Agile, because you are delivering to customers every couple weeks, everyone knows exactly where the project is and slips are caught (and adjusted for) quickly. In my experience, Waterfall produces weeks or months of 100 hour weeks at the end. Agile means that you might have to put in a couple 12 hour days at the end of the sprint.

A third advantage is that Agile projects tend to be more motivating. It is very hard for people to have any sort of sense of drive in a year long project. The deadline seems so far away and that lack of immediacy means that people tend to procrastinate, over-polish designs and otherwise just not work very productively. This is especially true because the initial months are spent on things people don't like to do, like spec documents. Because Agile always has a very immediate, concrete deadlines in the very near future, people tend to be more motivated. It is much harder to procrastinate with a concrete deadline for a fixed set of tasks due next week.

share|improve this answer
    
First argument, that is exactly what I am under impression agile is for. Dealing with constantly changing requirements. Also, about changing requirements early, this still has a huge possibility of messing with the architecture in hand, leading to reimplementing a lot of existing code. Second argument, can you please elaborate why waterfall projects cause death marches? Seems like a little discipline coupled with concise technicals specs can do wonders here. Third argument, what's the problem with building a waterfall project from bottom to top and being able to build it once in a while? –  tux91 May 13 '12 at 1:51
2  
My experience with Waterfall projects is that they are always on time for the first 90% of the planned time, at which point they are suddenly months behind. Agile's model of insisting on demos every sprint makes this much less likely. When people know they will be accountable in a week and a half, they are usually better motivated then when they will be held accountable in nine months. It's just human psychology. –  Steven Burnap May 13 '12 at 3:09
1  
Well, I guess I can't argue with experience, although my experience has been a bit different, with a good design in hand coding doesn't have many troubles along the way and the estimates seemed to be pretty good too. I still think that the resulting architecture will be more solid if using waterfall though. –  tux91 May 13 '12 at 3:36
2  
There are a few areas — mostly safety-critical ones, e.g., railway signaling, avionics — where the customers do read the specs very carefully. They're pretty rare, and tend to lead to vastly different development methodologies anyway. –  Donal Fellows May 13 '12 at 12:20
add comment

In response to the quote from the question, which is a fundamental misconception of anti-agile opponents

"... whereas waterfall allows you to perfect your architecture before you release anything." is a fallacy, let me fix it for you; "... whereas waterfall allows you to perfect your architecture so you never release anything."

The implication that Waterfall some how provides higher quality architecture is fundamentally false and completely disproved by empirical historical evidence.

If Facebook was designed in a waterfall fashion with 500 million users as a hard and fast requirement and not released until an architecture to support that was perfected no one would have ever heard of it today.

Same with YouTube, Twitter and countless other companies.

They got something that the customer could touch and respond to working and released it as quickly as possible. They got feedback and refined it and released it again; iterate. This is how in these three examples, they respond with only features customers accept and want and waste as little time as possible on things that customers find of little or no value.

Anyone with any significant years of software development experience will agree there is no such thing as a perfected architecture. Just one that starts out farther from entropy and big ball of mud than other alternatives. Agile acknowledges this and takes it into account. Building into the process, this as technical debt, rapid re-prioritization and refactoring.

share|improve this answer
3  
By using the word "never", are you saying that there are no software products out there that were done using waterfall? Also, why "never" release anything if you have a set of requirements for a particular milestone that you end up successfully implemented? –  tux91 May 13 '12 at 0:13
1  
@tux91 you make my point for me; nothing will ever be perfect given that needs change immediately after designs are put to paper and are obsolete before a single line of code is written. Thus the statement I quoted is a fallacy, you will never perfect an architecture and thus never release anything. –  Jarrod Roberson May 16 '12 at 14:33
1  
@tux91 still read for comprehension, I said, that there is no prefect architecture and if you don't release until there is as in the quote, then nothing will ever be released. I didn't say what you are claiming, period, this is in your head and your interpretation. What I said, is the argument that waterfall somehow provides better quality architecture is a fallacy and fundamentally flawed. And these arguments about NASA and waterfall and what not; besides being unrelated to programmers, killed 3 astronauts, on the ground for that matter, not a shining success story. –  Jarrod Roberson May 16 '12 at 15:55
1  
@tux91 wrt usage of "never", I am with Jarrod here: question says "waterfall allows you to perfect..." - which he counters with a totally appropriate (in this unrealistic "perfect" context) word "never". The only reason I didn't upvote is that I try to avoid voting on answers to not constructive questions –  gnat May 16 '12 at 16:38
1  
@gnat yeah I guess I didn't think when I used the word perfect, that's not want I actually meant –  tux91 May 16 '12 at 16:39
show 11 more comments

Scrum by itself does not prescribe any technical practices because it is a general framework.

Agile software development requires technical excellence from the team. This comes from following technical practices from XP (extreme programming). For instance, you must learn about refactoring, tdd, whole team, pair programming, incremental design, ci, collaboration etc.

Doing it like that makes it possible to handle change quickly and safely, which is also the primary reason for using agile development in the first place.

The one measurement of agile that matters is if you regularly (weekly, biweekly) manage to release valuable, working software to your customer. Can you do that? Then you're agile in my book.

share|improve this answer
    
What I don't get is this "possible to handle change quickly and safely", as it implies that a waterfall based project is harder to change. Why is that? It's just a codebase. Specify what you need to change, design that and how it fits into existing architecture, code, test, and release. Just don't call it a sprint, call it a milestone instead. Doesn't seem like it would take a longer time or present any more difficulty than agile. The difference is that you do more careful planning but don't need to do XP stuff that rigorously. –  tux91 May 13 '12 at 16:11
    
@tux91 Updated my answer. As for architecture, I recommend reading this or watching this at 26:20 about what we call "incremental design". –  Martin Wickman May 13 '12 at 19:20
    
Thanks for the links, will check it out –  tux91 May 13 '12 at 19:49
add comment

For me, the main benefit of agile is reducing risk in the project.

By delivering iteratively and getting lots of feedback from your user base, you're increasing the chance that you'll deliver something they actually want, and you'll do it by pragmatically delivering the most important features for them as early as possible. In theory, this will be done with better estimates and planning. This is obviously very appealing from the business perspective - much more so than just the releasable build.

You could argue that this constant change compromises your system and reduces quality, but I would say two things. 1) This change happens anyway on more traditional software delivery projects - it's just unaccounted for and occurs later on in the project which is when, as you pointed out, things are harder and more expensive to change. 2) Agile has a lot to say about dealing with this change in terms of using experienced motivated people and practices such as TDD, pairing and code reviews. Without that shift in mindset towards quality and continuous improvement, I accept that frequent change that agile implies could start to cause problems.

share|improve this answer
    
Yeah, that's exactly what I think of the only advantage of waterfall. There are times though when you don't want to show your product to anyone while it is in development stages, because it's just not ready, it doesn't have the selling points yet. Or if you have a pretty firm idea of what you want build in the end. Tests, pair programming, and code reviews don't guarantee you'll end up with good overall product architecture though, they are only done to guarantee that new features work properly. –  tux91 May 13 '12 at 16:20
add comment

In a lot of cases the architecture will need to be modified very significantly, which will also mean you'll need to re-implement all the features that relied on the older architecture.

If your architecture needs to be modified significantly as a result of a requirements change, you have either bad architecture or a bad requirements. Good architecture allows you to defer as many decisions as possible and decouple components of your system. Good (user) requirements aren't things like "use a different database".

So, let's operate under the assumption that we have a good working architecture. If that is the case, then agile development methodologies lose this "wash" with big-up-front-design methodologies. After all, a core feature of agile development methodologies is practices like TDD that promote loose coupling in code, allowing the project to continue at a high velocity whether it's near the beginning or very mature.

share|improve this answer
add comment

In a lot of cases the architecture will need to be modified very significantly, which will also mean you'll need to re-implement all the features that relied on the older architecture. So the whole point of reduced costs kinda goes away here.

Following an agile process, there is a better chance of catching this requirment before too much software has been developed (And needs to be changed.). When you go several months without working with the client and giving them something to look at, you catch these major architectural problems only after you "think" you're ready to deliver.

I'd rather try to implement these changes in a working application that has test coverage than a massive pile of code that doesn't even compile. While being head of technical support, I was handed a CD of an application that had been in months of development and it wouldn't even install. We could have found that problem the first week, but instead it was time to order diner in because it was a long night.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.