Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I have been using an agile methodology (SCRUM) for about three years now and I see certain advantages to it, especially in the short-term feedback at many levels (from customers having early access to implemented features, from testers that can test features as soon as they are implemented, from other developers that can provide very early feedback on new code through review, etc).

On the other hand, I have two open problems, the first of which I will try to explain in this question.

Problem : Difficulty to get a good design

I try to do refactoring as soon as the code gets messy, I write unit tests as much as I can (which does help to prevent bugs in general and when refactoring in particular). On the other hand, developing some complex feature in small increments, with daily commits and continuously rethinking the code when it gets unstructured is not allowing me to produce really good design.

The only well-designed module I could produce recently I obtained by taking a different approach: I analyzed the problem for a few days (I actually had had the problem going through my mind for a couple of months before I started to work on it seriously), sketched a rather detailed design of all the involved classes and their relationships for a couple more days, and then locked myself up in my office and wrote down the whole code by working without interruption for about three weeks. The result was the best thing I have produced in a while, with very few bugs that were rather easy to locate and to fix, and with a very clear design, which has not required any relevant changes since then.

So up to now I found it much more effective to get an overall picture of what I want to do beforehand than starting to write code in small increments in the hope that the big picture magically emerges in the process. With my best effort, the small-increment development approach has always led me to worse design.

Question: Has anyone had a similar experience? Am I applying SCRUM the wrong way or what should I pay attention to if I want to develop in small increments and still end up with a well-designed piece of software? Or should I schedule a design user story before starting the actual coding? Is this considered a good practice, at least for features that are more complex than average?

EDIT - NOTE

I am aware of the fact that good design is not something absolute and does not have a value on its own but it depends on the context, and that one should aim at a design that is good enough for the problem at hand.

For example, I do not care (too much) about good design if I have to implement a simple component that (1) must be ready as soon as possible, (2) is going to be used only once, (3) is not used by other parts of the system (YAGNI).

I do care about good design when a component (1) will be used several times and in several different releases of a product, (2) needs to be maintained and extended over time, (3) has a lot of other components depending on it.

share|improve this question
5  
The only well-designed module I could produce recently I obtained by taking a different approach -- You answered your own question. You still need some up-front design; you can't expect a good design to simply grow organically from refactoring. It doesn't work that way. –  Robert Harvey Jul 13 '12 at 22:19
1  
No. Because maybe I am not applying SCRUM correctly. –  Giorgio Jul 13 '12 at 22:20
3  
There's no such thing as "SCRUM correctly." There is only that process which produces the desired result. –  Robert Harvey Jul 13 '12 at 22:21
1  
That's why they're called "guidelines" :) Anyway, commit every day, have short (1 week to 1 month) sprints, rules like this don't speak at all to design. You still gotta have design. –  Robert Harvey Jul 13 '12 at 22:23
2  
let us continue this discussion in chat –  Giorgio Jul 13 '12 at 22:29

6 Answers 6

up vote 9 down vote accepted

On the other hand, developing some complex feature in small increments, with daily commits and continuously rethinking the code when it gets unstructured is not allowing me to produce really good design.

Then don't do that.

Not everything fits into the nice agile box. Often times a product will have a few complex things that can't be farmed out to individuals and cannot be completed in any sane manner within a sprint. Forcing them into that box will only cause trouble. But these should be few and far between. If you find that many of your components are like this, your product owner needs to work at breaking things up better (with your team). I am fine doing like what you did: take the story, design it, then hammer it out ASAP with the expectation that it will take a few weeks.

In the more general case, there are 3 things that I've seen done for getting good design in Agile:

  • Actually do design - A lot of places I've seen start their sprint and just start cowboying out code. You'll get bad design this way. Agile doesn't change the development process of plan - code - test - release, it just shortens it to more granular pieces. At the beginning of the sprint, sit down as needed and actually design your solution.

  • Have an architect/lead - Once your project is big enough, you'll end up with multiple scrum teams working on different pieces of the application. It is very useful to have someone (or multiple people depending on the size of the application) whose primary job is to know what all of the teams are doing from a design standpoint. They can answer questions and guide the teams towards a more harmonious over-arching design. Having each scrum team have a lead that knows most of what other teams are doing I've also seen and was very effective.

  • Be a pragmatist - I covered this one above. Agile is a tool, and like any tool; it doesn't apply to all problems. If it doesn't make sense to apply to some component, then don't apply it there. Your goal is to ship good software; don't forget it.

share|improve this answer
3  
+1 (+10 if I had more than one vote!) for "Forcing them into that box will only cause trouble." –  Giorgio Jul 15 '12 at 21:46

It is very possible to implement in small increments and end up with well-structured maintainable code. In theory, it's very simple: if you ensure that the code is well-structured after each change, it will always be well-structured. Your code is becoming poorly structured because you are continuing to add code when you should be refactoring.

No matter how much time you spend on design, eventually the requirements will change in some unanticipated way and you will have to write code that does not fit the original design. Then you will have to make an incremental change. If you have a good set of unit tests, and are skilled at refactoring, then you will be able to keep the code well-structured as you meet new requirements.

share|improve this answer
    
+1: OK. So I should schedule a refactoring story when I think there is a need for it and not add any new features until I have a sufficiently good design again. That's probably what we are missing in our process (in addition to the fact that, IMO, the increments we are developing are often too small). –  Giorgio Jul 13 '12 at 23:13
2  
actually you should add the technical debt story and discuss when to actually act on it with the Product Owner, this is what is meant by technical debt. –  Jarrod Roberson Jul 14 '12 at 0:01
4  
All projects I've seen where you put the responsibility of code quality in the hands of the product owner by creating "technical debt" stories or similar, code quality has always gotten prioritized so low as to seriously damage velocity and morale. I believe the team is the entity that takes responsibility of code quality. The team should estimate stories so there is room for delivering with preserved or, if necessary, lowered technical debt. –  Buhb Jul 15 '12 at 7:06
4  
"Refactoring story" or "Technical debt story" should not happen. Never. The cost of refactoring is part of the development and should not be a separate task. It should be done continuously in small steps, not planned, after the fact, stories. I know this, our team tried the refactoring stories, it's bad. When you start refactoring 'on-the-fly', you will see that refactoring becomes a part of your coding stile and not a separate task to do. –  Patkos Csaba Jul 15 '12 at 17:12
1  
If you believe that the codebase won't be able to conveniently handle story X without significant restructuring / rewriting, then this restructuring should be part of story X and the work needed for this restructuring should be taken into consideration when estimating story X. –  Buhb Jul 16 '12 at 11:14

"Well-designed" is subjective

What does "well designed" mean to you? to the Product Owner? to the Customer?

Is "well designed" a goal of the product owner? a goal of the customer? or just You?

Is what you consider "not well designed" still meeting the Product Owners expectations and making the customer happy? Then that is pretty "well designed".

Good Enough and YAGNI

Nothing in most Agile methodologies speaks of "well designed", because any system that the Product Owner accepts the stories as complete and the customers believe it meets their requirements is "well designed".

It is expected that the developers are professionals and will pragmatically use best practices, appropriate designs and idioms to implement the features and stories.

If you are not factoring in the time to do things correctly that is a developer problem, if the Product Owner is demanding things in less time that these can be done, it is their prerogative to do this, and your responsibility to educate them about the consequences in the form of technical debt stories.

SCRUM

The Agile Methodology that can be written down, isn't the Agile Methodology." - Jarrod Roberson

SCRUM is supposed to be a framework of tools to manage the total lifecycle of a software product. It isn't supposed to be a rigid set of things, just a good place to start and hopefully improve on.

Most shops I have worked in have what are called Sprint ZERO, Sprints for the senior members of the team to sketch out an overall architecture or theme of the product.

Stories that are larger than say 20 usually get broken down until they are actually a few 3 - 5 point Stories. One of these stories is, "As a team we need to meet to discuss how we are going to design this feature, so that we will have as little technical debt as possible given the time allotted."

Generally

In general a "well designed" system, is Good Enough and follows YAGNI.

Agile, and SCRUM in particular as an implementation of Agile are more about how to produce a product as transparently as possible to the Product Owner / Sponsors.

It isn't about anything technical design/architecture wise. It is more a philosophy about how to approach delivering software that meets the customers needs and expectations. If there isn't what you call "well designed" parts, that is not a failing per se, as it is not a fundamental part of the philosophy.

share|improve this answer
2  
Is "well designed" a goal of the product owner: Not a direct goal. Indirectly, yes: well-designed means easier to debug and maintain. I had to spend weeks finding critical bugs (that crashed the application) in messy, badly designed code. –  Giorgio Jul 13 '12 at 22:49
3  
What a depressing answer. It basically guarantees mediocre software enveloping the planet like a grey goo. –  Robert Harvey Jul 13 '12 at 22:50
    
@Giorgio that isn't a goal, that is an implied expectation. –  Jarrod Roberson Jul 13 '12 at 22:51
    
@RobertHarvey I know you have seen the Big Ball of Mud video and read the Paper. This is real life, SCRUM in particular is an acknowledgement of BBOM and embraces it in methodology by accepting the entropy as part of the process and managing it by trying to document it ( technical debut ) and slow it down ( refactoring ). Yes, you should start as far away from total BBOM / Entropy, but in real life that isn't always the case. –  Jarrod Roberson Jul 13 '12 at 22:54
1  
OK, but you can't eat an "implied expectation." That's just hand-waving. "It will be well architected because I'm a professional and I know what I'm doing." (using my best Bill Murray voice) Yeah, right. –  Robert Harvey Jul 13 '12 at 22:55

We've been working with scrum for several months, and I want to share my experience concerning your problem.

A few months ago I was given a big piece to implement. I had all the specifications ready, so I had to implement new features accordingly. The task was to take about 5-6 weeks (as I estimated). I spent the first week working on design only. The task was a little complicated: there was a main object which, as I concluded from the spec, had 15 different states, and the UI was to have different behaviours for every state.

I designed the whole workflow, and DB structure and classes afterwards.

I don't see any other approach in my case. If I dived into coding at once, I would have got a big nasty mess at the end _ almost impossible to maintain and extremely difficult to make any further changes. But the changes came in the next 2 weeks, so I had to rework the code. This was easy now, with the initial thought-out design. This saved our time, money and nerves.

After this experience I'm absolutely sure that its worth making an acceptable design in the beginning, than hope that with some miracle you'll have it at the end.

share|improve this answer
1  
+1: Very interesting answer. Agile supporters would tell you that you should have broken up your 6-week story into smaller stories. I once was given a similar advice and I replied that my six 1-week stories would have had lots of dependences between each other: because even if I change my work plan, I cannot change the problem domain. I got no answer on this: agile often assumes that you can break up your work on small, independent stories, but in the real world this is not always the case. –  Giorgio Nov 4 '13 at 7:35

Hind-sight is 20-20. If you had the information at you disposal at the beginning of the project to figure the whole thing out and then write the code in a few weeks, I suggest you do it.

You are not giving enough credit to all of the insights you gained, the approaches that were tried and failed/had to be changed, and the increased ability of the client/users to provide requirements enough credit.

Why would anyone with a history of successful water-fall projects, switch to an agile methodology?

share|improve this answer
    
"Why would anyone with a history of successful water-fall projects, switch to an agile methodology?": very good observation (+1). I think the choice between waterfall and agile should be related to the kind of projects one does: for projects with poorly defined requirements that need frequent prototypes and in which a prototype will likely become the final product, agile can be appropriate. For a project in which the requirements are more stable and where robustness and stability are more important, waterfall (or some variation) is probably a better choice. –  Giorgio Jul 10 '13 at 19:25

You always need the bigger picture of what the end goal should be, and what features are intended to be implemented and when, before you start a project. Working on stories as atomic tasks is asking for trouble. You need to keep future requirements in mind as much as possible.

share|improve this answer
    
Is it a good practice to schedule a design user story? –  Giorgio Jul 13 '12 at 22:24
    
@Giorgio: That's probably unnecessary but the Product Owner should at least give his team an overview of the customer's expectations of the project, before the project starts, and an explanation of how it was broken down as it was. –  James Jul 13 '12 at 22:30
1  
If someone downvotes please give a reason –  James Jul 14 '12 at 2:38
    
Downvoters rarely take the time to explain why they downvote. I find it pretty annoying. –  Giorgio Jul 9 '13 at 22:55

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.