Handle complexity in large software projects

Question

I am a lead developer in a larger software projects.

From time to time its getting hard to handle the complexity within this project. E. g.

• Have the whole big picture in mind all the time
• Keeping track of the teammates work results
• Doing Code Reviews
• Supply management with information etc.

Are there best practices/ time management techniques to handle these tasks? Are there any tools to support you having an overview?

Answer 1

The first and most important advice I can give you: Give your working week a structure.

That's how I do it:

Mondays:    Code Review and Functional Quality Assurance for things that have passed QA
Tuesdays:   Preparing Reports for Management and Stakeholders (Code Metrics Development over time, status of current stories/use cases)
Wednesdays: Put functional requirements for a use cases/story in a form that the team can understand (technical spec, column mappings, mandatory fields, ui, implementation design)
Thursdays:  Code a little bit, provide support for functional and technical issues
Fridays:    Code a little bit, provide support for functional and technical issues
Daily:      Ask each developer about status and impediments

I work extensively with Indian Developers at the moment who face real problems coding against a database that is in a foreign language. Also, they have a very hard time understanding the functional requirements. I try to give them technical specifications and encourage them to learn the business. I also found out the hard way that I need to monitor them each day and every day in order to keep code quality high. I get a glimpse on their commits every day in order to kill bad/wrong code and structures as early as possible. I also do detailed code reviews every monday.

Doing Code Reviews

As the code base grows larger, you cannot look at every single file. But for Java I use code quality metrics tools like: sonar, dependometer, ncss, checkstyle and findbugs, load the xml file results into a custom excel file to see the development over time (counts of issues of a certain type). If they reach a threshold, I identify the main source and talk to the developer(s). With these metrics, I can easily spot bad coding practices. Also, I review the code quality of the more important parts manually each monday. I have a custom script to run them all at once and also a custom build setup.

Keeping track of the teammates work results

Supply management with information etc.

Ask the developers daily about the status and current impediments. Those conversations should be short. Once a week you check the stuff marked as quality checked from QA and prepare a document for the stakeholders that includes:

• Current state of work items
• Code Quality Development over time (as graphs)
• Next steps
• Current impediments

Further, I always supply them with a document about our coding conventions and a document called "Definition of Done" so every developer knows when something is done and does not bother me to review something that has not been finished properly. The latter contains stuff like "Passes all the tests and statistifies acceptance criteria, code is xhtml compliant, build compiles without warnings and errors etc.";

But perhaps the most important tool for this is a ticket management system, like jira or trac to always have a good overview over the current state.

Have the whole big picture in mind all the time

That's tough but doable. Just have the big picture of the architecture in mind. In fact, you should've created it in the first place. When developer's ask me questions, I mostly consult my specification or my prepared Story/Use-Case documents as well as recent customer communication. Tools like mind maps help a lot. I also use UML every now and then to get a good glimpse on the big picture. But I do not go very much into detail with UML. I use it in a unconventional way, to communicate functional requirements to developers and for myself to lookup. I also use it to get a picture of the domain classes used and their relationsships.

See also this answer of mine for further information: Standard practices for an architect

Answer 2

In addition to the great answer by @Falcon I suggest you consider the project you work on is probably not large, as you have discussed it as if one person can oversee the entire project from the minute detail of code review to management reporting of the entire project.

There is a concept of "Thinking in the small" - where you have to remember everything about everything to make progress. If you can, it is a small project. The next shift to "think in the large", where you know you cannot keep it all in your head, so you don't try. There is not real middle ground, and you appear to have outgrown the "Thinking in the small" model. This requires quite a different approach to manage it and be successful.

The "big picture" is incompatible with detailed code level. Don't try - keep the high level data structures, high level concepts and interfaces in mind. No need to worry about the code if these are clearly understood. When working on code, built the knowledge and then "forget it" (As in, do not try to remember it).

If you really want to succeed, don't be involved in every detail - such as every code review. Let the team do the reviews. Initially, you sit in on a few to monitor and maintain the standards - but as time goes by, this should be delegated to one or two developers who have proven they are up to it.

Don't monitor every workers output every day. Keep an eye on things for sure - use reports generated from defect tracking and project planning tools to pickup outliers. Focus in teh exceptions.

As a rule, everything to need to know should come out of reporting. If code reviews are done correctly, a report comes out of each task with thinks like Defects (major/minor and route cause) listed and it's trivial to get a report broken down into project / developer/ months work / module etc. You then focus on the outliers.

Let go and delegate.....

Answer 3

It depends upon your responsibilities, which vary between companies when someone is assigned the "lead developer" role. If that includes management activities, like schedule, cost, customer, task assignment, approval....then you are going to have to learn to give up a lot of getting down and dirty in the details. You can try, but you'll end up doing a lousy job in the end.

On the other hand, if you are the technical lead then you absolutely need to know the details. Of course the size of the project affects how much of the details you can know and do. If the project has more than 4 or 5 developers then you probably won't be able to assign much (if any) modules to yourself. After that, then you need to pick your priorities for what you should focus on.

Usually my preferred priorities in reviewing/editing work is:

 1 - Common Code (If everyone is using it then it better be right)
 2 - Inexperienced/less skilled developers work

 3 - Then it really varies from there depending on the project, my confidence in other developers work and how important specific modules are to the project.

I take the approach that if I am the technical lead, then EVERY ASPECT of the design, code and testing is my responsibility. So while another developer may have done the work, if I approve it then it is as good as if I did the work. However, you very well may not have time to review every single aspect, so you have to use your judgement on what level of detail you have time to learn. For example, there are times when the best I can do is verify a module's interface, but not the inner details of the implementation. That may or may not be good enough depending on the project and the developers assigned to the project.

The above kind of answers your question in a round about way, being that it might not be possible for you to understand all the details. However, to more specifically answer your question, nothing helps understand a system better than coding or doing the work yourself. Unfortunately, as the lead, you probably won't have time to take on many substantive tasks yourself. Thus, what works for me, is that I have no qualms in making use of our CM system; I go in and modify the code to meet my level of standards. Sometimes that results in quite a rewrite/redesign. If that ticks off the original developer, then so be it; but they should have done a decent job that fit with the overall scheme anyways. The bottom line is that by giving yourself permission to modify the code, you can ensure that the original system vision is adhered to and that the quality of the code meets your standards and in doing so you should be able to grasp most of the details of what is happening in the system. Of course it helps immensely if you can count on one or two other developers that you don't have to focus too much attention on.