How to “back track”?

Question

I find that I start projects and, due to my lack of experience, find that old database structures and huge blocks of code are inefficient and memory-costly. However, by the time I realize a re-design of the entire project is needed, the project has grown to such a size that it is simply too late to go back and modify the project in its current state and requires a completely new project file and the whole shebang.

How should I prevent ruts such as this one, where it is too late to go back and modify the current project to fit specifications modified far down the road from the creation of the project?

(Apologies in advance for confusing grammar, it's been a long day here... as you can probably tell.)

Answer 1

There are a number of things that will help, but there's no silver bullet that makes reworking your project simple. In no particular order:

• Use version control. You should be able to go back to any revision of your project at any time. If you determine that you made a big design mistake two weeks ago and wish you could take it back, version control lets you go back.

• Avoid dependencies. If fixing your project is harder than starting over from scratch, it's very likely that the various parts of your program are far too interdependent. There are many patterns and strategies for making your code more modular and reducing dependencies between modules. Spend some time learning about good object oriented design (assuming you're working in the OO style). Isolating different functionality in separate modules means that problems should be better confined to specific modules, and improving your project means reworking a module or two instead of rewriting the whole project.

• Fix your code anyway. Fixing your code might seem like more work than starting over, but you'll probably learn a lot more by fixing your mistakes. It's hard to really understand problems until you have to fix them. Starting over may just lead to making the same old mistakes and needing to start over yet again sometime in the future.

• Review early, review often. If you don't notice problems until they're so broken that you have to abandon the project, you're not looking at your code hard enough, early enough, or often enough.

• Get help. You say that your lack of experience leads to bad decisions. Find some smart, experienced friends or colleagues who can help you make design decisions in the first place and review both your design and your code along the way. Explaining your work to other people can help even if your helpers aren't really all that smart or experienced. Other viewpoints are often helpful, and the act of explaining itself helps you see your work through someone else's eyes.

Answer 2

I seem to identify two common problems:

• Shifting requirements
• Bad database design.

What seems strange to me is that you have thorough performance problems which imply large rewrites- that is extremely unusual. If you are not profiling, when you find a performance problem it is common to try and solve the performance problems blindly, which is unlikely to work and very, very frustrating (the feeling you get is that you have rewritten everything, but things are still slow- this normally means that you haven't rewritten the important part). If you learn to use a profiler, you will find easily the true performance problems in your code and you will probably solve them easily.

Now, once that is solved, let's address the rest.

Shifting requirements are often unavoidable and often very costly. Database model design is vital.

What can you do:

• Learn and practise database modelling. Learn from your mistakes. Read some books. One I would recommend which might work for you is The Data Model Resource Book, which contains typical database models for business applications. It is not a substitute for modelling theory, but it will come in handy
• Devote more time to requirements gathering. Clients seldom can describe their data model in detail on their own- you need to help them- examples from the book I mentioned earlier might come in handy.
• Keep your database as normalized as possible. Normalized database designs are "simpler"- although they might involve extra tables, they tend to be much easier to refactor
• Use the single-responsibility principle, DRY... and other, least important software design principles :) It will drive you towards smaller, easier-to-refactor code.