Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

Reading SQLite source code is IMO mission impossible. Yet it is a usable piece of quite complex software (it's a full-blown embedded database after all) that can be downloaded, compiled and used from others code and it is constantly updated.

How do people manage to write and maintain such extremely complex and hard to read code?

share|improve this question

closed as primarily opinion-based by gnat, GlenH7, MichaelT, Giorgio, ratchet freak Apr 17 at 20:41

Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise.If this question can be reworded to fit the rules in the help center, please edit the question.

1  
I think, they don't :-D –  Pawka Oct 5 '10 at 10:06
2  
@Pawka: Where "they" doesn't include the SQLite people. –  Frank Shearar Oct 5 '10 at 10:34
10  
Even if the code is complex and hard to read, it may still be be relatively good code considering the complexity of the problem it solves. Writing simple and easy code that solves hard problems is often just impossible, no matter how closely you follow "best practices". Then it's even more important to make the code as simple and clear as it can be. –  Joonas Pulakka Oct 5 '10 at 11:05
1  
Has anyone ever seen a huge project that was easy to read? –  Jeff Davis Oct 5 '10 at 16:43
2  
Hiring interns, postdocs, etc and make them do it... –  DarenW Oct 7 '10 at 3:42

12 Answers 12

up vote 15 down vote accepted

There is a big difference between Complex and Complicated. The following link about sums it up. :)

http://codebetter.com/blogs/dru.sellers/archive/2009/09/25/complex-vs-complicated.aspx

On a more personal note, I work on a code base of more than 1 million lines of code. I have been in it from line 1 to its current state. The more farmilar you are (read longer you are in the code) the easier it becomes. I can not tell you what every line does, but I can tell you were to start looking for a given task or bug. It just comes naturally.

The moral of the story is that like any thing in the programming world it takes time. If you expect to look at SQLite and just know and understand it, you are kidding yourself. It is going to take time to figure out how everything works together. The difference between good code and bad code is how long that process takes.

Lastly, some developer do not have the ability to jump into a code base and start figuring it out. Many will feel overwhelmed at sheer size or architecture of the code base. Others will have no problem just jumping into it. It all depends on the developers strengths.

share|improve this answer

In the specific case of SQLite, the main tool they've chosen to use in development and maintenance is automated testing. They pride themselves on 100% coverage (branch coverage, not statement coverage) in their test suite. According to them, it's one of the best-tested software products in the world. So they know right away when something they've added or changed causes a regression, and can develop pretty fearlessly as a result of that.

http://sqlite.org/testing.html

Pretty staggering numbers--they have around 640 times as many lines of testing code as of production code.

EDIT: This question has been raised from the dead, it seems! And a little more than a year later, that same page reports they have 1177 times as many lines of testing code as production!

share|improve this answer
1  
No one else but me sees this as a problem rather than a point of pride? –  Stephen Oct 22 '10 at 21:40
1  
not really i assume. Database has to be reliable first of all. –  ts01 Oct 27 '10 at 20:09
5  
@Stephen, why would many test cases be a problem? –  user1249 Feb 14 '11 at 23:20
    
a waist of time? too many test is as bad as to little test (IMHO) –  Dainius Jul 13 '11 at 9:14
9  
How can a test be a waste of time if it covers a possible case where the code could fail? –  Ramhound Nov 15 '11 at 14:26
  • functionalities evolves over time.
    • new functionalities are driven by new customer requirements.
    • Old code was written in a way that allows future yet-to-be-implemented functionalities to be added without breaking old code.
  • domain experts (in this case, database is a well-known domain in Computer Science.)
  • community feedback
    • bugs
  • steep learning curve was not a problem for the well-prepared and persevere. It may take 6 months, 1 year or even longer to become comfortable, and that's what some people are able to put up with.
  • commercial motivations. without monetary support, very few people can invest the time and energy into the steep learning curve.
share|improve this answer

You do not need to have an intimate understanding of the whole project in order to be able to maintain it. Usually with large, complex software, people will have their own particular "areas" that they look after and they only have a 'passing' knowledge of the rest of the system.

SQLite is actually relatively small on the scale of "large software projects" but if you look at something like the Windows operating system, you'll have people who just work on the kernel, people who just work on the shell, people who just work on Internet Explorer, people who just work on the Window manager, etc etc. Someone who works in the "shell" isn't going to be able to fix a bug in the kernel at the drop of a hat.

There's also the benefit that these projects evolve over time: they didn't always start out this complicated. That means a new developer can usually be "trained" by more experienced developers.

When you join a large team of developers, you'll be given a particular aspect of the project to work on (maybe a bug or new feature) and you'll have another developer be you "buddy" for the first few iterations. Your buddy will have a good understanding of the area you're working and can help you find your way around.

For open source projects like SQLite, it's actually a little harder because there's no motivation for existing developers to "train" new developers. So you'll find you're on your own a bit more. But you can still find help on developer forums or mailing lists (e.g. just posting a question like "I'd like to implement such&such feature" or "I found bug XYZ, where do I start looking?" and you're likely to get some form of help.

share|improve this answer
    
Change the specifics, and this sounds a lot like my current job! The best part of this answer is the specialization and the need to develop over time. Also add in a pinch of humor about the whole thing. –  DarenW Oct 7 '10 at 3:44

There is a difference between hard to read and complex project. If a person doesn't understand deeply the language which the project was written or the domain of the project it doesn't means the code is badly written.

My advice:

  • Have sure you understand the language of the project. It's not enough know the language.
  • Learn every detail about the domain before put hands on code.

For me SQLite is far to be complex and nothing complicated. The domain of this project demands deep understanding about wide concepts.

share|improve this answer

One thing not mentioned in the other answers is "It comes naturally to them". Most people want to write good code but they are tuned to writing bad code. Some of the programmers I have met are naturally LINEAR thinkers + some times very wittingly they come up with complex solutions. Most of such people don't spend time reading or learning from the book they learn as and when the work demands it.

share|improve this answer
1  
LOL, I worked with soemone like this, if there were several ways to do something (and there always are) he would invariably choose the most complicated, convoluted solution. I don't think he was even aware of that. –  HLGEM Oct 5 '10 at 17:18

How do people manage to write and maintain such extremely complex and hard to read code?

If they are the original coders, and they keep maintaining it, they do not see it as "complex and hard to read". They probably see it as "simple and easy to read", since they wrote it themselves.

Any code takes time to understand it. It's just that some take longer than others :)

share|improve this answer

You can wrap your head around any code-base if you have - perseverance, patience and a methodical approach - but mainly perseverance :-)

share|improve this answer

Managing becomes a lot easier if there are things done always the same way. I do not know the SQLite code, but I'm working in an environment where there are multiple projects. Besides understanding the business case (eq logic), as everything is basically done the same way everywhere (database access, etc.) it's relatively easy to start working on another project. Long text short: Coding guidelines - in appropriate manner - make life and such code far more easy. This could be one of the helpers for the SQLite coders.

share|improve this answer
  • If you're the original author of the software, and you've worked with it for > 10 years, and you're a genius, then it may be possible to understand the whole thing. Big pieces of complex software (like SQLite or the Linux kernel) indeed usually have exactly one original author having a complete, deep understanding of it, even if other people have contributed.
  • If the software architecture is sane (high cohesion, low coupling), understanding the whole thing is not a prerequisite for making useful additions and modifications to it.
  • Understanding a RDBMS or OS are somewhat special cases, requiring deep understanding of the underlying CS principles. Not so with "ordinary" software applications.
share|improve this answer

They try to write it in a simple way but not a simplist way.

Going as much simple as you can makes things faster to understand/read, even if it can take some time.

share|improve this answer

The algorithm being implemented sets a lower limit on how simple the code for an implementation can be. If an abstract description of the algorithm to be implemented is extremely complicated and requires tons of different pieces of data to be coupled to each other, then the code implementing said algorithm can't be simple no matter who's writing it.

In other words, one reason why I sometimes write inscrutable code is because, given the algorithm I want to implement, I have no choice.

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.