Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

As a software developer, I've worked on projects ranging from tiny home-made apps to medium-size enterprise applications. In nearly every project I used a database or regretted choosing not to use it from the beginning.

Now, I am wondering a few things about the databases and their usage in general applications:

  • Why Windows itself doesn't use any "central" SQL database? For example:
    • Errors Reporting data is stored in a bunch of files,
    • Windows Update stores everything in flat files,
    • Icons cache is stored in a very strange single file which doesn't seem to be accessed through SQL, etc.
  • Why so many large applications avoid using databases? For example, wouldn't Microsoft Outlook gain by using a real database instead of reinventing the wheel by having its own format for .pst files and storing some data in registry?

If database adds an additional layer of an overall complexity and a tiny performance loss, it is a price of a huge advantage of making the code simpler in most circumstances, especially when it comes to the storage of small organized chunks of data instead of large binary streams. So why so few products are actually using databases? Probably the only application I know which actually uses Sqlite database is Firefox, and maybe Microsoft Exchange (but the last one is not a desktop application)?

Also, wouldn't a set of applications, like Microsoft Office or Microsoft Expression, benefit from having an unified SQL database, making it easier to deploy the applications, to update/upgrade the data, to share data between those applications, to make backups, etc.?

share|improve this question
4  
Speak for your own platform. ;) Many Mac/iPhone apps use CoreData, which commonly uses a SQLite database for storage. –  mipadi Nov 4 '10 at 14:44
7  
Databases are also just "a bunch of files". I'd argue that all the examples you presented are in fact databases, just not SQL-queryable relational databases. –  Allon Guralnek Nov 5 '10 at 21:27
7  
By database he means relational database. –  Olivier Jacot-Descombes Dec 22 '11 at 21:25

7 Answers 7

up vote 17 down vote accepted

1) Generally speaking the overhead of running a full RDBMS is too great and it would be adding needless load to the system and complexity.

Having one installed makes your life easier as a developer but makes the life of the owner of the machine worse as their machine is likely to run slower and with more issues. In developer vs. user confrontations the user should almost always win.

2) Many data stores have specific needs which are not met by something like SQL Server Express.

For instance, error logs should be written to the simplest possible thing to maximise the chance that the write will happen and the data will be available. SQL Server is never going to be that simple.

For more complex applications the argument tends to be more around optimising for very specific user cases - flat files can be lightning fast.

share|improve this answer
4  
I don't think he meant the kind of database where you need to install a whole server. SQLite works without a server and is widespread in open source applications. –  this.lau_ Aug 16 '11 at 8:56
    
@Laurent - if he didn't then why did he mark it as the accepted answer when he could have accepted the SQL Lite answer below? –  Jon Hopkins Aug 16 '11 at 9:37

Many applications embed SQLite. Quoting from their website:

SQLite is a software library that implements a self-contained, serverless, zero-configuration, transactional SQL database engine. SQLite is the most widely deployed SQL database engine in the world. The source code for SQLite is in the public domain.".

You just use one of the available APIs, create a database, tables, etc and the SQL engine will store everything in a binary file that you can put anywhere within the user's file system. Best of all, it's FLOSS.

share|improve this answer

Windows ships with and uses a database engine named Extensible Storage Engine (ESE).

ESE is used by Active Directory, Exchange Server, Windows Mail and Desktop Search amongst many other Windows services and applications.

For development, open source ESE wrappers exist:

With ESE shipping with every modern version of Windows you have no database deployment concerns. Architecturally ESE fills a similar role to embedded databases such as SQLite.

share|improve this answer

MainMa, I have noticed this too. I use to work for a company that has their applications running on DBISAM (it is a database for Delphi programs). I always pushed to move to SQL Server Express Edition for our clients – something free and quite easy to setup and always got nowhere. All I could surmise was that no one wanted to learn/support another database and spend time/money changing the products.
This always frustrated me cause 90% of the support calls came from data issues. Performance, volume of data, folder and file sharing issues all were attributed to the application having to many users doing things at the same time. A real database server is design to handle this. You see these same issues sharing an Access Database or excel spreadsheets over a network, etc. Those using embedded database with 1 user or just a few it is okay – but if your application is holding information important to your clients organization and will need to be shared by many (like financial, customer, inventory, etc ) put their data in a real database – it will save you many headaches down the road.

share|improve this answer

Databases offer flexibility and like every feature there is almost certainly a cost. If a flat-file will work, then you can assume no one is going to need to query the data in unanticipated ways like in an ad hoc report.

Databases also do a good job of handling multiple connections/inputs. Again, if this isn't a requirement for your desktop app, why add the additional complexity? Other than through the user interface/the actual installation of Outlook itself, how many other applications or users are trying to write to the Outlook file on your computer?

It would be easier if all data were in a relational database from a "I want to be able to query all data any way I see fit with a single language like SQL" perspective, but this is rarely a priority with many applications.

Most programmers will look at it from:

  • I know what the app is suppose to do, this is all it is suppose to do

  • my app is the only entitiy that will ever access and/or manipulate these data

  • a simple file structure of my own design will work

  • for at least one application I'd like to avoid relying on other people's code and have complete control.

share|improve this answer
1  
+1. You're right: there are no multiple connections for desktop apps (in most cases), which reduces the benefit of an SQL database. –  MainMa Nov 5 '10 at 2:19

Why Windows itself doesn't use any "central" SQL database?

Actually, this idea has been in development at Microsoft for some time. It is called WinFS. It is an intriguing technology, and it is available to MSDN subscribers. Some days soon, I'm going to install it on a VM and play around with it.

share|improve this answer
2  
I still remember WinFS being the "next big thing" for Windows (at the time) Longhorn, back when I still knew how to use a PC. :-) –  Denis de Bernardy Jun 10 '11 at 9:38

Relational databases are not the most efficient data structure for every domain. Lots of desktop applications do use embedded databases, you just can't readily tell they are doing it if all you see are a couple of data files.

share|improve this answer
    
That's the point: why not just using Microsoft SQL Server (Express? Or a special license for a single localhost SQL Server preinstalled on every Windows platform?) instead of embedded (not SQL?) databases or an own extraterrestrial binary format like Microsoft Outlook does? –  MainMa Nov 4 '10 at 13:27
    
Because as he said SQL isn't optimal for everything. Many application specific data stores will he highly optimised for one task and as a result will be faster than SQL with a smaller footprint. –  Jon Hopkins Nov 4 '10 at 14:02
3  
@MainMa: You've now just added an additional dependency: the user has to have SQL Server installed and running. This isn't trivial. If a special localhost SQL Server was preinstalled on all Windows computers, it might be different, but that doesn't happen. –  David Thornley Nov 4 '10 at 17:35
    
SQLServer Compact is easy to install: just copy a couple of files. –  MarkJ May 14 '12 at 6:09

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.