Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I've recently been struggling with the concept of what file structure to use when creating save-files for a desktop applications. When I say "save-files" I mean something like *.doc files. Files that contain the work that the user put into the program and want to be able to load and resume their work at a later time.

Things like binary serialization are easy, though if and when the model for the applications gets re-factored or expanded, that breaks (I believe) the files saved with the older format. Things like XML and JSON are more resilient to future change, but I feel like I'm falling into the trap of using XML as my "golden hammer" so-to-speak. A database seems overkill for basically all of the applications I develop.

What are some common options, and I'm sure there's many, that I'm not considering? Can you describe when each of these options is best used and why its the best choice?

share|improve this question

marked as duplicate by gnat, Kilian Foth, Jalayn, Yusubov, GlenH7 Jun 7 '13 at 13:05

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

what do you mean "save-files"? – Saeed Neamati Jul 25 '11 at 13:32
@Saeed Edited the question to help explain what I mean. – Anthony Jul 25 '11 at 13:34
up vote 4 down vote accepted

Binary files don't have to break. You could have file readers for different versions of the file, or try to make future versions add to the existing data model rather than alter it. That way old files can probably still be read. You'd also probably want to have the file keep a "version number" so the file reader knows what version of the data model it's working with (guessing can go horribly wrong). I don't know what kind of data this is, but you could also use a simple strucutred text file, or something like YAML.

In general, some considerations (there are many more I probably can't think of right now) to make are:

  • How complex is the data? Some data formats, such as 3D geometry can be easily stored in very simple text files.

  • Will programs other than your own be using it? This could be a good reason to use XML and send other client progams a schema file so they can very easily read the schema and the data files.

  • Will the data model likely change in the future? This one can be hard to manage if it's not possible to get a clear idea of future changes. Again, XML could also mitigate this, but you say you want to avoid "XML Golden Hammer" syndrome, and if the file format is very simple, a version number on the first line might be good enough for a text or binary format of the file.

  • How big is the data file? If the file is very large you might want to compress the resulting output which will decrease disc usage that will increase loading time. How much does it matter to the end user?

share|improve this answer
Thanks for the feedback. I've considered keeping track of the "version" number inside the data file, though never hearing it from someone with more experience than myself, struggled with whether it was acceptable or not (or if there were other ways of managing the issue). – Anthony Jul 25 '11 at 17:26

Have you considered lightweight embedded databases for your file format, specifically SQLite or ESENT?

SQLite as a file format.

  • transactional
  • SQL queries
  • ORM friendly

Or perhaps ESENT (Windows only)

  • high performance
  • table based storage
  • transactional
  • already available on all modern versions of Windows; no deployment concerns.
  • libraries for .NET and C++
share|improve this answer
My experiences with databases is slim. Most of my work here lately has been with C# and I've been enjoying the features that LINQ-to-SQL provides even with MS-SQL CE, but every time I think about using it, usually it adds more complexity to the project than its worth. Would love to be able to work on something where its the "right" choice so-to-speak. – Anthony Jul 25 '11 at 17:23
You're going to have to define the format of your file somehow. A database provides you a solid language for defining the format of your files and for reading and writing your data to the format. You can create a separate database file for each 'document' with a file extension associated with your application. Anything simpler and you're looking at serializing your objects to binary files which will give you version headaches. I don't consider XML file storage to be simpler than a SQL database, especially when you consider transactions. – Ed James Jul 26 '11 at 8:22

Not the answer you're looking for? Browse other questions tagged or ask your own question.