Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

After having had endless issues with INI-style configuration files and parsing them correctly (let along getting binary data right), a crazy idea crossed my mind recently:

What if an INI file like

# settings.ini
[Windows]
DirectorySeparator='\\'
PathSeparator=';'

[Linux]
DirectorySeparator='/'
PathSeparator=':'

Would rather be stored in the file system; each group is a directory, each key/value pair is a file, so

settings/Windows/DirectorySeparator (contains just `\`)
settings/Windows/PathSeparator      (contains just ';')
... you get the idea ...

Disadvantages:

  • Possibly slow due to IO work, but maybe file system caches help?
  • Running low of inodes on some file systems?
  • Group/Key names may only consists of characters which are also valid for directory/filenames, may be rather restrictive depending on the FS.

Advantages:

  • Arbitrarily deeply nested groups
  • No worries about escaping, binary data is easy. Just have each file contain the raw data.
  • Automatic locking, access control and whatever the filesystem provides
  • Easy to access with most programming languages, even by hand

I'm very tempted to try this in my next project, I wonder whether anybody else has experience with this approach already?

share|improve this question
    
That is almost exactly what the Windows registry does. –  Ross Patterson Feb 13 '13 at 11:39
    
@RossPatterson: Well except that the Windows registry doesn't actually use the file system directly but rather builds a hierarchical configuration file on top of that (according to this answer). –  Frerich Raabe Feb 13 '13 at 11:44
1  
@RossPatterson: the Windows Registry doesn't use the filesystem, it is a filesystem, but one that is not accessible using the normal filesystem APIs. There is/was a Linux filesystem driver for the Windows Registry that would allow you to mount the Registry just like a regular filesystem, and it turns out that when accessed in that way, the Registry is surprisingly pleasant to work with. –  Jörg W Mittag Feb 13 '13 at 11:53
    
This is like how the linux kernel exposes configuration and information in the /sys pseudo-filesystem. –  Bjarke Freund-Hansen Feb 13 '13 at 12:11
    
@JörgWMittag I did say "almost" :-) –  Ross Patterson Feb 13 '13 at 13:09
add comment

5 Answers

That would be super obnoxious. You:

  • Wouldn't be able to see them all at once
  • Wouldn't be able to place comments explaining why things are set the way they are
  • Wouldn't be able to have documentation on what commands do, or what possible options are right in the configuration
  • It would make backing up/version controlling your configs more obnoxious.

Using a library to store and read stuff in XML (ugh) or JSON or INI files is so easy there is no excuse to come up with other stuff.

Filesystems are meant to store and organize files. Don't abuse it and come up with a half-baked configuration alternative when there are plenty of configuration libraries available.

share|improve this answer
    
I'd also suggest a small database store (eg. SQLite). But then, you can't put comments in those, either. And correct me if I'm wrong, but there's no explicit support for comments in JSON files. –  detly Feb 13 '13 at 0:49
    
All this is true only if you do not use file system designed for that purposes. I easily can imagine some mounted FS with XML file backend (or easily exportable to it). NTFS for example allows you to associate streams where you can store your metadata (comments for example). –  c-smile Feb 13 '13 at 6:59
    
Agreed, this problem can be easily solved by using an actual library. I'd throw in a vote for YAML, it's a superset of JSON (familiar, human readable, etc), and it has implementations / bindings for dozens of languages. –  Daniel B Feb 13 '13 at 7:02
    
About seeing them all at once - I can just (possibly recursively) list all files, no? And backing up/version controlling the configuration doesn't seem difficult to me either. Just check in the entire directory tree and/or rsync the directory. Not much more difficult than checking in/backing up a single file. I agree with your second and third points though. The lack of 'meta data' is dissatisfying. –  Frerich Raabe Feb 13 '13 at 7:16
    
In fact, having a single (possibly big) file can be quite obnoxious with version controlling as well. Depending on the file format, you can get conflicts all the time if there is more than one party changing the configuration. Having finer-grained files (and relying on the locking which your operating system already provides) helps with that. –  Frerich Raabe Feb 13 '13 at 7:20
show 7 more comments

This is a perfectly fine way of representing tree-shaped data. A file system is a tree database, why re-implement one on top of it?

The most well-known implementation of this idea is the Windows Registry. Its main flaw is that it implements a filesystem alongside the filesystem and doesn't support the filesystem API, which means you can't use filesystem tools such as find or grep on it.

Elektra is a project which implements a global configuration key-value database similar to the Windows Registry in a portable manner. It uses a storage format very similar to the one you describe.

The Linux sysfs virtual filesystem which is used to both expose data about the kernel object graph and change settings also uses a "one-value-per-file" format and uses symlinks extensively to represent graph-shaped data.

The Gatling web server even uses filesystem metadata for configuration. Here's how you configure two virtual hosts:

mkdir www.example.com:80
mkdir internal.example.com:8080

Here's how you configure a redirect:

ln -s http://www.google.com/mail gmail

Here's how you tell Gatling that a file is a server side script, i.e. that it should be executed and not served:

chmod +x script

Here's how you tell Gatling to serve or not serve a file:

chmod o+r serve_this
chmod o-r dont_serve_that

Gatling appears to have been inspired by qmail.

share|improve this answer
    
Indeed, what Gatling does appears to be pretty much what I had in mind! So maybe it's not that bad an approach as the other answers make it look. +1 for mentioning that. –  Frerich Raabe Feb 13 '13 at 7:18
    
@FrerichRaabe: I added another example: the Linux sysfs virtual filesystem. –  Jörg W Mittag Feb 13 '13 at 11:50
add comment

In addition to an excellent answer provided by @whatsisname:

  • It won't be possible to make two elements of the same name;
  • It won't be possible to share the config;
  • It won't be possible to make custom order of elements;
  • Different file systems have different limitations as per file names. They also have reserved names, like CON or NUL for Windows;
  • Also, file systems may be case-sensitive or not.

I completely concur with the idea of using a ready-made library for working with configuration files.

Regarding your concern of storing large chunks of binary data, most database-driven config libraries provide with that feature. If chosen a library that doesn't, using a file/directory reference plus individual binary files seems to be the best idea.

share|improve this answer
1  
You usually can't have two settings with the same name in the same group of INI files either. As for sharing - wouldn't this just be a matter of symlinking or so? I concur with the other concerns you raise though, +1 for that. –  Frerich Raabe Feb 13 '13 at 7:17
    
@FrerichRaabe This is very correct. I was just thinking that OP suggested INI as a general example. Other approaches (e.g. XML) don't have this limitation. Symlinking is good within the same system or network, but what about sending it via email or, for instance, asking a question here at StackExchange? "I have a problem, here's my config..." –  bytebuster Feb 13 '13 at 9:34
add comment

This has in fact been done. For example, qmail control files are individual files that contain a single line with the configuration setting.

The fact that this is not a very popular method of configuration is supported by reasons given in other answers to this question.

share|improve this answer
    
+1 for providing an example of the project which does something like that. –  Frerich Raabe Feb 13 '13 at 7:21
    
Ah, yes. DJB is a big inspiration for Fefe, I believe he got the idea for Gatling's configuration system from qmail. –  Jörg W Mittag Feb 13 '13 at 11:46
add comment

Here are some considerations not mentioned so far, that may or may not be relevant, depending on the application you would want to use this for.

Integrity check of the configuration

With a single config file, take a cryptographic hash. Done. If it differs next time you read the file, then you know it was manually modified or got corrupted.

With a filesystem based config, you need to traverse the whole tree, reading items in a specified order before feeding them one by one to the hash. Not only the file content, but also the directory and file names. Quite some things to do.

Related to this,

Robustness

Even if you have integrity checks, then there is so much more that could go wrong within a file system. Just think about a user looking through the files, moving some files/directories to other locations, with his uncareful mouse movements and a nervous finger.

How would you handle a link to a parent directory that throws you into an infinite loop when you traverse it unchecked. With the right corruption that is not checked by your application, it may as well be rendered completely broken, requiring an uninstall, manual cleanup and a reinstall.

A backup of a corrupted config file is so much easier to restore.

Talking about backups,

Power users and administrators

What may they want to do?

  • Modify it in ways that are useful, but not possible with your builtin configuration dialogs.
  • Backup and restore of the config.
  • Export/import of the config.
  • Transfer the config to a friend or to their second machine.
  • Rollout a mandatory config to a large number of machines and users.

With a single, text based config file, everything you need is already built into the OS: Copy, move and rename of files, a text editor, copy and paste. The transfer is as easy as attaching a file to a mail.

With the filesystem, everything that is not provided by your application is hard to do. Manually modifying a text option? Better take care not to add an additional End Of Line. Modifying a binary option? Hmm, hex editor?

Transfer the config by mail? You need additional tools: Build a zip file and attach it. Then hope that the binary options inside won't trigger the mail virus scanner and that it gets through.

For all practical purposes, a filesystem based config is as good as a black box for a non-programmer.

EDIT: Cluster size

A separate file for each option may use quite a lot of hard disk space. When each 5 byte option eats a full 64KB cluster, and you have a lot of them... Do the math.

share|improve this answer
    
Thanks for these aspects, but I'm not convinced. Integrity check? Not relevant in practice, I think. The vast majority of programs I know doesn't even bother storing a cryptographic hash of configuration files. Robustness? I'd argue that having a user tinker with some text file which needs to adhere some specific format (say: an INI file) is no less dangerous. Easy to forget a quotation mark, or to forget escaping one. Backing up a file system subtree is hardly more difficult than backing up a single file. –  Frerich Raabe Feb 13 '13 at 22:34
    
As for the power user use-cases: I think those advantages you give for a single text file "Copy, move and rename of files, a text editor, copy and paste" apply to file system trees just as well. Except that you don't need a text editor in many cases, a simple "echo foo > settings/windows/yoyomode" may be enough. Copying files and directories around, which would be the equivalent of copy & pasting text, is a common operation as well. –  Frerich Raabe Feb 13 '13 at 22:36
    
@Frerich Raabe: "Not relevant in practice, I think." Mmhh, famous last words. ;) Wait a moment, there is one more point, I'm editing... –  Secure Feb 13 '13 at 22:51
    
@FrerichRaabe: We're all aware that it could work. We are all just arguing that your schemes disadvantages dominate its advantages. –  whatsisname Feb 13 '13 at 22:58
    
@Frerich Raabe: BTW, I'm not trying to "convince" you. You asked for pros and cons, and that's it. By all means, go ahead and research this idea, if you really want. A lot of success stories came from people who didn't listen to the frogs (stopsinning.net/frogs.htm). Oh well, a lot of failure stories as well, because they ignored the knowledge and experience from the past. But that's life. –  Secure Feb 13 '13 at 23:21
show 1 more comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.