Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I would like to access in read and write mode to a zip without decompressing it on disk: what options do I have?

I need to perform the usual IO actions reserved for a filesystem like reading, writing new file, deleting them.

If it's possible I would like to stick with the zlib library.

share|improve this question

closed as not a real question by Jarrod Roberson, Walter, Matthieu, Jim G., Mark Trapp Aug 31 '12 at 1:14

It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center.If this question can be reworded to fit the rules in the help center, please edit the question.

    
What OS are you targeting? and more importantly Why? –  Jarrod Roberson Aug 16 '12 at 6:30
    
@JarrodRoberson for convenience, if i don't have to unzip the file i can buy some time, cpu power and disk performance, for example in the notebooks the disk is pretty slow and in some netbooks the CPU is really low powered; avoiding the unzipping phase with a direct access to the zip can make my app more snappy and less dipendent from the hardware performance. –  user827992 Aug 16 '12 at 16:10

2 Answers 2

I'm not sure which operating system you're aiming for, but you can build your FS driver in FUSE for linux or WinFUSE for Windows, which allow you to write your own filesystem drivers in userspace. You write your code which translates filesystem calls to zlib calls, and you're good to go.

There are, however, a lot of caveats that you should be aware of. Keeping a zip-based filesystem means you're replacing disk storage for CPU activity, to unzip the files when needed. You would often be surprised, however, at how frequently data is accessed, and you should plan accordingly, so you don't end up re-extracting the same files over and over again.

For instance, Windows Explorer doesn't use only the file extension to determine a file's icon - it digs into the file to find an icon handler, and uses that, so an Excel file saved as XML, for instance, will still get the Excel icon. If you're implementing your own filesystem you should be aware of things like that, and maybe save a fast cache of the first 1k or so of every file so you won't have to extract it all whenever Windows Explorer wants a peek. (This example courtesy of Raymond Chen)

This is just one example, and I'm sure that other programs have their own idiosyncracies. Best idea is to start with a naive implementation and start testing it under real-world conditions, profile and see what triggers excessive re-extractions.

share|improve this answer
    
i don't want to implement my own filesystem, i just want a library that can give me commands like foo::listFiles(file.zip) to list all the files in a zip or foo::addNew(file.doc) to add some new files. Also i want to stick with zlib for just 1 reason: it's the most portable library i have ever seen, i want to target at least Windows, MAC and Linux all at once. –  user827992 Aug 16 '12 at 16:33

Virtual FileSystem via API

There are things like Zip FileSystem strewn all over the internet, this one was the first hit on a simple Google search.

There is also efforts to use zlib with boost::filesystem as well, don't see a specific working example, but if you do boost it might be worth a consideration.

Virtual FileSystem via FUSE at OS Level

If you don't have a hard and fast reason to do something in C++ specifically, FUSE has read/write Zip support on every major OS.

Why?

The only way to get rid of the "on disk I/O" is to move the file off disk into memory, and it would be much more efficient and cheaper to just put an SSD into the machine just to deal with these files until they become static and get moved back to spinning media.

Moot to Worse IOPS

Either way, all of this is pretty much moot; none of this will keep you from having to actually decompress, remove things and re-compress to write new files to the archive when a file changes. This will actually mean more IOPS, lots more, than having contents unzipped, work with them normally and then zip them up when completed. Zip just doesn't work any other way.

share|improve this answer
    
To your last point: You could possibly cache the edits, deletes and news until a "commit" of some sort, preferably in a persistent way (presuming a program, and not a driver). I know loads of times I would rather have pointed a program directly at a zip file rather than going through the hassle of unpacking and repacking. –  Max Aug 16 '12 at 6:41
1  
@Max you miss the point of the last section, there is no way to get around "the hassle of unpack and repacking", you can hide it, but it still happens. And it can happen in a very very degenerate way if done in a naive way. –  Jarrod Roberson Aug 16 '12 at 6:51
    
Ah yes, that is true. I simply meant from a users point of view... –  Max Aug 16 '12 at 7:01
    
this library (ZFS) looks old and discontinued, i'm thinking more and more about serialize the data directly into my executable, this ZIP thing is good for the user but it requires too much I/O. –  user827992 Aug 16 '12 at 16:40

Not the answer you're looking for? Browse other questions tagged or ask your own question.