Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I am writing an application that works with satellite images, and my boss asked me to look at some of the commercial applications, and see how they behave. I found a strange behavior and then as I was looking, I found it in other standard applications as well.

These programs first write to the temp folder, and then copy it to the intended destination.

Example: 7zip first extracts to the temp folder, and then copies the extracted data to the location that you had asked it to extract the data to.

I see several problems with this approach:

  1. The temp folder might not have enough space, while the intended location might have that much space.

  2. If it is a large file, it can take a non-negligible amount of time for the copy operation.

I thought about it a lot, but I couldn't see one single positive point to doing this. Am I missing something, or is there a real benefit to doing this?

share|improve this question
    
If the application requires it, do it. Otherwise, do what makes sense. Your brain is telling you that its not needed in this case. I'm in the camp of doing the least amount of work possible as you can tell by my answer so I recommend doing the least amount of work possible for the app. –  Jason Sebring Jun 16 '12 at 19:40
    
Are you sure it is "copying" the file and not "moving" it? Big difference. –  frankc Jun 29 '12 at 20:00

2 Answers 2

up vote 75 down vote accepted

A few reasons I can think of:

  • On most platforms, file moves are atomic, but file writes are not (especially if you can't write all the data in one go). So if you have the typical producer/consumer pattern (one process produces files, the other watches a directory and picks up everything it finds), writing to a temp folder first and only then moving to the real location means the consumer can never see an unfinished file.
  • If the process that writes the file dies halfway through, you have a broken file on your disk. If it's in a real location, you have to take care of cleaning it up yourself, but if it's in a temp location, the OS will take care of it.
  • If the file happens to be created while a backup job is running, the job may pick up an incomplete file; temp directories are generally excluded from backups, so the file will only be included once moved to the final destination.
  • The temp directory may be on a fast-but-volatile filesystem (e.g. a ramdisk), which can be beneficial for things like downloading several chunks of the same file in parallel, or doing in-place processing on the file with lots of seeks. Also, temp directories tend to cause more fragmentation than directories with less frequent reads, writes, and deletes, and keeping the temp directory on a separate partition can help keep fragmentation of the other partitions down.

TL;DR - it mostly boils down to atomicity, that is, you want to make it so that (at the final location) the file is either complete or not there at all at any given time.

share|improve this answer
11  
If the temp directory is on a separate partition, you lose the atomicity. –  yfeldblum Jun 16 '12 at 16:43
13  
Some programs will extract/copy to the intended directory, but they will use a temporary file extension (e.g. .tmp) and rename it when complete. –  Dan Diplo Jun 16 '12 at 16:43
    
@DanDiplo: On Windows, it probably doesn't matter either way... –  tdammers Jun 16 '12 at 17:02
5  
Sometime writing to a temp file is also useful when several instances of your program could try to upload the same file at the same time, and you cannot rely on the file system to provide the right locking level (like poorly-defined cloud storage API). Using temp file will make sure the resulting file won't end up with a mix of data from both uploads. Of course this is just another instance of atomicity. –  Krzysztof Kozielczyk Jun 16 '12 at 19:18
1  
Dan Diplo's case is useful when updating an existing file. You don't want to replace the old one until the new one is successfully written, lest the write of the new one fail or something else read the new one before it's fully written. –  RalphChapin Jun 18 '12 at 14:29

This seems to be an issue in Windows, more specifically related to how the drag-drop is managed.

The developers of the WINSCP client have developed their own shell extension, which overrides this drag-drop behavior and allows dropping the file to the right folder immediately. They explain the trick in their documentation and, more interesting, what is the problem and how they solved it.

Here is the interesting part:

Windows drag&drop mechanics does not allow source application of drag&drop operation to find out easily, where the files are dropped. It is up to target application (Windows Explorer usually) to transfer files to destination. It is rather reasonable, because source application can hardly transfer files to all possible destinations. Keep in mind that you can drop files not only to a directory, but even to ZIP file (or any other archive), remote directory (via FTP, SFTP, SCP, …), trash, …

Obviously even Windows Explorer (or any other target application, like WinZip) cannot download files from any possible source (particularly it does not know SFTP/SCP).

Also, specifically for 7Zip: user ray023 answers this question in the SuperUser Stack Q&A: http://superuser.com/a/422463

Basically, if instead of drag-dropping your file you use the "extract here" method available in bith 7-ZIP and Winrar, the files are directly extractly to the right directory.

share|improve this answer
1  
The question was not "Why does 7zip use temp files?" The actual question was "Why are temp files commonly used in software development?" –  Phil Jun 16 '12 at 20:22
    
@Phil The example given is, from what I understand, what happens when you drag one file from an archive into a folder on the system, without explicitely asking to "extract it". Yet, I re-read the question, and yes, I think you are right in that the author wanted to know more about the utility of temporary folders, which by the way was masterfuly answered by tdammers (+1 voted by me too) –  Jalayn Jun 16 '12 at 23:11
    
While your answer wasn't exactly what i was looking for, +1, for solving my problem with 7zip –  Devdatta Tengshe Jun 17 '12 at 11:42
    
@DevdattaTengshe thank you –  Jalayn Jun 17 '12 at 12:33
1  
Good comment basically boils down to: why does a program use a temp folder? Because when drag and dropping it doesn't know the destination folder and just "gives it" to windows, then windows takes over and puts the file in the right place. –  Pieter B Jun 29 '12 at 8:06

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.