Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I am creating an application that maintains a database of files of a certain type in a given folder (and all subfolders) Initially the program will recurse the folders and add any file it finds of that type to the database.

I want the application to have the ability to re-scan the folder and add any files that were not there the last time the folders were scanned.

It can't use the date created property of the file because there is a high chance of a file being added to the folders that isn't a new file.

I am wondering what the most efficient way of doing this is, and if there is a way that doesn't involve checking each file is in the database already (which, if there are 5000 files would mean 5000 queries of a list 5000 items in size, or 25 million 'checks' for the sql engine to perform)

I suppose a more specific question to acheive the same goal would be - is there a property of a file (in Microsoft Windows) that will reliably tell you when that file arrived in that folder.

Edit: The app would not be running all the time, so monitoring the folder for change events is not an option. A typical scenario might be. Run the app. get new files. close the app. A week later (after normal computer usage and files being added to the folder) run the app again, look for changes since the app was last used.

share|improve this question
2  
You could try monitoring the folder for changes and perform updates in near real-time, instead of trying to diff it. msdn.microsoft.com/en-us/library/… –  Robert Harvey Nov 30 '12 at 16:28
    
"I suppose a more specific question to acheive the same goal would be - is there a property of a file (in Microsoft Windows) that will reliably tell you when that file arrived in that folder." I don't think there is such an API, and it might not help anyway. How would you detect files that were deleted or renamed? –  Charles E. Grant Nov 30 '12 at 17:25
    
Which language? C# has msdn.microsoft.com/en-us/library/… If java, this could help: stackoverflow.com/questions/285955/… –  Luiz Angelo Nov 30 '12 at 19:14

1 Answer 1

Looking on your profile on StackOverflow, seems you're (or were) a C++ Builder developer, so I suggest to use the Shell API to get notified of modifications on given folder. If you go down this path, creating an service-like (or even a real windows service) program would be my path.

Edit: another way is getting a "hash"/"signature" calculated from the name of the files and some other attribute. So you calculate it each time you scan and compare with one you saved on DB.

share|improve this answer
    
That sounds like something that would require the application to be running all the time. Which it won't be. –  MrVimes Nov 30 '12 at 17:07

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.