Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

Say you have this task in an application that consists of two parts:

1) Delivery of a file to an external system (not in your control)
2) A database transaction in our system, with information about said delivery (what was sent, when and by whom)

If any of these two parts fail, our system's state will be inconsistent. If a file has been sent and the db has no information about it, it's an error. Likewise, if the file transfer failed, but the information in the database suggests otherwise, that is also an error.

How would you approach this task to ensure that the database is left in a consistent state after each time the task is attempted?

I'm inexperienced in dealing with problems like this in a proper manner - I could always use the naïve approach and pretend writing to the database will never fail. :-) I wish to do better than that, though.

The two basic variations I see here both leave something to be desired (pseudocode):

A:

try:
    commitTransaction()
    deliverFile()
catch TransactionError:
    // No problem, delivery was never attempted.
catch DeliveryError:
    // The system will now say that it was delivered, 
    // even though it wasn't.

B:

try:
    deliverFile()
    commitTransaction()
catch TransactionError:
    // The system will now say nothing was delivered,
    // even though it was.
catch DeliveryError:
    // Easy enough to handle:
    rollbackTransaction()

I'm sure there is an acceptable solution to this, and that I just can't spot it?

share|improve this question
    
I'd appreciate help in giving this a good title and proper tags. –  perp Mar 28 '11 at 18:16
    
do you have any way of determining if they received the file? If not, then the best you can do is to track the "sending" of the file. I know when I try to send a letter whether or not I put it in the mailbox, not whether it actually ended up in theirs. –  Morgan Herlocker Mar 28 '11 at 18:30
    
Knowing that deliverFile() succeeds is good enough in my case. It submits the file to a third-party messaging service's message queue. If the service accepts the submission, successful delivery is "guaranteed"... or so they say. I have no choice but to trust them. –  perp Mar 28 '11 at 18:41
    
In other words, knowing that the letter has been put in the mailbox is enough. –  perp Mar 28 '11 at 18:49

2 Answers 2

Read about the Two phase commit protocol, and the three phase to get an idea how commercial database systems tackle the problem you are describing. Then you can use that knowledge to model a solution.

Regardless, you will need some way to perform some kind of hash or CRC on the data at the external system, otherwise you will never be able to reliably resolve the state of the system. If you cannot do a file integrety check on the remote machine your program challenge resembles the Two Generals' Problem

share|improve this answer

You need to have two different status. If you have tasks A and B. Your status should indicate if just task A was completed or if task A and B were complete.

So in place of:

deliverFile()
commitTransaction()

You should have something like:

deliverFile()
setStatus('file delivered')
commitTransaction()
setStatus('complete')

This way if the job fails half way through or at the beginning it is clear.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.