Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I have an program that collects data from other servers. I don’t have access to these servers so I can’t change anything. The servers can give incorrect data, I can solve the problem but I’m a little bit stuck on “when to do the correction”.

I download the data from the server, the data gets processed and stored on the hard-drive. When the same data is needed an there is no change, then I use the old data. So this is basically caching.

There are 2 places where I could apply the correction.

  • When I receive the data from the other servers;
  • After I cache the data. So right before I use it.

When I apply the correction on the downloaded data directly, then the server doesn’t need to do much calculations. It’s only corrected once.

When I apply the correction after I took it from the cache or when isn’t cached after I cache the data. Then I need more CPU power. But this gives me the advantage to access the original data at all time. I could fall back when I made a mistake and apply a correct correction.

So the question is, when should I apply the correction? And why should I do it that way.

share|improve this question

1 Answer 1

up vote 2 down vote accepted

This is going to be a question of performance and dependent on your expected application usage. If your application is going to be downloading the data infrequently but distributing it to users frequently I would do my calculations when writing the data to disk. If you are going to be downloading a lot of data sets from the servers which may never be accessed by a user I would move the processing to when you are getting the data from disk / from a cache.

Quick note here:

When I apply the correction after I took it from the cache or when isn’t cached after I cache the data. Then I need more CPU power. But this gives me the advantage to access the original data at all time. I could fall back when I made a mistake and apply a correct correction.

There is nothing that says that you couldn't keep both a raw copy of the data as well as a modified one while applying the correction at the time of downloading the data.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.