Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

A company produces software (and hardware) that is used to both perform automatic adjustments on electronic test equipment as well as perform calibrations of the same equipment. The results of the calibrations are put onto a certificate of calibration that is sent to the customer along with the equipment.

This calibration certificate states various conditions of the calibration, such as what hardware (models/serial numbers) and software (version) was used to perform the calibration, as well as things like environmental conditions, etc.

Making the assumption that the software used to produce the data (and listed on the calibration certificate) used on the certificate of calibration must have gone through a "test/release" process and must be considered "released" software - does this also mean that the software used for adjustment must also be released?

I believe that the method (software/environmental conditions/etc) used or present during adjustment doesn't matter, all that really matters is the end result of the calibration, the conditions present during the calibration, and whether or not the equipment was within the specifications.

The real question I'm hoping to get answered: Is there a reputable source (e.g. NIST or somewhere similar) that addresses this question? (I have searched...)

The thinking is that during high volume production runs, the "unreleased" system can be used to perform adjustments, as long as a released system is used to perform the calibrations, since the time required to perform the adjustments is much longer than the calibration. This unreleased system will eventually become released for use, but currently is not.

Also, please not that there is a distinction between "adjustment" and "calibration". The definition from BIPM International vocabulary of metrology, 2.39:

Operation that, under specified conditions, in a first step, establishes a relation between the quantity values with measurement uncertainties provided by measurement standards and corresponding indications with associated measurement uncertainties (of the calibrated instrument or secondary standard) and, in a second step, uses this information to establish a relation for obtaining a measurement result from an indication.

Followed by NOTE 2 (emphasis in original text):

Calibration should not be confused with adjustment of a measuring system, often mistakenly called "self-calibration", nor with verification of calibration

As a side note, I'm not sure why this got down voted. It's regarding software and it's use before and after release for use. I believe there is a best practice that can be applied and this is (hopefully) not primarily opinion based.

share|improve this question
    
If your question is merely "where can I find a NIST source," then it is off-topic here. We're not a search engine. –  Robert Harvey Oct 29 '13 at 19:26
    
I suppose I should have mentioned that I could not find an answer to this specific question utilizing search engines, or sites like NIST or A2LA. –  Steve Oct 29 '13 at 19:38
    
Regarding your edit: Yes, this is a software site. Nobody here is going to tell you to rely on unproven software for a critical operation. Just sayin'; you don't need OSHA or NIST to tell you that. –  Robert Harvey Oct 29 '13 at 20:32
    
Yes, but do you ever write a utility to automate generating, let's say, a lookup table for trig functions? The numbers in the table matter, not really the implementation of the sin() function used. And that utility doesn't always go through V&V and acceptance testing beyond your own unit testing, right? –  Steve Oct 29 '13 at 20:36
2  
Yes, it does. I never rely solely on my own testing, if the software is going to be relied upon for anything other than my own curiosity. I wrote the software, after all; I already know it works. :) –  Robert Harvey Oct 29 '13 at 20:40

2 Answers 2

It sounds like you are leaning solely on the results of equipment calibrations to serve as Verification and Validation for a piece of unreleased software.

If the only requirement in the software design specification for the unreleased software is that it must be capable of making adjustments to a piece of equipment under calibration, then that may be a valid approach. But seldom does software have only a single such requirement. If there are other software requirements, then I would say that this approach is not valid.

It is these kinds of rationalizations that eventually lead to problems like the Ariane 5 explosion, or the Therac-25 accidents, or sudden acceleration in Toyota vehicles.

share|improve this answer
    
The thinking is that during high volume production runs, the "unreleased" system can be used to perform adjustments, as long as a released system is used to perform the calibrations, since the time required to perform the adjustments is much longer than the calibration. –  Steve Oct 29 '13 at 19:37
    
Whether you can get away with it or not does depend in large part on the nature of the equipment being calibrated. If it is ordinary VOM's, there might not be a problem at all. If it involves the calibration of durable medical equipment or equipment that could cause loss of life or injury if it fails, you might run afoul of laws or government regulations. –  Robert Harvey Oct 29 '13 at 19:40
    
That's similar to an example I used - if we're calibrating a meter stick against a standard, does it really matter if a saw or a laser was used to trim the length of the meter stick? All that really matters is how long the resulting stick is. –  Steve Oct 29 '13 at 19:41
    
Ah, but both the saw and the laser went through a QA process. –  Robert Harvey Oct 29 '13 at 19:42
    
The saw didn't necessarily need to go through a QA process. I could have simply broke the stick to the right length - did my hand/arm system need to go through a QA process? Also - the example given isn't to accomplish verification or validation, but to temporarily support a production process. –  Steve Oct 29 '13 at 19:44

Adjustment is the process of re calibration. Calibration is the state of the systems accuracy and errors. From a regulatory framework, it is rare to worry how the adjustment was done, and common to worry about the calibration. Hence the need for calibration certificates. (It is common to worry that adjustment cannot be done after calibration, I will ignore that for now....)

A simple example might be the calibration of a fuel pump at your local service station. You care that is is calibrated correctly, as does the law. Typically its done by filling test bucket, but maybe its done putting a flow meter on the pump nozzle while filling a car. In both cases, the test equipment must also be calibrated (usually to a higher standard). However, if the pump is reading wrong and needs adjustment, it does not matter if this is done by a setting in software, turning a trim pot or replacing a sensor.

In a production environment it is not normally required (within a regulatory framework) that the adjustment process be rigorous, i.e. you can use whatever means you want in production to adjust, including unreleased automated software, provided the system is calibrated to the regulatory/commercial requirements. However, in my experience, the commercial risk and cost of unreleased software in high volume production lines means it just does not happen, as therefore the two tasks are often performed by the same system.

Note: Some frameworks with strict control (life critical systems etc) will have regulatory oversight of adjustment and calibration. In this case the answer does not stand.

share|improve this answer
    
if the pump is reading wrong and needs adjustment, it does not matter if this is done by a setting in software, turning a trim pot or replacing a sensor. -- I think it goes a little deeper than that. The question is not about the mechanism of adjustment; it's about whether that mechanism can be automated safely with non-certified software. –  Robert Harvey Oct 29 '13 at 20:30
    
@Robert is correct. Calibration is measuring how close the adjustments came to "true". How the adjustments got there might not matter. This second statement is the crux of the matter. –  Steve Oct 29 '13 at 20:33
    
Tx answer edited. –  mattnz Oct 29 '13 at 20:47

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.