Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

Due to data protection, I cannot discuss fine details of the work itself so apologies

PROBLEM CASE

Sometimes my software projects require merging/integration with third party (customer or other suppliers) software. these software are often in linkable executables or object code (requires that my source code is retargeted and linked with it). When I get the executables or object code, I cannot validate its operation fully without integrating it with my system.

My initial idea is that executables are not meant to be unit tested, they are meant to be linkable with other system, but what is the guarantee that post-linkage and integration behaviour will be okay? There is also no sufficient documentation available (from the customer) to indicate how to go about integrating the executables or object files.

I know this is philosophical question, but apparently not enough research could be found at this moment to conclude to a solution. I was hoping that people could help me go to the right direction by suggesting approaches. To start, I have found out that Avionics OEM software is often rehosted and retargeted by third parties e.g. simulator makers. I wonder how they test them. Surely, the source code will not be supplied due to IPR rgulations.

UPDATE

I have received reasonable and very useful suggestions regarding this area. My current struggle has shifted into testing 3rd party OBJECT code that needs to be linked with my own source code (retargeted) on my host machine. How can I even test object code? Surely, I need to link them first to even think about doing anything. Is it the post-link behaviour that needs to be determined and scripted (using perl,Tcl, etc.) so that inputs and outputs could be verified? No clue!! :(

thanks,

share|improve this question
    
With the aircraft simulators, chances are that the source code of the avionics package is available to the simulator builder, but under a strict non-disclosure agreement. –  Bart van Ingen Schenau Jun 21 '13 at 10:34
    
@BartvanIngenSchenau Without going into more details, I can confirm that in most cases, there is no source code. The total point for having the executables or object code is that you protect your IPR. Also, it does not require them to supply any hardware with it. And I can confirm this because of my existing knowledge and experience in training and sim. industry. I will be interested to find out if you have seen something different. –  hagubear Jun 21 '13 at 10:47
    
YMMV. In the field of consumer electronics, where I come from, libraries always come with source (even if you are not allowed to modify it), because there is just too much variety in hardware platforms and OSes to effectively provide binary distributions for all combinations. (let alone the problems with providing a binary for the proprietary OSes) –  Bart van Ingen Schenau Jun 21 '13 at 11:07
    
@BartvanIngenSchenau agreed on the consumer electronics bit.....I just wonder how to test these things... may be me being over ambitious and stupid, but what else to do if no source code is available? –  hagubear Jun 21 '13 at 11:46
1  
@hgubear: I don't have an answer to that, but my guess would be the same way that you test the OS or standard library: You only test it at integration-level and above and trust (or require proof of) its correct functioning. (proofs to be delivered by the supplier of the software) –  Bart van Ingen Schenau Jun 21 '13 at 11:51

1 Answer 1

up vote 1 down vote accepted

Treat the executable like a black box. Make sure you have a software test harness that you can plug the executable into and verify it meets your expectations. That test harness doesn't need to represent your full code suite. It just needs to handle the cases you care about.

Looking at it from a broader level, you've been given a library that purports to meet a contract or an API. You need to be able to verify that the executable will meet the contract in the manner that you expect it to. Since you don't have the source code to visually inspect the implementation, you need to consider something else. To me, this screams for having a test suite of unit / integration tests to validate behavior.

Simply assuming that the black box executable will perform as promised is a foolhardy assumption. If the executable is mature, then it's a less risky proposition but there is still some risk. Your end users / clients won't care if your software fails or if the black box fails. To them, it is one and the same. From their point of view, your software is at fault.

Have a test harness. Create a step in your acceptance procedures to run new executables through the harness. Ideally, nothing will ever turn up as wrong. I've yet to see the ideal situation in software development though.

share|improve this answer
    
good answer and sorry as I cannot upvote you yet die to low reputation...it seems like I have to sit down with the requirements document and find out an explicit list of acceptance tests for those exec. And .obj files –  hagubear Jun 21 '13 at 15:06
    
That's the path I would follow. Regrettably, mild paranoia is your friend in this case. Trust nothing that you haven't verified. –  GlenH7 Jun 21 '13 at 15:25
    
Another problem is that the customer can always argue "BECAUSE you are getting our source code, anything you derive from our source code or link to it is going to be our IP"...kills you from your PoV. –  hagubear Jun 24 '13 at 7:57

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.