Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

Assume you are using APIs from a vendor, how to make sure their API is working as expected?

My main concern is sometimes the vendor pushed the changes to their code and break the API, we want to have some sort of automatic software to test them continually. How to deal with this?

share|improve this question
    
Depending on the language, there may be tools that can help (I'm thinking Pex for C# libraries/APIs). –  Steve Evers Dec 21 '12 at 16:53
add comment

3 Answers

Short answer: you need a test suite for a third-party vendor API - so you will have to develop one.

Don't expect anyone else to do it for you, and don't expect a "magic bullet" for generating automatically the right tests.

Some things you could try additionally:

  • ask the vendor if they provide a list of "breaking changes" for each new release
  • ask them how they care about API compatibility/inform them that this is an important feature for you
  • check if the API provides specific testing hooks, logging output or something like that for parts which could not be tested easily either
  • wrap important API calls with your own logging code, writing input and related output of the API to a log file, this will make it easier to debug things if something unexpected happens
  • add assertions to API calls to check pre- and postconditions, so if a new release of the API shows up unexpected behaviour within your application, you get informed early by an error message

If these things work or not depends on who is your vendor and what kind of API you have in mind. An API which produces some inspectable output like files is much easier to test than an API which controls some physical device where you have to observe the behaviour of the thing to decide whether the API call was successful or not.

share|improve this answer
    
I totally agree, but it seems to me, that the questioner hasn't experience with testing units and doesn't know the scheme of work with them. I mean: find critical points, write test units, run all tests, debug, during debugging find new critical points, write new units, repeat the last 4 steps after every API change. –  Gangnus Dec 21 '12 at 9:17
    
@Gangnus: IMHO the OP did not tell us anything about his former experiences with unit testing. If he has issues with that, I am sure he knows to ask a more specific question. Furthermore, the topic here is not "unit tests", but "automated integration tests". Those require typically a different scheme than, for example unit-testing in a TDD style manner. –  Doc Brown Dec 21 '12 at 10:25
    
Yes, he hadn't said that. But if he asks "how to make sure their API is working as expected" not mentioning test unites, "it seems to me", that he doesn't know them. As for automated integration tests, he doesn't know them even with higher probability, he mentioned merely "some sort of automatic software". You are awaiting from the people the same knowledges as yours, but in these themes 99% of programmers (including me) know far less. and 90% far far far less. –  Gangnus Dec 21 '12 at 11:06
add comment

Based on the poster's phrasing, it's more than just testing, IMO. After you write your unit test for the API and make sure everything is working as expected, you need to monitor third-party APIs so you catch issues before the users do. That's the real risk with third-party APIs - it's not your code and you have no control over how much testing was done on the API or when/if it changes.

(Disclaimer: product names used here) If you use soapUI to write your API tests, those tests can be re-used in AlertSite as an operational monitor to make sure the API keeps working as expected. If it fails the test, you can get alerted before your users call you and complain that your app doesn't work.

share|improve this answer
add comment

Implement learning tests for your area of interest (features that you plan to use). Learning tests are integration tests that are written by the developer against the public contract of the API. The tests should not be written against the internal implementation details even if the source code for the API is available. This kind of learning tests serves two purpose -

  1. It dramatically improves your understanding of the third-party API.
  2. The tests help to verify whether the claimed new version is actually backward compatible or not.
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.