Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I recently took on a contract job that will largely involve generating data feeds from my client's data and sending it to external partners, via feeds and API invocations.

I've always found this particular kind of coding to be frustrating, for a number of reasons:

  • You have to rely on specs, which often diverge from the actual implementation or simply don't go into sufficient detail
  • You have little to no ability to see into the external system, to ensure that your data has been accepted and interpreted properly
  • If there's any kind of problem with data you've generated, it's not always straightforward to back out the changes and try again
  • External testing and staging environments frequently diverge from production logic and the production database

What are some best practices for doing this kind of work?

Creating mocks and stubs won't necessarily resolve my problems, as my real concern is: do my feeds have the correct effect on another system, which is basically a black box? Mocks simply duplicate my assumptions.

share|improve this question
add comment

1 Answer

up vote 1 down vote accepted

I've done a bit of work in recent years with EDI, and it has all the problems you've mentioned. The biggest problem is that partners don't follow their own specs and when you point it out, they say, "this is a deviation" but they never document it.

Here's how I deal with it and manage to sleep at night:

  • Create a framework for building and maintaining lots of unit tests. At first you'll just test the spec plus your assumptions, but eventually (every time you find a deviation) you have to write another batch of tests to check those new facts.
  • Your system must log everything. It must log everything that happened, when it happened, who did it, what it received, what it wrote, where, etc. You will need this when you go toe-to-toe with your counterpart on the other end, not just to troubleshoot, but as evidence to prove that your system is doing everything it's supposed to do. Confirmations are particularly important to log. These logs are invaluable when you go to write new tests.
  • Build in the ability to retry with modifications. When something happens, you'll want to be able to tweak the data and submit it again.
  • Avoid intermediaries (aka "3rd parties"). Unfortunately in our case we actually have 3 different systems our data travels through between us and the ultimate destination (two of which are called "value added resellers", or VARs, which is complete BS because from my perspective they add no value at all). When something goes wrong, we get contacted by the end-user at the other end, and then we have to go up one level on our end, ask them to check, and then they go up a level and talk to their VAR, who has to talk to the other VAR, etc. It's frustrating. If you have the ability to communicate with the other system directly, do it.
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.