Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

A question that has been vexing me lately has been about how to effectively test (end-to-end) features in a distributed system. Particuarly, how to effectively manage (through time) test data for feature testing.

The system in question is a typical SOA setup. The composition is done in JavaScript when call to several REST APIs. Each service is built as an independent block. Each service has some kind of persistent storage (SQL Server in most cases).

The main issue at the moment is how to approach test data when testing end-to-end features. Functional end-to-end testing occurs through the UI, and it is therefore necessary for test data to be set up before the test run (this could be manual or automated testing). As is typical in a distributed system, identifiers from one service are used as a link in another service. So, some level of synchronization needs to be present in the data to effectively test.

What is the best way to manage and set up this data after a successful deployment to a test environment?

For example, is it better to manage this test data inside each service? Or package it together with the testing suite? Does that testing suite exist as a separate project?

I'm interested in design guidance about how to store and manage this test data as the application features evolve.

share|improve this question

2 Answers 2

Trying to provision for a widely distributed system is a hard thing to do.

Assumption 1: You don't have control over the databases at the level where you can reset them readily. Institutional test databases tend to be used, often heavily, by a number of separate development/maintenance groups. Hence, you cannot wipe/restore all of the databases without extensive coordination.

Assumption 2: You have support from the data stewards of each repository for necessary testing; hence, you can call on data managers to work with you on data sets. The data manager might allocate a range of test ZIP codes, or usernames, or similar.

Assumption 3: You are able to manage the data sets you are working with in each repository domain. For instance, in a contact database, you might be able to add an address of "1234 Testers Lane", or, perhaps "Suite CamoTester1, 1234 Birch Street" when ZIP code checking is being done.

Assumption 4: You are able to establish clear mappings between systems that exchange data. While not completely necessary, it will leave holes in your testing if you can't arrange this.

Set up provisioning sheets (MS Excel is your friend) with worksheets for each repository. You may end up with hundreds of data cases for each repository. Nobody said it would be easy! Now you have to come up with a way for your data not to explode combinatorially on you -- see assumption 4. Take data from sheet 1 for repository 1, determine if it is a source or sink for repository 2, whether it needs to coordinate with repository 2. Continue on down the line.

I have experience setting up testing for the connection between two ERP systems. Likely, the size of the problem was similar to what you are facing, although yours is more complex.

share|improve this answer

If your prod, dev, and test environments are seperated then this should be relatively easy. The database should also be in test, which allows you to import a known dataset into the database for testing. Instead of import, you could restore a db snapshot from backup before starting test.

In the past, I have bundled test data in the application test suite. This is either a set of SQL scripts to insert data or a db backup to be restored. My test suite was a seperate project but was always matched the application release. So, I would not use the v1.7 tests against the v1.9 app, unless I explicitly wanted to simulate an upgrade scenario.

share|improve this answer
    
In a distributed application, however, I have many, many databases (and other storage). Are you suggesting that I would have db backups for all of them? There could be 20-30 databases involved. If so, what are some ways to keep the test data in step with the application as it evolves? –  Davin Tryon Feb 25 at 23:55

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.