Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

We develop small to large visualization projects for different tasks and industries and sometimes while rewriting them a couple of times in the process we hit walls because we discover that we need to add a lot of code to support new requirements. Now we have established a design process that seems to work well (at least we reduced the development time for each new project quite a bit), but we're still left scratching our heads around this question: what exactly should we test when testing visualizations?

  • If everything that we want to explore is on the screen (bounded visualizations)?
  • If the data is ok - if data is valid (that's one of the nice things about visualizations you can spot errors in your datasets)?
  • Usability?
  • User interaction?
  • Code quality?

I can tell you for sure that a simple check of the code quality is certainly not enough! Is there a classic paper / book about how to test visualizations? Also do you happen to know about classic design patterns for visualizations (except the obvious ones like Pub-Sub)?

share|improve this question
up vote 1 down vote accepted

What are you "testing" for? Are you testing for accuracy of the visualization, in other words, whether the visualization matches the system it describes? In this case, consider a View Model: an object structure identical to the visualization. Write code that compares it to either (1) the view model to the system it describes or (2) a known correct visualization for a certain system and repeat for as many different visualizations as required.

If you want to test the visualizations themselves, specifically how they appear on the screen, this is more challenging. You can "screen scrape" (compare the bitmap representation) but this could be foiled by changes in rendering engines between OS or library upgrades among a number of things. Depending on the graphics library and operating system used, a GUI testing library may be available to help. See for a list.

Are you testing whether the visualization is at the appropriate level of abstraction or whether it makes sense to customers? This is not really "testing". This is more UI or visualization design and is a much bigger topic.

share|improve this answer
You correctly identified some of the challenges. I really love your answer. Can you provide more details about "GUI testing library"? Did you used something like this? – paxRoman Sep 14 '12 at 17:05
I guess you are talking about something like this: . Thanks for the idea! – paxRoman Sep 14 '12 at 17:07
@paxRoman There are many, many such tools and I have added some details to the answer. I have used Selenium and WatiN for web work before but nothing for native GUIs for several years. – akton Sep 15 '12 at 1:15
thanks! we would like to test visualizations in the browser - therefore we really need to check for browser bugs and we need a normal Web testing environment... – paxRoman Sep 15 '12 at 10:07
@paxRoman Then consider Selenium ( or WatiN ( Selenium ( can parallelize testing if needed, too. – akton Sep 15 '12 at 11:14

You could read through the excellent treatise The Grammar of Graphics, which elaborates on different properties of visualizations and how they convey or obstruct the flow of information to the observer. Although the book does not list a test plan for visualizations, it certainly brings to light to connection between code, model, media and perception. For me it was a most inspiring read.

In my opinion, visualizations are neither inherently right nor wrong, but there are certainly best practices. Eg., having an understanding of the perception of color will help you find good ordinal color scales.

share|improve this answer
Yes I read most of the important books and articles from Edward Tufte, Stephen Few, Leland Wilkinson, Jeffrey Heer, Mike Bostock, John Resig, and many others...including books about art and psychology by the likes of Rudolf Arnheim, Ernst Gombrich, Eric Kandel...None of them however insists too much on testing visualizations...:( exactly as you's like they're not good or wrong...but some of them certainly have bugs and we need to fix them and just classic software engineering doesn't apply because there might some other subtle bugs... – paxRoman Sep 14 '12 at 8:23

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.