Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

Was the suite of UI automation (called automation from here on out) stable? So were the tests deterministic? The requirement is if you ran the tests N times in a row you would get the same results N times in a row?

What was the goal automation testing exactly? Testing product overall? Stress testing the product: trying to see how long it could be poked until it crashed. Something else?

How did the automation go about testing? Did the automation drive a build of the product? Or did the automation use a custom test build of the product with the UI decoupled from the rest of the product? What were the benefits or drawbacks of either approach?

Did the automation use internal hooks or accessibility APIs to drive the product or perhaps something else? For the case of internal hooks did they skip the UI elements and simply fire the event handlers or use another approach?

Did the automation only verify the state of the UI or was internal state of the product also verified during the automation?

How long (wallclock time) and about how many steps did a typical test go through?

How did you solve the problem of "waiting"? Any non-trivial program will, at some point, need to wait for something to load from disk or the network or for something to compute. Presumably UI action can invoke this long running operation to happen. How would an automated test get notified of its completion so that the automation could verify the result and/or make forward progress with the test?

How much time was spent maintaining automation once it was in place?

Were the tests and test framework (if you wrote the automation framework yourself) more or less complicated that the feature/product you were trying to test?

Was the automation worth it overall? Was there too much time spent maintaining the tests? Were the tests too difficult to write in the first place?

share|improve this question

closed as not constructive by ChrisF Aug 11 '11 at 10:21

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.If this question can be reworded to fit the rules in the help center, please edit the question.

1  
Um, not a real question as it has no single answer, since it askes everyone for their personal experience –  Oxinabox Aug 11 '11 at 9:16
    
Please ask specific, answerable questions. –  ChrisF Aug 11 '11 at 10:21
    
You are asking a lot. It would probably be better to split this question into a few separated ones. Also, see Good subjective, bad subjective for advice on how to get the best possible answers. –  blubb Aug 11 '11 at 10:27
    
You should break this up and ask at sqa.stackexchange.com - don't forget to search first, since half your questions have already been answered there (and vote up the questions and answers!). If you are looking at writing UI automation and want something stable and maintainable, you should look up WatiN, Selenium, and the Page Object design pattern. Record-and-Playback requires less development skill but is more fragile. –  Ethel Evans Aug 11 '11 at 21:49

1 Answer 1

yes. yes. yes.

TDD. yes. no. no.

database-driven test-runner. yes. no. tested product just like a user.

no. no.

no.

a few minutes, and a few hundred.

callbacks. events.

about 12%

less.

yes. no. no.

ADDENDUM: not my intention to mock you, but to illustrate that this is not one question

share|improve this answer
    
First, I believe you answered no to internal hooks and accessibility APIs for driving the product. What were the technicals behind test driving the product exactly? Second, how were callbacks and events communicated to the test? For example if the test tells the product to load a document how exactly is the test notified the documented has finished loading or failed to load? –  nullcodename Aug 11 '11 at 8:13
    
Also, could you clarify the "no" answer to "Did the automation only verify the state of the UI or was internal state of the product also verified during the automation?" To clarify the question: what were the tests verifying? UI? Internal state? Both? –  nullcodename Aug 11 '11 at 8:17
    
Although Good subjective, bad subjective is concerned with questions, it still shows how you can improve this answer a lot. –  blubb Aug 11 '11 at 10:25
1  
@Simon Two downvotes is a small price to pay for having a sense of humor. This was a terrible question, and deserved no better. ;-) –  Steven A. Lowe Aug 11 '11 at 21:32
1  
@Steven: I agree, and I did find it humorous too. However, given that the OP is a very new user, your answer may give him the impression that mocking someone like this is an accepted practice in this community, hence my downvote. –  blubb Aug 12 '11 at 7:05

Not the answer you're looking for? Browse other questions tagged or ask your own question.