<< Test Planning Summary | Test Cases >>

Test Preparation

<< Test Planning Summary | Test Cases >>

T e s t P r e p a r a t i o n

Test Scripting

There are several schools of thought to test scripting.

In risk averse industries such as defence and finance there is a tendency to emphasise scripting
tests before they are executed. These industries are more concerned with the potential loss from
a software defect than the potential gain from introducing a new piece of software. As a
consequence there is a heavy emphasis on verifiable test preparation (although observance of this
verification might only be lip-service!) And in some industries, external compliance issues (legal
compliance for example) mean that a script-heavy approach is mandated.

On the other hand, in Commercial-Off-The-Shelf (COTS) software development, a looser
approach is normally taken. Since speed-to-market is more important than the risk of a single
software defect, there is considerable latitude in the test approach. Specific test cases may not be
documented or loosely documented and testers will be given a great deal of freedom in how they
perform their testing.

The ultimate extension of this is exploratory or unscripted testing.

In this form of testing, there is a considerable amount of preparation done but test cases are not
pre-scripted. The tester uses their experience and a structured method to 'explore' the software
and uncover defects. They are free to pursue areas which they think are more risky than others.

Scripting, it is argued, is a waste of time. On a big project the amount of time spent on scripting
can actually exceed the amount of time in execution. If you have an experienced, educated tester
with the right set of tools and the right mindset, it would be more effective and more cost efficient
to let them get at the software right away and find some defects.

This concept is almost heresy in some camps.

I sit somewhere in the middle of this debate.

Preparation is essential. Some scripting is good but too much is not. Exploratory testing relies on

good testers and fails without them. But a lot of time can be 'wasted' in scripting methodologies by

writing scripts that will never find a defect.

I do think that 'exploratory testing' produces better (thinking) testers than script heavy

methodologies. In script heavy methodologies there is a tendency to believe the hard work is over

when the scripting is done. A monkey could then execute the scripts and find defects this is a

dubious conclusion at best.

But sometimes all you have are monkeys and it is a more efficient use of resources to allow your

experienced testers to script, and use the simians to execute.

In the end, don't let adherence to a particular methodology blind you to the possibilities of other

approaches. Train your testers in all the possibilities and let them use their judgement.

There is also an important legal aspect to this as Cem Kaner points out in his book "Testing
Computer Software". If you are responsible for releasing a piece of software that causes financial
loss you could be liable for damages. Further, if you cannot prove that you have conducted due
diligence through adequate testing you may be guilty of professional negligence. One of the goals
of test preparation therefore is to provide an audit trail which shows the efforts you have made to
verify the correct behaviour of the software.