Update 04-May-2003:

Factors affecting test-case effectiveness

Figure 1.

There are 6 factors (in 3 groups) affecting test-case effectiveness: (A) Test-design phase factors - (1) Incorrect test specifications, (2) Incomplete test design, (3) Incomplete test suite; (B) Test-execution phase factors - (4) Test execution problems; (C) Test-planning phase factors - (5) Incorrect functional specifications, (6) Incomplete functional specifications.

All these factors will reflect on test-case effectiveness.

Test-case escape classification logic

Figure 2.

During the analysis of the test results one of tricks will be classification of test-case escapes based on the shown logic (Fig.2). This will allow to analyze not only the quality of the product being tested but used test quality and assumption quality during the test preparation.

By using algorithm shown on the Fig.2 all test escapes can be grouped into 6 groups corresponding to 6 factors above.

Test Escape Analysis

Figure 3.

On Fig.3 shown example of the test-case escape analysis. This is not a real example.

As we can see from the diagram, escapes does not always indicate product fails, but caused by either incomplete functional specification, or by incorrect functional specification.

Product has been developed based on improper functional description.

Test escapes in the example shows importance of the Requirements Analysis at the beginning of the project.

Given sample project should be restarted from the very first stages. Time is lost, and the designed project does not correspond to User expectation.