Code-Coverage Tool Finds Defects Before Deployment

Code testing is a haphazard undertaking for many software developers. It often involves running through the application's functionality a few times to see if it deals with the cases you've designed it to handle. Testing code in this manner can uncover many problems, but you'll have a hard time eliminating all the defects if you don't execute every line of code in the application at least once during your tests.

Rational Software Corp. recently introduced Visual PureCoverage, a tool that should take some of the manual work out of executing application code during the test phase of your projects.

Code coverage tests determine which lines of code are executed. This common technique helps ensure that all lines of code have been run at least once, which can reduce undetected defects before the product is deployed into the production environment.

Visual PureCoverage supports code coverage testing for Windows NT-based developers working in Visual C++, Visual Basic, and Java. Both Visual C++ and Visual Basic code must be instrumented, a process where Visual PureCoverage creates a copy of the executable program and its libraries and then inserts additional code into each module to monitor code coverage. You then execute the code-either standalone or within the Visual PureCoverage environment-and the coverage information is automatically recorded.

Java code is not instrumented because it's not compiled into a native executable file. For Java applets, Visual PureCoverage uses the Microsoft Java Virtual Machine to monitor and report coverage information back to Visual PureCoverage.

Although having a Java code coverage tool is fine for creating new applications, relying on the Microsoft JVM may cause problems for some developers. The release notes say Visual PureCoverage requires Build 2553 or later of the Microsoft JVM, which is later than the version distributed with Internet Explorer 4.0. A copy of this JVM is also included on the Visual PureCoverage CD.

When a program is executed from within Visual PureCoverage, it displays a module status window, which is continuously updated to show how many routines have been executed during the test. It also displays a call-distribution bar graph that shows how often certain routines have been executed during the test run. I used this window to determine when I had executed every routine in the program.

If you're not interested in recording coverage data for certain portions of the test, Visual PureCoverage lets you pause and resume collecting coverage information during the test. You might use this option, for example, if you already know you have good code coverage testing on the initialization part of your application. An API is also provided to let you incorporate pause and resume instructions directly into your code.

After completing the test run, Visual PureCoverage displays the coverage browser window, which provides statistics about the routines and lines of code executed during the test run. This view is organized hierarchically-by module and by routine-using a tree view. For each module and routine shown in this window, Visual PureCoverage displays the routines executed and the lines of code run in each routine-as well as those not run.

Users can also change the information shown in this window to reduce the clutter. Double-clicking the routines in this window opens the code viewer, which shows a color-coded view of the lines of codes hit and missed, which makes it easy to spot code that failed to run.

In a single test, you probably won't run all the code in your program. As a result, Visual PureCoverage lets you run another test and merge the results from subsequent tests into a single combined output. The merged results are especially useful if your testing policy is to ensure that at least 99% of the code in your application has been executed by the tests.

In addition to testing applications through the Visual PureCoverage user interface, you can also execute the same code coverage technology in a nongraphical manner as part of an automated test script. By combining this technology with automated testing scripts and merging the test results, you can ensure not only that your automatic tests have run, but that they executed a meaningful amount of code in your application. This feature will be invaluable for quality assurance groups that use automated regression testing.

Two features I really want with this type of tool-but that aren't provided with Visual PureCoverage-are the ability to graphically view code coverage results and branch analysis. Visual PureCoverage provides only hierarchical text and code-level views of the test results. Users aren't given a graphical view or an easy way to copy this information into a spreadsheet.

Branch analysis is another type of coverage test that indicates whether all of the possible branches of code have been run, such as "if-then-else" blocks.

Developers who sporadically test their code might want to add Rational's code coverage product to their testing environment to gain an added degree of consistency in the quality assurance process.

By: Andy Feibus, CustomBytes