The Five Essentials for Software Testing

The following advice should help clarify your thinking about software testing and help you improve the effectiveness and efficiency of your testing. It is helpful to think about software testing in terms of five essential elements:

1. A test strategy that tells you what types of testing and the amount of testing you think will work best at finding the defects that are lurking in the software

2. A testing plan of the actual testing tasks you will need to execute to carry out that strategy

3. Test cases that have been prepared in advance in the form of detailed examples you will use to check that the software will actually meet its requirements

4. Test data consisting of both input test data and database test data to use while you are executing your test cases, and

5. A test environment which you will use to carry out your testing.

If any one of these five elements is missing or inadequate, your test effort will most likely fall far short of what you could otherwise achieve.

Test Strategy

The purpose of testing is to find defects, not to pass easy tests. A test strategy basically tells you which types of testing seem best to do, the order in which to perform them, the proposed sequence of execution, and the optimum amount of effort to put into each test objective to make your testing most effective. A test strategy is based on the prioritized requirements and any other available information about what is important to the customers. Because you will always face time and resource constraints, a test strategy faces up to this reality and tells you how to make the best use of whatever resources you do have to locate most of the worst defects. Without a test strategy, you are apt to waste your time on less fruitful testing and miss using some of your most powerful testing options. You should create the test strategy at about the middle of the design phase as soon as the requirements have settled down.

Testing Plan

A testing plan is simply that part of your project plan that deals with the testing tasks. It details who will do which tasks, starting when, ending when, taking how much effort, and depending on which other tasks. It provides a complete list of all the things that need to be done for testing, including all the preparation work during all of the phases before testing. It shows the dependencies among the tasks to clearly create a critical path without surprises. You will be able to start filling in the details of your testing plan as soon as your test strategy is completed. Both your test strategy and testing plan are subject to change as the project evolves. Modify your strategy first, if you need to, and then your testing plan.

Test Cases

Your test cases (and automated test scripts if called for by your strategy) are prepared based on the strategy which tells you how much of each type of testing to do. Test cases are developed based on prioritized requirements and acceptance criteria for the software, keeping in mind the customer’s emphasis on quality dimensions and the project’s latest risk assessment of what could go wrong. Except for a small amount of ad hoc testing, all of your test cases should be prepared in advance of the start of testing. There are many different approaches to developing test cases. Test case development is an activity performed in parallel with software development. It is just as difficult to do a good job of coming up with test cases as it is to program the system itself. In addition to figuring out what steps to take to test the system, you need to know the requirements and business rules well enough to predict exactly what the expected results should be. Without expected results to compare to actual results, you will not be able to say whether a test will pass or fail. A good test case checks to make sure requirements are being met and has a good chance of uncovering defects.

Test Data

In addition to the steps to perform to execute your test cases, you also need to systematically come up with test data to use. This often equals sets of names, addresses, product orders, or whatever other information the system uses. Since you are probably going to test query functions, change functions and delete functions, you will most likely need a starting database of data in addition to the examples to input. Consider how many times you might need to go back to the starting point of the database to restart the testing and how many new customer names you will need for all the testing in your plan. Test data development is usually done simultaneously with test case development.

Test Environment

You will need a place to do the testing and the right equipment to use. Unless the software is very simple, one PC will not suffice. You will need all of the components of the system as close as possible to what it will eventually be. Test environments may be scaled-down versions of the real thing, but all the parts need to be there for the system to actually run. Building a test environment usually involves setting aside separate regions on mainframe computers and/or servers, networks and PCs that can be dedicated to the test effort and that can be reset to restart testing as often as needed. Sometimes lab rooms of equipment are set aside, especially for performance or usability testing. A wish list of components that will be needed is part of the test strategy, which then needs to be reality checked as part of the test planning process. Steps to set up the environment are part of the testing plan and need to be completed before testing begins.