Re-running tests to check that bugs are fixed is an essential part of the test management process. Teams grow, the amount of data generated by completed testcases increases and the number of raised defects expands too. As a result it becomes more difficult to keep track of the retesting. There are several different ways to approach this.
Firstly we can identify all failed cases from a previous cycle and just re run those testcases. This approach has one serious flaw in that defects raised outside of the QA effort don’t then get retested. Having said that this is the simplest approach to execute. We talk more about this approach in practice in this managing retests blog post.
The second approach is based on finding all defects that are in a resolved state and making sure they have been tested before they are closed out. This is usually covered by having defect status values like ‘fixed’ and ‘verified’. Where the developer moves a defect to a fixed status. The QA engineer then pulls out all the bugs that are fixed and is responsible for checking the fixes. Once checked the state is moved to verified. The problem with this approach is that it can be difficult for the QA engineer to identify tests that are relevant.
This is where a test management system with good capabilities for managing traceability comes in to play. Whilst maintaining traceability can seem like a significant overhead at the start it’s situations like this where that overhead pays off significantly. With good traceability in place we can identify the following:
1. From defects in a fixed state we can see which tests we ran that resulted in the defect being raised. If you have a significant amount defects to verify then identifying the relevant testcases can be a time consuming manual process.
2. where we have defects in a fixed state and there is no link to a testcase we know we have to write the testcase before we can verify the defect fix. This is a really important step in the process because it enables us to build a regression pack that can be used to verify futures builds of the application.
A good test management tool should support both of these approaches. In the following video we look at how QAComplete can be used to identify all defects that are fixed and then identify relevant testcases.
Retesting is an absolutely essential part of the test management process. Once the volume of data starts increasing, having a good system in place to identify and track of retests ensures that no software is shipped with bugs that haven’t really been fixed. Key to this though is understanding the way to filter and extract the right data from your test management tool.
