The art of test management can be considered a niche topic within the realm of Software Testing. Yet we can drill down on test management further and look at the development of reports and dashboards as a topic in it’s own right too. All manner of data, statistics and information can be reported on. The more data we enter as part of our process the more information we have to report on. We have tools capable of collecting everything from pass/fail ratios to the exact time taken for execution. In short we’re spoilt for choice when it comes to creating dashboards and reports.
The trick is sorting through all of this data to report on what is important. And to make sure that the important information you’re reporting on is correct. If you think of your reporting as a map you’re following then you need to be using an accurate map. You need to ensure that you’re heading towards the correct destination and that you’re following the right route. If the map is wrong then who knows where your map is taking you. If your map is right but you’re following the wrong route then your wasting time and effort. If your reports and dashboards are giving you the wrong data or data presented poorly, then who knows where your testing is taking you. Probably in the direction of a product release, at best delayed by wasted time. At worst a product release with serious defects.
It’s usually useful to categorise the test management reports and dashboards you need in terms of the reports end users. So categories would include managers (high level status reporting), team leads (reporting on functional areas) and testers (data showing assigned responsibilities). For example the test manager and project manager will find an execution trending report (both manual and automated) essential. A trending report would show the overall progress of tests completed and their status on a time line. This way the managers can see that yesterday 100 tests were completed and only 5 off those failed. This is a useful report for all of the team to see high level progress. It’s essential for the test management team to see high level progress.
From the team leads perspective a report that shows assignment and status is going to be crucial. With an assignment report the lead can see who’s got the largest volume of tests assigned to them and how they are progressing. This gives the lead the chance to redistribute the assignment if needed. The lead can also assign tests based on a team members specialist technical knowledge.
From the testers perspective his or her primary requirement is going to be to see which tests are assigned to him or her. Unlike the managers and leads these reports are going to need to show data across all projects. So whilst a manager or lead will be focused on seeing information at a project level the tester needs to have visibility of assigned cases across all projects. With this the testers reporting requirements are also likely to need to show priorities of assignments too. Without a report to show which are the most important tests to run first the team members are going to be left guessing as to the importance of different projects and individual testcases within those projects.
It’s thinking through reports and dashboards at this level that makes the difference. The difference between following the shortest route or veering off on the wrong route to your destination. Building these reports should be based around clear requirements capture of the users. And implemented in a similar way to the way you deliver product features for your customers.
With an effective test management tool, building the reports that the team needs should be straight forward. No decent tool is without the capability to create these reports. That’s the easy bit. The difficult bit is making sure you create the reports that you really need.
With these tools you have the ability to report on data accurately and present the information your team needs. In this way you’re driving your project with a map that’s been correctly drawn and you’ve picked the best route to get to your destination. Make sure you use your test management tool in such a way that it delivers an accurate map and shows you the best route to your destination.
