A healthy test management system requires maintenance. When I say healthy I mean a system that gives you accurate reports, doesn’t add a significant overhead to the testers work load and accurately mirrors your real world process.
Rarely does a company implement a test management tool then leave it untouched for years. You’re aiming to improve over time and get more out of your tool as your experience grows. You should be looking to make continuous improvements to the process and tool over time.
First it’s best to take a step back and look at what it is you want to achieve. Consider your goals. Why are you using a test management system? “To track the tests we’re running” is not a good answer. You need to be more specific. Answers like these are more useful:
These are all good goals. With clear goals you can start to think about what you need from your test management tool, the reports and the data that needs to captured. With this we can start to define tasks that take us towards those goals. The tasks we should be carrying out on a regular basis. So for example common maintenance tasks might include…
Data Quality – Are the users entering accurate information? Are testcases written well? Are some users better at keeping information up to data than others? Data quality is key if your reporting is going to accurately reflect what is going on in the real world.
Process Improvement – Using a test management tool implies that users are having to spend time and effort entering information. Forms with mandatory fields where the data isn’t relevant, excessive capture of irrelevant information, work flow that doesn’t fit what someone is doing. All of this reduces a QA engineers enthusiasm for working with a tool and their desire to enter accurate data. Understand that this all takes time away from what really matters. What really matters is carrying out testing.
Expansion – Rarely is it just the QA team that have to execute and record tests. Developers, end users, UAT teams and internal staff will use a product internally before they release the product. Often these other areas of QA are poorly structured and organised (usually because people don’t have a QA background). If you want to improve these areas of testing expanding the usage of a test management tool to these teams can often help. I’m not suggesting you make this some heavyweight, process driven, nightmare that turns people off the moment they see it. Rather that you might get some UAT resource on board by reusing or cloning some existing testcases. Then they can see the benefits of using the system early on. In the end this helps overall tracking and organisation of the effort from start to finish of the project life cycle.
Creating Reports – One of the key reasons we want to capture and organise this information in the first place is so that we can report on it. With accurate reports we know that we’re on track to meet targets or that we need to take corrective action. The question is what exactly is it that we want to see and are we creating briefs that we can act on. Reports for reports sake are a waste of time. We need to know that we’re on track and that we don’t need to change anything is okay. And we need to know when we’re not on track so we can adjust x and y. So consider what it is you want to track and ask the question “is this data that we’ll be able to act on”.
Once we’ve identified a list of maintenance tasks we can start to look at how frequently we’ll carry out these tasks. Remember this is all about continuous improvement. It’s not about kicking of a large project to improve everything in one go. Set up regular half hourly sessions to review data entered (e.g. testcases written) with a different QA engineer within the team every Monday morning. Review forms and workflow setup with a QA team lead on the first of each month. Talk to the UAT lead 3 weeks before UAT kicks off to find out what their requirements are and see how a test management tool will help support them. Review reports and formats with key stakeholders at the end of each month to see if the information being delivered is supporting the project goals
Using and maintaining test management tools imposes an overhead. People spend significant amounts of time putting data in and getting information out. What’s the point if the time spent doesn’t outweigh the benefits? Having clear goals helps you see if you have the balance right between time put it in relation to the the benefits you’re receiving. With those goals in mind you can then put in a little effort on a regular basis to see continuous improvements in your test management process.
